Lucene Revolution 2012 is now done, and the talk Robert and I gave went well! We showed how we are using automata ( FSA s and FST s) to make great improvements throughout Lucene. You can view the slides here . This was the first time I used Google Docs exclusively for a talk, and I was impressed! The real-time collaboration was awesome: we each could see the edits the other was...
Read More About Finite State Automata in Lucene »

Lucene's TokenStream class produces the sequence of tokens to be indexed for a document's fields. The API is an iterator: you call incrementToken to advance to the next token, and then query specific attributes to obtain the details for that token. For example, CharTermAttribute holds the text of the token; OffsetAttribute has the character start and end offset into the original string...
Read More About Lucene's TokenStreams are actually graphs! »

I'm happy to announce that two Lucene Google Summer of Code projects were accepted for this summer! The first project ( LUCENE-3312 ), proposed by Nikola Tanković, will separate StorableField out of IndexableField , and also fix the longstanding confusing trap that one can retrieve a Document at search time and re-index it, without losing anything. It's unfortunately not true ! ...
Read More About Lucene has two Google Summer of Code students! »

One of the very first thing users encounter when using Apache Solr is its schema. Here they configure the fields that their Documents will contain and the field types which define amongst other things, how field data will be analyzed. Solr’s schema is often touted as one of its major features and you will find it used in almost every Solr component. Yet at the same time, users of Apache Lucene...
Read More About On Schemas and Lucene »

Result grouping and faceting are in essence two different search features. Faceting counts the number of hits for specific field values matching the current query. Result grouping groups documents together with a common property and places these documents under a group. These groups are used as the hits in the search result. Usually result grouping and faceting are used together and a lot of...
Read More About Faceting & result grouping »