Chapter 6. Manual indexing

6.1. Indexing

It is sometimes useful to index an entity even if this entity is not inserted or updated to the database. This is for example the case when you want to build your index for the first time. FullTextSession.index() allows you to do so.

Example 6.1. Indexing an entity via FullTextSession.index()

FullTextSession fullTextSession = Search.getFullTextSession(session);
Transaction tx = fullTextSession.beginTransaction();
for (Customer customer : customers) {
    fullTextSession.index(customer);
}
tx.commit(); //index are written at commit time    

For maximum efficiency, Hibernate Search batches index operations and executes them at commit time. If you expect to index a lot of data, however, you need to be careful about memory consumption since all documents are kept in a queue until the transaction commit. You can potentially face an OutOfMemoryException. To avoid this exception, you can use fullTextSession.flushToIndexes(). Every time fullTextSession.flushToIndexes() is called (or if the transaction is committed), the batch queue is processed (freeing memory) applying all index changes. Be aware that once flushed changes cannot be rolled back.

Note

hibernate.search.worker.batch_size has been deprecated in favor of this explicit API which provides better control

Other parameters which also can affect indexing time and memory consumption are:

  • hibernate.search.[default|<indexname>].indexwriter.[batch|transaction].max_buffered_docs
  • hibernate.search.[default|<indexname>].indexwriter.[batch|transaction].max_field_length
  • hibernate.search.[default|<indexname>].indexwriter.[batch|transaction].max_merge_docs
  • hibernate.search.[default|<indexname>].indexwriter.[batch|transaction].merge_factor
  • hibernate.search.[default|<indexname>].indexwriter.[batch|transaction].ram_buffer_size
  • hibernate.search.[default|<indexname>].indexwriter.[batch|transaction].term_index_interval

These parameters are Lucene specific and Hibernate Search is just passing these parameters through - see Section 3.8, “Tuning Lucene indexing performance” for more details.

Example 6.2. Efficiently indexing a given class (useful for index (re)initialization)

fullTextSession.setFlushMode(FlushMode.MANUAL);
fullTextSession.setCacheMode(CacheMode.IGNORE);
transaction = fullTextSession.beginTransaction();
//Scrollable results will avoid loading too many objects in memory
ScrollableResults results = fullTextSession.createCriteria( Email.class )
    .setFetchSize(BATCH_SIZE)
    .scroll( ScrollMode.FORWARD_ONLY );
int index = 0;
while( results.next() ) {
    index++;
    fullTextSession.index( results.get(0) ); //index each element
    if (index % BATCH_SIZE == 0) {
        fullTextSession.flushToIndexes(); //apply changes to indexes
        fullTextSession.clear(); //clear since the queue is processed
    }
}
transaction.commit();

Try to use a batch size that guarantees that your application will not run out of memory.

6.2. Purging

It is equally possible to remove an entity or all entities of a given type from a Lucene index without the need to physically remove them from the database. This operation is named purging and is also done through the FullTextSession.

Example 6.3. Purging a specific instance of an entity from the index

FullTextSession fullTextSession = Search.getFullTextSession(session);
Transaction tx = fullTextSession.beginTransaction();
for (Customer customer : customers) {
    fullTextSession.purge( Customer.class, customer.getId() );
}
tx.commit(); //index are written at commit time    

Purging will remove the entity with the given id from the Lucene index but will not touch the database.

If you need to remove all entities of a given type, you can use the purgeAll method. This operation remove all entities of the type passed as a parameter as well as all its subtypes.

Example 6.4. Purging all instances of an entity from the index

FullTextSession fullTextSession = Search.getFullTextSession(session);
Transaction tx = fullTextSession.beginTransaction();
fullTextSession.purgeAll( Customer.class );
//optionally optimize the index
//fullTextSession.getSearchFactory().optimize( Customer.class );
tx.commit(); //index are written at commit time    

It is recommended to optimize the index after such an operation.

Note

Methods index, purge and purgeAll are available on FullTextEntityManager as well.