Solr Multithreaded concurrent atomic updates
Solr Multithreaded concurrent atomic updates problem:
Solr has few limitations for the data ingestion, as it doesn't provide row level lock over document.
I face this problem while uploading data in bulk to solr5 in multithread environment and I solved it by solrj client side lock.
When concurrent threads try to make atomic update on a multivalued field of a document at the same time, few threads changes get overridden and it happens because last thread update take sometime to get indexed.
Data ingestion scenario:
There are two tables in RDBMS and I need to denormalize in solr, Steps I was following for atomic/partial document update-
1- Fetch the existing document.
2- Update the single value fields if required and add/set the new values to multivlaued fields.
3- Update the final document back to solr.
Read full article from Solr Multithreaded concurrent atomic updates
No comments:
Post a Comment