Sharded architecture / pagination problem · Issue #175 · bloomberg/lucene-solr
But because the rescoring happens locally in each shard, I believe we have still problems with pagination in a shared environment.
Of course it is a minor issue that will appear only if you rerank few docs per shard or in deep pagination examples.
The problem resides in the fact that the aggregator of the results from the shards, will merge search results from different shards and then rank according to the score.
Each shard will return results in the proper order ( first reranked docs and following the original scored ones).
But score wise, rescored docs can have a smaller score than the original scored.
When the aggregation happens, original scored ones can surpass in the ranking the rescored ones.
I discussed briefly with Diego and his suggestion is to normalise the scores to make the re--ranked ones always with an higher score of the original scored.
I do agree this can be a possible solution to be consistent and maintain the ordinal nature of the score in Solr, that we are currently breaking at the reRankDocs point.
Read full article from Sharded architecture / pagination problem · Issue #175 · bloomberg/lucene-solr
No comments:
Post a Comment