DocValues - Apache Solr Reference Guide - Apache Software Foundation

DocValues - Apache Solr Reference Guide - Apache Software Foundation

The standard way that Solr builds the index is with an inverted index. This style builds a list of terms found in all the documents in the index and next to each term is a list of documents that the term appears in (as well as how many times the term appears in that document). This makes search very fast - since users search by terms, having a ready list of term-to-document values makes the query process faster.

For other features that we now commonly associate with search, such as sorting, faceting, and highlighting, this approach is not very efficient. The faceting engine, for example, must look up each term that appears in each document that will make up the result set and pull the document IDs in order to build the facet list. In Solr, this is maintained in memory, and can be slow to load (depending on the number of documents, terms, etc.).

In Lucene 4.0, a new approach was introduced. DocValue fields are now column-oriented fields with a document-to-value mapping built at index time. This approach promises to relieve some of the memory requirements of the fieldCache and make lookups for faceting, sorting, and grouping much faster.

<field name="manu_exact" type="string" indexed="false" stored="false" docValues="true" />

DocValues are only available for specific field types. The types chosen determine the underlying Lucene docValue type that will be used. The available Solr field types are:

• String fields of type StrField.
  
  • If the field is single-valued (i.e., multi-valued is false), Lucene will use the SORTED type.
  • If the field is multi-valued, Lucene will use the SORTED_SET type.
• Any Trie* fields.
  
  • If the field is single-valued (i.e., multi-valued is false), Lucene will use the NUMERIC type.
  • If the field is multi-valued, Lucene will use the SORTED_SET type.
• UUID fields

The default implementation employs a mixture of loading some things into memory and keeping some on disk.
<fieldType name="string_in_mem_dv" class="solr.StrField" docValues="true" docValuesFormat="Memory" />

Lucene index back-compatibility is only supported for the default codec. If you choose to customize the docValuesFormat in your schema.xml, upgrading to a future version of Solr may require you to either switch back to the default codec and optimize your index to rewrite it into the default codec before upgrading, or re-build your entire index from scratch after upgrading.

Read full article from DocValues - Apache Solr Reference Guide - Apache Software Foundation