Lucene 4.X 倒排索引原理与实现: (3) Term Dictionary和Index文件 (FST详细解析)_搜索引擎技术_酷勤网



我们来看最复杂的部分,就是Term Dictionary和Term Index文件,Term Dictionary文件的后缀名为tim,Term Index文件的后缀名是tip,格式如图所示。

image

Term Dictionary文件首先是一个Header,接下来是PostingsHeader,这两个的格式一致,但是保存的是不同的信息。SkipInterval是跳跃表的跳的幅度,MaxSkipLevels是跳跃表的层数,SkipMinimun是应用跳跃表的最小倒排表长度,接下来就是Term的部分了。

在tim文件中,Term是分成Block进行保存的,如何将Term进行分块,则需要和tip文件配合。Term Index文件对于每一个Field都保存一个FSTIndex来帮助快速定位tim文件中属于这个Field的Term的位置,由于FSTIndex的长度不同,为了快速定位某个Field的位置,则应用指针列表规则,为每一个Field保存了指向这个Field的FSTIndex的指针。


Read full article from Lucene 4.X 倒排索引原理与实现: (3) Term Dictionary和Index文件 (FST详细解析)_搜索引擎技术_酷勤网


No comments:

Post a Comment

Labels

Algorithm (219) Lucene (130) LeetCode (97) Database (36) Data Structure (33) text mining (28) Solr (27) java (27) Mathematical Algorithm (26) Difficult Algorithm (25) Logic Thinking (23) Puzzles (23) Bit Algorithms (22) Math (21) List (20) Dynamic Programming (19) Linux (19) Tree (18) Machine Learning (15) EPI (11) Queue (11) Smart Algorithm (11) Operating System (9) Java Basic (8) Recursive Algorithm (8) Stack (8) Eclipse (7) Scala (7) Tika (7) J2EE (6) Monitoring (6) Trie (6) Concurrency (5) Geometry Algorithm (5) Greedy Algorithm (5) Mahout (5) MySQL (5) xpost (5) C (4) Interview (4) Vi (4) regular expression (4) to-do (4) C++ (3) Chrome (3) Divide and Conquer (3) Graph Algorithm (3) Permutation (3) Powershell (3) Random (3) Segment Tree (3) UIMA (3) Union-Find (3) Video (3) Virtualization (3) Windows (3) XML (3) Advanced Data Structure (2) Android (2) Bash (2) Classic Algorithm (2) Debugging (2) Design Pattern (2) Google (2) Hadoop (2) Java Collections (2) Markov Chains (2) Probabilities (2) Shell (2) Site (2) Web Development (2) Workplace (2) angularjs (2) .Net (1) Amazon Interview (1) Android Studio (1) Array (1) Boilerpipe (1) Book Notes (1) ChromeOS (1) Chromebook (1) Codility (1) Desgin (1) Design (1) Divide and Conqure (1) GAE (1) Google Interview (1) Great Stuff (1) Hash (1) High Tech Companies (1) Improving (1) LifeTips (1) Maven (1) Network (1) Performance (1) Programming (1) Resources (1) Sampling (1) Sed (1) Smart Thinking (1) Sort (1) Spark (1) Stanford NLP (1) System Design (1) Trove (1) VIP (1) tools (1)

Popular Posts