how does the huffman tree work in word2vec?
There are two major approaches to training the word2vec model. One is the so called "hierarchical softmax" and the other is a process called "Noise Contrastive Estimation". For the "hierarchical softmax" method, a Huffman binary tree is used [Mikolov et al. 2013a; Mikolov et al. 2013b].
Before reading about word2vec, I was familiar with Huffman coding as a means of lossless data compression, but I was confused about how exactly the tree is constructed, and then how it is used in word2vec's "hierarchical softmax" method.
Read full article from how does the huffman tree work in word2vec?
No comments:
Post a Comment