See also GloVe, wiki2vec, WordRank, sense2vec. A Continuous embedding method.

Software and Libraries #

• https://code.google.com/p/word2vec/ by Tomas Mikolov
• Gensim
• TensorFlow
  • Vector Representations of Words

Intro and Theory #

• A Neural Probabilistic Language Model
• Distributed Representations of Words and Phrases and their Compositionality

It was recently shown that the word vectors capture many linguistic regularities, for example vector operations vector('Paris') - vector('France') + vector('Italy') results in a vector that is very close to vector('Rome'), and vector('king') - vector('man') + vector('woman') is close to vector('queen') [3, 1].

• Paper/Levy2014: Neural World Embedding as Implicit Matrix Factorization
• Paper/Levy2014a: Linguistic Regularities in Sparse and Explicit Word Representations
• word2vec Parameter Learning Explained

Methods #

• Skip-gram

Tutorials #

• http://rare-technologies.com/word2vec-tutorial/
• Deep Learning Basics: Neural Networks, Backpropagation and Stochastic Gradient Descent
  • Word2Vec Tutorial Part I: The Skip-Gram Model
• 한국어와 NLTK, Gensim의 만남
• Demystifying Word2Vec
• Jurafsky and Martin: Vector Semantics, Part II
• Chris McCormick Word2Vec Tutorial - The Skip-Gram Model
• TensorFlow Vector Representations of Words

Articles #

• http://deeplearning4j.org/word2vec.html#crazy

• what does the output vector of a word in word2vec represent?

• Semantic Word Embeddings

  • Word Embeddings: Explaining their properties

• MESSING AROUND WITH WORD2VEC

Presentations #

• Machine Perception with Neural Networks by Ilya Sutskever
• Text By the Bay 2015: Chris Moody, A Word is Worth a Thousand Vectors

Topics #

Paragaraph vectors #

• Distributed Representations of Sentences and Documents
  • slides

• Document Embedding with Paragraph Vectors

• Deep learning with paragraph2vec

  • Doc2vec tutorial

Comparing multiple sets of vectors #

• Code for the bilingual lexicon induction experiments
• Paper/Hamilton2016
  • http://nlp.stanford.edu/projects/histwords/

Optimal dimension #

• On the Dimensionality of Word Embedding

Sentiment analysis #

• Learning Word Vectors for Sentiment Analysis

Application to networks #

• Paper/Perozzi2014

Scalability #

• Network-Efficient Distributed Word2vec Training System for Large Vocabularies

Gender bias #

• Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings

Clinical concepts #

• Clinical concept embedding

Incoming Links #

Related Articles (Article 0) #

• Continuous embedding
• Deep learning
• Gender bias
• Gensim
• Hierarchical softmax
• Neural network
• Paper/Perozzi2014
• Sentiment analysis
• Skip-gram
• Tomas Mikolov
• Word embedding
• WordRank
• sense2vec
• wiki2vec

Related Scholarly Articles (ScholarlyArticle 1) #

• Paper/Levy2014
• Paper/Levy2014a

Suggested Pages #

• 0.310 Graph embedding
• 0.154 Network geometry
• 0.126 Softmax function
• 0.054 tf-idf
• 0.045 Softmax
• 0.039 Sentence embedding
• 0.026 Lexical feature selection
• 0.024 tfidf
• 0.022 Recurrent neural network
• 0.018 Finkbeiner test
• More suggestions...