July 31, 2017

Enriching Word Vectors with Subword Information

TACL, Association for Computational Linguistics (ACL 2017)

By: Armand Joulin, Edouard Grave, Piotr Bojanowski, Tomas Mikolov

Abstract

Continuous word representations, trained on large unlabeled corpora are useful for many natural language processing tasks. Popular models that learn such representations ignore the morphology of words, by assigning a distinct vector to each word. This is a limitation, especially for languages with large vocabularies and many rare words. In this paper, we propose a new approach based on the skipgram model, where each word is represented as a bag of character n-grams.