Compounding-aware Word Embedding for Improved Semantic Representation

Abstract

Existing word embedding approaches may not adequately capture the inherent complexities of a language, e.g. the word compounding phenomenon. While a class of data-driven approaches has been shown to be effective in embedding words of languages that are relatively simple as per inflections and compounding characteristics (e.g. English), an open area of investigation is ways of integrating language-specific characteristics within the framework of an embedding model. In this work, we explore how words in a highly agglutinative language, e.g. German, can be embedded more effectively by additionally taking into account the contexts around the constituents of a compound word. We propose a word transformation based generalization of the skip-gram algorithm to address these relationships between a compound word and its constituents. Our experiments on standard German word-pair similarity datasets and polarity classification of German compounds confirm our hypothesis that modeling contextual relationships between a compound word and its constituents can improve word representations.