Vylomova, E; Rimell, L; Cohn, T; Baldwin, T; Erk, K; Smith, NA
(The Association for Computational Linguistics, 2016)
Recent work on word embeddings has shown that simple vector subtraction over
pre-trained embeddings is surprisingly effective at capturing different lexical
relations, despite lacking explicit supervision. Prior work has evaluated this
intriguing result using a word analogy prediction formulation and hand-selected
relations, but the generality of the finding over a broader range of lexical
relation types and different learning settings has not been evaluated. In this
paper, we carry out such an evaluation in two learning settings: (1) spectral
clustering to induce word relations, and (2) supervised learning to classify
vector differences into relation types. We find that word embeddings capture a
surprising amount of information, and that, under suitable supervised training,
vector subtraction generalises well to a broad range of relations, including
over unseen lexical items.