Not All Neural Embeddings are Born Equal

不是所有的神经（网络）嵌入都是天生平等的

Abstract

Neural languagemodels learn word representations that capture rich linguistic and conceptualinformation. Here we investigate the embeddings learned by neural machinetranslation models. We show that translation-based embeddings outperform thoselearned by cutting-edge monolingual models at single-language tasks requiringknowledge of conceptual similarity and/or syntactic role. The findings suggestthat, while monolingual models learn information about how concepts arerelated, neural-translation models better capture their true ontologicalstatus.

神经网络语言模型学习词表示来描述丰富的语言和概念信息。这里我们观察通过神经机器翻译模型的（词）嵌入（表示）。我们证明基于翻译的嵌入比那些通过删除边的单个语言模型在单一语言上需要概念相似性和/或语义角色知识的任务上表现更出色。这个发现说明，单一语言模型学习概念是如何关系的知识表现不错，而神经网络翻译模型更好地描述本体的状态。

这篇文章非常有意思。就是观察词的嵌入式表示有那些特征，当然，这里的观察要设计实验，去看嵌入式词表示到底哪里好，哪里不好。