本文为荷兰阿姆斯特丹大学（作者：Thomas Kipf）的博士论文，共178页。

在这篇论文中，我们提出了一种新颖的方法来利用结构化数据进行机器学习。我们所提出的方法主要基于以图形式构造神经网络的模型表示和计算主题，这使得从具有显式和隐式模块结构的数据学习时可以改进泛化性能。

我们的贡献如下：

•我们提出了将图卷积网络（GCN）（Kipf和Welling，2017；第3章）用于图结构数据中节点的半监督分类。GCN是一种图神经网络，在图中执行参数化的消息传递操作，建模为谱图卷积的一阶近似。GCN发表时在许多无向图数据集中实现了节点级分类任务的最新性能。

•我们提出了图自动编码器（GAE）（Kipf和Welling，2016；第4章），用于图结构数据中的无监督学习和链接预测。GAE利用一个基于图神经网络的编码器和一个基于成对记分函数重建图链接的解码器。我们进一步提出了一个以概率生成模型为框架，利用变分推理进行训练的模型变量，我们称之为变分GAE。GAE和变分GAE特别适合于在没有节点标记的图上进行表示学习。

•我们提出关系GCN（Schlichtkrull and Kipf et al.，2018；第5章），将GCN模型扩展到具有多个边缘类型的有向关系图。关系GCN非常适合于对关系数据进行建模，我们展示了它在知识库中半监督实体分类中的应用。

•我们提出了神经关系推理（NRI）（Kipf和Fetaya等人，2018；第6章）来发现交互系统中潜在的关系结构。NRI将图神经网络与基于图中边缘类型的概率潜在变量模型相结合。我们将NRI应用于相互作用的动力学系统，如物理学中的多粒子系统。

•我们提出了组合模仿学习和执行（CompILE）（Kipf等人，2019；第7章），这是序列行为数据中结构发现的模型。CompILE使用一种新的可微序列分割机制来发现与自动编码有意义的子序列或行为子程序。潜在代码可以被执行和重新组合以产生新的行为。

•我们提出了对比训练的结构化世界模型（C-SWM）（Kipf等人，2020；第8章），用于在没有监督的情况下从原始像素观测值学习环境的对象因子化模型。C-SWM使用图神经网络将环境表示成图的形式，其中节点表示对象，边表示受某个动作影响的成对关系或交互作用。C-SWM采用无像素损失的对比学习方法进行训练，非常适合于具有合成结构的环境学习模型。

In this thesis, Deep Learning with Graph-Structured Representations, we proposenovel approaches to machine learning with structured data. Our proposedmethods are largely based on the theme of structuring the representations andcomputations of neural network-based models in the form of a graph, whichallows for improved generalization when learning from data with both explicitand implicit modular structure.

Our contributions are as follows:

• We propose graph convolutional networks (GCNs) (Kipf and Welling,2017; Chapter 3) for semi-supervised classification of nodes in graph-structured data. GCNs are a form of graph neural network that perform parameterized message-passing operations in a graph, modeled as afirst-order approximation to spectral graph convolutions. GCNs achievedstate-of-the-art performance in node-level classification tasks in a numberof undirected graph datasets at the time of publication.

• We propose graph auto-encoders (GAEs) (Kipf and Welling, 2016; Chapter 4) for unsupervised learning and link prediction in graph-structureddata. GAEs utilize an encoder based on graph neural networks and adecoder that reconstructs links in a graph based on a pairwise scoringfunction. We further propose a model variant framed as a probabilisticgenerative model that is trained using variational inference, which wename variational GAE. GAEs and variational GAEs are particularly suitedfor representation learning on graphs in the absence of node labels.

• We propose relational GCNs (Schlichtkrull and Kipf et al., 2018; Chapter5) that extend the GCN model to directed, relational graphs with multiple edge types. Relational GCNs are well-suited for modeling relationaldata and we demonstrate an application to semi-supervised entity classification in knowledge bases.

• We propose neural relational inference (NRI) (Kipf and Fetaya et al., 2018;Chapter 6) for discovery of latent relational structure in interacting systems. NRI combines graph neural networks with a probabilistic latentvariable model over edge types in a graph. We apply NRI to model interacting dynamical systems, such as multi-particle systems in physics.

• We propose compositional imitation learning and execution (CompILE)(Kipf et al., 2019; Chapter 7), a model for structure discovery in sequentialbehavioral data. CompILE uses a novel differentiable sequence segmentation mechanism to discover and auto-encode meaningful sub-sequencesor sub-programs of behavior in the context of imitation learning. Latentcodes can be executed and re-composed to produce novel behavior.

• We propose contrastively-trained structured world models (C-SWMs)(Kipf et al., 2020; Chapter 8) for learning object-factorized models of environments from raw pixel observations without supervision. C-SWMsuse graph neural networks to structure the representation of an environment in the form of a graph, where nodes represent objects and edgesrepresent pairwise relations or interactions under the influence of an action. C-SWMs are trained using contrastive learning without pixel-basedlosses and are well-suited for learning models of environments with compositional structure.

1.  引言

2.  项目背景

3.  用于半监督分类的图卷积网络

4.  基于图自编码的链接预测

5.  基于图卷积网络的模型关系数据

6.  用于交互系统的神经关系推理

7.  组合模仿学习与执行

8.  结构化世界模型的对比学习

9.  结论

更多精彩文章请关注公众号：