This article is an extensive survey of the applications of graph neural networks (GNN) to the domain of natural language processing (NLP).
In it, the authors introduce a taxonomy to decompose the matter into its different components and give a comprehensive list of the recent work done for each component: graph construction, graph representation learning, encoder-decoder processes, and applications.
In Section 2, some of the classical non-GNN graph-based algorithms used in NLP are recalled, together with a list of their applications: random walk, graph clustering methods, graph matching, and label propagation algorithm.
Section 3 gives the foundational ideas of GNNs in full mathematical details, such as filtering, message passing, pooling, etc., as well as the mathematical building blocks of GNNs.
Section 4 focuses on the construction of graphs in the context of NLP and follows a detailed taxonomy. A main distinction is made between static and dynamic constructions. While the former require domain knowledge and manual inputs, the latter allow for automatic learning of the graph structure/adjacency matrix. Many models from recent literature are introduced with their formulas.
Section 5 is dedicated to representation learning, that is, the techniques involved in computing vector representations of the graph data obtained by the methods from the previous section.
In Section 6, the different encoder-decoder architectures are described, such as graph-to-sequence and graph-to-graph models.
In each of the above sections the models and algorithms are formally described together with their technical aspects.
Section 7 contains a long list of specific applications of GNNs in NLP. For each application its object and methods are explained. The authors also list the metrics and benchmark datasets against which the different models are evaluated. These applications are classified into 12 general categories, detailed in Table 7.1.
Finally, the survey ends with a description of challenges and future directions.
Overall this survey is a good introduction to the field, containing many valuable details. The text contains a wealth of references taken from the article’s thorough bibliography.