Till startsida
Webbkarta
Till innehåll Läs mer om hur kakor används på gu.se

Learning to compose spatial relations with grounded neural language models

Poster (konferens)
Författare Mehdi Ghanimifard
Simon Dobnik
Publicerad i Second International Workshop on Models and Representations in Spatial Cognition (MRSC)
Publiceringsår 2017
Publicerad vid Institutionen för filosofi, lingvistik och vetenskapsteori
Språk en
Länkar www.cog.uni-tuebingen.de/workshop.h...
www.uni-tuebingen.de/index.php?eID=...
https://gup-server.ub.gu.se/v1/asse...
Ämnesord spatial cognition, machine learning, deep learning, neural networks, language model, grounding, language, perception
Ämneskategorier Kognitionsforskning, Lingvistik, Datalogi, Datorlingvistik

Sammanfattning

Neural language models are common in recent applications of neural networks in machine translation, speech recognition, and image captioning. Vector representation of linguistic units (such as words) are inputs to these models which learn to generate meaningful compositions of them. The intermediate composed symbolic representations (word vectors) can also be grounded in meaningful composed representations of another modality (images, or sensors). An interesting question is what is the correspondence between linguistic compositions and compositions in another modality. Common evaluation metrics do not express sufficiently the performance of different neural models in learning compositions. They tell us about the performance of the learned representations on the evaluation dataset but they do not say anything about the internal structure of these representations. The question we want to answer is to what degree neural language models learn how to compose grounded spatial expressions and to what degree the learned model corresponds to the model that generated the composed data. We produce a synthetic dataset of composed spatial templates corresponding to composed linguistic expressions using a known compositional function. We use simple spatial templates from Logan and Sadler (1996). Our neural network learns composed spatial templates from individual locations of target-landmark pairs incrementally grounded in the sequence of words that have been passed to it. To evaluate the performance of the network we compare generations from different setups of grounded neural language models with the original composed templates and consider their similarity with the (known) function that produced them.

Sidansvarig: Webbredaktion|Sidan uppdaterades: 2012-09-11
Dela:

På Göteborgs universitet använder vi kakor (cookies) för att webbplatsen ska fungera på ett bra sätt för dig. Genom att surfa vidare godkänner du att vi använder kakor.  Vad är kakor?