Analysis of Language Change in Collaborative Instruction Following

@inproceedings{Effenberger2022AnalysisOL,
  title={Analysis of Language Change in Collaborative Instruction Following},
  author={Anna Effenberger and Eva Yan and Rhia Singh and Alane Suhr and Yoav Artzi},
  booktitle={SCIL},
  year={2022}
}
We analyze language change over time in a collaborative, goal-oriented instructional task, where utility-maximizing participants form conventions and increase their expertise. Prior work studied such scenarios mostly in the context of reference games, and consistently found that language complexity is reduced along multiple dimensions, such as utterance length, as conventions are formed. In contrast, we find that, given the ability to increase instruction utility, instructors increase language… 

Pragmatics in Grounded Language Learning: Phenomena, Tasks, and Modeling Approaches

People rely heavily on context to enrich meaning beyond what is literally said, enabling concise but effective communication. To interact successfully and naturally with people, user-facing artificial

From partners to populations: A hierarchical Bayesian account of coordination and convention

A mixed-effects logic regression model including intercepts and random effects of trial number for each simulated trajectory finds that communicative accuracy steadily improves over time across all conditions, and examines the effective vocabulary sizes used by speakers in each condition.

MUG: Interactive Multimodal Grounding on User Interfaces

M UG, a novel interactive task for multimodal grounding where a user and an agent work collaboratively on an interface screen, allows multiple rounds of interactions such that upon seeing the agent responses, the user can give further commands for the agent to refine or even correct its actions.

References

SHOWING 1-10 OF 21 REFERENCES

Characterizing the dynamics of learning in repeated reference games

This work releases an open corpus of extended dyadic interactions in a classic repeated reference game task where pairs of participants had to coordinate on how to refer to initially difficult-to-describe tangram stimuli.

Executing Instructions in Situated Collaborative Interactions

This work introduces a learning approach focused on recovery from cascading errors between instructions, and modeling methods to explicitly reason about instructions with multiple goals, and observes how users adapt to the system abilities.

Convergence of Syntactic Complexity in Conversation

Using corpus data of spoken dialogue, we examine the convergence of syntactic complexity levels between interlocutors in natural conversations, as it occurs within spans of topic episodes. The

Convention-formation in iterated reference games

Results from a large-scale, multi-player replication of the classic tangrams task are presented, focusing on three foundational properties of conventions: arbitrariness, stability, and reduction of utterance length over time, which motivate a theory of convention-formation where agents assume others are using language with such knowledge.

Changes in reference phrases as a function of frequency of usage in social interaction: a preliminary study

Pairs of subjects interacted in a problem-solving task which required them to communicate about ambiguous figures. The length of the reference phrase for each figure was calculated. A negative

Goal-driven Answers in the Cards Dialogue Corpus

The resolvedness conditions for questions are not semantically invariant, but rather heavily influenced by the goals and intentions of the discourse participants. One of the chief innovations of

Modeling Expert Effects and Common Ground Using Questions Under Discussion

The QUD-graph model enables us to quantify the experthood of a speaker relative to a fixed domain and to characterize the ways in which rich common ground facilitates more efficient communication.

Corpus Evidence for Preference-Driven Interpretation

The Cards corpus of task-oriented dialogues is presented and it is shown how it can inform study of the ways in which discourse is goal- and preference-driven.

Pragmatic Language Interpretation as Probabilistic Inference

Colors in Context: A Pragmatic Neural Model for Grounded Language Understanding

We present a model of pragmatic referring expression interpretation in a grounded communication task (identifying colors from descriptions) that draws upon predictions from two recurrent neural