Analysis of Language Change in Collaborative Instruction Following

@inproceedings{Effenberger2022AnalysisOL,
  title={Analysis of Language Change in Collaborative Instruction Following},
  author={Anna Effenberger and Eva Yan and Rhia Singh and Alane Suhr and Yoav Artzi},
  booktitle={SCIL},
  year={2022}
}
We analyze language change over time in a collaborative, goal-oriented instructional task, where utility-maximizing participants form conventions and increase their expertise. Prior work studied such scenarios mostly in the context of reference games, and consistently found that language complexity is reduced along multiple dimensions, such as utterance length, as conventions are formed. In contrast, we find that, given the ability to increase instruction utility, instructors increase language… 

MUG: Interactive Multimodal Grounding on User Interfaces

We present M UG , a novel interactive task for multimodal grounding where a user and an agent work collaboratively on an interface screen. Prior works modeled multimodal UI grounding in one round:

From partners to populations: A hierarchical Bayesian account of coordination and convention

TLDR
A mixed-effects logic regression model including intercepts and random effects of trial number for each simulated trajectory finds that communicative accuracy steadily improves over time across all conditions, and examines the effective vocabulary sizes used by speakers in each condition.

References

SHOWING 1-10 OF 21 REFERENCES

Characterizing the dynamics of learning in repeated reference games

TLDR
This work releases an open corpus of extended dyadic interactions in a classic repeated reference game task where pairs of participants had to coordinate on how to refer to initially difficult-to-describe tangram stimuli.

Executing Instructions in Situated Collaborative Interactions

TLDR
This work introduces a learning approach focused on recovery from cascading errors between instructions, and modeling methods to explicitly reason about instructions with multiple goals, and observes how users adapt to the system abilities.

Convergence of Syntactic Complexity in Conversation

Using corpus data of spoken dialogue, we examine the convergence of syntactic complexity levels between interlocutors in natural conversations, as it occurs within spans of topic episodes. The

Convention-formation in iterated reference games

TLDR
Results from a large-scale, multi-player replication of the classic tangrams task are presented, focusing on three foundational properties of conventions: arbitrariness, stability, and reduction of utterance length over time, which motivate a theory of convention-formation where agents assume others are using language with such knowledge.

Changes in reference phrases as a function of frequency of usage in social interaction: a preliminary study

Pairs of subjects interacted in a problem-solving task which required them to communicate about ambiguous figures. The length of the reference phrase for each figure was calculated. A negative

Goal-driven Answers in the Cards Dialogue Corpus

The resolvedness conditions for questions are not semantically invariant, but rather heavily influenced by the goals and intentions of the discourse participants. One of the chief innovations of

Modeling Expert Effects and Common Ground Using Questions Under Discussion

TLDR
The QUD-graph model enables us to quantify the experthood of a speaker relative to a fixed domain and to characterize the ways in which rich common ground facilitates more efficient communication.

Naturalizing a Programming Language via Interactive Learning

TLDR
This work starts with a core programming language and allows users to “naturalize” the core language incrementally by defining alternative, more natural syntax and increasingly complex concepts in terms of compositions of simpler ones.

Corpus Evidence for Preference-Driven Interpretation

TLDR
The Cards corpus of task-oriented dialogues is presented and it is shown how it can inform study of the ways in which discourse is goal- and preference-driven.

Pragmatic Language Interpretation as Probabilistic Inference