Catalytic Role of Noise and Necessity of Inductive Biases in the Emergence of Compositional Communication

Prelegent(ci)

Łukasz Kuciński

Afiliacja

IM PAN

Termin

21 października 2021 12:15

Informacje na temat wydarzenia

online seminar: meet.google.com/ooi-zxye-dxa

Seminarium

Seminarium "Uczenie maszynowe"

---

 We will talk about our recent compositionality paper accepted at NeurIPS 2021. Communication is compositional if complex signals can be represented as a combi-nation of simpler subparts. In this paper, we theoretically show that inductive biases on both the training framework and the data are needed to develop a compositional communication. Moreover, we prove that compositionality spontaneously arises in the signaling games, where agents communicate over a noisy channel. We experi-mentally confirm that a range of noise levels, which depends on the model and the data, indeed promotes compositionality. Finally, we provide a comprehensive study of this dependence and report results in terms of recently studied compositionality metrics: topographical similarity, conflict count, and context independence

---