Novel Programming Interfaces

As robots become more commonly used in everyday scenarios, a wide range of individuals will need appropriate tools to easily and efficiently create programs that specify what their robot should do or how it should behave. This work seeks to make programming human robot interactions more natural for non-roboticists by employing programming language techniques, such as program synthesis, to translate multimodal user input into a full robot program.

---

[HRI 2023] Sketching Robot Programs On the Fly

David Porfirio, Laura Stegner, Maya Cakmak, Allison Sauppé, Aws Albarghouthi, and Bilge Mutlu

Resources: PDF - Video - Presentation - Github - OSF

We present a novel, multimodal, and on-the-fly development system, Tabula. Inspired by a formative design study with a prototype, Tabula leverages a combination of spoken language for specifying the core of a robot task and sketching for contextualizing the core. The result is that developers can script partial, sloppy versions of robot programs to be completed and refined by a program synthesizer.

---

[CHI 2021] Figaro: A Tabletop Authoring Environment for Human-Robot Interaction

David Porfirio, Laura Stegner, Maya Cakmak, Allison Sauppé, Aws Albarghouthi, and Bilge Mutlu

Resources: PDF - Video - Presentation - Github

Figaro is a tabletop authoring environment where demonstrators use instrumented figurines to play out scenes of human-robot interactions. The positions, movements, behaviors, and speech of the figurines is recorded for each scene. Figaro then synthesizes a full human-robot interaction program that can be executed on a robot.

[back]