Reactile: Programming Swarm User Interfaces through Direct Physical Manipulation

Ryo Suzuki , Jun Kato , Mark D. Gross , Tom Yeh



We explore a new approach to programming swarm user interfaces (Swarm UI) by leveraging direct physical manipulation. Existing Swarm UI applications are written using a robot programming framework: users work on a computer screen and think in terms of low-level controls. In contrast, our approach allows programmers to work in physical space by directly manipulating objects and think in terms of high-level interface design. Inspired by current UI programming practices, we introduce a four-step workflow-create elements, abstract attributes, specify behaviors, and propagate changes-for Swarm UI programming. We propose a set of direct physical manipulation techniques to support each step in this workflow. To demonstrate these concepts, we developed Reactile, a Swarm UI programming environment that actuates a swarm of small magnets and displays spatial information of program states using a DLP projector. Two user studies-an in-class survey with 148 students and a lab interview with eight participants-confirm that our approach is intuitive and understandable for programming Swarm UIs.

Keywords:  Direct ManipulationTangible ProgrammingSwarm User InterfacesProgramming By Demonstration


Ryo Suzuki, Jun Kato, Mark D. Gross, Tom YehReactile: Programming Swarm User Interfaces through Direct Physical ManipulationIn Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI '18)ACM, New York, NY, USA  Page: 1-13.  DOI: