RealitySketch: Embedding Responsive Graphics and Visualizations in AR through Dynamic Sketching

Ryo Suzuki , Rubaiat Habib Kazi , Li-Yi Wei , Stephen DiVerdi , Wilmot Li , Daniel Leithinger



We present RealitySketch, an augmented reality interface for sketching interactive graphics and visualizations. In recent years, an increasing number of AR sketching tools enable users to draw and embed sketches in the real world. However, with the current tools, sketched contents are inherently static, floating in mid air without responding to the real world. This paper introduces a new way to embed dynamic and responsive graphics in the real world. In RealitySketch, the user draws graphical elements on a mobile AR screen and binds them with physical objects in real-time and improvisational ways, so that the sketched elements dynamically move with the corresponding physical motion. The user can also quickly visualize and analyze real-world phenomena through responsive graph plots or interactive visualizations. This paper contributes to a set of interaction techniques that enable capturing, parameterizing, and visualizing real-world motion without pre-defined programs and configurations. Finally, we demonstrate our tool with several application scenarios, including physics education, sports training, and in-situ tangible interfaces.

Keywords:  Augmented RealityEmbedded Data VisualizationReal Time AuthoringSketching InterfacesTangible Interaction


Ryo Suzuki, Rubaiat Habib Kazi, Li-Yi Wei, Stephen DiVerdi, Wilmot Li, Daniel LeithingerRealitySketch: Embedding Responsive Graphics and Visualizations in AR through Dynamic SketchingIn Proceedings of the Annual ACM Symposium on User Interface Software and Technology (UIST '20)ACM, New York, NY, USA  Page: 1-16.  DOI: