Abstract

We introduce Augmented Physics, a machine learning-integrated authoring tool designed for creating embedded interactive physics simulations from static textbook diagrams. Leveraging recent advancements in computer vision, such as Segment Anything and Multi-modal LLMs, our web-based system enables users to semi-automatically extract diagrams from physics textbooks and generate interactive simulations based on the extracted content. These interactive diagrams are seamlessly integrated into scanned textbook pages, facilitating interactive and personalized learning experiences across various physics concepts, such as optics, circuits, and kinematics. Drawing from an elicitation study with seven physics instructors, we explore four key augmentation strategies: 1) augmented experiments, 2) animated diagrams, 3) bi-directional binding, and 4) parameter visualization. We evaluate our system through technical evaluation, a usability study (N=12), and expert interviews (N=12). Study findings suggest that our system can facilitate more engaging and personalized learning experiences in physics education.

Reference

Abstract

Sustainable fabrication approaches and biomaterials are increasingly being used in HCI to fabricate interactive devices. However, the majority of the work has focused on integrating electronics. This paper takes a sustainable approach to exploring the fabrication of biochemical sensing devices. Firstly, we contribute a set of biochemical formulations for biological and environmental sensing with bio-sourced and environment-friendly substrate materials. Our formulations are based on a combination of enzymes derived from bacteria and fungi, plant extracts and commercially available chemicals to sense both liquid and gaseous analytes: glucose, lactic acid, pH levels and carbon dioxide. Our novel holographic sensing scheme allows for detecting the presence of analytes and enables quantitative estimation of the analyte levels. We present a set of application scenarios that demonstrate the versatility of our approach and discuss the sustainability aspects, its limitations, and the implications for bio-chemical systems in HCI.

Reference

Abstract

We explore the potential for subtle on-hand gesture and microgesture interactions for map navigation with augmented reality (AR) devices. We describe a design exercise and follow-up elicitation study in which we identified on-hand gestures for cartographic interaction primitives. Microgestures and on-hand interactions are a promising space for AR map navigation as they offers always-available, tactile, and memorable spaces for interaction. Our findings show a clear set of microgesture interaction patterns that are well suited for supporting map navigation and manipulation. In particular, we highlight how the properties of various microgestures align with particular cartographic interaction tasks. We also describe our experience creating an exploratory proof-of-concept AR map prototype which helped us identify new opportunities and practical challenges for microgesture control. Finally, we discuss how future AR map systems could benefit from on-hand and microgesture input schemes.

Reference

Abstract

We examine input visualizations, visual representations that are designed to collect (and represent) new data rather than encode preexisting datasets. Information visualization is commonly used to reveal insights and stories within existing data. As a result, most contemporary visualization approaches assume existing datasets as the starting point for design, through which that data is mapped to visual encodings. Meanwhile, the implications of visualizations as inputs and as data sources have received little attention—despite the existence of visual and physical examples stretching back centuries. In this paper, we present a design space of 50 input visualizations analyzing their visual representation, data, artifact, context, and input. Based on this, we identify input modalities, purposes of input visualizations, and a set of design considerations. Finally, we discuss the relationship between input visualization and traditional visualization design and suggest opportunities for future research to better understand these visual representations and their potential.

Reference

Abstract

We explore the potential of generative AI text-to-image models to help designers efficiently craft unique, representative, and demographically diverse anthropographics that visualize data about people. Currently, creating data-driven iconic images to represent individuals in a dataset often requires considerable design effort. Generative text-to-image models can streamline the process of creating these images, but risk perpetuating designer biases in addition to stereotypes latent in the models. In response, we outline a conceptual workflow for crafting anthropographic assets for visualizations, highlighting possible sources of risk and bias as well as opportunities for reflection and refinement by a human designer. Using an implementation of this workflow with Stable Diffusion and Google Colab, we illustrate a variety of new anthropographic designs that showcase the visual expressiveness and scalability of these generative approaches. Based on our experiments, we also identify challenges and research opportunities for new AI-enabled anthropographic visualization tools.

Reference

Abstract

We introduce Augmented Math, a machine learning-based approach to authoring AR explorable explanations by augmenting static math textbooks without programming. To augment a static document, our system first extracts mathematical formulas and figures from a given document using optical character recognition (OCR) and computer vision. By binding and manipulating these extracted contents, the user can see the interactive animation overlaid onto the document through mobile AR interfaces. This empowers non-technical users, such as teachers or students, to transform existing math textbooks and handouts into on-demand and personalized explorable explanations. To design our system, we first analyzed existing explorable math explanations to identify common design strategies. Based on the findings, we developed a set of augmentation techniques that can be automatically generated based on the extracted content, which are 1) dynamic values, 2) interactive figures, 3) relationship highlights, 4) concrete examples, and 5) step-by-step hints. To evaluate our system, we conduct two user studies: preliminary user testing and expert interviews. The study results confirm that our system allows more engaging experiences for learning math concepts.

Reference

Abstract

This paper introduces HoloBots, a mixed reality remote collaboration system that augments holographic telepresence with synchronized mobile robots. Beyond existing mixed reality telepresence, HoloBots lets remote users not only be visually and spatially present, but also physically engage with local users and their environment. HoloBots allows the users to touch, grasp, manipulate, and interact with the remote physical environment as if they were co-located in the same shared space. We achieve this by synchronizing holographic user motion (Hololens 2 and Azure Kinect) with tabletop mobile robots (Sony Toio). Beyond the existing physical telepresence, HoloBots contributes to an exploration of broader design space, such as object actuation, virtual hand physicalization, world-in-miniature exploration, shared tangible interfaces, embodied guidance, and haptic communication. We evaluate our system with twelve participants by comparing it with hologram-only and robot-only conditions. Both quantitative and qualitative results confirm that our system significantly enhances the level of co-presence and shared experience, compared to the other conditions.

Reference

Abstract

We introduce RealityCanvas, a mobile AR sketching tool that can easily augment real-world physical motion with responsive hand-drawn animation. Recent research in AR sketching tools has enabled users to not only embed static drawings into the real world but also dynamically animate them with physical motion. However, existing tools often lack the flexibility and expressiveness of possible animations, as they primarily support simple line-based geometry. To address this limitation, we explore both expressive and improvisational AR sketched animation by introducing a set of responsive scribble animation techniques that can be directly embedded through sketching interactions: 1) object binding, 2) flip-book animation, 3) action trigger, 4) particle effects, 5) motion trajectory, and 6) contour highlight. These six animation effects were derived from the analysis of 172 existing video-edited scribble animations. We showcase these techniques through various applications, such as video creation, augmented education, storytelling, and AR prototyping. The results of our user study and expert interviews confirm that our tool can lower the barrier to creating AR-based sketched animation, while allowing creative, expressive, and improvisational AR sketching experiences.

Reference

Abstract

Despite the advances and ubiquity of digital communication media such as videoconferencing and virtual reality, they remain oblivious to the rich intentions expressed by users. Beyond transmitting audio, videos, and messages, we envision digital communication media as proactive facilitators that can provide unobtrusive assistance to enhance communication and collaboration. Informed by the results of a formative study, we propose three key design concepts to explore the systematic integration of intelligence into communication and collaboration, including the panel substrate, language-based intent recognition, and lightweight interaction techniques. We developed CrossTalk, a videoconferencing system that instantiates these concepts, which was found to enable a more fluid and flexible communication and collaboration experience.

Reference

Abstract

In this paper, we introduce Physica, a tangible physics simulation system and approach based on tabletop mobile robots. In Physica, each tabletop robot can physically represent distinct simulated objects that are controlled through an underlying physics simulation, such as gravitational force, molecular movement, and spring force. It aims to bring the benefits of tangible and haptic interaction into explorable physics learning, which was traditionally only available on screen-based interfaces. The system utilizes off-the-shelf mobile robots (Sony Toio) and an open-source physics simulation tool (Teilchen). Built on top of them, we implement the interaction software pipeline that consists of 1) an event detector to reflect tangible interaction by users, and 2) target speed control to minimize the gap between the robot motion and simulated moving objects. To present the potential for physics education, we demonstrate various application scenarios that illustrate different forms of learning using Physica. In our user study, we investigate the effect and the potential of our approach through a perception study and interviews with physics educators.

Reference

Abstract

Anthropographics are human-shaped visualizations that aim to emphasize the human importance of datasets and the people behind them. However, current anthropographics tend to employ homogeneous human shapes to encode data about diverse demographic groups. Such anthropographics can obscure important differences between groups and contemporary designs exemplify the lack of inclusive approaches for representing human diversity in visualizations. In response, we explore the creation of demographically diverse anthropographics that communicate the visible diversity of demographically distinct populations. Building on previous anthropographics research, we explore strategies for visualizing datasets about people in ways that explicitly encode diversity—illustrating these approaches with examples in a variety of visual styles. We also critically reflect on strategies for creating diverse anthropographics, identifying social and technical challenges that can result in harmful representations. Finally, we highlight a set of forward-looking research opportunities for advancing the design and understanding of diverse anthropographics.

Reference

Abstract

We present ChameleonControl, a real-human teleoperation system for scalable remote instruction in hands-on classrooms. In contrast to the existing video or AR/VR-based remote hands-on education, ChameleonControl uses a real human as a surrogate of a remote instructor. Building on existing human-based telepresence approaches (e.g. ChameleonMask), we contribute a novel method to teleoperate a human surrogate through synchronized mixed reality (MR) hand gestural navigation and verbal communication. By overlaying the remote instructor's virtual hands in the local user's MR view, the remote instructor can guide and control the local user as if they were physically present. This allows the local user/surrogate to synchronize their hand movements and gestures with the remote instructor, effectively ``teleoperating'' a real human. We evaluate our system through the in-the-wild deployment for physiotherapy classrooms, as well as lab-based experiments for other application domains such as mechanical assembly, sign language, and cooking lessons. The study results confirm that our approach can increase engagement and the sense of co-presence, showing potential for the future of remote hands-on classrooms.

Reference

Abstract

This paper introduces Teachable Reality, an augmented reality (AR) prototyping tool for creating interactive tangible AR applications with arbitrary everyday objects. Teachable Reality leverages vision-based interactive machine teaching (e.g., Teachable Machine), which captures real-world interactions for AR prototyping. It identifies the user-defined tangible and gestural interactions using an on-demand computer vision model. Based on this, the user can easily create functional AR prototypes without programming, enabled by a trigger-action authoring interface. Therefore, our approach allows the flexibility, customizability, and generalizability of tangible AR applications that can address the limitation of current marker-based approaches. We explore the design space and demonstrate various AR prototypes, which include tangible and deformable interfaces, context-aware assistants, and body-driven AR applications. The results of our user study and expert interviews confirm that our approach can lower the barrier to creating functional AR prototypes while also allowing flexible and general-purpose prototyping experiences.

Reference

Abstract

We contribute interaction techniques for augmenting mixed reality (MR) visualizations with smartphone proxies. By combining head-mounted displays (HMDs) with mobile touchscreens, we can augment low-resolution holographic 3D charts with precise touch input, haptics feedback, high-resolution 2D graphics, and physical manipulation. Our approach aims to complement both MR and physical visualizations. Most current MR visualizations suffer from unreliable tracking, low visual resolution, and imprecise input. Data physicalizations on the other hand, although allowing for natural physical manipulation, are limited in dynamic and interactive modification. We demonstrate how mobile devices such as smartphones or tablets can serve as physical proxies for MR data interactions, creating dynamic visualizations that support precise manipulation and rich input and output. We describe 6 interaction techniques that leverage the combined physicality, sensing, and output capabilities of HMDs and smartphones, and demonstrate those interactions via a prototype system. Based on an evaluation, we outline opportunities for combining the advantages of both MR and physical charts.

Reference

Abstract

This paper introduces VR Haptics at Home, a method of repurposing everyday objects in the home to provide casual and on-demand haptic experiences. Current VR haptic devices are often expensive, complex, and unreliable, which limits the opportunities for rich haptic experiences outside research labs. In contrast, we envision that, by repurposing everyday objects as passive haptics props, we can create engaging VR experiences for casual uses with minimal cost and setup. To explore and evaluate this idea, we conducted an in-the-wild study with eight participants, in which they used our proof-of-concept system to turn their surrounding objects such as chairs, tables, and pillows at their own homes into haptic props. The study results show that our method can be adapted to different homes and environments, enabling more engaging VR experiences without the need for complex setup process. Based on our findings, we propose a possible design space to showcase the potential for future investigation.

Reference

Abstract

Gestural interaction with freehands and while grasping an everyday object enables always-available input. To sense such gestures, minimal instrumentation of the user’s hand is desirable. However, the choice of an effective but minimal IMU layout remains challenging, due to the complexity of the multi-factorial space that comprises diverse finger gestures, objects and grasps. We present SparseIMU, a rapid method for selecting minimal inertial sensor-based layouts for effective gesture recognition. Furthermore, we contribute a computational tool to guide designers with optimal sensor placement. Our approach builds on an extensive microgestures dataset that we collected with a dense network of 17 inertial measurement units (IMUs). We performed a series of analyses, including an evaluation of the entire combinatorial space for freehand and grasping microgestures (393K layouts), and quantified the performance across different layout choices, revealing new gesture detection opportunities with IMUs. Finally, we demonstrate the versatility of our method with four scenarios.

Reference

Abstract

This paper introduces Sketched Reality, an approach that com- bines AR sketching and actuated tangible user interfaces (TUI) for bi-directional sketching interaction. Bi-directional sketching enables virtual sketches and physical objects to affect each other through physical actuation and digital computation. In the existing AR sketching, the relationship between virtual and physical worlds is only one-directional --- while physical interaction can affect virtual sketches, virtual sketches have no return effect on the physical objects or environment. In contrast, bi-directional sketching interaction allows the seamless coupling between sketches and actuated TUIs. In this paper, we employ tabletop-size small robots (Sony Toio) and an iPad-based AR sketching tool to demonstrate the concept. In our system, virtual sketches drawn and simulated on an iPad (e.g., lines, walls, pendulums, and springs) can move, actuate, collide, and constrain physical Toio robots, as if virtual sketches and the physical objects exist in the same space through seamless coupling between AR and robot motion. This paper contributes a set of novel interactions and a design space of bi-directional AR sketching. We demonstrate a series of potential applications, such as tangible physics education, explorable mechanism, tangible gaming for children, and in-situ robot programming via sketching.

Reference

Abstract

We present RealityTalk, a system that augments real-time live presentations with speech-driven interactive virtual elements. Augmented presentations leverage embedded visuals and animation for engaging and expressive storytelling. However, existing tools for live presentations often lack interactivity and improvisation, while creating such effects in video editing tools require significant time and expertise. RealityTalk enables users to create live augmented presentations with real-time speech-driven interactions. The user can interactively prompt, move, and manipulate graphical elements through real-time speech and supporting modalities. Based on our analysis of 177 existing video-edited augmented presentations, we propose a novel set of interaction techniques and then incorporated them into RealityTalk. We evaluate our tool from a presenter’s perspective to demonstrate the effectiveness of our system.

Reference

Abstract

In this paper, we present Mixels, programmable magnetic pixels that can be rapidly fabricated using an electromagnetic printhead mounted on an off-the-shelve 3-axis CNC machine. The ability to program magnetic material pixel-wise with varying magnetic force enables Mixels to create new tangible, tactile, and haptic interfaces. To facilitate the creation of interactive objects with Mixels, we provide a user interface that lets users specify the high-level magnetic behavior and that then computes the underlying magnetic pixel assignments and fabrication instructions to program the magnetic surface. Our custom hardware add-on based on an electromagnetic printhead and hall effect sensor clips onto a standard 3-axis CNC machine and can both write and read magnetic pixel values from magnetic material. Our evaluation shows that our system can reliably program and read magnetic pixels of various strengths, that we can predict the behavior of two interacting magnetic surfaces before programming them, that our electromagnet is strong enough to create pixels that utilize the maximum magnetic strength of the material being programmed, and that this material remains magnetized when removed from the magnetic plotter.

Reference

Abstract

Designers and makers are increasingly interested in leveraging bio-based and bio-degradable ‘do-it-yourself’ (DIY) materials for sustainable prototyping. Their self-produced bioplastics possess compelling properties such as self-adhesion but have so far not been functionalized to create soft interactive devices, due to a lack of DIY techniques for the fabrication of functional electronic circuits and sensors. In this paper, we contribute a DIY approach for creating Interactive Bioplastics that is accessible to a wide audience, making use of easy-to-obtain bio-based raw materials and familiar tools. We present three types of conductive bioplastic materials and their formulation: sheets, pastes and foams. Our materials enable additive and subtractive fabrication of soft circuits and sensors. Furthermore, we demonstrate how these materials can substitute conventional prototyping materials, be combined with off-the-shelf electronics, and be fed into a sustainable material ‘life-cycle’ including disassembly, re-use, and re-melting of materials. A formal characterization of our conductors highlights that they are even on-par with commercially available carbon-based conductive pastes.

Reference

Abstract

We introduce UltraBots, a system that combines ultrasound haptic feedback and robotic actuation for large-area mid-air haptics for VR. Ultrasound haptics can provide precise mid-air haptic feedback and versatile shape rendering, but the interaction area is often limited by the small size of the ultrasound devices, restricting the possible interactions for VR. To address this problem, this paper introduces a novel approach that combines robotic actuation with ultrasound haptics. More specifically, we will attach ultrasound transducer arrays to tabletop mobile robots or robotic arms for scalable, extendable, and translatable interaction areas. We plan to use Sony Toio robots for 2D translation and/or commercially available robotic arms for 3D translation. Using robotic actuation and hand tracking measured by a VR HMD (ex: Oculus Quest), our system can keep the ultrasound transducers underneath the user’s hands to provide on-demand haptics. We demonstrate applications with workspace environments, medical training, education and entertainment.

Reference

Abstract

This paper introduces a method to generate highly selective encodings that can be magnetically programmed onto physical modules to enable them to self-assemble in chosen configurations. We generate these encodings based on Hadamard matrices, and show how to design the faces of modules to be maximally attractive to their intended mate, while remaining maximally agnostic to other faces. We derive guarantees on these bounds, and verify their attraction and agnosticism experimentally. Using cubic modules whose faces have been covered in soft magnetic material, we show how inexpensive, passive modules with planar faces can be used to selectively self-assemble into target shapes without geometric guides. We show that these modules can be easily re-programmed for new target shapes using a CNC-based magnetic plotter, and demonstrate self-assembly of 8 cubes in a water tank.

Reference

Abstract

We present EvoIsland, a scalable interactive evolutionary user interface framework inspired by the spatially isolated land masses seen on Earth. Our generalizable interaction system encourages creators to spatially explore a wide range of design possibilities through the combination, separation, and rearrangement of hexagonal tiles on a grid. As these tiles are grouped into islandlike clusters, localized populations of designs form through an underlying evolutionary system. The interactions that take place within EvoIsland provide content creators with new ways to shape, display and assess populations in evolutionary systems that produce a wide range of solutions with visual phenotype outputs.

Reference

Abstract

We take the well-established use of physical scale models in architecture and identify new opportunities for using them to interactively visualize and examine multiple streams of geospatial data. Overlaying, comparing, or integrating visualizations of complementary data sets in the same physical space is often challenging given the constraints of various data types and the limited design space of possible visual encodings. Our vision of “simultaneous worlds” uses physical models as a substrate upon which visualizations of multiple data streams can be dynamically and concurrently integrated. To explore the potential of this concept, we created three design explorations that use an illuminated campus model to integrate visualizations about building energy use, climate, and movement paths on a university campus. We use a research through design approach, documenting how our interdisciplinary collaborations with domain experts, students, and architects informed our designs. Based on our observations, we characterize the benefits of models for 1) situating visualizations, 2) composing visualizations, and 3) manipulating and authoring visualizations. Our work highlights the potential of physical models to support embodied exploration of spatial and non-spatial visualizations through fluid interactions.

Reference

Abstract

We explore the design and utility of situated manual self-tracking visualizations on dedicated displays that integrate data tracking into existing practices and physical environments. Situating self-tracking tools in relevant locations is a promising approach to enable reflection on and awareness of data without needing to rely on sensorized tracking or personal devices. In both a long-term autobiographical design process and a co-design study with six participants, we rapidly prototyped and deployed 30 situated self-tracking applications over a ten month period. Grounded in the experience of designing and living with these trackers, we contribute findings on logging and data entry, the use of situated displays, and the visual design and customization of trackers. Our results demonstrate the potential of customizable dedicated self-tracking visualizations that are situated in relevant physical spaces, and suggest future research opportunities and new potential applications for situated visualizations.

Reference

Abstract

We explore the use of cinematic “pre-visualization” (previs) techniques as a rapid ideation and design futuring method for human computer interaction (HCI) research. Previs approaches, which are widely used in animation and film production, use digital design tools to create medium-fidelity videos that capture richer interaction, motion, and context than sketches or static illustrations. When used as a design futuring method, previs can facilitate rapid, iterative discussions that reveal tensions, challenges, and opportunities for new research. We performed eight one-week design futuring sprints, in which individual HCI researchers collaborated with a lead designer to produce concept sketches, storyboards, and videos that examined future applications of their research. From these experiences, we identify recurring themes and challenges and present a One Week Futuring Workbook that other researchers can use to guide their own futuring sprints. We also highlight how variations of our approach could support other speculative design practices.

Reference

Abstract

Skin is a promising interaction medium and has been widely explored for mobile, and expressive interaction. Recent research in HCI has seen the development of Epidermal Computing Devices: ultra-thin and non-invasive devices which reside on the user’s skin, offering intimate integration with the curved surfaces of the body, while having physical and mechanical properties that are akin to skin, expanding the horizon of on-body interaction. However, with rapid technological advancements in multiple disciplines, we see a need to synthesize the main open research questions and opportunities for the HCI community to advance future research in this area. By systematically analyzing Epidermal Devices contributed in the HCI community, physical sciences research and from our experiences in designing and building Epidermal Devices, we identify opportunities and challenges for advancing research across five themes. This multi-disciplinary synthesis enables multiple research communities to facilitate progression towards more coordinated endeavors for advancing Epidermal Computing.

Reference

Abstract

This paper contributes to a taxonomy of augmented reality and robotics based on a survey of 460 research papers. Augmented and mixed reality (AR/MR) have emerged as a new way to enhance human-robot interaction (HRI) and robotic interfaces (e.g., actuated and shape-changing interfaces). Recently, an increasing number of studies in HCI, HRI, and robotics have demonstrated how AR enables better interactions between people and robots. However, often research remains focused on individual explorations and key design strategies, and research questions are rarely analyzed systematically. In this paper, we synthesize and categorize this research field in the following dimensions: 1) approaches to augmenting reality; 2) characteristics of robots; 3) purposes and benefits; 4) classification of presented information; 5) design components and strategies for visual augmentation; 6) interaction techniques and modalities; 7) application domains; and 8) evaluation strategies. We formulate key challenges and opportunities to guide and inform future research in AR and robotics.

Reference

Abstract

Art is not Research. Research is not Art. is a multimedia, multi-site participatory installation by a collective of artists and researchers from Calgary, Toronto, and Lancaster; it is informed by these contexts. It reflects the tensions between how “participants” are treated in participatory art and interaction research. It offers a framework through which we can explore how epistemologies might evolve in a blending between Art and Research. Visitors download the paper to read, critically reflect on the relationship between art and research, and experientially engage with the material through a series of creative prompts. A performance variation of the piece will be performed in-person and online through the ACM SIGCHI Conference on Human Factors in Computing Systems alt.chi track.

Reference

Abstract

In this paper, we survey the emerging design space of expandable structures in robotics, with a focus on how such structures may improve human-robot interactions. We detail various implementation considerations for researchers seeking to integrate such structures in their own work and describe how expandable structures may lead to novel forms of interaction for a variety of different robots and applications, including structures that enable robots to alter their form to augment or gain entirely new capabilities, such as enhancing manipulation or navigation, structures that improve robot safety, structures that enable new forms of communication, and structures for robot swarms that enable the swarm to change shape both individually and collectively. To illustrate how these considerations may be operationalized, we also present three case studies from our own research in expandable structure robots, sharing our design process and our findings regarding how such structures enable robots to produce novel behaviors that may capture human attention, convey information, mimic emotion, and provide new types of dynamic affordances.

Reference

Abstract

HapticBots introduces a novel encountered-type haptic approach for Virtual Reality (VR) based on multiple tabletop-size shape-changing robots. These robots move on a tabletop and change their height and orientation to haptically render various surfaces and objects on-demand. Compared to previous encountered-type haptic ap- proaches like shape displays or robotic arms, our proposed approach has an advantage in deployability, scalability, and generalizability---these robots can be easily deployed due to their compact form factor. They can support multiple concurrent touch points in a large area thanks to the distributed nature of the robots. We propose and evaluate a novel set of interactions enabled by these robots which include: 1) rendering haptics for VR objects by providing just-in- time touch-points on the user’s hand, 2) simulating continuous surfaces with the concurrent height and position change, and 3) enabling the user to pick up and move VR objects through graspable proxy objects. Finally, we demonstrate HapticBots with various ap- plications, including remote collaboration, education and training, design and 3D modeling, and gaming and entertainment.

Reference

Abstract

We explore how the lens of fictional superpowers can help characterize how visualizations empower people and provide inspiration for new visualization systems. Researchers and practitioners often tout visualizations' ability to “make the invisible visible” and to “enhance cognitive abilities.” Meanwhile superhero comics and other modern fiction often depict characters with similarly fantastic abilities that allow them to see and interpret the world in ways that transcend traditional human perception. We investigate the intersection of these domains, and show how the language of superpowers can be used to characterize existing visualization systems and suggest opportunities for new and empowering ones. We introduce two frameworks: The first characterizes seven underlying mechanisms that form the basis for a variety of visual superpowers portrayed in fiction. The second identifies seven ways in which visualization tools and interfaces can instill a sense of empowerment in the people who use them. Building on these observations, we illustrate a diverse set of “visualization superpowers” and highlight opportunities for the visualization community to create new systems and interactions that empower new experiences with data. Material and illustrations are available under CC-BY 4.0 at osf.io/8yhfz.

Reference

Abstract

In the near future, mixed traffic consisting of manual and autonomous vehicles (AVs) will be common. Questions surrounding how vulnerable road users such as pedestrians in wheelchairs (PWs) will make crossing decisions in these new situations are underexplored. We conducted a remote co-design study with one of the researchers of this work who has the lived experience as a powered wheelchair user and applied inclusive design practices. This allowed us to identify and reflect on interface design ideas that can help PWs make safe crossing decisions at intersections. Through an iterative five-week study, we implemented interfaces that can be placed on the vehicle, on the wheelchair, and on the street infrastructure and evaluated them during the co-design sessions using a VR simulator testbed. Informed by our findings, we discuss design insights for implementing inclusive interfaces to improve interactions between autonomous vehicles and vulnerable road users.

Reference

Abstract

Machine learning and predictive algorithms find patterns in large stores of data and make predictions which corporations and governments use to support decision-making. Yet, the system's representation of reality can be more influential to outcomes than the complexities of daily life. They become problematic when they undermine the inclusivity of public decision making, and when their use perpetuates social or economic inequality. To address these challenges, the public must be able to participate in discourse about the implications of algorithmic systems. I propose a series of participatory installations exploring the impacts of algorithmic systems, providing contexts for active exploration of these concerns. I will conduct phenomenographic interviews to better understand how visitors experience art installations about technical topics, providing insight for subsequent installations. I will consolidate the results into a set of best practices about engaging the public on these topics.

Reference

Abstract

We present I/O Bits, a prototype personal informatics system that explores the potential for user-driven and situated self-tracking. With simple tactile inputs and small e-paper visualizations, I/O Bits are dedicated physical devices that allow individuals to track and visualize different kinds of personal activities in-situ. This is in contrast to most self-tracking systems, which automate data collection, centralize information displays, or integrate into multi-purpose devices like smartwatches or mobile phones. We report findings from an e-paper visualization workshop and a prototype deployment where participants constructed their own I/O Bits and used them to track a range of personal data. Based on these experiences, we contribute insights and opportunities for situated and user-driven personal informatics.

Reference

Talk

Abstract

Worlds-in-Miniature (WiMs) are interactive worlds within a world and combine the advantages of an input space, a cartographicmap, and an overview+detail interface. They have been used across the extended virtuality spectrum for a variety of applications.Building on an analysis of examples of WiMs from the research literature we contribute a design space for WiMs based on sevendesign dimensions. Further, we expand upon existing definitions of WiMs to provide a definition that applies across the extendedreality spectrum. We identify the design dimensions of size-scope-scale, abstraction, geometry, reference frame, links, multiples, andvirtuality. Using our framework we describe existing Worlds-in-Miniature from the research literature and reveal unexplored researchareas. Finally, we generate new examples of WiMs using our framework to fill some of these gaps. With our findings, we identifyopportunities that can guide future research into WiMs.

Reference

Abstract

Immersive Analytics is a quickly evolving field that unites several areas such as visualisation, immersive environments, and human-computer interaction to support human data analysis with emerging technologies. This research has thrived over the past years with multiple workshops, seminars, and a growing body of publications, spanning several conferences. Given the rapid advancement of interaction technologies and novel application domains, this paper aims toward a broader research agenda to enable widespread adoption. We present 17 key research challenges developed over multiple sessions by a diverse group of 24 international experts, initiated from a virtual scientific workshop at ACM CHI 2020. These challenges aim to coordinate future work by providing a systematic roadmap of current directions and impending hurdles to facilitate productive and effective applications for Immersive Analytics.

Reference

Abstract

We present an autobiographical design journey exploring the experience of returning to long-term single player games. Continuing progress from a previously saved game, particularly when substantial time has passed, is an understudied area in games research. To begin our exploration in this domain, we investigated what the return experience is like first-hand. By returning to four long-term single player games played extensively in the past, we revealed a phenomenon we call The Pivot Point, a ‘eureka’ moment in return gameplay. The pivot point anchors our design explorations, where we created prototypes to leverage the pivot point in reconnecting with the experience. These return experiences and subsequent prototyping iterations inform our understanding of how to design better returns to gameplay, which can benefit both producers and consumers of long-term single player games.

Reference

Abstract

We introduce three novel data visualizations, called perspective charts, based on the concept of size constancy in linear perspective projection. Bar charts are a popular and commonly used tool for the interpretation of datasets, however, representing datasets with multi-scale variation is challenging in a bar chart due to limitations in viewing space. Each of our designs focuses on the static representation of datasets with large ranges with respect to important variations in the data. Through a user study, we measure the effectiveness of our designs for representing these datasets in comparison to traditional methods, such as a standard bar chart or a broken-axis bar chart, and state-of-the-art methods, such as a scale-stack bar chart. The evaluation reveals that our designs allow pieces of data to be visually compared at a level of accuracy similar to traditional visualizations. Our designs demonstrate advantages when compared to state-of-the-art visualizations designed to represent datasets with large outliers.

Reference

Abstract

Architects and designers communicate their ideas within a range of representational methods. No single instance of these methods, either in the form of orthographic projections or perspectival representation, can address all questions regarding the design, but as a whole, they demonstrate a comprehensive range of information about the building or object they intend to represent. This explicates an inevitable degree of deficiency in representation, regardless of its type. In addition, perspective-based optical illusions manipulate our spatial perception by deliberately misrepresenting the reality. In this regard, they are not new concepts to architectural representation. As a consequence, Optically Illusive Architecture (OIA) is proposed, not as a solution to fill the gap between the representing and represented spaces, but as a design paradigm whose concept derives from and accounts for this gap. By OIA we aim to cast light to an undeniable role of viewpoints in designing architectural spaces. The idea is to establish a methodology in a way that the deficiency of current representational techniques—manifested as specific thread of optical illusions—flourishes into thoughtful results embodied as actual architectural spaces. Within our design paradigm, we define a framework to be able to effectively analyze its precedents, generate new space, and evaluate their efficiencies. Moreover, the framework raises a hierarchical set of questions to differentiate OIA from a visual gimmick. Furthermore, we study two OIA-driven environments, by conducting empirical studies using Virtual Reality (VR). These studies bear essential information, in terms of design performance, and the public’s ability to engage and interact with an OIA space, prior to the actual fabrication of the structures.

Reference

Abstract

We present the SmartCampus visualization tool, representing spatiotemporal data of over 200 student pathways and restpoints on a university campus. Based on our experiences with SmartCampus, we also propose a task-based framework that de-scribes how practicing urban designers (specifically, architects) can use human movement data visualizations in their work. Although extensive amounts of location data are produced daily by smartphones, existing geospatial tools are not customized to specifically support high-level urban design tasks. To help identify opportunities in urban design for visualizing human movement data from devices such as smartphones, we used our SmartCampus prototype to facilitate a series of 3 participatory design sessions (3 participants), a targeted online survey (14 participants), and semi-structured interviews (6 participants) with architectural experts. Our findings showcase the need for location analysis tools tailored to concrete urban design practices, and also highlight opportunities for Smart City researchers interested in developing domain specific, visualization tools.

Reference

Abstract

Empathy tools are experiences designed to evoke empathetic responses by placing the user in another’s lived and felt experience. The problem is that designers do not have a common vocabulary to describe empathy tool experiences; consequently, it is difficult to compare/contrast empathy tool designs or to think about their efficacy. To address this problem, we analyzed 26 publications on empathy tools to develop a descriptive framework for designers of empathy tools. Based on our analysis, we found that empathy tools can be described along three dimensions: (i) the amount of agency the tool allows, (ii) the user’s perspective while using the tool, and (iii) the type of sensations that are experienced. We show that this framework can be used to describe a wide variety of empathy tools and provide recommendations for empathy tool designers, as well as techniques for measuring the efficacy of an empathy tool experience.

Reference

Abstract

We present RealitySketch, an augmented reality interface for sketching interactive graphics and visualizations. In recent years, an increasing number of AR sketching tools enable users to draw and embed sketches in the real world. However, with the current tools, sketched contents are inherently static, floating in mid air without responding to the real world. This paper introduces a new way to embed dynamic and responsive graphics in the real world. In RealitySketch, the user draws graphical elements on a mobile AR screen and binds them with physical objects in real-time and improvisational ways, so that the sketched elements dynamically move with the corresponding physical motion. The user can also quickly visualize and analyze real-world phenomena through responsive graph plots or interactive visualizations. This paper contributes to a set of interaction techniques that enable capturing, parameterizing, and visualizing real-world motion without pre-defined programs and configurations. Finally, we demonstrate our tool with several application scenarios, including physics education, sports training, and in-situ tangible interfaces.

Reference

Abstract

We focus on the problem of simulating the haptic infrastructure of a virtual environment (i.e. walls, doors). Our approach relies on multiple ZoomWalls---autonomous robotic encounter-type haptic wall-shaped props---that coordinate to provide haptic feedback for room-scale virtual reality. Based on a user's movement through the physical space, ZoomWall props are coordinated through a predict-and-dispatch architecture to provide just-in-time haptic feedback for objects the user is about to touch. To refine our system, we conducted simulation studies of different prediction algorithms, which helped us to refine our algorithmic approach to realize the physical ZoomWall prototype. Finally, we evaluated our system through a user experience study, which showed that participants found that ZoomWalls increased their sense of presence in the VR environment. ZoomWalls represents an instance of autonomous mobile reusable props, which we view as an important design direction for haptics in VR.

Reference

Abstract

We present PufferBot, an aerial robot with an expandable structure that may expand to protect a drone’s propellers when the robot is close to obstacles or collocated humans. PufferBot is made of a custom 3D printed expandable scissor structure, which utilizes a one degree of freedom actuator with rack and pinion mechanism. We propose four designs for the expandable structure, each with unique charac- terizations which may be useful in different situations. Finally, we present three motivating scenarios in which PufferBot might be useful beyond existing static propeller guard structures.

Reference

Abstract

We compare physical and virtual reality (VR) versions of simple data visualizations and explore how the addition of virtual annotation and filtering tools affects how viewers solve basic data analysis tasks. We report on two studies, inspired by previous examinations of data physicalizations. The first study examines differences in how viewers interact with physical hand-scale, virtual hand-scale, and virtual table-scale visualizations and the impact that the different forms had on viewer’s problem solving behavior. A second study examines how interactive annotation and filtering tools might support new modes of use that transcend the limitations of physical representations. Our results highlight challenges associated with virtual reality representations and hint at the potential of interactive annotation and filtering tools in VR visualizations.

Reference

Abstract

We present TRAVIS II, an augmented acoustic violin with touch sensors integrated into its 3D printed fingerboard that track left-hand finger gestures in real time. The fingerboard has four strips of conductive PLA filament which produce an electric signal when fingers press down on each string. While these sensors are physically robust, they are mechanically assembled and thus easy to replace if damaged. The performer can also trigger presets via four FSRs attached to the body of the violin. The instrument is completely wireless, giving the performer the freedom to move throughout the performance space. While the sensing fingerboard is installed in place of the traditional fingerboard, all other electronics can be removed from the augmented instrument, maintaining the aesthetics of a traditional violin. Our design allows violinists to naturally create music for interactive performance and improvisation without requiring new instrumental techniques. In this paper, we describe the design of the instrument, experiments leading to the sensing fingerboard, and performative applications of the instrument.

Reference

Talk

Abstract

Natural disasters cause long-lasting mental health problems such as PTSD in children. Following the 2011 Earthquake and Tsunami in Japan, we witnessed a shift of toy block play behavior in young children who suffered from stress after the disaster. The behavior reflected their emotional responses to the traumatic event. In this paper, we explore the feasibility of using data captured from block-play to assess children's stress after a major natural disaster. We prototyped sets of sensor-embedded toy blocks, AssessBlocks, that automate quantitative play data acquisition. During a three-year period, the blocks were dispatched to fifty-two post-disaster children. Within a free play session, we captured block features, a child's playing behavior, and stress evaluated by several methods. The result from our analysis reveal correlations between block play features and stress measurements and show initial promise of using the effectiveness of using AssessBlocks to assess children's stress after a disaster. We provide detailed insights into the potential as well as the challenges of our approach and unique conditions. From these insights we summarize guidelines for future research in automated play assessment systems that support children's mental health.

Reference

Abstract

We built a chatbot system–Audience Bot–that simulates an audience for novice live streamers to engage with while streaming. New live streamers on platforms like Twitch are expected to perform and talk to themselves, even while no one is watching. We ran an observational lab study on how Audience Bot assists novice live streamers as they acclimate to multitasking–simultaneously playing a video game while performing for a (simulated) audience.

Reference

Abstract

RoomShift is a room-scale dynamic haptic environment for virtual reality, using a small swarm of robots that can move furniture. RoomShift consists of nine shape-changing robots: Roombas with mechanical scissor lifts. These robots drive beneath a piece of furniture to lift, move and place it. By augmenting virtual scenes with physical objects, users can sit on, lean against, place and otherwise interact with furniture with their whole body; just as in the real world. When the virtual scene changes or users navigate within it, the swarm of robots dynamically reconfigures the physical environment to match the virtual content. We describe the hardware and software implementation, applications in virtual tours and architectural design and interaction techniques.

Reference

Abstract

We were approached by a group of healthcare providers who are involved in the care of chronic patients looking for potential technologies to facilitate the process of reviewing patient-generated data during clinical visits. Aiming at understanding the healthcare providers’ attitudes towards reviewing patient-generated data, we (1) conducted a focus group with a mixed group of healthcare providers. Next, to gain the patients’ perspectives, we (2) interviewed eight chronic patients, collected a sample of their data and designed a series of visualizations representing patient data we collected. Last, we (3) sought feedback on the visualization designs from healthcare providers who requested this exploration. We found four factors shaping patient-generated data: data & context, patient’s motivation, patient’s time commitment, and patient’s support circle. Informed by the results of our studies, we discussed the importance of designing patient-generated visualizations for individuals by considering both patient and healthcare provider rather than designing with the purpose of generalization and provided guidelines for designing future patient-generated data visualizations.

Reference

Abstract

We present a mixed-methods study of viewers on their practices and motivations around watching mukbang — video streams of people eating large quantities of food. Viewers' experiences provide insight on future technologies for multisensorial video streams and technology-supported commensality (eating with others). We surveyed 104 viewers and interviewed 15 of them about their attitudes and reflections on their mukbang viewing habits, their physiological aspects of watching someone eat, and their perceived social relationship with mukbangers. Based on our findings, we propose design implications for remote commensality, and for synchronized multisensorial video streaming content.

Reference

Talk

Abstract

We are interested in the ways pedestrians will interact with autonomous vehicle (AV) in a future AV transportation ecosystem, when nonverbal cues from the driver such as eye movements, hand gestures, etc. are no longer provided. In this work, we examine a subset of this challenge: interaction between pedestrian with reduced mobility (PRM) and AV. This study explores interface designs between AVs and people in a wheelchair to help them interact with AVs by conducting a preliminary design study. We have assessed the data collected from the study using qualitative analysis and presented different findings on AV-PRM interactions. Our findings reflect on the importance of visual interfaces, changes to the wheelchair and the creative use of the street infrastructure.

Reference

Abstract

We describe a design space of view manipulation interactions for small data-driven contextual visualizations (word-scale visualizations). These interaction techniques support an active reading experience and engage readers through exploration of embedded visualizations whose placement and content connect them to specific terms in a document. A reader could, for example, use our proposed interaction techniques to explore word-scale visualizations of stock market trends for companies listed in a market overview article. When readers wish to engage more deeply with the data, they can collect, arrange, compare, and navigate the document using the embedded word-scale visualizations, permitting more visualization-centric analyses. We support our design space with a concrete implementation, illustrate it with examples from three application domains, and report results from two experiments. The experiments show how view manipulation interactions helped readers examine embedded visualizations more quickly and with less scrolling and yielded qualitative feedback on usability and future opportunities.

Reference

Abstract

Autonomous vehicles (AVs) will redefine interactions between road users. Presently, cyclists and drivers communicate through implicit cues (vehicle motion) and explicit but imprecise signals (hand gestures, horns). Future AVs could consistently communicate awareness and intent and other feedback to cyclists based on their sensor data. We present an exploration of AV-cyclist interaction, starting with preliminary design studies which informed the implementation of an immersive VR AV-cyclist simulator, and the design and evaluation of a number of AV-cyclist interfaces. Our findings suggest that AV-cyclist interfaces can improve rider confidence in lane merging scenarios. We contribute an AV-cyclist immersive simulator, insights on trade-offs of various aspects of AV-cyclist interaction design including modalities, location, and complexity, and positive results suggesting improved rider confidence due to AV-cyclist interaction. While we are encouraged by the potential positive impact AV-cyclist interfaces can have on cyclist culture, we also emphasize the risks over-reliance can pose to cyclists.

Reference

Talk

Abstract

Large-scale shape-changing interfaces have great potential, but creating such systems requires substantial time, cost, space, and efforts, which hinders the research community to explore interactions beyond the scale of human hands. We introduce modular inflatable actuators as building blocks for prototyping room-scale shape-changing interfaces. Each actuator can change its height from 15cm to 150cm, actuated and controlled by air pressure. Each unit is low-cost (8 USD), lightweight (10 kg), compact (15 cm), and robust, making it well-suited for prototyping room-scale shape transformations. Moreover, our modular and reconfigurable design allows researchers and designers to quickly construct different geometries and to explore various applications. This paper contributes to the design and implementation of highly extendable inflatable actuators, and demonstrates a range of scenarios that can leverage this modular building block.

Reference

Abstract

We present the second iteration of a Touch-Responsive Augmented Violin Interface System, called TRAVIS II, and two compositions that demonstrate its expressivity. TRAVIS II is an augmented acoustic violin with touch sensors integrated into its 3-D printed fingerboard that track left-hand finger gestures in real time. The fingerboard has four strips of conductive PLA filament that produce an electric signal when fingers press down on each string. Although these sensors are physically robust, they are mechanically assembled and thus easy to replace if damaged. The performer can also trigger presets via four sensors attached to the body of the violin. The instrument is completely wireless, giving the performer the freedom to move throughout the performance space. Although the sensing fingerboard is installed in place of the traditional fingerboard, all other electronics can be removed from the augmented instrument, maintaining the aesthetics of a traditional violin. Our design allows violinists to naturally create music for interactive performance and improvisation without requiring new instrumental techniques. The first author composed two compositions to highlight TRAVIS II: “Dream State” and “Kindred Dichotomy.” Both of these compositions involve improvisation in their creative process and include interactive visuals. In this article we describe the design of the instrument, experiments leading to the sensing fingerboard, performative applications of the instrument, and compositional considerations for the resultant pieces.

Reference

Abstract

Pen-based input is often treated as auxiliary to mobile devices. We posit that cross-device interactions can inspire and extend the design space of pen-based interactions into new, expressive directions. We realize this through WatchPen, a smartwatch mounted on a passive, capacitive stylus that: (1) senses the usage context and leverages it for expression (e.g., changing colour), (2) contains tools and parameters within the display, and (3) acts as an on-demand output. As a result, it provides users with a dynamic relationship between inputs and outputs, awareness of current tool selection and parameters, and increased expressive match (e.g., added ability to mimic physical tools, showing clipboard contents). We discuss and reflect upon a series of interaction techniques that demonstrate WatchPen within a drawing application. We highlight the expressive power of leveraging multiple sensing and output capabilities across both the watch-augmented stylus and the tablet surface.

Reference

Abstract

We introduce shape-changing swarm robots. A swarm of self-transformable robots can both individually and collectively change their configuration to display information, actuate objects, act as tangible controllers, visualize data, and provide physical affordances. ShapeBots is a concept prototype of shape-changing swarm robots. Each robot can change its shape by leveraging small linear actuators that are thin (2.5 cm) and highly extendable (up to 20cm) in both horizontal and vertical directions. The modular design of each actuator enables various shapes and geometries of self-transformation. We illustrate potential application scenarios and discuss how this type of interface opens up possibilities for the future of ubiquitous and distributed shape-changing interfaces.

Reference

Abstract

Complex data visualization design projects often entail collaboration between people with different visualization-related skills. For example, many teams include both designers who create new visualization designs and developers who implement the resulting visualization software. We identify gaps between data characterization tools, visualization design tools, and development platforms that pose challenges for designer-developer teams working to create new data visualizations. While it is common for commercial interaction design tools to support collaboration between designers and developers, creating data visualizations poses several unique challenges that are not supported by current tools. In particular, visualization designers must characterize and build an understanding of the underlying data, then specify layouts, data encodings, and other data-driven parameters that will be robust across many different data values. In larger teams, designers must also clearly communicate these mappings and their dependencies to developers, clients, and other collaborators. We report observations and reflections from five large multidisciplinary visualization design projects and highlight six data-specific visualization challenges for design specification and handoff. These challenges include adapting to changing data, anticipating edge cases in data, understanding technical challenges, articulating data-dependent interactions, communicating data mappings, and preserving the integrity of data mappings across iterations. Based on these observations, we identify opportunities for future tools for prototyping, testing, and communicating data-driven designs, which might contribute to more successful and collaborative data visualization design.

Reference

Talk

Abstract

We present results from two studies comparing the performance of four different navigation techniques (flight, teleportation, world-in-miniature, and 3D cone-drag) and their combinations in large virtual reality map environments. While prior work has individually examined each of these techniques in other settings, our study presents the first direct comparison between them in large open environments, as well as one of the first comparisons in the context of current-generation virtual reality hardware. Our first study compared common techniques (flight, teleportation, and world-in-miniature) for search and navigation tasks. A follow-up study compared these techniques against 3D cone drag, a direct-manipulation navigation technique used in contemporary tools like Google Earth VR. Our results show the strength of flight as a stand-alone navigation technique, but also highlight five specific ways in which viewers can combine teleportation, world-in-miniature, and 3D cone drag with flight, drawing on the relative strengths of each technique to compensate for the weaknesses of others.

Reference

Abstract

We present Mutation: performing arts based approach that can help decrease the cognitive load associated with cyborg transitioning. Cyborgs are human-machine hybrids with organic and mechatronic body parts that can be implanted or worn. The transition into and out of experiencing additional body parts is not fully understood. Our goal is to draw from performing arts techniques in order to help decrease the cognitive load associated with becoming and unbecoming a cyborg. Actors constantly shift between states, whether from one character to another, or from pre- to post- performance. We contribute a straightforward adaptation of classic performing art practices to cyborg transitioning, and a study where actors used these protocols in order to enter a cyborg state, perform as a cyborg, and then exit the cyborg state. Our work on Mutation suggests that classic performing art practices can be useful in cyborg transitioning, as well as in other technology augmented experiences.

Reference

Abstract

We discuss Logical Conclusion, an analog interactive installation which presents issues surrounding the social impacts of algorithms used by corporations and governments via logic puzzles with physical elements that visitors manipulate to solve. We present the combination of physicality and participation as promising tools to engage the public with the ways that complex technologies interact with society. We also pose questions regarding how such strategies might be extended by the addition of responsive tangible computing elements.

Reference

Abstract

We report on findings from seven design workshops that used ideation and sketching activities to prototype new situated visualizations - representations of data that are displayed in proximity to the physical referents (such as people, objects, and locations) to which the data is related. Designing situated visualizations requires a fine-grained understanding of the context in which the visualizations are placed, as well as an exploration of different options for placement and form factors, which existing methods for visualization design do not account for. Focusing on small displays as a target platform, we reflect on our experiences of using a diverse range of sketching activities, materials, and prompts. Based on these observations, we identify challenges and opportunities for sketching and ideating situated visualizations. We also outline the space of design activities for situated visualization and highlight promising methods for both designers and researchers.

Reference

Abstract

Astral is a prototyping tool for authoring mobile and smart object interactive behaviours. It mirrors selected display contents of desktop applications onto mobile devices (smartphones and smartwatches), and streams/remaps mobile sensor data to desktop input events (mouse or keyboard) to manipulate selected desktop contents. This allows designers to use familiar desktop applications (e.g. PowerPoint, AfterEffects) to prototype rich interactive behaviours. Astral combines and integrates display mirroring, sensor streaming and input remapping, where designers can exploit familiar desktop applications to prototype, explore and fine-tune dynamic interactive behaviours. With Astral, designers can visually author rules to test real-time behaviours while interactions take place, as well as after the interaction has occurred. We demonstrate Astral's applicability, workflow and expressiveness within the interaction design process through both new examples and replication of prior approaches that illustrate how various familiar desktop applications are leveraged and repurposed.

Reference

Abstract

AV-pedestrian interaction will impact pedestrian safety, etiquette, and overall acceptance of AV technology. Evaluating AV-pedestrian interaction is challenging given limited availability of AVs and safety concerns. These challenges are compounded by "mixed traffic" conditions: studying AV-pedestrian interaction will be difficult in traffic consisting of vehicles varying in autonomy level. We propose immersive pedestrian simulators as design tools to study AV-pedestrian interaction, allowing rapid prototyping and evaluation of future AV-pedestrian interfaces. We present OnFoot: a VR-based simulator that immerses participants in mixed traffic conditions and allows examination of their behavior while controlling vehicles' autonomy-level, traffic and street characteristics, behavior of other virtual pedestrians, and integration of novel AV-pedestrian interfaces. We validated OnFoot against prior simulators and Wizard-of-Oz studies, and conducted a user study, manipulating vehicles' autonomy level, interfaces, and pedestrian group behavior. Our findings highlight the potential to use VR simulators as powerful tools for AV-pedestrian interaction design in mixed traffic.

Reference

Abstract

We introduce MorphIO, entirely soft sensing and actuation modules for programming by demonstration of soft robots and shape-changing interfaces.MorphIO's hardware consists of a soft pneumatic actuator containing a conductive sponge sensor.This allows both input and output of three-dimensional deformation of a soft material.Leveraging this capability, MorphIO enables a user to record and later playback physical motion of programmable shape-changing materials.In addition, the modular design of MorphIO's unit allows the user to construct various shapes and topologies through magnetic connection.We demonstrate several application scenarios, including tangible character animation, locomotion experiment of a soft robot, and prototyping tools for animated soft objects.Our user study with six participants confirms the benefits of MorphIO, as compared to the existing programming paradigm.

Reference

Abstract

Drawing upon multiple disciplines, avant-garde fashion-tech teams push the boundaries between fashion and technology. Many are well trained in envisioning aesthetic qualities of garments, but few have formal training on designing and fabricating technologies themselves. We introduce Mannequette, a prototyping tool for fashion-tech garments that enables teams to experiment with interactive technologies at early stages of their design processes. Mannequette provides an abstraction of light-based outputs and sensor-based inputs for garments through a DJ mixer-like interface that allows for dynamic changes and recording/playback of visual effects. The base of Mannequette can also be incorporated into the final garment, where it is then connected to the final components. We conducted an 8-week deployment study with eight design teams who created new garments for a runway show. Our results revealed Mannequette allowed teams to repeatedly consider new design and technical options early in their creative processes, and to communicate more effectively across disciplinary backgrounds.

Reference

Abstract

We investigate how people discover the functionality of an interactive visualization that was designed for the general public. While interactive visualizations are increasingly available for public use, we still know little about how the general public discovers what they can do with these visualizations and what interactions are available. Developing a better understanding of this discovery process can help inform the design of visualizations for the general public, which in turn can help make data more accessible. To unpack this problem, we conducted a lab study in which participants were free to use their own methods to discover the functionality of a connected set of interactive visualizations of public energy data. We collected eye movement data and interaction logs as well as video and audio recordings. By analyzing this combined data, we extract exploration strategies that the participants employed to discover the functionality in these interactive visualizations. These exploration strategies illuminate possible design directions for improving the discoverability of a visualization's functionality.

Reference

Abstract

We investigate enhancing virtual haptic experiences by applying Stochastic Resonance or SR noise to the user's hands. Specifically, we focus on improving users' ability to discriminate between virtual textures modelled from nine grits of real sandpaper in a virtual texture discrimination task. We applied mechanical SR noise to the participant's skin by attaching five flat actuators to different points on their hand. By fastening a linear voice-coil actuator and a 6-DOF haptic device to participants' index finger, we enabled them to interact and feel virtual sandpapers while inducing different levels of SR noise. We hypothesize that SR will improve their discrimination performance.

Reference

Abstract

We introduce three lightweight interactive camera control techniques for 3D terrain maps on touch devices based on a look-from metaphor (Discrete Look-From-At, Continuous Look-From-Forwards, and Continuous Look-From-Towards). These techniques complement traditional touch screen pan, zoom, rotate, and pitch controls allowing viewers to quickly transition between top-down, oblique, and ground-level views. We present the results of a study in which we asked participants to perform elevation comparison and line-of-sight determination tasks using each technique. Our results highlight how look-from techniques can be integrated on top of current direct manipulation navigation approaches by combining several direct manipulation operations into a single look-from operation. Additionally, they show how look-from techniques help viewers complete a variety of common and challenging map-based tasks.

Reference

Abstract

While improvised theatre (improv) is often performed on a bare stage, improvisers sometimes incorporate physical props to inspire new directions for a scene and to enrich their performance. A tech booth can improvise light and sound technical elements, but coordinating with improvisers' actions on-stage is challenging. Our goal is to inform the design of an augmented prop that lets improvisers tangibly control light and sound technical elements while performing. We interviewed five professional improvisers about their use of physical props in improv, and their expectations of a possible augmented prop that controls technical theatre elements. We propose a set of guidelines for the design of an augmented prop that fits with the existing world of unpredictable improvised performance.

Reference

Abstract

Wind simulations are typically one-off implementations for specific applications. We introduce WindyWall, a platform for creative design and exploration of wind simulations. WindyWall is a three-panel 90-fan array that encapsulates users with 270? of wind coverage. We describe the design and implementation of the array panels, discussing how the panels can be re-arranged, where various wind simulations can be realized as simple effects. To understand how people perceive "wind" generated from WindyWall, we conducted a pilot study of wind magnitude perception using different wind activation patterns from WindyWall. Our findings suggest that: horizontal wind activations are perceived more readily than vertical ones, and that people's perceptions of wind are highly variable-most individuals will rate airflow differently in subsequent exposures. Based on our findings, we discuss the importance of developing a method for characterizing wind simulations, and provide design directions for others using fan arrays to simulate wind.

Reference

Abstract

Providing data visualization authoring tools for the general public remains an ongoing challenge. Inspired by block-printing, we explore how visualization stamps as a physical visualization authoring tool could leverage both visual freedom and ease of repetition. We conducted a workshop with two groups---visualization experts and non-experts---where participants authored visualizations on paper using hand-carved stamps made from potatoes and sponges. The low-fidelity medium freed participants to test new stamp patterns and accept mistakes. From the created visualizations, we observed several unique traits and uses of block-printing tools for visualization authoring, including: modularity of patterns, annotation guides, creation of multiple patterns from one stamp, and various techniques to apply data onto paper. We discuss the issues around expressivity and effectiveness of block-printed stamps in visualization authoring, and identify implications for the design and assembly of primitives in potential visualization stamp kits, as well as applications for future use in non-digital environments.

Reference

Abstract

Freehand gesture interaction has long been proposed as a `natural' input method for Augmented Reality (AR) applications, yet has been little explored for intensive applications like multiscale navigation. In multiscale navigation, such as digital map navigation, pan and zoom are the predominant interactions. A position-based input mapping (e.g. grabbing metaphor) is intuitive for such interactions, but is prone to arm fatigue. This work focuses on improving digital map navigation in AR with mid-air hand gestures, using a horizontal intangible map display. First, we conducted a user study to explore the effects of handedness (unimanual and bimanual) and input mapping (position-based and rate-based). From these findings we designed DiveZoom and TerraceZoom, two novel hybrid techniques that smoothly transition between position- and rate-based mappings. A second user study evaluated these designs. Our results indicate that the introduced input-mapping transitions can reduce perceived arm fatigue with limited impact on performance.

Reference

Talk

Abstract

We examine the potential for immersive unit visualizations—interactive virtual environments populated with objects representing individual items in a dataset. Our virtual reality prototype highlights how immersive unit visualizations can allow viewers to examine data at multiple scales, support immersive exploration, and create affective personal experiences with data.

Reference

Abstract

This paper introduces Dynamic 3D Printing, a fast and reconstructable shape formation system. Dynamic 3D Printing can assemble an arbitrary three-dimensional shape from a large number of small physical elements. Also, it can disassemble the shape back to elements and reconstruct a new shape. Dynamic 3D Printing combines the capabilities of 3D printers and shape displays: Like conventional 3D printing, it can generate arbitrary and graspable three-dimensional shapes, while allowing shapes to be rapidly formed and reformed as in a shape display. To demonstrate the idea, we describe the design and implementation of Dynablock, a working prototype of a dynamic 3D printer. Dynablock can form a three-dimensional shape in seconds by assembling 3,000 9 mm blocks, leveraging a 24 x 16 pin-based shape display as a parallel assembler. Dynamic 3D printing is a step toward achieving our long-term vision in which 3D printing becomes an interactive medium, rather than the means for fabrication that it is today. In this paper, we explore possibilities for this vision by illustrating application scenarios that are difficult to achieve with conventional 3D printing or shape display systems.

Reference

Talk

Abstract

In improvisational theatre (improv), actors perform unscripted scenes together, collectively creating a narrative. Audience suggestions introduce randomness and build audience engagement, but can be challenging to mediate at scale. We present Robot Improv Puppet Theatre (RIPT), which includes a performance robot (Pokey) who performs gestures and dialogue in short-form improv scenes based on audience input from a mobile interface. We evaluated RIPT in several initial informal performances, and in a rehearsal with seven professional improvisers. The improvisers noted how audience prompts can have a big impact on the scene - highlighting the delicate balance between ambiguity and constraints in improv. The open structure of RIPT performances allows for multiple interpretations of how to perform with Pokey, including one-on-one conversations or multi-performer scenes. While Pokey lacks key qualities of a good improviser, improvisers found his serendipitous dialogue and gestures particularly rewarding.

Reference

Abstract

Because gesture design for augmented reality (AR) remains idiosyncratic, people cannot necessarily use gestures learned in one AR application in another. To design discoverable gestures, we need to understand what gestures people expect to use. We explore how the scale of AR affects the gestures people expect to use to interact with 3D holograms. Using an elicitation study, we asked participants to generate gestures in response to holographic task referents, where we varied the scale of holograms from desktop-scale to room-scale objects. We found that the scale of objects and scenes in the AR experience moderates the generated gestures. Most gestures were informed by physical interaction, and when people interacted from a distance, they sought a good perspective on the target object before and during the interaction. These results suggest that gesture designers need to account for scale, and should not simply reuse gestures across different hologram sizes.

Reference

Abstract

Electronic fashion (eFashion) garments use technology to augment the human body with wearable interaction. In developing ideas, eFashion designers need to prototype the role and behavior of the interactive garment in context; however, current wearable prototyping toolkits require semi-permanent construction with physical materials that cannot easily be altered. We present Bod-IDE, an augmented reality 'mirror' that allows eFashion designers to create virtual interactive garment prototypes. Designers can quickly build, refine, and test on-the-body interactions without the need to connect or program electronics. By envisioning interaction with the body in mind, eFashion designers can focus more on reimagining the relationship between bodies, clothing, and technology.

Reference

Abstract

This paper presents a set of lessons learnt from introducing maker culture and DIY paradigms to 'at-risk' students (age 12-14). Our goal is to engage 'at-risk' students through maker culture activities. While improved technology literacy is one of the outcomes we also wanted the learners to use technology to realize concepts and ideas, and to gain freedom of thinking similar to creators, artists and designers. We present our study and a set of high level suggestions to enable thinking about how maker culture activities can facilitate engagement and creative use of technology by 1) thinking about creativity in task, 2) facilitating different entry points, 3) the importance of personal relevance, and 4) relevance to education.

Reference

Abstract

We present PEP (Printed Electronic Papercrafts), a set of design and fabrication techniques to integrate electronic based interactivities into printed papercrafts via 3D sculpting. We explore the design space of PEP, integrating four functions into 3D paper products: actuation, sensing, display, and communication, leveraging the expressive and technical opportunities enabled by paper-like functional layers with a stack of paper. We outline a seven-step workflow, introduce a design tool we developed as an add-on to an existing CAD environment, and demonstrate example applications that combine the electronic enabled functionality, the capability of 3D sculpting, and the unique creative affordances by the materiality of paper.

Reference

Abstract

We explore a new approach to programming swarm user interfaces (Swarm UI) by leveraging direct physical manipulation. Existing Swarm UI applications are written using a robot programming framework: users work on a computer screen and think in terms of low-level controls. In contrast, our approach allows programmers to work in physical space by directly manipulating objects and think in terms of high-level interface design. Inspired by current UI programming practices, we introduce a four-step workflow-create elements, abstract attributes, specify behaviors, and propagate changes-for Swarm UI programming. We propose a set of direct physical manipulation techniques to support each step in this workflow. To demonstrate these concepts, we developed Reactile, a Swarm UI programming environment that actuates a swarm of small magnets and displays spatial information of program states using a DLP projector. Two user studies-an in-class survey with 148 students and a lab interview with eight participants-confirm that our approach is intuitive and understandable for programming Swarm UIs.

Reference

Abstract

Based on an analysis of 49 popular contemporary video games, we develop a descriptive framework of visual interaction cues in video games. These cues are used to inform players what can be interacted with, where to look, and where to go within the game world. These cues vary along three dimensions: the purpose of the cue, the visual design of the cue, and the circumstances under which the cue is shown. We demonstrate that this framework can also be used to describe interaction cues for augmented reality applications. Beyond this, we show how the framework can be used to generatively derive new design ideas for visual interaction cues in augmented reality experiences.

Reference

Talk

Abstract

Remote collaborators working together on physical objects have difficulty building a shared understanding of what each person is talking about. Conventional video chat systems are insufficient for many situations because they present a single view of the object in a flattened image. To understand how this limited perspective affects collaboration, we designed the Remote Manipulator (ReMa), which can reproduce orientation manipulations on a proxy object at a remote site. We conducted two studies with ReMa, with two main findings. First, a shared perspective is more effective and preferred compared to the opposing perspective offered by conventional video chat systems. Second, the physical proxy and video chat complement one another in a combined system: people used the physical proxy to understand objects, and used video chat to perform gestures and confirm remote actions.

Reference

Abstract

People often enjoy sharing outdoor activities together such as walking and hiking. However, when family and friends are separated by distance it can be difficult if not impossible to share such activities. We explore this design space by investigating the benefits and challenges of using a telepresence robot to support outdoor leisure activities. In our study, participants participated in the outdoor activity of geocaching where one person geocached with the help of a remote partner via a telepresence robot. We compared a wide field of view (WFOV) camera to a 360° camera. Results show the benefits of having a physical embodiment and a sense of immersion with the 360° view. Yet challenges related to a lack of environmental awareness, safety issues, and privacy concerns resulting from bystander interactions. These findings illustrate the need to design telepresence robots with the environment and public in mind to provide an enhanced sensory experience while balancing safety and privacy issues resulting from being amongst the general public.

Reference

Abstract

Toolkit research plays an important role in the field of HCI, as it can heavily influence both the design and implementation of interactive systems. For publication, the HCI community typically expects toolkit research to include an evaluation component. The problem is that toolkit evaluation is challenging, as it is often unclear what 'evaluating' a toolkit means and what methods are appropriate. To address this problem, we analyzed 68 published toolkit papers. From our analysis, we provide an overview of, reflection on, and discussion of evaluation methods for toolkit contributions. We identify and discuss the value of four toolkit evaluation strategies, including the associated techniques that each employs. We offer a categorization of evaluation strategies for toolkit researchers, along with a discussion of the value, potential limitations, and trade-offs associated with each strategy.

Reference

Talk

Abstract

Drivers use nonverbal cues such as vehicle speed, eye gaze, and hand gestures to communicate awareness and intent to pedestrians. Conversely, in autonomous vehicles, drivers can be distracted or absent, leaving pedestrians to infer awareness and intent from the vehicle alone. In this paper, we investigate the usefulness of interfaces (beyond vehicle movement) that explicitly communicate awareness and intent of autonomous vehicles to pedestrians, focusing on crosswalk scenarios. We conducted a preliminary study to gain insight on designing interfaces that communicate autonomous vehicle awareness and intent to pedestrians. Based on study outcomes, we developed four prototype interfaces and deployed them in studies involving a Segway and a car. We found interfaces communicating vehicle awareness and intent: (1) can help pedestrians attempting to cross; (2) are not limited to the vehicle and can exist in the environment; and (3) should use a combination of modalities such as visual, auditory, and physical.

Reference

Talk

Abstract

In the coming years, emergency calling services in North America will begin to incorporate new modalities for reporting emergencies, including video-based calling. The challenge is that we know little of how video calling systems should be designed and what benefits or challenges video calling might bring. We conducted observations and contextual interviews within three emergency response call centres to investigate these points. We focused on the work practices of call takers and dispatchers. Results show that video calls could provide valuable contextual information about a situation and help to overcome call taker challenges with information ambiguity, location, deceit, and communication issues. Yet video calls have the potential to introduce issues around control, information overload, and privacy if systems are not designed well. These results point to the need to think about emergency video calling along a continuum of visual modalities ranging from audio calls accompanied with images or video clips to one-way video streams to two-way video streams where camera control and camera work need to be carefully designed.

Reference

Abstract

In the physical world, teammates develop situation awareness about each other's location, status, and actions through cues such as gaze direction and ambient noise. To support situation awareness, distributed multiplayer games provide awareness cues - information that games automatically make available to players to support cooperative gameplay. The design of awareness cues can be extremely complex, impacting how players experience games and work with teammates. Despite the importance of awareness cues, designers have little beyond experiential knowledge to guide their design. In this work, we describe a design framework for awareness cues, providing insight into what information they provide, how they communicate this information, and how design choices can impact play experience. Our research, based on a grounded theory analysis of current games, is the first to provide a characterization of awareness cues, providing a palette for game designers to improve design practice and a starting point for deeper research into collaborative play.

Reference

Abstract

In this paper, we discuss the role of the movement trajectory and velocity enabled by our tele-robotic system (ReMa) for remote collaboration on physical tasks. Our system reproduces changes in object orientation and position at a remote location using a humanoid robotic arm. However, even minor kinematics differences between robot and human arm can result in awkward or exaggerated robot movements. As a result, user communication with the robotic system can become less efficient, less fluent and more time intensive.

Reference

Abstract

For people with visual impairments, tactile graphics are an important means to learn and explore information. However, raised line tactile graphics created with traditional materials such as embossing are static. While available refreshable displays can dynamically change the content, they are still too expensive for many users, and are limited in size. These factors limit wide-spread adoption and the representation of large graphics or data sets. In this paper, we present FluxMaker, an inexpensive scalable system that renders dynamic information on top of static tactile graphics with movable tactile markers. These dynamic tactile markers can be easily reconfigured and used to annotate static raised line tactile graphics, including maps, graphs, and diagrams. We developed a hardware prototype that actuates magnetic tactile markers driven by low-cost and scalable electromagnetic coil arrays, which can be fabricated with standard printed circuit board manufacturing. We evaluate our prototype with six participants with visual impairments and found positive results across four application areas: location finding or navigating on tactile maps, data analysis, and physicalization, feature identification for tactile graphics, and drawing support. The user study confirms advantages in application domains such as education and data exploration.

Reference

Abstract

We compare the effectiveness of 2D maps and 3D terrain models for visibility tasks and demonstrate how interactive dynamic viewsheds can improve performance for both types of terrain representations. In general, the two-dimensional nature of classic topographic maps limits their legibility and can make complex yet typical cartographic tasks like determining the visibility between locations difficult. Both 3D physical models and interactive techniques like dynamic viewsheds have the potential to improve viewers' understanding of topography, but their impact has not been deeply explored. We evaluate the effectiveness of 2D maps, 3D models, and interactive viewsheds for both simple and complex visibility tasks. Our results demonstrate the benefits of the dynamic viewshed technique and highlight opportunities for additional tactile interactions. Based on these findings we present guidelines for improving the design and usability of future topographic maps and models.

Reference

Talk

Abstract

We present an observational study of physical prototype critique that highlights some of the challenges of communicating physical behaviors and materiality at a distance. Geographically distributed open hardware communities often conduct user feedback and peer critique sessions via video conference. However, people have difficulty using current video conferencing tools to demonstrate and critique physical designs. To examine the challenges of remote critique, we conducted an observational lab study in which participants critiqued pairs of physical prototypes (prosthetic hands) for a face-to-face or remote collaborator. In both conditions, participants' material experiences were an important part of their critique, however their attention was divided between interacting with the prototype and finding strategies to communicate `invisible' features. Based on our findings, we propose design implications for remote collaboration tools that support the sharing of material experiences and prototype critique.

Reference

Abstract

In this paper we consider various genres of citizen science from the perspective of citizen participants. As a mode of scientific inquiry, citizen science has the potential to "scale up" scientific data collection efforts and increase lay engagement with science. However, current technological directions risk losing sight of the ways in which citizen science is actually practiced. As citizen science is increasingly used to describe a wide range of activities, we begin by presenting a framework of citizen science genres. We then present findings from four interlocking qualitative studies and technological interventions of community air quality monitoring efforts, examining the motivations and capacities of citizen participants and characterizing their alignment with different types of citizen science. Based on these studies, we suggest that data acquisition involves complex multi-dimensional tradeoffs, and the commonly held view that citizen science systems are a win-win for citizens and science may be overstated.

Reference

Abstract

In this paper we analyze the role of physical scale models in the architectural design process and apply insights from architecture for the creation and use of data physicalizations. Based on a survey of the architecture literature on model making and ten interviews with practicing architects, we describe the role of physical models as a tool for exploration and communication. From these observations, we identify trends in the use of physical models in architecture, which have the potential to inform the design of data physicalizations. We identify four functions of architectural modeling that can be directly adapted for use in the process of building rich data models. Finally, we discuss how the visualization community can apply observations from architecture to the design of new data physicalizations.

Reference

Abstract

Interactive, smart objects – customized to individuals and uses – are central to many movements, such as tangibles, the internet of things (IoT), and ubiquitous computing. Yet, rapid prototyping both the form and function of these custom objects can be problematic, particularly for those with limited electronics or programming experience. Designers often need to embed custom circuitry; program its workings; and create a form factor that not only reflects the desired user experience but can also house the required circuitry and electronics. To mitigate this, we created Pineal, a design tool that lets end-users: (1) modify 3D models to include a smart watch or phone as its heart; (2) specify high-level interactive behaviours through visual programming; and (3) have the phone or watch act out such behaviours as the objects' "smarts". Furthermore, a series of prototypes show how Pineal exploits mobile sensing and output, and automatically generates 3D printed form-factors for rich, interactive, objects.

Reference

Talk

Abstract

Do-it-yourself (DIY) inspired activities have gained popularity as a means of creative expression and self-directed learning. However, DIY culture is difficult to implement in places with limited technology infrastructure and traditional learning cultures. Our goal is to understand how learners in such a setting react to DIY activities. We present observations from a physical computing workshop with 12 students (13-15 years old) conducted at a high school in India. We observed unique challenges for these students when tackling DIY activities: a high monetary and psychological cost to exploration, limited independent learning resources, difficulties with finding intellectual courage and assumed technical language proficiency. Our participants, however, overcome some of these challenges by adopting their own local strategies: resilience, nonverbal and verbal learning techniques, and creating documentation and fallback circuit versions. Based on our findings, we discuss a set of lessons learned about makerspaces in a context with socio-technical challenges.

Reference

Abstract

We contribute an investigation of the design and function of word-scale graphics and visualizations embedded in text documents. Word-scale graphics include both data-driven representations such as word-scale visualizations and sparklines, and non-data-driven visual marks. Their design, function, and use has so far received little research attention. We present the results of an open ended exploratory study with nine graphic designers. The study resulted in a rich collection of different types of graphics, data provenance, and relationships between text, graphics, and data. Based on this corpus, we present a systematic overview of word-scale graphic designs, and examine how designers used them. We also discuss the designers' goals in creating their graphics, and characterize how they used word-scale graphics to visualize data, add emphasis, and create alternative narratives. Building on these examples, we discuss implications for the design of authoring tools for word-scale graphics and visualizations, and explore how new authoring environments could make it easier for designers to integrate them into documents.

Reference