Abstract

In this work, we introduce a formal design approach derived from the performing arts to design robot group behaviour. In our first experiment, we worked with professional actors, directors, and non-specialists using a participatory design approach to identify common group behaviour patterns. In a follow-up studio work, we identified twelve common group movement patterns, transposed them into a performance script, built a scale model to support the performance process, and evaluated the patterns with a senior actor under studio conditions. We evaluated our refined models with 20 volunteers in a user study in the third experiment. Results from our affective circumplex modelling suggest that the patterns elicit positive emotional responses from the users. Also, participants performed better than chance in identifying the motion patterns without prior training. Based on our results, we propose design guidelines for social robots’ behaviour and movement design to improve their overall comprehensibility in interaction.

Reference

Abstract

We introduce RealityCanvas, a mobile AR sketching tool that can easily augment real-world physical motion with responsive hand-drawn animation. Recent research in AR sketching tools has enabled users to not only embed static drawings into the real world but also dynamically animate them with physical motion. However, existing tools often lack the flexibility and expressiveness of possible animations, as they primarily support simple line-based geometry. To address this limitation, we explore both expressive and improvisational AR sketched animation by introducing a set of responsive scribble animation techniques that can be directly embedded through sketching interactions: 1) object binding, 2) flip-book animation, 3) action trigger, 4) particle effects, 5) motion trajectory, and 6) contour highlight. These six animation effects were derived from the analysis of 172 existing video-edited scribble animations. We showcase these techniques through various applications, such as video creation, augmented education, storytelling, and AR prototyping. The results of our user study and expert interviews confirm that our tool can lower the barrier to creating AR-based sketched animation, while allowing creative, expressive, and improvisational AR sketching experiences.

Reference

Abstract

We introduce UltraBots, a system that combines ultrasound haptic feedback and robotic actuation for large-area mid-air haptics for VR. Ultrasound haptics can provide precise mid-air haptic feedback and versatile shape rendering, but the interaction area is often limited by the small size of the ultrasound devices, restricting the possible interactions for VR. To address this problem, this paper introduces a novel approach that combines robotic actuation with ultrasound haptics. More specifically, we will attach ultrasound transducer arrays to tabletop mobile robots or robotic arms for scalable, extendable, and translatable interaction areas. We plan to use Sony Toio robots for 2D translation and/or commercially available robotic arms for 3D translation. Using robotic actuation and hand tracking measured by a VR HMD (ex: Oculus Quest), our system can keep the ultrasound transducers underneath the user’s hands to provide on-demand haptics. We demonstrate applications with workspace environments, medical training, education and entertainment.

Reference

Abstract

We explore the use of cinematic “pre-visualization” (previs) techniques as a rapid ideation and design futuring method for human computer interaction (HCI) research. Previs approaches, which are widely used in animation and film production, use digital design tools to create medium-fidelity videos that capture richer interaction, motion, and context than sketches or static illustrations. When used as a design futuring method, previs can facilitate rapid, iterative discussions that reveal tensions, challenges, and opportunities for new research. We performed eight one-week design futuring sprints, in which individual HCI researchers collaborated with a lead designer to produce concept sketches, storyboards, and videos that examined future applications of their research. From these experiences, we identify recurring themes and challenges and present a One Week Futuring Workbook that other researchers can use to guide their own futuring sprints. We also highlight how variations of our approach could support other speculative design practices.

Reference

Abstract

This paper contributes to a taxonomy of augmented reality and robotics based on a survey of 460 research papers. Augmented and mixed reality (AR/MR) have emerged as a new way to enhance human-robot interaction (HRI) and robotic interfaces (e.g., actuated and shape-changing interfaces). Recently, an increasing number of studies in HCI, HRI, and robotics have demonstrated how AR enables better interactions between people and robots. However, often research remains focused on individual explorations and key design strategies, and research questions are rarely analyzed systematically. In this paper, we synthesize and categorize this research field in the following dimensions: 1) approaches to augmenting reality; 2) characteristics of robots; 3) purposes and benefits; 4) classification of presented information; 5) design components and strategies for visual augmentation; 6) interaction techniques and modalities; 7) application domains; and 8) evaluation strategies. We formulate key challenges and opportunities to guide and inform future research in AR and robotics.

Reference

Abstract

We explore how the lens of fictional superpowers can help characterize how visualizations empower people and provide inspiration for new visualization systems. Researchers and practitioners often tout visualizations' ability to “make the invisible visible” and to “enhance cognitive abilities.” Meanwhile superhero comics and other modern fiction often depict characters with similarly fantastic abilities that allow them to see and interpret the world in ways that transcend traditional human perception. We investigate the intersection of these domains, and show how the language of superpowers can be used to characterize existing visualization systems and suggest opportunities for new and empowering ones. We introduce two frameworks: The first characterizes seven underlying mechanisms that form the basis for a variety of visual superpowers portrayed in fiction. The second identifies seven ways in which visualization tools and interfaces can instill a sense of empowerment in the people who use them. Building on these observations, we illustrate a diverse set of “visualization superpowers” and highlight opportunities for the visualization community to create new systems and interactions that empower new experiences with data. Material and illustrations are available under CC-BY 4.0 at osf.io/8yhfz.

Reference

Abstract

In the near future, mixed traffic consisting of manual and autonomous vehicles (AVs) will be common. Questions surrounding how vulnerable road users such as pedestrians in wheelchairs (PWs) will make crossing decisions in these new situations are underexplored. We conducted a remote co-design study with one of the researchers of this work who has the lived experience as a powered wheelchair user and applied inclusive design practices. This allowed us to identify and reflect on interface design ideas that can help PWs make safe crossing decisions at intersections. Through an iterative five-week study, we implemented interfaces that can be placed on the vehicle, on the wheelchair, and on the street infrastructure and evaluated them during the co-design sessions using a VR simulator testbed. Informed by our findings, we discuss design insights for implementing inclusive interfaces to improve interactions between autonomous vehicles and vulnerable road users.

Reference

Abstract

We present an autobiographical design journey exploring the experience of returning to long-term single player games. Continuing progress from a previously saved game, particularly when substantial time has passed, is an understudied area in games research. To begin our exploration in this domain, we investigated what the return experience is like first-hand. By returning to four long-term single player games played extensively in the past, we revealed a phenomenon we call The Pivot Point, a ‘eureka’ moment in return gameplay. The pivot point anchors our design explorations, where we created prototypes to leverage the pivot point in reconnecting with the experience. These return experiences and subsequent prototyping iterations inform our understanding of how to design better returns to gameplay, which can benefit both producers and consumers of long-term single player games.

Reference

Abstract

We compare physical and virtual reality (VR) versions of simple data visualizations and explore how the addition of virtual annotation and filtering tools affects how viewers solve basic data analysis tasks. We report on two studies, inspired by previous examinations of data physicalizations. The first study examines differences in how viewers interact with physical hand-scale, virtual hand-scale, and virtual table-scale visualizations and the impact that the different forms had on viewer’s problem solving behavior. A second study examines how interactive annotation and filtering tools might support new modes of use that transcend the limitations of physical representations. Our results highlight challenges associated with virtual reality representations and hint at the potential of interactive annotation and filtering tools in VR visualizations.

Reference

Abstract

Natural disasters cause long-lasting mental health problems such as PTSD in children. Following the 2011 Earthquake and Tsunami in Japan, we witnessed a shift of toy block play behavior in young children who suffered from stress after the disaster. The behavior reflected their emotional responses to the traumatic event. In this paper, we explore the feasibility of using data captured from block-play to assess children's stress after a major natural disaster. We prototyped sets of sensor-embedded toy blocks, AssessBlocks, that automate quantitative play data acquisition. During a three-year period, the blocks were dispatched to fifty-two post-disaster children. Within a free play session, we captured block features, a child's playing behavior, and stress evaluated by several methods. The result from our analysis reveal correlations between block play features and stress measurements and show initial promise of using the effectiveness of using AssessBlocks to assess children's stress after a disaster. We provide detailed insights into the potential as well as the challenges of our approach and unique conditions. From these insights we summarize guidelines for future research in automated play assessment systems that support children's mental health.

Reference

Abstract

We are interested in the ways pedestrians will interact with autonomous vehicle (AV) in a future AV transportation ecosystem, when nonverbal cues from the driver such as eye movements, hand gestures, etc. are no longer provided. In this work, we examine a subset of this challenge: interaction between pedestrian with reduced mobility (PRM) and AV. This study explores interface designs between AVs and people in a wheelchair to help them interact with AVs by conducting a preliminary design study. We have assessed the data collected from the study using qualitative analysis and presented different findings on AV-PRM interactions. Our findings reflect on the importance of visual interfaces, changes to the wheelchair and the creative use of the street infrastructure.

Reference

Abstract

Autonomous vehicles (AVs) will redefine interactions between road users. Presently, cyclists and drivers communicate through implicit cues (vehicle motion) and explicit but imprecise signals (hand gestures, horns). Future AVs could consistently communicate awareness and intent and other feedback to cyclists based on their sensor data. We present an exploration of AV-cyclist interaction, starting with preliminary design studies which informed the implementation of an immersive VR AV-cyclist simulator, and the design and evaluation of a number of AV-cyclist interfaces. Our findings suggest that AV-cyclist interfaces can improve rider confidence in lane merging scenarios. We contribute an AV-cyclist immersive simulator, insights on trade-offs of various aspects of AV-cyclist interaction design including modalities, location, and complexity, and positive results suggesting improved rider confidence due to AV-cyclist interaction. While we are encouraged by the potential positive impact AV-cyclist interfaces can have on cyclist culture, we also emphasize the risks over-reliance can pose to cyclists.

Reference

Talk

Abstract

We present Mutation: performing arts based approach that can help decrease the cognitive load associated with cyborg transitioning. Cyborgs are human-machine hybrids with organic and mechatronic body parts that can be implanted or worn. The transition into and out of experiencing additional body parts is not fully understood. Our goal is to draw from performing arts techniques in order to help decrease the cognitive load associated with becoming and unbecoming a cyborg. Actors constantly shift between states, whether from one character to another, or from pre- to post- performance. We contribute a straightforward adaptation of classic performing art practices to cyborg transitioning, and a study where actors used these protocols in order to enter a cyborg state, perform as a cyborg, and then exit the cyborg state. Our work on Mutation suggests that classic performing art practices can be useful in cyborg transitioning, as well as in other technology augmented experiences.

Reference

Abstract

AV-pedestrian interaction will impact pedestrian safety, etiquette, and overall acceptance of AV technology. Evaluating AV-pedestrian interaction is challenging given limited availability of AVs and safety concerns. These challenges are compounded by "mixed traffic" conditions: studying AV-pedestrian interaction will be difficult in traffic consisting of vehicles varying in autonomy level. We propose immersive pedestrian simulators as design tools to study AV-pedestrian interaction, allowing rapid prototyping and evaluation of future AV-pedestrian interfaces. We present OnFoot: a VR-based simulator that immerses participants in mixed traffic conditions and allows examination of their behavior while controlling vehicles' autonomy-level, traffic and street characteristics, behavior of other virtual pedestrians, and integration of novel AV-pedestrian interfaces. We validated OnFoot against prior simulators and Wizard-of-Oz studies, and conducted a user study, manipulating vehicles' autonomy level, interfaces, and pedestrian group behavior. Our findings highlight the potential to use VR simulators as powerful tools for AV-pedestrian interaction design in mixed traffic.

Reference

Abstract

Electronic fashion (eFashion) garments use technology to augment the human body with wearable interaction. In developing ideas, eFashion designers need to prototype the role and behavior of the interactive garment in context; however, current wearable prototyping toolkits require semi-permanent construction with physical materials that cannot easily be altered. We present Bod-IDE, an augmented reality 'mirror' that allows eFashion designers to create virtual interactive garment prototypes. Designers can quickly build, refine, and test on-the-body interactions without the need to connect or program electronics. By envisioning interaction with the body in mind, eFashion designers can focus more on reimagining the relationship between bodies, clothing, and technology.

Reference

Abstract

This paper presents a set of lessons learnt from introducing maker culture and DIY paradigms to 'at-risk' students (age 12-14). Our goal is to engage 'at-risk' students through maker culture activities. While improved technology literacy is one of the outcomes we also wanted the learners to use technology to realize concepts and ideas, and to gain freedom of thinking similar to creators, artists and designers. We present our study and a set of high level suggestions to enable thinking about how maker culture activities can facilitate engagement and creative use of technology by 1) thinking about creativity in task, 2) facilitating different entry points, 3) the importance of personal relevance, and 4) relevance to education.

Reference

Abstract

Remote collaborators working together on physical objects have difficulty building a shared understanding of what each person is talking about. Conventional video chat systems are insufficient for many situations because they present a single view of the object in a flattened image. To understand how this limited perspective affects collaboration, we designed the Remote Manipulator (ReMa), which can reproduce orientation manipulations on a proxy object at a remote site. We conducted two studies with ReMa, with two main findings. First, a shared perspective is more effective and preferred compared to the opposing perspective offered by conventional video chat systems. Second, the physical proxy and video chat complement one another in a combined system: people used the physical proxy to understand objects, and used video chat to perform gestures and confirm remote actions.

Reference

Abstract

Drivers use nonverbal cues such as vehicle speed, eye gaze, and hand gestures to communicate awareness and intent to pedestrians. Conversely, in autonomous vehicles, drivers can be distracted or absent, leaving pedestrians to infer awareness and intent from the vehicle alone. In this paper, we investigate the usefulness of interfaces (beyond vehicle movement) that explicitly communicate awareness and intent of autonomous vehicles to pedestrians, focusing on crosswalk scenarios. We conducted a preliminary study to gain insight on designing interfaces that communicate autonomous vehicle awareness and intent to pedestrians. Based on study outcomes, we developed four prototype interfaces and deployed them in studies involving a Segway and a car. We found interfaces communicating vehicle awareness and intent: (1) can help pedestrians attempting to cross; (2) are not limited to the vehicle and can exist in the environment; and (3) should use a combination of modalities such as visual, auditory, and physical.

Reference

Talk

Abstract

In this paper, we discuss the role of the movement trajectory and velocity enabled by our tele-robotic system (ReMa) for remote collaboration on physical tasks. Our system reproduces changes in object orientation and position at a remote location using a humanoid robotic arm. However, even minor kinematics differences between robot and human arm can result in awkward or exaggerated robot movements. As a result, user communication with the robotic system can become less efficient, less fluent and more time intensive.

Reference

Abstract

We compare the effectiveness of 2D maps and 3D terrain models for visibility tasks and demonstrate how interactive dynamic viewsheds can improve performance for both types of terrain representations. In general, the two-dimensional nature of classic topographic maps limits their legibility and can make complex yet typical cartographic tasks like determining the visibility between locations difficult. Both 3D physical models and interactive techniques like dynamic viewsheds have the potential to improve viewers' understanding of topography, but their impact has not been deeply explored. We evaluate the effectiveness of 2D maps, 3D models, and interactive viewsheds for both simple and complex visibility tasks. Our results demonstrate the benefits of the dynamic viewshed technique and highlight opportunities for additional tactile interactions. Based on these findings we present guidelines for improving the design and usability of future topographic maps and models.

Reference

Talk

Abstract

Do-it-yourself (DIY) inspired activities have gained popularity as a means of creative expression and self-directed learning. However, DIY culture is difficult to implement in places with limited technology infrastructure and traditional learning cultures. Our goal is to understand how learners in such a setting react to DIY activities. We present observations from a physical computing workshop with 12 students (13-15 years old) conducted at a high school in India. We observed unique challenges for these students when tackling DIY activities: a high monetary and psychological cost to exploration, limited independent learning resources, difficulties with finding intellectual courage and assumed technical language proficiency. Our participants, however, overcome some of these challenges by adopting their own local strategies: resilience, nonverbal and verbal learning techniques, and creating documentation and fallback circuit versions. Based on our findings, we discuss a set of lessons learned about makerspaces in a context with socio-technical challenges.

Reference

Abstract

People are increasingly using mobile video to communicate, collaborate, and share experiences while on the go. Yet this presents challenges in adequately sharing camera views with remote users. In this paper, we study the use of semi-autonomous drones for video conferencing, where an outdoor user (using a smartphone) is connected to a desktop user who can explore the environment from the drone's perspective. We describe findings from a study where pairs collaborated to complete shared navigation and search tasks. We illustrate the benefits of providing the desktop user with a view that is elevated, manipulable, and decoupled from the outdoor user. In addition, we articulate how participants overcame challenges in communicating environmental information and navigational cues, negotiated control of the view, and used the drone as a tool for sharing experiences. This provides a new way of thinking about mobile video conferencing where cameras that are decoupled from both users play an integral role in communication, collaboration, and sharing experiences.

Reference

Abstract

Software is typically developed incrementally and released in stages. Planning these releases involves deciding which features of the system should be implemented for each release. This is a complex planning process involving numerous trade-offs-constraints and factors that often make decisions difficult. Since the success of a product depends on this plan, it is important to understand the trade-offs between different release plans in order to make an informed choice. We present STRATOS, a tool that simultaneously visualizes several software release plans. The visualization shows several attributes about each plan that are important to planners. Multiple plans are shown in a single layout to help planners find and understand the trade-offs between alternative plans. We evaluated our tool via a qualitative study and found that STRATOS enables a range of decision-making processes, helping participants decide on which plan is most optimal.

Reference

Abstract

Lens visualization has been a prominent research area in the visualization community, fueled by the continuous need to mitigate visual clutter and occlusion resulting from the increasingly large datasets. Interactive lenses for 3D data, particularly, challenge visualization designers to conceive design strategies that facilitates the analysis of dense, multifaceted data with spatial referents. Given their relevance, the overarching research goal of this dissertation is to investigate how visualization lenses may support 3D data exploration and analysis—across both conventional and immersive environments. To this end, we begin with (1) modeling lenses by conducting a systematic review of existing lenses operating within spatial contexts. From this survey, we derive a design space that captures core design dimensions and choices involved in constructing spatially-embedded visualization lenses. Building upon this theoretical foundation, we proceed with (2) designing lenses, through the design, development, and evaluation of four immersive lenses tailored to support distinct forms of 3D data analysis: i) a deformable quadric lens for heterogeneous feature analysis, ii) a dual-imagery lens for multivariate, multi-geometry analysis, iii) a view-dependent lens for reservoir uncertainty analysis, and iv) a shape-conformal lens for spatiotemporal urban data analysis. Afterwards, we probe (3) evaluating lenses by complementing our prior theoretical and practical endeavors with empirical evidence from a controlled study comparing the efficacy of four lens-based designs in supporting time-varying urban data analysis. Throughout the course of our research agenda, we document domain-specific lessons learned and translate them into generalizable design guidelines intended to inform the broader visualization community. Finally, this dissertation concludes by suggesting future research directions for advancing visualization lenses in 3D data exploration and analysis—across both traditional and immersive environments. In the long term, we hope that this dissertation serve valuable reference points for visualization researchers and practitioners operating at either a theoretical, design, or empirical level.

Reference

Abstract

This dissertation explores the intersection of Drama, virtual reality (VR), and Human-Computer Interaction (HCI), examining their parallels and collective impact on shaping immersive digital experiences. Throughout our research, we ask: how can VR support traditional performance practices? And how can we leverage existing performance practices to support wholistic user engagement? From our interdisciplinary position we identify gaps in existing research and offer novel solutions and innovative frameworks for navigating and structuring experiences within VR. In pursuit of answers to our research questions, we contribute C1) Performance-Based Multimodal Methodological Approaches for HCI, and C2) A Demonstration of Interdisciplinary Possibilities to the landscape of HCI VR research. We propose a symbiotic relationship between the user's cognitive engagement and the digital environment's architectural and interaction design can be enhanced through the application of practices and principles from Drama and performance. The practical application of these theoretical constructs is showcased in C3) Thresholding Protocols for Digital State Change, and C4) The Virtual Rehearsal Suite (VRS), an immersive VR environment that supports solo performance training, demonstrating how Drama and performance methodologies can enhance the user's experience, offering tested perspectives and techniques that promote interaction, presence, and embodiment. Starting with our related works, we identify our interdisciplinary position with a foundation that draws from both the academic and artistic communities interested in VR as a domain, research, and performance tool. We then cast a wide net to understand the dimensions of virtual technologies and their impact on user experience with subsequent chapters investigating the layers of reality, immersion, embodiment, performance rituals, and thresholding concepts. Each chapter contributes to the identification of gaps and parallels across research domains and the discussion of how Drama and performance can elevate the understanding and advancement of VR and HCI systems. The dissertation concludes that the confluence of Drama and performance practice with Interaction Design holds the potential to shape the future aesthetics and experiential facets of virtual environments. The embrace of VR as both a tool and a medium for creative expression is positioned as a transformative leap forward in both HCI and Drama, heralding a new era of digital interaction that needs to embrace the full spectrum of human experience for success and longevity. Our work positions actors as interaction specialists, capable of existing in multiple realities at once and providing insightful reflections on their experiences in iterative processes. The VRS study demonstrates this ability, while emphasizing that virtual environments are not merely technological constructs but complex experiential spaces where the physical and digital converge, challenging traditional perceptions of reality. It highlights the importance of centralizing the user as key to creating compelling virtual experiences that con only be achieved through meticulously designed interactions that resonate with the user's sensory and cognitive faculties. Our study underscores the efficacy of VR in supporting actor training with minimal digital interventions, facilitating a seamless transition into and out of VR, enhancing focus during VR engagement, and addressing issues such as VR sickness. It highlights the centralization of the human element as the pivotal factor in VR creation, emphasizing that VR environments should cater to the nuanced spectrum of human emotions, behaviours, and social interactions.

Reference

Abstract

Dexterous and natural haptic interaction with the environment in Virtual Reality promises a new era of embodied and intuitive computing. But among the remaining challenges stands the difficulty of natural wall interactions. Personal haptic devices for natural wall interaction in virtual reality should be portable and should provide passive, body-scale interactions. However, existing techniques fall short: Room-scale proxies lack portability, wearable robotic arms are energy-intensive and induce friction, and existing hand-scale passive interaction techniques are unsuitable for continuous large-scale renders. In this thesis, we introduce PantographHaptics, a technique which uses the scaling properties of a pantograph to passively render body-scale surfaces. A pantograph is a classical linkage mechanism which can enlarge or shrink designs by coordinating nodes to move in scaled, geometrically similar paths. To our knowledge, no prior work has applied the pantograph mechanism to large-scale immersive haptics. PantographHaptics is a novel method for passively achieving body-scale haptics which uses a pantograph to scale up a small positional constraint into an encounterable midair render. We present the conceptual foundation underpinning of PantographHaptics by describing the operation of the pantograph mechanism and detailing how we apply it for haptics. Then we verify the PantographHaptics technique through two prototypes: HapticLever, a grounded system, and Feedbackpack, a wearable device. We detail the designs, implementations, and technical evaluations of both prototypes, and we highlight the challenges and solutions involved in their development. We evaluate these prototypes with user evaluations, which contribute assessments of their interaction fidelity, investigations of their usability, comparisons of their performance against other haptic modalities, and recorded participant experiences of using the devices. By introducing and verifying PantographHaptics, we show that this novel technique is a viable and promising approach for interactions with large surfaces. By documenting the development of our prototype artifacts and reporting user experiences with the devices, we contribute a foundation for future research.

Reference

Abstract

Virtual reality can be a highly immersive experience due to its realistic visual presentation. This immersive state is useful for applications including education, training, and entertainment. To enhance the state of immersion provided by virtual reality further, devices capable of simulating touch and force have been researched to allow not only a visual and audio experience but a haptic experience as well. Such research has investigated many approaches to generating haptics for virtual reality but often does not explore how to create an immersive haptic experience using them. In this thesis, we present a discussion on four proposed areas of the virtual reality haptic experience design process using a demonstration methodology. To investigate the application of haptic devices, we designed a modular ungrounded haptic system which was used to create a general-purpose device capable of force-based feedback and used it in the three topics of exploration. The first area explored is the application of existing haptic theory for aircraft control to the field of virtual reality drone control. The second area explored is the presence of the size-weight sensory illusion within virtual reality when using a simulated haptic force. The third area explored is how authoring within a virtual reality medium can be used by a designer to create VR haptic experiences. From these explorations, we begin a higher-level discussion of the broader process of creating a virtual reality haptic experience. Using the results of each project as a representation of our proposed design steps, we discuss not only the broader concepts the steps contribute to the process and their importance, but also draw connections between them. By doing this, we present a more holistic approach to the large-scale design of virtual reality haptic experiences and the benefits we believe it provides.

Reference

Abstract

Autonomous Vehicle (AV) technology promises to revolutionize human life. The promise of AVs includes reduced highway congestion, more efficient energy usage, and cheaper goods and services. However, without careful design, removing human drivers from vehicles will eliminate the natural communication channels which enable pedestrians to navigate safely. This thesis aims to design, present, and study anthropomorphic interfaces for autonomous vehicles, with the objective of enabling AVs to communicate with pedestrians through non-verbal cues. Non-verbal human communication is vital in human relationships. People use non-verbal communication when speech is impractical, such as when interacting with vehicles. When looking into ways in which AVs can use non-verbal communication to interact with pedestrians, we were inspired by the prospect of using anthropomorphic interfaces. This concept is well explored in Human-Robot Interaction (HRI) but has not been investigated in the context of AVs. For this thesis, we explored the design of anthropomorphic interfaces for autonomous vehicles. First, we proposed three types of anthropomorphic interfaces for AVs: facial expressions, hand gestures, and humanoid torsos. We developed a design space for each category using sketches and a low-fi prototype. Then, to research the benefits and limitations of anthropomorphic AVs, we implemented our AV interfaces in a Virtual Reality (VR) environment and developed two testbeds to evaluate their feasibility and scalability. Finally, we conducted two studies using the two testbeds. We investigated the study results using immersive analytics alongside traditional methods and revealed that anthropomorphic AVs could be helpful in AV-pedestrian interaction when designed by specific guidelines. Since we studied anthropomorphic AVs in VR, we were interested in the possibilities of analyzing the data of our study in an immersive environment. We designed a VR prototype specifically to analyze the data collected from the anthropomorphic AV study. The prototype provided basic immersive analytics features for the AV study data. We conducted an expert session with two domain experts to evaluate our immersive analytics prototype. The study contributed insights into the opportunities and challenges of utilizing immersive analytics to analyze AV studies.

Reference

Abstract

In the near future, mixed traffic consisting of manual and autonomous vehicles (AVs) will be common. Autonomous vehicles with advanced technology offer opportunities for innovative designs and introduce communication challenges for vulnerable road users such as pedestrians and cyclists. Our goal is to explore the emerging new domain of interaction between different road users and autonomous vehicles in a future AV transportation ecosystem. This led us to conduct the thesis following these two themes: 1) understanding design opportunities for external automotive displays (EADs) of AVs; 2) exploring the design of interactions between vulnerable road users (VRUs) and AVs. In theme 1, our work extends contemporary research into visualizations and related applications for autonomous vehicles. Focusing on external car bodies as a design space we introduce a set of EADs. EADs show visualizations to share context and user-specific information and offer opportunities for interaction between users and AVs. We conducted a design study to explore design concepts for EADs to provide services to different road users: pedestrians, passengers, and drivers of other vehicles. Based on the design study, we prototyped four EADs in virtual reality (VR) to demonstrate the potential of our approach. This exploration contributes to our vision for EADs, a design critique of the prototypes, and a discussion of the possible impact and future usage of external automotive displays. In theme 2, we are interested in the ways pedestrians will interact with autonomous vehicles in the absence of non-verbal cues from the driver (such as eye movements, hand gestures, etc.). Crossing streets in these new situations could be more dangerous for VRUs without a proper communication medium. We examined a subset of this challenge with two groups of pedestrians: interaction between AVs and pedestrians with hearing aids (PHAs), and pedestrians in wheelchairs (PWs). First, we worked with hearing aid users as a preliminary exploration of this research. We conduct a co-design study with a co-designer with hearing impairment who has lived experience of wearing hearing aid enhancements. This study contributes several insights and design recommendations on how potential audio cues can be designed to enhance direct communications between PHAs and AVs. For the second part of our research, we designed interactions between pedestrians in wheelchairs and AVs. From an early exploration of potential interface designs through a design study with interaction designers, we prototyped different interfaces in VR. Then, we evaluated the implemented simulations during a co-design study with a powered wheelchair user following inclusive design practices. We identify and reflect on interface design ideas that can help PWs make safe crossing decisions at intersections and discuss design insights for implementing different inclusive interfaces.

Reference

Abstract

Autonomous vehicle research today places an emphasis on developing better sensors and algorithms to enable the vehicle to localize itself in the environment, plan routes, and control its movement. Surveying the general public reveals optimism about the technology but also some skepticism about its ability to communicate with vulnerable road users such as pedestrians and cyclists. In today's interaction with vehicles at crosswalks, pedestrians rely on cues originating from the vehicle and the driver. Vehicle cues relate to its kinematics such as speed and stopping distance while driver cues are concerned with communication such as eye gaze and contact, head and body movement, and hand gestures. In autonomous vehicles, however, a driver is not expected to be on-board to provide cues to pedestrians. We attempted to tackle the problem of designing novel ways to facilitate autonomous vehicle-pedestrian interaction at crosswalks. We propose interfaces which communicate an autonomous vehicle's awareness and intent as a means of helping pedestrians make safe crossing decisions. Through our exploration, we make several contributions. First, we propose a design space for building interfaces using different cue modalities and cue locations. From an early exploration of this design space, we prototype interfaces designed to facilitate autonomous vehicle-pedestrian interaction. The interaction between vehicles and pedestrians will become more challenging during the transition period until all vehicles on the road are fully autonomous. During this period which we term mixed traffic, vehicles of varying levels of autonomy will occupy roads, some of which will have drivers, others such as semi-autonomous which may have distracted drivers, and fully autonomous vehicles which may or may not have drivers. To study this problem, we contribute a virtual reality-based pedestrian simulator. Our final contribution relates to the evaluation of interfaces in the real and virtual world where we found their inclusion helped pedestrians make safe crossing decisions.

Reference

Abstract

Traditional flat topographic maps are difficult to understand due to the distortion and compromise of the 3-dimensional (3D) spatial representation when it is folded into lower-dimension media (e.g. 2D). During the process, the x-y coordinate of a location can be captured but its physical elevation must be transformed using some visualization techniques, resulting in noticeable cognitive effort in comprehending the original geometric and geographic properties of the original terrain. In this manuscript-based dissertation, I present a collection of my past publications that aim to increase the readability of topographic maps by restoring the original spatiality of the terrain - including the elevations - with a physical map representation and then superimpose additional data visualization on top of it. In this way, the entire terrain topology is kept in a scaled physical representation, allowing users to view it with natural human perceptions. Additionally, user gestures can be tracked in real-time as a sketch-based input to allow novel dynamic interaction of the map interface and data manipulation of the spatial information. Through the chapters, I present the aforementioned concept, named interactive topographic interface, along with a few applications of it in different academic and industrial environments. I also report the design and results of a user study that compares the interface with traditional flat topographic maps. In the long-term, I hope that research mentioned in this dissertation inspires future interactive physical cartography to not only improve map comprehension but also facilitate better spatial and situational awareness over the map interface, resulting in an evolved map usefulness.

Reference

Abstract

Acquiring, storing, processing, synthesizing, visualizing, and interpreting data are core to scientific knowledge discovery. This data life cycle is common to many diverse fields such as medicine and petroleum engineering. A wide range of techniques may be used for interacting with and visualizing data. Immersive technologies such as augmented reality and virtual reality show great potential to enhance important workplace activities such as collaboration. To this end an immersive, collaborative tool for visualizing reservoir data is discussed. Some collaborative scenarios using these technologies are then described. It is important to carefully consider how these technologies will be incorporated into a professional setting to ensure tools based on these technologies will provide a high quality user experience while meeting the security needs of industry. In order to further this goal, some of the architectural considerations of a collaboration tool that uses a variety of technologies from the reality-virtuality continuum are explored. A prototype tool is then presented that has been developed for collaborating over petroleum reservoir scenarios involving sensitive data. This tool incorporates visual protection mechanisms to facilitate collaboration while providing enhanced control over information disclosure. A user feedback session was performed with reservoir engineering subject matter experts, and the results from this exploratory evaluation are reported.

Reference

Abstract

Interactive garments enable new forms of communication between our bodies and with other people. In electronic fashion (eFashion) design, interactive garments on high fashion runways envision how people might use interactive technologies to enhance our clothing with new sensing and output capabilities. Researchers and fashion designers have since explored new interactive textiles that enable aesthetics-driven, interactive, and new material properties to explore on clothing. While there exist physical tools to implement interactive garments and software tools to create the visual aesthetic of a garment, these tools cannot yet enable designers to use new eFashion technologies in their garments because they require engineering expertise and specialized laboratory equipment. In this thesis, I explore the use of computer-aided prototyping tools to develop interactive eFashion garments. I present case studies with makers and two experienced eFashion designers about their design practices and formulate design guidelines for prototyping tools. I then present two prototyping tools for implementation and exploration of interactive garments. Finally, I discuss future work for physical and virtual prototyping tools in eFashion.

Reference

Abstract

Simulation and immersive environments are gaining popularity in various contexts. Arguably, such interactive systems have the potential to benefit many users in a variety of education and training scenarios. However, some of these systems especially with the lack of skilled instructors are still faced by challenges of operational complexity, the incorporation of different technologies and features, and the limited availability of performance measures and feedback. Therefore, the design of these systems would benefit from integrating experiential aspects and essential educational aids. For example, users of such learning systems, especially the novice ones, can be better supported by a smoother learning curve, detailed guidance features, the availability of feedback and performance reporting, and the integration of engaging & reflective capabilities. In essence, we recognize a need to re-explore learning aids and how they impact design, usage, and overall learning experience in interactive immersive environments.
The goal of this dissertation is to mediate experiential learning in interactive immersive environments. This includes exploring existing and novel learning aids that would facilitate learning with improved engagement and immersion, enrich learners with insightful reflections, better support novice users’ learning and training needs, and ultimately enhance the overall experience.
To achieve this goal, we utilized existing learning models and simulation-based training approaches and proposed a framework of learning aids to mediate learning in interactive immersive environments. Working closely with domain expert collaborators, we designed, implemented and evaluated four new interactive immersive prototypes in an attempt to validate the practicality of our aids. The first prototype, NeuroSimVR, is a stereoscopic visualization augmented with educational aids to support how medical users learn about a common back surgery procedure. The second prototype, ReflectiveSpineVR, is an immersive virtual reality surgical simulation with innovative
interaction history capabilities that aim to empower users’ memories and enable deliberate repetitive practice as needed. The third prototype, JackVR, is an interactive immersive training system, utilizing novel gamification elements, and aims to support oil-and-gas experts in the process of landing oil rigs. Our fourth prototype, RoboTeacher, involves a humanoid robot instructor for teaching people industrial assembly tasks. In our prototypes, we presented novel learning aids, visualization, and interaction techniques that are new to many of the current immersive learning tools. We conclude this dissertation with lessons learned and guidelines for designing with learning aids in future research directions that target interactive experiential environments.

Reference

Abstract

The Maker Movement aims to democratize technological practices and promises many benefits for people including improved technical literacy, a means for self-expression and agency, and an opportunity to become more than consumers of technology. As part of the Maker Movement, people build hobbyist and utilitarian projects by themselves using programmable electronics (e.g., microcontroller, sensors, actuators) and software tools. While the Maker Movement is gaining momentum globally, some people are left out. Constraints such as material limitations, educational culture restrictions, and emotional or behavioral difficulties can often limit people from taking part in the Maker Movement. We refer to the systematic investigation of how diverse people respond to making-centered activities within constraints as an exploration of making within constraints.
In this dissertation, we (1) study how people respond to creating physical objects by themselves within constraints and, (2) investigate how to design technology that can help makers within constraints. We conducted an observational study in an impoverished school in India and identified the students' challenges and their strategies for making within material and educational culture constraints. We conducted a second study with at-promise youth in Canada and identified a set of lessons learned to engage youth within emotional and behavioral constraints in making-centered activities. Leveraging our observations, we proposed Augmented Reality (AR)-mediated prototyping as a way to address material constraints. AR-mediated prototyping can help makers to build, program, interact with and iterate on physical computing projects that combine both real-world and stand-in virtual electronic components. We designed, implemented, and evaluated a technology probe, Polymorphic Cube (PMC), as an instance of our vision. Our results show that PMC helped participants prototype despite missing I/O electronic components, and highlighted how AR-mediated prototyping extends to exploring project ideas, tinkering with implementation, and making with others.
Informed by our empirical and design explorations, we suggest a set of characteristics of constraints and implications for designing future technologies for makers within constraints. In the long-term, we hope that this research will inspire interaction designers to develop new tools that can help resolve constraints for making.

Reference

Abstract

Transparent displays are ‘see-through’ screens: a person can simultaneously view both the graphics on the screen and real-world content visible through the screen. Interactive transparent displays can serve as an important medium supporting face-to-face collaboration, where people interact with both sides of the display and work together. Such displays enhance workspace awareness, which smooths collaboration: when a person is working on one side of a transparent display, the person on the other side can see the other's hand gestures, gaze, and what s/he is currently manipulating on the shared screen. Even so, we argue that in order to provide effective support for collaboration, designing such transparent displays must go beyond current offerings. We propose using two-sided transparent displays, which can present different content on both sides. The displays should also accept interactive input on both sides and visually augment users’ actions when display transparency is reduced. We operationalized these design requirements with our two-sided transparent display prototype, FACINGBOARD-II, and devised a palette of supportive interaction techniques. Through empirical studies, we found that the workspace awareness provided by transparent displays is compromised with degrading display transparency, and that visually enhancing user actions can compensate for this awareness loss.

Reference

Abstract

What if one could paint music? Music is often shaped by the medium we use to interact with it, and thus the development of different ways to compose and experience music can open up new creative possibilities to people. This somewhat whimsical proposition of painting music, and a quest to find a new medium to create, experience, and interact with music, gave impetus and shape to the development of a musical interface known as Vuzik. Vuzik is an interface for creating and visualizing music through painting gestures on a large interactive surface. This thesis presents the vision, design and implementation of Vuzik. It then describes explorations of it as a tool for music education, and for the performance and creation of electronic music. Finally, it presents evaluation efforts of Vuzik in its performance of each of these explorations, and reflects on their implications regarding the nature and potential of Vuzik.

Reference

Reference

Reference

Reference

Reference

Reference