Abstract

This paper is an invitation to HCI designers and artist-researchers to weave together practice-based ways of knowing with others’ experiences with their work, focused on understanding cognitive and aesthetic impressions garnered during exhibitions. It describes the first author's process for weaving together the warp (practice-based ways of knowing) and weft (interview-based insights into others’ experiences) by leveraging the exhibition site as a place to generate knowledge with attendees experiencing their work. Understanding others’ experiences informs the experimentation that practice-based knowledge generation is founded on, deepening and enriching the resulting work.

Reference

Abstract

We present perspectives from 10 autistic Twitch streamers regarding their experiences as livestreamers and how autism uniquely colors their experiences. Livestreaming offers a social online experience distinct from in-person, face-to-face communication, where autistic people tend to encounter challenges. Our reflexive thematic analysis of interviews with 10 participants showcases autistic livestreamers’ perspectives in their own words. Our findings center on the importance of having streamers establishing connections with other, sharing autistic identities, controlling a space for social interaction, personal growth, and accessibility challenges. In our discussion, we highlight the crucial value of having a medium for autistic representation, as well as design opportunities for streaming platforms to onboard autistic livestreamers and to facilitate livestreamers communication with their audience.

Reference

Abstract

Art is not Research. Research is not Art. is a multimedia, multi-site participatory installation by a collective of artists and researchers from Calgary, Toronto, and Lancaster; it is informed by these contexts. It reflects the tensions between how “participants” are treated in participatory art and interaction research. It offers a framework through which we can explore how epistemologies might evolve in a blending between Art and Research. Visitors download the paper to read, critically reflect on the relationship between art and research, and experientially engage with the material through a series of creative prompts. A performance variation of the piece will be performed in-person and online through the ACM SIGCHI Conference on Human Factors in Computing Systems alt.chi track.

Reference

Abstract

We explore how the lens of fictional superpowers can help characterize how visualizations empower people and provide inspiration for new visualization systems. Researchers and practitioners often tout visualizations' ability to “make the invisible visible” and to “enhance cognitive abilities.” Meanwhile superhero comics and other modern fiction often depict characters with similarly fantastic abilities that allow them to see and interpret the world in ways that transcend traditional human perception. We investigate the intersection of these domains, and show how the language of superpowers can be used to characterize existing visualization systems and suggest opportunities for new and empowering ones. We introduce two frameworks: The first characterizes seven underlying mechanisms that form the basis for a variety of visual superpowers portrayed in fiction. The second identifies seven ways in which visualization tools and interfaces can instill a sense of empowerment in the people who use them. Building on these observations, we illustrate a diverse set of “visualization superpowers” and highlight opportunities for the visualization community to create new systems and interactions that empower new experiences with data. Material and illustrations are available under CC-BY 4.0 at osf.io/8yhfz.

Reference

Abstract

In the near future, mixed traffic consisting of manual and autonomous vehicles (AVs) will be common. Questions surrounding how vulnerable road users such as pedestrians in wheelchairs (PWs) will make crossing decisions in these new situations are underexplored. We conducted a remote co-design study with one of the researchers of this work who has the lived experience as a powered wheelchair user and applied inclusive design practices. This allowed us to identify and reflect on interface design ideas that can help PWs make safe crossing decisions at intersections. Through an iterative five-week study, we implemented interfaces that can be placed on the vehicle, on the wheelchair, and on the street infrastructure and evaluated them during the co-design sessions using a VR simulator testbed. Informed by our findings, we discuss design insights for implementing inclusive interfaces to improve interactions between autonomous vehicles and vulnerable road users.

Reference

Abstract

Machine learning and predictive algorithms find patterns in large stores of data and make predictions which corporations and governments use to support decision-making. Yet, the system's representation of reality can be more influential to outcomes than the complexities of daily life. They become problematic when they undermine the inclusivity of public decision making, and when their use perpetuates social or economic inequality. To address these challenges, the public must be able to participate in discourse about the implications of algorithmic systems. I propose a series of participatory installations exploring the impacts of algorithmic systems, providing contexts for active exploration of these concerns. I will conduct phenomenographic interviews to better understand how visitors experience art installations about technical topics, providing insight for subsequent installations. I will consolidate the results into a set of best practices about engaging the public on these topics.

Reference

Abstract

Architects and designers communicate their ideas within a range of representational methods. No single instance of these methods, either in the form of orthographic projections or perspectival representation, can address all questions regarding the design, but as a whole, they demonstrate a comprehensive range of information about the building or object they intend to represent. This explicates an inevitable degree of deficiency in representation, regardless of its type. In addition, perspective-based optical illusions manipulate our spatial perception by deliberately misrepresenting the reality. In this regard, they are not new concepts to architectural representation. As a consequence, Optically Illusive Architecture (OIA) is proposed, not as a solution to fill the gap between the representing and represented spaces, but as a design paradigm whose concept derives from and accounts for this gap. By OIA we aim to cast light to an undeniable role of viewpoints in designing architectural spaces. The idea is to establish a methodology in a way that the deficiency of current representational techniques—manifested as specific thread of optical illusions—flourishes into thoughtful results embodied as actual architectural spaces. Within our design paradigm, we define a framework to be able to effectively analyze its precedents, generate new space, and evaluate their efficiencies. Moreover, the framework raises a hierarchical set of questions to differentiate OIA from a visual gimmick. Furthermore, we study two OIA-driven environments, by conducting empirical studies using Virtual Reality (VR). These studies bear essential information, in terms of design performance, and the public’s ability to engage and interact with an OIA space, prior to the actual fabrication of the structures.

Reference

Abstract

Empathy tools are experiences designed to evoke empathetic responses by placing the user in another’s lived and felt experience. The problem is that designers do not have a common vocabulary to describe empathy tool experiences; consequently, it is difficult to compare/contrast empathy tool designs or to think about their efficacy. To address this problem, we analyzed 26 publications on empathy tools to develop a descriptive framework for designers of empathy tools. Based on our analysis, we found that empathy tools can be described along three dimensions: (i) the amount of agency the tool allows, (ii) the user’s perspective while using the tool, and (iii) the type of sensations that are experienced. We show that this framework can be used to describe a wide variety of empathy tools and provide recommendations for empathy tool designers, as well as techniques for measuring the efficacy of an empathy tool experience.

Reference

Abstract

We present TRAVIS II, an augmented acoustic violin with touch sensors integrated into its 3D printed fingerboard that track left-hand finger gestures in real time. The fingerboard has four strips of conductive PLA filament which produce an electric signal when fingers press down on each string. While these sensors are physically robust, they are mechanically assembled and thus easy to replace if damaged. The performer can also trigger presets via four FSRs attached to the body of the violin. The instrument is completely wireless, giving the performer the freedom to move throughout the performance space. While the sensing fingerboard is installed in place of the traditional fingerboard, all other electronics can be removed from the augmented instrument, maintaining the aesthetics of a traditional violin. Our design allows violinists to naturally create music for interactive performance and improvisation without requiring new instrumental techniques. In this paper, we describe the design of the instrument, experiments leading to the sensing fingerboard, and performative applications of the instrument.

Reference

Talk

Abstract

We built a chatbot system–Audience Bot–that simulates an audience for novice live streamers to engage with while streaming. New live streamers on platforms like Twitch are expected to perform and talk to themselves, even while no one is watching. We ran an observational lab study on how Audience Bot assists novice live streamers as they acclimate to multitasking–simultaneously playing a video game while performing for a (simulated) audience.

Reference

Abstract

We were approached by a group of healthcare providers who are involved in the care of chronic patients looking for potential technologies to facilitate the process of reviewing patient-generated data during clinical visits. Aiming at understanding the healthcare providers’ attitudes towards reviewing patient-generated data, we (1) conducted a focus group with a mixed group of healthcare providers. Next, to gain the patients’ perspectives, we (2) interviewed eight chronic patients, collected a sample of their data and designed a series of visualizations representing patient data we collected. Last, we (3) sought feedback on the visualization designs from healthcare providers who requested this exploration. We found four factors shaping patient-generated data: data & context, patient’s motivation, patient’s time commitment, and patient’s support circle. Informed by the results of our studies, we discussed the importance of designing patient-generated visualizations for individuals by considering both patient and healthcare provider rather than designing with the purpose of generalization and provided guidelines for designing future patient-generated data visualizations.

Reference

Abstract

We present a mixed-methods study of viewers on their practices and motivations around watching mukbang — video streams of people eating large quantities of food. Viewers' experiences provide insight on future technologies for multisensorial video streams and technology-supported commensality (eating with others). We surveyed 104 viewers and interviewed 15 of them about their attitudes and reflections on their mukbang viewing habits, their physiological aspects of watching someone eat, and their perceived social relationship with mukbangers. Based on our findings, we propose design implications for remote commensality, and for synchronized multisensorial video streaming content.

Reference

Talk

Abstract

We are interested in the ways pedestrians will interact with autonomous vehicle (AV) in a future AV transportation ecosystem, when nonverbal cues from the driver such as eye movements, hand gestures, etc. are no longer provided. In this work, we examine a subset of this challenge: interaction between pedestrian with reduced mobility (PRM) and AV. This study explores interface designs between AVs and people in a wheelchair to help them interact with AVs by conducting a preliminary design study. We have assessed the data collected from the study using qualitative analysis and presented different findings on AV-PRM interactions. Our findings reflect on the importance of visual interfaces, changes to the wheelchair and the creative use of the street infrastructure.

Reference

Abstract

Autonomous vehicles (AVs) will redefine interactions between road users. Presently, cyclists and drivers communicate through implicit cues (vehicle motion) and explicit but imprecise signals (hand gestures, horns). Future AVs could consistently communicate awareness and intent and other feedback to cyclists based on their sensor data. We present an exploration of AV-cyclist interaction, starting with preliminary design studies which informed the implementation of an immersive VR AV-cyclist simulator, and the design and evaluation of a number of AV-cyclist interfaces. Our findings suggest that AV-cyclist interfaces can improve rider confidence in lane merging scenarios. We contribute an AV-cyclist immersive simulator, insights on trade-offs of various aspects of AV-cyclist interaction design including modalities, location, and complexity, and positive results suggesting improved rider confidence due to AV-cyclist interaction. While we are encouraged by the potential positive impact AV-cyclist interfaces can have on cyclist culture, we also emphasize the risks over-reliance can pose to cyclists.

Reference

Talk

Abstract

We present the second iteration of a Touch-Responsive Augmented Violin Interface System, called TRAVIS II, and two compositions that demonstrate its expressivity. TRAVIS II is an augmented acoustic violin with touch sensors integrated into its 3-D printed fingerboard that track left-hand finger gestures in real time. The fingerboard has four strips of conductive PLA filament that produce an electric signal when fingers press down on each string. Although these sensors are physically robust, they are mechanically assembled and thus easy to replace if damaged. The performer can also trigger presets via four sensors attached to the body of the violin. The instrument is completely wireless, giving the performer the freedom to move throughout the performance space. Although the sensing fingerboard is installed in place of the traditional fingerboard, all other electronics can be removed from the augmented instrument, maintaining the aesthetics of a traditional violin. Our design allows violinists to naturally create music for interactive performance and improvisation without requiring new instrumental techniques. The first author composed two compositions to highlight TRAVIS II: “Dream State” and “Kindred Dichotomy.” Both of these compositions involve improvisation in their creative process and include interactive visuals. In this article we describe the design of the instrument, experiments leading to the sensing fingerboard, performative applications of the instrument, and compositional considerations for the resultant pieces.

Reference

Abstract

Pen-based input is often treated as auxiliary to mobile devices. We posit that cross-device interactions can inspire and extend the design space of pen-based interactions into new, expressive directions. We realize this through WatchPen, a smartwatch mounted on a passive, capacitive stylus that: (1) senses the usage context and leverages it for expression (e.g., changing colour), (2) contains tools and parameters within the display, and (3) acts as an on-demand output. As a result, it provides users with a dynamic relationship between inputs and outputs, awareness of current tool selection and parameters, and increased expressive match (e.g., added ability to mimic physical tools, showing clipboard contents). We discuss and reflect upon a series of interaction techniques that demonstrate WatchPen within a drawing application. We highlight the expressive power of leveraging multiple sensing and output capabilities across both the watch-augmented stylus and the tablet surface.

Reference

Abstract

We discuss Logical Conclusion, an analog interactive installation which presents issues surrounding the social impacts of algorithms used by corporations and governments via logic puzzles with physical elements that visitors manipulate to solve. We present the combination of physicality and participation as promising tools to engage the public with the ways that complex technologies interact with society. We also pose questions regarding how such strategies might be extended by the addition of responsive tangible computing elements.

Reference

Abstract

Astral is a prototyping tool for authoring mobile and smart object interactive behaviours. It mirrors selected display contents of desktop applications onto mobile devices (smartphones and smartwatches), and streams/remaps mobile sensor data to desktop input events (mouse or keyboard) to manipulate selected desktop contents. This allows designers to use familiar desktop applications (e.g. PowerPoint, AfterEffects) to prototype rich interactive behaviours. Astral combines and integrates display mirroring, sensor streaming and input remapping, where designers can exploit familiar desktop applications to prototype, explore and fine-tune dynamic interactive behaviours. With Astral, designers can visually author rules to test real-time behaviours while interactions take place, as well as after the interaction has occurred. We demonstrate Astral's applicability, workflow and expressiveness within the interaction design process through both new examples and replication of prior approaches that illustrate how various familiar desktop applications are leveraged and repurposed.

Reference

Abstract

We investigate enhancing virtual haptic experiences by applying Stochastic Resonance or SR noise to the user's hands. Specifically, we focus on improving users' ability to discriminate between virtual textures modelled from nine grits of real sandpaper in a virtual texture discrimination task. We applied mechanical SR noise to the participant's skin by attaching five flat actuators to different points on their hand. By fastening a linear voice-coil actuator and a 6-DOF haptic device to participants' index finger, we enabled them to interact and feel virtual sandpapers while inducing different levels of SR noise. We hypothesize that SR will improve their discrimination performance.

Reference

Abstract

While improvised theatre (improv) is often performed on a bare stage, improvisers sometimes incorporate physical props to inspire new directions for a scene and to enrich their performance. A tech booth can improvise light and sound technical elements, but coordinating with improvisers' actions on-stage is challenging. Our goal is to inform the design of an augmented prop that lets improvisers tangibly control light and sound technical elements while performing. We interviewed five professional improvisers about their use of physical props in improv, and their expectations of a possible augmented prop that controls technical theatre elements. We propose a set of guidelines for the design of an augmented prop that fits with the existing world of unpredictable improvised performance.

Reference

Abstract

Providing data visualization authoring tools for the general public remains an ongoing challenge. Inspired by block-printing, we explore how visualization stamps as a physical visualization authoring tool could leverage both visual freedom and ease of repetition. We conducted a workshop with two groups---visualization experts and non-experts---where participants authored visualizations on paper using hand-carved stamps made from potatoes and sponges. The low-fidelity medium freed participants to test new stamp patterns and accept mistakes. From the created visualizations, we observed several unique traits and uses of block-printing tools for visualization authoring, including: modularity of patterns, annotation guides, creation of multiple patterns from one stamp, and various techniques to apply data onto paper. We discuss the issues around expressivity and effectiveness of block-printed stamps in visualization authoring, and identify implications for the design and assembly of primitives in potential visualization stamp kits, as well as applications for future use in non-digital environments.

Reference

Abstract

In improvisational theatre (improv), actors perform unscripted scenes together, collectively creating a narrative. Audience suggestions introduce randomness and build audience engagement, but can be challenging to mediate at scale. We present Robot Improv Puppet Theatre (RIPT), which includes a performance robot (Pokey) who performs gestures and dialogue in short-form improv scenes based on audience input from a mobile interface. We evaluated RIPT in several initial informal performances, and in a rehearsal with seven professional improvisers. The improvisers noted how audience prompts can have a big impact on the scene - highlighting the delicate balance between ambiguity and constraints in improv. The open structure of RIPT performances allows for multiple interpretations of how to perform with Pokey, including one-on-one conversations or multi-performer scenes. While Pokey lacks key qualities of a good improviser, improvisers found his serendipitous dialogue and gestures particularly rewarding.

Reference

Abstract

Electronic fashion (eFashion) garments use technology to augment the human body with wearable interaction. In developing ideas, eFashion designers need to prototype the role and behavior of the interactive garment in context; however, current wearable prototyping toolkits require semi-permanent construction with physical materials that cannot easily be altered. We present Bod-IDE, an augmented reality 'mirror' that allows eFashion designers to create virtual interactive garment prototypes. Designers can quickly build, refine, and test on-the-body interactions without the need to connect or program electronics. By envisioning interaction with the body in mind, eFashion designers can focus more on reimagining the relationship between bodies, clothing, and technology.

Reference

Abstract

Based on an analysis of 49 popular contemporary video games, we develop a descriptive framework of visual interaction cues in video games. These cues are used to inform players what can be interacted with, where to look, and where to go within the game world. These cues vary along three dimensions: the purpose of the cue, the visual design of the cue, and the circumstances under which the cue is shown. We demonstrate that this framework can also be used to describe interaction cues for augmented reality applications. Beyond this, we show how the framework can be used to generatively derive new design ideas for visual interaction cues in augmented reality experiences.

Reference

Talk

Abstract

Remote collaborators working together on physical objects have difficulty building a shared understanding of what each person is talking about. Conventional video chat systems are insufficient for many situations because they present a single view of the object in a flattened image. To understand how this limited perspective affects collaboration, we designed the Remote Manipulator (ReMa), which can reproduce orientation manipulations on a proxy object at a remote site. We conducted two studies with ReMa, with two main findings. First, a shared perspective is more effective and preferred compared to the opposing perspective offered by conventional video chat systems. Second, the physical proxy and video chat complement one another in a combined system: people used the physical proxy to understand objects, and used video chat to perform gestures and confirm remote actions.

Reference

Abstract

Toolkit research plays an important role in the field of HCI, as it can heavily influence both the design and implementation of interactive systems. For publication, the HCI community typically expects toolkit research to include an evaluation component. The problem is that toolkit evaluation is challenging, as it is often unclear what 'evaluating' a toolkit means and what methods are appropriate. To address this problem, we analyzed 68 published toolkit papers. From our analysis, we provide an overview of, reflection on, and discussion of evaluation methods for toolkit contributions. We identify and discuss the value of four toolkit evaluation strategies, including the associated techniques that each employs. We offer a categorization of evaluation strategies for toolkit researchers, along with a discussion of the value, potential limitations, and trade-offs associated with each strategy.

Reference

Talk

Abstract

In this paper, we discuss the role of the movement trajectory and velocity enabled by our tele-robotic system (ReMa) for remote collaboration on physical tasks. Our system reproduces changes in object orientation and position at a remote location using a humanoid robotic arm. However, even minor kinematics differences between robot and human arm can result in awkward or exaggerated robot movements. As a result, user communication with the robotic system can become less efficient, less fluent and more time intensive.

Reference

Abstract

We present an observational study of physical prototype critique that highlights some of the challenges of communicating physical behaviors and materiality at a distance. Geographically distributed open hardware communities often conduct user feedback and peer critique sessions via video conference. However, people have difficulty using current video conferencing tools to demonstrate and critique physical designs. To examine the challenges of remote critique, we conducted an observational lab study in which participants critiqued pairs of physical prototypes (prosthetic hands) for a face-to-face or remote collaborator. In both conditions, participants' material experiences were an important part of their critique, however their attention was divided between interacting with the prototype and finding strategies to communicate `invisible' features. Based on our findings, we propose design implications for remote collaboration tools that support the sharing of material experiences and prototype critique.

Reference

Abstract

Interactive, smart objects – customized to individuals and uses – are central to many movements, such as tangibles, the internet of things (IoT), and ubiquitous computing. Yet, rapid prototyping both the form and function of these custom objects can be problematic, particularly for those with limited electronics or programming experience. Designers often need to embed custom circuitry; program its workings; and create a form factor that not only reflects the desired user experience but can also house the required circuitry and electronics. To mitigate this, we created Pineal, a design tool that lets end-users: (1) modify 3D models to include a smart watch or phone as its heart; (2) specify high-level interactive behaviours through visual programming; and (3) have the phone or watch act out such behaviours as the objects' "smarts". Furthermore, a series of prototypes show how Pineal exploits mobile sensing and output, and automatically generates 3D printed form-factors for rich, interactive, objects.

Reference

Talk

Abstract

Do-it-yourself (DIY) inspired activities have gained popularity as a means of creative expression and self-directed learning. However, DIY culture is difficult to implement in places with limited technology infrastructure and traditional learning cultures. Our goal is to understand how learners in such a setting react to DIY activities. We present observations from a physical computing workshop with 12 students (13-15 years old) conducted at a high school in India. We observed unique challenges for these students when tackling DIY activities: a high monetary and psychological cost to exploration, limited independent learning resources, difficulties with finding intellectual courage and assumed technical language proficiency. Our participants, however, overcome some of these challenges by adopting their own local strategies: resilience, nonverbal and verbal learning techniques, and creating documentation and fallback circuit versions. Based on our findings, we discuss a set of lessons learned about makerspaces in a context with socio-technical challenges.

Reference

Abstract

People are increasingly using mobile video to communicate, collaborate, and share experiences while on the go. Yet this presents challenges in adequately sharing camera views with remote users. In this paper, we study the use of semi-autonomous drones for video conferencing, where an outdoor user (using a smartphone) is connected to a desktop user who can explore the environment from the drone's perspective. We describe findings from a study where pairs collaborated to complete shared navigation and search tasks. We illustrate the benefits of providing the desktop user with a view that is elevated, manipulable, and decoupled from the outdoor user. In addition, we articulate how participants overcame challenges in communicating environmental information and navigational cues, negotiated control of the view, and used the drone as a tool for sharing experiences. This provides a new way of thinking about mobile video conferencing where cameras that are decoupled from both users play an integral role in communication, collaboration, and sharing experiences.

Reference

Abstract

We discuss touch-based navigation of 3D visualizations in a combined monoscopic and stereoscopic viewing environment. We identify a set of interaction modes, and a workflow that helps users transition between these modes to improve their interaction experience. In our discussion we analyze, in particular, the control-display space mapping between the different reference frames of the stereoscopic and monoscopic displays. We show how this mapping supports interactive data exploration, but may also lead to conflicts between the stereoscopic and monoscopic views due to users' movement in space; we resolve these problems through synchronization. To support our discussion, we present results from an exploratory observational evaluation with domain experts in fluid mechanics and structural biology. These experts explored domain-specific datasets using variations of a system that embodies the interaction modes and workflows; we report on their interactions and qualitative feedback on the system and its workflow.

Reference

Talk

Abstract

Makers participate in remixing culture by drawing inspiration from, combining, and adapting designs for physical objects. To examine how makers remix each others' designs on a community scale, we analyzed metadata from over 175,000 digital designs from Thingiverse, the largest online design community for digital fabrication. Remixed designs on Thingiverse are predominantly generated designs from Customizer a built-in web app for adjusting parametric designs. However, we find that these designs do not elicit subsequent user activity and the authors who generate them tend not to contribute additional content to Thingiverse. Outside of Customizer, influential sources of remixing include complex assemblies and design primitives, as well as non-physical resources posing as physical designs. Building on our findings, we discuss ways in which online maker communities could become more than just design repositories and better support collaborative remixing.

Reference

Talk

Abstract

Performative wearable technology has become a powerful medium for storytelling, advocacy, and social change. At the intersection of fashion and human-computer interaction (HCI), performative wearable technology explores how wearable technology can be utilized as an expressive tool, enabling designers to convey embodied and personal experiences. This thesis explores two distinct categories of performative wearables: (1) wearable empathy tools, designed to immerse the wearer in an experience of another, and (2) fashion-technology garments, which utilize performance and spectacle to engage audiences with social issues visually. By examining these two approaches, this research aims to understand how performative contexts influence the design process of performative wearables addressing social issues and how contextual factors impact the final designs. To achieve this, I employ qualitative research methods to explore designers' design strategies to create empathy tools and fashion-technology runway garments. I focus on the use of Research through Design (RtD) methodologies, where I incorporate first-person autobiographical accounts, soma design, and artifact evaluation to document and critically reflect on the design and impact of two performative wearable projects: HACKLES.empathy}and HACKLES.runway. The first, an empathy tool, is designed to simulate and communicate the lived experience of anxiety that I, as a woman, feel when walking alone at night, aiming to foster a deeper understanding through embodied interaction. The second, a runway garment, translates the same narrative into a spectacle-driven fashion-technology piece intended for audience engagement and public discourse on safety and gendered experiences in urban spaces. Through comparative analysis, my research shows that the design process for performative wearables is heavily affected by and informed by how the garment will be experienced. In addition, first-person experiences embedded in the design of empathy tools and fashion-technology garments lead to rich expressions of social issues. Personal experiences are crucial in these designs, and designers who are not inspired by personal experiences should collaborate with individuals who have. This thesis contributes to ongoing discourse in HCI, design research, and interactive fashion by synthesizing insights from fashion activism, embodied interaction, and first-person HCI design. The work proposes a design framework for performative wearables that helps designers navigate the trade-offs between individual experience and visual communication in wearable technology for social awareness. Ultimately, this research highlights the potential of wearable technologies to transcend their functional utility, instead positioning performative wearables as a critical medium for storytelling, activism, and empathy-building.

Reference

Abstract

Materials have long served as a cornerstone of human creativity and the evolution of our built environment. However, the limited diversity of available materials and the increasing depletion of finite resources, exacerbated by fragmented modeling, analysis, and production processes, highlight the urgent need for integrated and selective methodologies. The emergence of computational media and advanced materials introduces new possibilities, enabling architects to interactively and efficiently conceive, evaluate, and produce well-performing products and processes. This thesis introduces PASSAGE, a comprehensive computational workflow designed to digitally configure, select, and produce material compositions for architectural applications based on customized performance criteria. At the center of PASSAGE lies an interactive workflow that automates the configuration and production of architectural objects through the strategic manipulation of material compositions. This workflow supports designers by an interactive interface, allowing for generation of performance datasets to meet specific user requirements in near-real time. The research presented in this thesis encompasses the design, development, and prototyping of the workflow using existing commercial software and fabrication technologies. The work includes the parametric and simulation modeling of geometry, material properties, and fabrication processes, all of which interact dynamically with a database, the designer, and fabrication machinery to ensure comprehensive control over the design, analysis, and production of architectural object models. The effectiveness of PASSAGE is demonstrated through the development of a visual programing code and a case study focused on the creation of a freeform-heterogeneous chair. This case study, realized through the workflow and a Grasshopper code, serves to evaluate and verify the integrated process from design to production using a 3D printer. The dual goals of this research are to empower architects with an interactive, efficient, and scalable computational workflow and to establish a holistic, integrated approach to material composition of architectural products.

Reference

Abstract

This dissertation explores the intersection of Drama, virtual reality (VR), and Human-Computer Interaction (HCI), examining their parallels and collective impact on shaping immersive digital experiences. Throughout our research, we ask: how can VR support traditional performance practices? And how can we leverage existing performance practices to support wholistic user engagement? From our interdisciplinary position we identify gaps in existing research and offer novel solutions and innovative frameworks for navigating and structuring experiences within VR. In pursuit of answers to our research questions, we contribute C1) Performance-Based Multimodal Methodological Approaches for HCI, and C2) A Demonstration of Interdisciplinary Possibilities to the landscape of HCI VR research. We propose a symbiotic relationship between the user's cognitive engagement and the digital environment's architectural and interaction design can be enhanced through the application of practices and principles from Drama and performance. The practical application of these theoretical constructs is showcased in C3) Thresholding Protocols for Digital State Change, and C4) The Virtual Rehearsal Suite (VRS), an immersive VR environment that supports solo performance training, demonstrating how Drama and performance methodologies can enhance the user's experience, offering tested perspectives and techniques that promote interaction, presence, and embodiment. Starting with our related works, we identify our interdisciplinary position with a foundation that draws from both the academic and artistic communities interested in VR as a domain, research, and performance tool. We then cast a wide net to understand the dimensions of virtual technologies and their impact on user experience with subsequent chapters investigating the layers of reality, immersion, embodiment, performance rituals, and thresholding concepts. Each chapter contributes to the identification of gaps and parallels across research domains and the discussion of how Drama and performance can elevate the understanding and advancement of VR and HCI systems. The dissertation concludes that the confluence of Drama and performance practice with Interaction Design holds the potential to shape the future aesthetics and experiential facets of virtual environments. The embrace of VR as both a tool and a medium for creative expression is positioned as a transformative leap forward in both HCI and Drama, heralding a new era of digital interaction that needs to embrace the full spectrum of human experience for success and longevity. Our work positions actors as interaction specialists, capable of existing in multiple realities at once and providing insightful reflections on their experiences in iterative processes. The VRS study demonstrates this ability, while emphasizing that virtual environments are not merely technological constructs but complex experiential spaces where the physical and digital converge, challenging traditional perceptions of reality. It highlights the importance of centralizing the user as key to creating compelling virtual experiences that con only be achieved through meticulously designed interactions that resonate with the user's sensory and cognitive faculties. Our study underscores the efficacy of VR in supporting actor training with minimal digital interventions, facilitating a seamless transition into and out of VR, enhancing focus during VR engagement, and addressing issues such as VR sickness. It highlights the centralization of the human element as the pivotal factor in VR creation, emphasizing that VR environments should cater to the nuanced spectrum of human emotions, behaviours, and social interactions.

Reference

Abstract

Autonomous Vehicle (AV) technology promises to revolutionize human life. The promise of AVs includes reduced highway congestion, more efficient energy usage, and cheaper goods and services. However, without careful design, removing human drivers from vehicles will eliminate the natural communication channels which enable pedestrians to navigate safely. This thesis aims to design, present, and study anthropomorphic interfaces for autonomous vehicles, with the objective of enabling AVs to communicate with pedestrians through non-verbal cues. Non-verbal human communication is vital in human relationships. People use non-verbal communication when speech is impractical, such as when interacting with vehicles. When looking into ways in which AVs can use non-verbal communication to interact with pedestrians, we were inspired by the prospect of using anthropomorphic interfaces. This concept is well explored in Human-Robot Interaction (HRI) but has not been investigated in the context of AVs. For this thesis, we explored the design of anthropomorphic interfaces for autonomous vehicles. First, we proposed three types of anthropomorphic interfaces for AVs: facial expressions, hand gestures, and humanoid torsos. We developed a design space for each category using sketches and a low-fi prototype. Then, to research the benefits and limitations of anthropomorphic AVs, we implemented our AV interfaces in a Virtual Reality (VR) environment and developed two testbeds to evaluate their feasibility and scalability. Finally, we conducted two studies using the two testbeds. We investigated the study results using immersive analytics alongside traditional methods and revealed that anthropomorphic AVs could be helpful in AV-pedestrian interaction when designed by specific guidelines. Since we studied anthropomorphic AVs in VR, we were interested in the possibilities of analyzing the data of our study in an immersive environment. We designed a VR prototype specifically to analyze the data collected from the anthropomorphic AV study. The prototype provided basic immersive analytics features for the AV study data. We conducted an expert session with two domain experts to evaluate our immersive analytics prototype. The study contributed insights into the opportunities and challenges of utilizing immersive analytics to analyze AV studies.

Reference

Abstract

I propose a persona framework and five guidelines for designing interactive technology tools that help self-teaching, hobbyist musicians independently develop musical understanding by leveraging and augmenting their existing musical behaviour. Using perspectives from informal music education, self-directed learning, and the serious leisure perspective, I define 'self-teaching, hobbyist musicians' as self-directed, experiential learners who build musical understanding and skills in opportunistic, constructionist ways, primarily for leisure. However, self-directed music learning inevitably produces gaps in music theory understanding, affecting musical literacy and perception, which can overall increase the difficulty of picking up new musical knowledge and skills over time. Previous HCI-Music research focuses on motor skill acquisition or augmenting expressivity, while theory-teaching systems neglect practical musical behaviours, learning context, and practices in music pedagogy. To better inform this domain, I interviewed eight musicians to learn about their resources, strategies, and challenges in their idiosyncratic learning processes of music and music theory. Given my findings, this thesis contributes (1) a discussion of hobbyist musicians as technology users and a persona framework to guide design: "The user is a {musician} who learns {music theory competency} while doing {practical musical behaviour}"; and (2) guidelines for designing technology for self-teaching, hobbyist musicians. For each guideline, I offer and discuss speculative, concept sketches of example interactive tools. Finally, I discuss the limitations of the research presented in this thesis and opportunities for future work.

Reference

Abstract

In this thesis, I propose methods for repurposing existing hardware and software to enable designers to create live interactive prototypes for smart interactive objects without the need to write code or create custom circuitry. The advent of ubiquitous computing brought the promise of interactive artifacts that integrate into our everyday lives. While this has led to a myriad of “smart objects”, the problem is that it is difficult for interaction designers to devise interactive behaviours for such objects. For example, how might an interaction designer prototype behaviours for a smart speaker? How can they go beyond voice responses and, for instance, animate lights to show that the speaker is listening, or searching for an answer on the web? Designers today face three challenges: (1) needing multiple expertise of designing behaviour, form, circuitry, and programming the functionality; (2) lacking software tools to author fine-tuned dynamic behaviours; and (3) needing closer-to-product representations to physically manipulate the prototype. I overcome this gap through a method and two interactive systems. I propose a design metaphor: Soul–Body Prototyping, which suggests leveraging off-the-shelf mobile phones and watches to create smart object prototypes. By enclosing the mobile device (“soul”) into a physical enclosure (“body”), the designer can exploit the mobile device’s rich sensing, outputs, and internet connectivity. I then operationalize Soul–Body Prototyping through two proof-of-concept prototyping tools. Pineal features trigger-action behaviours which automatically generate 3D models for physical forms. These forms fit a mobile device and expose the necessary inputs and outputs. Astral is a tool where designers can mirror a portion of the desktop’s screen onto a mobile device, and create mappings that convert live mobile sensor data into mouse or keyboard events. Thus, the mobile device remote controls (and repurposes) familiar desktop applications for dynamic behaviour prototyping. Overall, my work contributes an alternative way to prototype smart interactive objects, which informs the design of future prototyping tools. Moreover, I investigate fundamental questions such as the meaning of interactive behaviour, as well as evaluation methods for prototyping tools and toolkits in HCI research.

Reference

Abstract

The tactile qualities of technology are often neglected by interaction designers who favour digital functionality, even though materiality plays a huge role in defining humans’ surroundings. Multiple benefits emerge from tactile interfaces and this thesis concentrates on three of them: 1. tactile interfaces can communicate visual, emotional, or sensorial information when visual and hearing senses are already overloaded with information; 2. they can guide users’ mobility through an environment, such as using hands to assist in walking through a dark room; and, 3. they can benefit computer-based applications for visually-impaired people. I focus on three themes – materiality, physical computing, and human touch behaviour – which I investigate through a collaborative project with four other design students. I conduct experime nts to explore three ways of using different tactile materials to create (computationally) interactive tactile interfaces (for example, Conductive Silicone, Touch-Sound Synthesizers, Reactive Chair). This led to two final installations, the first (“Tactile Room”) focusing on how to create tactile cues to guide the people through a non-visual interactive layer of the space, and the second (“Growth”) oriented to designing tactile cues to invite the people to interact with the interactive system. This Research through Design approach enables me to demonstrate how tactile interactivity enhances engagement with digital information. This research expands designers’ range of materials and fabrication techniques, including the elements of physical computing, to prototype new tactile interfaces.

Reference

Abstract

In the evolving world of virtual reality (VR), haptics plays an important role. Haptics enables individuals to feel and experience the virtual world and to immerse fully in the virtual environment. Haptics has a long history in which researchers have created various types of devices. Virtual reality brings in the need for highly portable and wearable devices which are limited in weight and grounding. Within the restrictive weight and size limitations of mobile and wearable devices limits the haptic rendering capabilities. This thesis attempts to overcome the limitations by developing a better understanding of the biology of touch via sensorimotor neuroscience to improve touch perception in virtual textures. In this research, we explore the receptor cells that encode touch information from mechanical stimuli and send them to the brain. Sensorimotor neuroscience explores the functional role of the different type of receptor cells in the human body. We use this biological perspective to create a classification of wearable haptic devices for the fingertip and the hand. The second part of the research explores the neuroscience concept of stochastic resonance that is proved to improve light touch sensation. We create a system comprising of hardware and software to evaluate the impact of stochastic resonance in a virtual texture discrimination task. I conclude this thesis by exploring the future directions of the research presented in this thesis and summarizing the two main contributions of this thesis: first is to develop a neuroscience-based classification of wearable haptic devices for the fingertip and the hand and second is to build a platform to evaluate the effect of mechanical SR in discriminating virtual textures.

Reference

Abstract

While improvised theatre (improv) is often performed on a bare stage, improvisers sometimes incorporate physical props and technical elements to inspire new directions for a scene and to enrich their performance. For improvising technical elements such as light and sound, a control booth is used in a theatre space. However, coordinating with improvisers’ action on-stage is challenging as there is a disconnect between the technicians and the performers. My goal is to inform the design of a technology augmented prop that brings the capabilities of the control booth directly to the hands of the performers as a tangible user interface. I interviewed five professional improvisers about their use of physical props and technical elements in improv, and their expectations of performer-controlled technology. I propose a set of guidelines for the design of a technology augmented prop that is integrated into the existing world of improvised theatre.

Reference

Abstract

In our data-rich society, it is increasingly important that all people are able to use and understand data. Large data sets commonly require expert knowledge to design and disseminate accessible and information-rich visualizations, resulting in visualizations that work well for experts but are less accessible for the general public. My research question addresses how we can support the use of visualization to increase data accessibility for the general public. One approach is to encourage people to self-author data representations suited to their own comprehension needs using simple techniques. However, providing data visualization authoring tools for the general public remains an ongoing challenge. My thesis explores the use of physical tools—specifically, rulers and block-printing stamps—as novel methods of authoring data visualizations, leveraging the advantages of ready-made visualization templates while providing freedom to personalize visual elements. To first explore the possibility of designing physical tools for authoring data visualizations, I present prototypes for several modifiable, computationally-fabricated ruler and stamp designs, created with the goal of allowing users to quickly create repeating visual elements when authoring visual elements on paper. From my design efforts, stamps show promise as low-effort, easy-to-create tools; I therefore conducted a workshop study to understand how people approach visualization authoring when given the ability to create their own physical template tools. In this study, participants authored visualizations on paper using hand-carved stamps made from potatoes and sponges. My results show that participants were able to author meaningful data visualizations from their self-created stamps, as well as several unique traits and uses of block-printing stamps. I conclude the thesis by discussing issues around expressivity and effectiveness of personalizing physical authoring tools, identify implications for the design and assembly of primitives in potential visualization authoring kits, and applications for physical authoring tools in the bigger scope of data democratization.

Reference

Abstract

Establishing strong patient-provider communication during clinical visits can have positive impacts on patient health outcomes. On the other hand, miscommunication between patients and healthcare providers can cause harm or in extreme cases cause death to patients. Factors such as healthcare providers' limited time, inefficient clinical environments, and patients' difficulties in communicating their concerns can be the causes of this miscommunication. In this thesis, I explore the design of visualizations to facilitate communication between healthcare providers and patients during clinical visits. In the first part (i) of this thesis, I present the results of a literature review I have conducted to expand our understanding of patients' and providers' communication challenges during in-clinic visits. In the second phase (ii) I discuss the results of interviews with healthcare providers and I contrast and compare patients' and providers' perspectives in the context of each other to unveil the roots of their communication challenges. Among the communication challenges we identified, I focus on exploring the challenges and the realities patients and healthcare providers face tracking and sharing patient-generated health data. In the third part of this dissertation (iii), I discuss the results of a series of interviews and focus groups with patients and healthcare providers I have conducted to gain a better understanding of patient-generated data communication challenges. I leverage this understanding to propose potential visualization designs representing patient-generated data collections to improve the process of reviewing and communicating these data between patients and healthcare providers. In the fourth part of this dissertation (iv), I discuss the results of the interviews with healthcare providers seeking their reflection on the proposed visualization designs. Finally, in collaboration with our healthcare provider team in Alberta Healthcare Services, I implement the prototypes of a number of carefully selected visualization designs. In the last part of this dissertation (v), I outline insights, lessons learned, and future research directions that arise from these studies and the design process. I hope this research provides more support for considering patients' and healthcare providers' individualities when designing technologies and visualizations in healthcare settings.

Reference

Abstract

Interactive garments enable new forms of communication between our bodies and with other people. In electronic fashion (eFashion) design, interactive garments on high fashion runways envision how people might use interactive technologies to enhance our clothing with new sensing and output capabilities. Researchers and fashion designers have since explored new interactive textiles that enable aesthetics-driven, interactive, and new material properties to explore on clothing. While there exist physical tools to implement interactive garments and software tools to create the visual aesthetic of a garment, these tools cannot yet enable designers to use new eFashion technologies in their garments because they require engineering expertise and specialized laboratory equipment. In this thesis, I explore the use of computer-aided prototyping tools to develop interactive eFashion garments. I present case studies with makers and two experienced eFashion designers about their design practices and formulate design guidelines for prototyping tools. I then present two prototyping tools for implementation and exploration of interactive garments. Finally, I discuss future work for physical and virtual prototyping tools in eFashion.

Reference

Abstract

I present an observational study of physical prototype critique that highlights some of the challenges of communicating physical behaviors and materiality at a distance. Geographically distributed open hardware communities often conduct user feedback and peer critique sessions via video conference. However, people have difficulty using current video conferencing tools to demonstrate and critique physical designs. To examine the challenges of remote critique, I conducted an observational lab study in which participants critiqued pairs of physical prototypes (prosthetic hands) for a face-to-face or remote collaborator. In both conditions, participants’ material experiences were an important part of their critique, however their attention was divided between interacting with the prototype and finding strategies to communicate ‘invisible’ features. Based on my findings, I propose design implications for remote collaboration tools that support the sharing of material experiences and prototype critique.

Reference

Abstract

The Maker Movement aims to democratize technological practices and promises many benefits for people including improved technical literacy, a means for self-expression and agency, and an opportunity to become more than consumers of technology. As part of the Maker Movement, people build hobbyist and utilitarian projects by themselves using programmable electronics (e.g., microcontroller, sensors, actuators) and software tools. While the Maker Movement is gaining momentum globally, some people are left out. Constraints such as material limitations, educational culture restrictions, and emotional or behavioral difficulties can often limit people from taking part in the Maker Movement. We refer to the systematic investigation of how diverse people respond to making-centered activities within constraints as an exploration of making within constraints.
In this dissertation, we (1) study how people respond to creating physical objects by themselves within constraints and, (2) investigate how to design technology that can help makers within constraints. We conducted an observational study in an impoverished school in India and identified the students' challenges and their strategies for making within material and educational culture constraints. We conducted a second study with at-promise youth in Canada and identified a set of lessons learned to engage youth within emotional and behavioral constraints in making-centered activities. Leveraging our observations, we proposed Augmented Reality (AR)-mediated prototyping as a way to address material constraints. AR-mediated prototyping can help makers to build, program, interact with and iterate on physical computing projects that combine both real-world and stand-in virtual electronic components. We designed, implemented, and evaluated a technology probe, Polymorphic Cube (PMC), as an instance of our vision. Our results show that PMC helped participants prototype despite missing I/O electronic components, and highlighted how AR-mediated prototyping extends to exploring project ideas, tinkering with implementation, and making with others.
Informed by our empirical and design explorations, we suggest a set of characteristics of constraints and implications for designing future technologies for makers within constraints. In the long-term, we hope that this research will inspire interaction designers to develop new tools that can help resolve constraints for making.

Reference