Abstract

We explore the potential for subtle on-hand gesture and microgesture interactions for map navigation with augmented reality (AR) devices. We describe a design exercise and follow-up elicitation study in which we identified on-hand gestures for cartographic interaction primitives. Microgestures and on-hand interactions are a promising space for AR map navigation as they offers always-available, tactile, and memorable spaces for interaction. Our findings show a clear set of microgesture interaction patterns that are well suited for supporting map navigation and manipulation. In particular, we highlight how the properties of various microgestures align with particular cartographic interaction tasks. We also describe our experience creating an exploratory proof-of-concept AR map prototype which helped us identify new opportunities and practical challenges for microgesture control. Finally, we discuss how future AR map systems could benefit from on-hand and microgesture input schemes.

Reference

Abstract

We examine input visualizations, visual representations that are designed to collect (and represent) new data rather than encode preexisting datasets. Information visualization is commonly used to reveal insights and stories within existing data. As a result, most contemporary visualization approaches assume existing datasets as the starting point for design, through which that data is mapped to visual encodings. Meanwhile, the implications of visualizations as inputs and as data sources have received little attention—despite the existence of visual and physical examples stretching back centuries. In this paper, we present a design space of 50 input visualizations analyzing their visual representation, data, artifact, context, and input. Based on this, we identify input modalities, purposes of input visualizations, and a set of design considerations. Finally, we discuss the relationship between input visualization and traditional visualization design and suggest opportunities for future research to better understand these visual representations and their potential.

Reference

Abstract

We explore the potential of generative AI text-to-image models to help designers efficiently craft unique, representative, and demographically diverse anthropographics that visualize data about people. Currently, creating data-driven iconic images to represent individuals in a dataset often requires considerable design effort. Generative text-to-image models can streamline the process of creating these images, but risk perpetuating designer biases in addition to stereotypes latent in the models. In response, we outline a conceptual workflow for crafting anthropographic assets for visualizations, highlighting possible sources of risk and bias as well as opportunities for reflection and refinement by a human designer. Using an implementation of this workflow with Stable Diffusion and Google Colab, we illustrate a variety of new anthropographic designs that showcase the visual expressiveness and scalability of these generative approaches. Based on our experiments, we also identify challenges and research opportunities for new AI-enabled anthropographic visualization tools.

Reference

Abstract

We introduce Augmented Math, a machine learning-based approach to authoring AR explorable explanations by augmenting static math textbooks without programming. To augment a static document, our system first extracts mathematical formulas and figures from a given document using optical character recognition (OCR) and computer vision. By binding and manipulating these extracted contents, the user can see the interactive animation overlaid onto the document through mobile AR interfaces. This empowers non-technical users, such as teachers or students, to transform existing math textbooks and handouts into on-demand and personalized explorable explanations. To design our system, we first analyzed existing explorable math explanations to identify common design strategies. Based on the findings, we developed a set of augmentation techniques that can be automatically generated based on the extracted content, which are 1) dynamic values, 2) interactive figures, 3) relationship highlights, 4) concrete examples, and 5) step-by-step hints. To evaluate our system, we conduct two user studies: preliminary user testing and expert interviews. The study results confirm that our system allows more engaging experiences for learning math concepts.

Reference

Abstract

Anthropographics are human-shaped visualizations that aim to emphasize the human importance of datasets and the people behind them. However, current anthropographics tend to employ homogeneous human shapes to encode data about diverse demographic groups. Such anthropographics can obscure important differences between groups and contemporary designs exemplify the lack of inclusive approaches for representing human diversity in visualizations. In response, we explore the creation of demographically diverse anthropographics that communicate the visible diversity of demographically distinct populations. Building on previous anthropographics research, we explore strategies for visualizing datasets about people in ways that explicitly encode diversity—illustrating these approaches with examples in a variety of visual styles. We also critically reflect on strategies for creating diverse anthropographics, identifying social and technical challenges that can result in harmful representations. Finally, we highlight a set of forward-looking research opportunities for advancing the design and understanding of diverse anthropographics.

Reference

Abstract

This paper introduces Teachable Reality, an augmented reality (AR) prototyping tool for creating interactive tangible AR applications with arbitrary everyday objects. Teachable Reality leverages vision-based interactive machine teaching (e.g., Teachable Machine), which captures real-world interactions for AR prototyping. It identifies the user-defined tangible and gestural interactions using an on-demand computer vision model. Based on this, the user can easily create functional AR prototypes without programming, enabled by a trigger-action authoring interface. Therefore, our approach allows the flexibility, customizability, and generalizability of tangible AR applications that can address the limitation of current marker-based approaches. We explore the design space and demonstrate various AR prototypes, which include tangible and deformable interfaces, context-aware assistants, and body-driven AR applications. The results of our user study and expert interviews confirm that our approach can lower the barrier to creating functional AR prototypes while also allowing flexible and general-purpose prototyping experiences.

Reference

Abstract

We contribute interaction techniques for augmenting mixed reality (MR) visualizations with smartphone proxies. By combining head-mounted displays (HMDs) with mobile touchscreens, we can augment low-resolution holographic 3D charts with precise touch input, haptics feedback, high-resolution 2D graphics, and physical manipulation. Our approach aims to complement both MR and physical visualizations. Most current MR visualizations suffer from unreliable tracking, low visual resolution, and imprecise input. Data physicalizations on the other hand, although allowing for natural physical manipulation, are limited in dynamic and interactive modification. We demonstrate how mobile devices such as smartphones or tablets can serve as physical proxies for MR data interactions, creating dynamic visualizations that support precise manipulation and rich input and output. We describe 6 interaction techniques that leverage the combined physicality, sensing, and output capabilities of HMDs and smartphones, and demonstrate those interactions via a prototype system. Based on an evaluation, we outline opportunities for combining the advantages of both MR and physical charts.

Reference

Abstract

We present RealityTalk, a system that augments real-time live presentations with speech-driven interactive virtual elements. Augmented presentations leverage embedded visuals and animation for engaging and expressive storytelling. However, existing tools for live presentations often lack interactivity and improvisation, while creating such effects in video editing tools require significant time and expertise. RealityTalk enables users to create live augmented presentations with real-time speech-driven interactions. The user can interactively prompt, move, and manipulate graphical elements through real-time speech and supporting modalities. Based on our analysis of 177 existing video-edited augmented presentations, we propose a novel set of interaction techniques and then incorporated them into RealityTalk. We evaluate our tool from a presenter’s perspective to demonstrate the effectiveness of our system.

Reference

Abstract

We present EvoIsland, a scalable interactive evolutionary user interface framework inspired by the spatially isolated land masses seen on Earth. Our generalizable interaction system encourages creators to spatially explore a wide range of design possibilities through the combination, separation, and rearrangement of hexagonal tiles on a grid. As these tiles are grouped into islandlike clusters, localized populations of designs form through an underlying evolutionary system. The interactions that take place within EvoIsland provide content creators with new ways to shape, display and assess populations in evolutionary systems that produce a wide range of solutions with visual phenotype outputs.

Reference

Abstract

We take the well-established use of physical scale models in architecture and identify new opportunities for using them to interactively visualize and examine multiple streams of geospatial data. Overlaying, comparing, or integrating visualizations of complementary data sets in the same physical space is often challenging given the constraints of various data types and the limited design space of possible visual encodings. Our vision of “simultaneous worlds” uses physical models as a substrate upon which visualizations of multiple data streams can be dynamically and concurrently integrated. To explore the potential of this concept, we created three design explorations that use an illuminated campus model to integrate visualizations about building energy use, climate, and movement paths on a university campus. We use a research through design approach, documenting how our interdisciplinary collaborations with domain experts, students, and architects informed our designs. Based on our observations, we characterize the benefits of models for 1) situating visualizations, 2) composing visualizations, and 3) manipulating and authoring visualizations. Our work highlights the potential of physical models to support embodied exploration of spatial and non-spatial visualizations through fluid interactions.

Reference

Abstract

We explore the design and utility of situated manual self-tracking visualizations on dedicated displays that integrate data tracking into existing practices and physical environments. Situating self-tracking tools in relevant locations is a promising approach to enable reflection on and awareness of data without needing to rely on sensorized tracking or personal devices. In both a long-term autobiographical design process and a co-design study with six participants, we rapidly prototyped and deployed 30 situated self-tracking applications over a ten month period. Grounded in the experience of designing and living with these trackers, we contribute findings on logging and data entry, the use of situated displays, and the visual design and customization of trackers. Our results demonstrate the potential of customizable dedicated self-tracking visualizations that are situated in relevant physical spaces, and suggest future research opportunities and new potential applications for situated visualizations.

Reference

Abstract

We explore the use of cinematic “pre-visualization” (previs) techniques as a rapid ideation and design futuring method for human computer interaction (HCI) research. Previs approaches, which are widely used in animation and film production, use digital design tools to create medium-fidelity videos that capture richer interaction, motion, and context than sketches or static illustrations. When used as a design futuring method, previs can facilitate rapid, iterative discussions that reveal tensions, challenges, and opportunities for new research. We performed eight one-week design futuring sprints, in which individual HCI researchers collaborated with a lead designer to produce concept sketches, storyboards, and videos that examined future applications of their research. From these experiences, we identify recurring themes and challenges and present a One Week Futuring Workbook that other researchers can use to guide their own futuring sprints. We also highlight how variations of our approach could support other speculative design practices.

Reference

Abstract

We explore how the lens of fictional superpowers can help characterize how visualizations empower people and provide inspiration for new visualization systems. Researchers and practitioners often tout visualizations' ability to “make the invisible visible” and to “enhance cognitive abilities.” Meanwhile superhero comics and other modern fiction often depict characters with similarly fantastic abilities that allow them to see and interpret the world in ways that transcend traditional human perception. We investigate the intersection of these domains, and show how the language of superpowers can be used to characterize existing visualization systems and suggest opportunities for new and empowering ones. We introduce two frameworks: The first characterizes seven underlying mechanisms that form the basis for a variety of visual superpowers portrayed in fiction. The second identifies seven ways in which visualization tools and interfaces can instill a sense of empowerment in the people who use them. Building on these observations, we illustrate a diverse set of “visualization superpowers” and highlight opportunities for the visualization community to create new systems and interactions that empower new experiences with data. Material and illustrations are available under CC-BY 4.0 at osf.io/8yhfz.

Reference

Abstract

We present I/O Bits, a prototype personal informatics system that explores the potential for user-driven and situated self-tracking. With simple tactile inputs and small e-paper visualizations, I/O Bits are dedicated physical devices that allow individuals to track and visualize different kinds of personal activities in-situ. This is in contrast to most self-tracking systems, which automate data collection, centralize information displays, or integrate into multi-purpose devices like smartwatches or mobile phones. We report findings from an e-paper visualization workshop and a prototype deployment where participants constructed their own I/O Bits and used them to track a range of personal data. Based on these experiences, we contribute insights and opportunities for situated and user-driven personal informatics.

Reference

Talk

Abstract

Worlds-in-Miniature (WiMs) are interactive worlds within a world and combine the advantages of an input space, a cartographicmap, and an overview+detail interface. They have been used across the extended virtuality spectrum for a variety of applications.Building on an analysis of examples of WiMs from the research literature we contribute a design space for WiMs based on sevendesign dimensions. Further, we expand upon existing definitions of WiMs to provide a definition that applies across the extendedreality spectrum. We identify the design dimensions of size-scope-scale, abstraction, geometry, reference frame, links, multiples, andvirtuality. Using our framework we describe existing Worlds-in-Miniature from the research literature and reveal unexplored researchareas. Finally, we generate new examples of WiMs using our framework to fill some of these gaps. With our findings, we identifyopportunities that can guide future research into WiMs.

Reference

Abstract

Immersive Analytics is a quickly evolving field that unites several areas such as visualisation, immersive environments, and human-computer interaction to support human data analysis with emerging technologies. This research has thrived over the past years with multiple workshops, seminars, and a growing body of publications, spanning several conferences. Given the rapid advancement of interaction technologies and novel application domains, this paper aims toward a broader research agenda to enable widespread adoption. We present 17 key research challenges developed over multiple sessions by a diverse group of 24 international experts, initiated from a virtual scientific workshop at ACM CHI 2020. These challenges aim to coordinate future work by providing a systematic roadmap of current directions and impending hurdles to facilitate productive and effective applications for Immersive Analytics.

Reference

Abstract

We introduce three novel data visualizations, called perspective charts, based on the concept of size constancy in linear perspective projection. Bar charts are a popular and commonly used tool for the interpretation of datasets, however, representing datasets with multi-scale variation is challenging in a bar chart due to limitations in viewing space. Each of our designs focuses on the static representation of datasets with large ranges with respect to important variations in the data. Through a user study, we measure the effectiveness of our designs for representing these datasets in comparison to traditional methods, such as a standard bar chart or a broken-axis bar chart, and state-of-the-art methods, such as a scale-stack bar chart. The evaluation reveals that our designs allow pieces of data to be visually compared at a level of accuracy similar to traditional visualizations. Our designs demonstrate advantages when compared to state-of-the-art visualizations designed to represent datasets with large outliers.

Reference

Abstract

We present the SmartCampus visualization tool, representing spatiotemporal data of over 200 student pathways and restpoints on a university campus. Based on our experiences with SmartCampus, we also propose a task-based framework that de-scribes how practicing urban designers (specifically, architects) can use human movement data visualizations in their work. Although extensive amounts of location data are produced daily by smartphones, existing geospatial tools are not customized to specifically support high-level urban design tasks. To help identify opportunities in urban design for visualizing human movement data from devices such as smartphones, we used our SmartCampus prototype to facilitate a series of 3 participatory design sessions (3 participants), a targeted online survey (14 participants), and semi-structured interviews (6 participants) with architectural experts. Our findings showcase the need for location analysis tools tailored to concrete urban design practices, and also highlight opportunities for Smart City researchers interested in developing domain specific, visualization tools.

Reference

Abstract

We compare physical and virtual reality (VR) versions of simple data visualizations and explore how the addition of virtual annotation and filtering tools affects how viewers solve basic data analysis tasks. We report on two studies, inspired by previous examinations of data physicalizations. The first study examines differences in how viewers interact with physical hand-scale, virtual hand-scale, and virtual table-scale visualizations and the impact that the different forms had on viewer’s problem solving behavior. A second study examines how interactive annotation and filtering tools might support new modes of use that transcend the limitations of physical representations. Our results highlight challenges associated with virtual reality representations and hint at the potential of interactive annotation and filtering tools in VR visualizations.

Reference

Abstract

We describe a design space of view manipulation interactions for small data-driven contextual visualizations (word-scale visualizations). These interaction techniques support an active reading experience and engage readers through exploration of embedded visualizations whose placement and content connect them to specific terms in a document. A reader could, for example, use our proposed interaction techniques to explore word-scale visualizations of stock market trends for companies listed in a market overview article. When readers wish to engage more deeply with the data, they can collect, arrange, compare, and navigate the document using the embedded word-scale visualizations, permitting more visualization-centric analyses. We support our design space with a concrete implementation, illustrate it with examples from three application domains, and report results from two experiments. The experiments show how view manipulation interactions helped readers examine embedded visualizations more quickly and with less scrolling and yielded qualitative feedback on usability and future opportunities.

Reference

Abstract

Complex data visualization design projects often entail collaboration between people with different visualization-related skills. For example, many teams include both designers who create new visualization designs and developers who implement the resulting visualization software. We identify gaps between data characterization tools, visualization design tools, and development platforms that pose challenges for designer-developer teams working to create new data visualizations. While it is common for commercial interaction design tools to support collaboration between designers and developers, creating data visualizations poses several unique challenges that are not supported by current tools. In particular, visualization designers must characterize and build an understanding of the underlying data, then specify layouts, data encodings, and other data-driven parameters that will be robust across many different data values. In larger teams, designers must also clearly communicate these mappings and their dependencies to developers, clients, and other collaborators. We report observations and reflections from five large multidisciplinary visualization design projects and highlight six data-specific visualization challenges for design specification and handoff. These challenges include adapting to changing data, anticipating edge cases in data, understanding technical challenges, articulating data-dependent interactions, communicating data mappings, and preserving the integrity of data mappings across iterations. Based on these observations, we identify opportunities for future tools for prototyping, testing, and communicating data-driven designs, which might contribute to more successful and collaborative data visualization design.

Reference

Talk

Abstract

We report on findings from seven design workshops that used ideation and sketching activities to prototype new situated visualizations - representations of data that are displayed in proximity to the physical referents (such as people, objects, and locations) to which the data is related. Designing situated visualizations requires a fine-grained understanding of the context in which the visualizations are placed, as well as an exploration of different options for placement and form factors, which existing methods for visualization design do not account for. Focusing on small displays as a target platform, we reflect on our experiences of using a diverse range of sketching activities, materials, and prompts. Based on these observations, we identify challenges and opportunities for sketching and ideating situated visualizations. We also outline the space of design activities for situated visualization and highlight promising methods for both designers and researchers.

Reference

Abstract

We investigate how people discover the functionality of an interactive visualization that was designed for the general public. While interactive visualizations are increasingly available for public use, we still know little about how the general public discovers what they can do with these visualizations and what interactions are available. Developing a better understanding of this discovery process can help inform the design of visualizations for the general public, which in turn can help make data more accessible. To unpack this problem, we conducted a lab study in which participants were free to use their own methods to discover the functionality of a connected set of interactive visualizations of public energy data. We collected eye movement data and interaction logs as well as video and audio recordings. By analyzing this combined data, we extract exploration strategies that the participants employed to discover the functionality in these interactive visualizations. These exploration strategies illuminate possible design directions for improving the discoverability of a visualization's functionality.

Reference

Abstract

We introduce three lightweight interactive camera control techniques for 3D terrain maps on touch devices based on a look-from metaphor (Discrete Look-From-At, Continuous Look-From-Forwards, and Continuous Look-From-Towards). These techniques complement traditional touch screen pan, zoom, rotate, and pitch controls allowing viewers to quickly transition between top-down, oblique, and ground-level views. We present the results of a study in which we asked participants to perform elevation comparison and line-of-sight determination tasks using each technique. Our results highlight how look-from techniques can be integrated on top of current direct manipulation navigation approaches by combining several direct manipulation operations into a single look-from operation. Additionally, they show how look-from techniques help viewers complete a variety of common and challenging map-based tasks.

Reference

Abstract

Freehand gesture interaction has long been proposed as a `natural' input method for Augmented Reality (AR) applications, yet has been little explored for intensive applications like multiscale navigation. In multiscale navigation, such as digital map navigation, pan and zoom are the predominant interactions. A position-based input mapping (e.g. grabbing metaphor) is intuitive for such interactions, but is prone to arm fatigue. This work focuses on improving digital map navigation in AR with mid-air hand gestures, using a horizontal intangible map display. First, we conducted a user study to explore the effects of handedness (unimanual and bimanual) and input mapping (position-based and rate-based). From these findings we designed DiveZoom and TerraceZoom, two novel hybrid techniques that smoothly transition between position- and rate-based mappings. A second user study evaluated these designs. Our results indicate that the introduced input-mapping transitions can reduce perceived arm fatigue with limited impact on performance.

Reference

Talk

Abstract

We examine the potential for immersive unit visualizations—interactive virtual environments populated with objects representing individual items in a dataset. Our virtual reality prototype highlights how immersive unit visualizations can allow viewers to examine data at multiple scales, support immersive exploration, and create affective personal experiences with data.

Reference

Abstract

We compare the effectiveness of 2D maps and 3D terrain models for visibility tasks and demonstrate how interactive dynamic viewsheds can improve performance for both types of terrain representations. In general, the two-dimensional nature of classic topographic maps limits their legibility and can make complex yet typical cartographic tasks like determining the visibility between locations difficult. Both 3D physical models and interactive techniques like dynamic viewsheds have the potential to improve viewers' understanding of topography, but their impact has not been deeply explored. We evaluate the effectiveness of 2D maps, 3D models, and interactive viewsheds for both simple and complex visibility tasks. Our results demonstrate the benefits of the dynamic viewshed technique and highlight opportunities for additional tactile interactions. Based on these findings we present guidelines for improving the design and usability of future topographic maps and models.

Reference

Talk

Abstract

In this paper we consider various genres of citizen science from the perspective of citizen participants. As a mode of scientific inquiry, citizen science has the potential to "scale up" scientific data collection efforts and increase lay engagement with science. However, current technological directions risk losing sight of the ways in which citizen science is actually practiced. As citizen science is increasingly used to describe a wide range of activities, we begin by presenting a framework of citizen science genres. We then present findings from four interlocking qualitative studies and technological interventions of community air quality monitoring efforts, examining the motivations and capacities of citizen participants and characterizing their alignment with different types of citizen science. Based on these studies, we suggest that data acquisition involves complex multi-dimensional tradeoffs, and the commonly held view that citizen science systems are a win-win for citizens and science may be overstated.

Reference

Abstract

In this paper we analyze the role of physical scale models in the architectural design process and apply insights from architecture for the creation and use of data physicalizations. Based on a survey of the architecture literature on model making and ten interviews with practicing architects, we describe the role of physical models as a tool for exploration and communication. From these observations, we identify trends in the use of physical models in architecture, which have the potential to inform the design of data physicalizations. We identify four functions of architectural modeling that can be directly adapted for use in the process of building rich data models. Finally, we discuss how the visualization community can apply observations from architecture to the design of new data physicalizations.

Reference

Abstract

We contribute an investigation of the design and function of word-scale graphics and visualizations embedded in text documents. Word-scale graphics include both data-driven representations such as word-scale visualizations and sparklines, and non-data-driven visual marks. Their design, function, and use has so far received little research attention. We present the results of an open ended exploratory study with nine graphic designers. The study resulted in a rich collection of different types of graphics, data provenance, and relationships between text, graphics, and data. Based on this corpus, we present a systematic overview of word-scale graphic designs, and examine how designers used them. We also discuss the designers' goals in creating their graphics, and characterize how they used word-scale graphics to visualize data, add emphasis, and create alternative narratives. Building on these examples, we discuss implications for the design of authoring tools for word-scale graphics and visualizations, and explore how new authoring environments could make it easier for designers to integrate them into documents.

Reference

Abstract

We introduce embedded data representations, the use of visual and physical representations of data that are deeply integrated with the physical spaces, objects, and entities to which the data refers. Technologies like lightweight wireless displays, mixed reality hardware, and autonomous vehicles are making it increasingly easier to display data in-context. While researchers and artists have already begun to create embedded data representations, the benefits, trade-offs, and even the language necessary to describe and compare these approaches remain unexplored. In this paper, we formalize the notion of physical data referents – the real-world entities and spaces to which data corresponds – and examine the relationship between referents and the visual and physical representations of their data. We differentiate situated representations, which display data in proximity to data referents, and embedded representations, which display data so that it spatially coincides with data referents. Drawing on examples from visualization, ubiquitous computing, and art, we explore the role of spatial indirection, scale, and interaction for embedded representations. We also examine the tradeoffs between non-situated, situated, and embedded data displays, including both visualizations and physicalizations. Based on our observations, we identify a variety of design challenges for embedded data representation, and suggest opportunities for future research and applications.

Reference

Talk

Abstract

Makers participate in remixing culture by drawing inspiration from, combining, and adapting designs for physical objects. To examine how makers remix each others' designs on a community scale, we analyzed metadata from over 175,000 digital designs from Thingiverse, the largest online design community for digital fabrication. Remixed designs on Thingiverse are predominantly generated designs from Customizer a built-in web app for adjusting parametric designs. However, we find that these designs do not elicit subsequent user activity and the authors who generate them tend not to contribute additional content to Thingiverse. Outside of Customizer, influential sources of remixing include complex assemblies and design primitives, as well as non-physical resources posing as physical designs. Building on our findings, we discuss ways in which online maker communities could become more than just design repositories and better support collaborative remixing.

Reference

Talk

Abstract

We explore interactive relief shearing, a set of non-intrusive, direct manipulation interactions that expose depth and shape information in terrain maps using ephemeral animations. Reading and interpreting topography and relief on terrain maps is an important aspect of map use, but extracting depth information from 2D maps is notoriously difficult. Modern mapping software attempts to alleviate this limitation by presenting digital terrain using 3D views. However, 3D views introduce occlusion, complicate distance estimations, and typically require more complex interactions. In contrast, our approach reveals depth information via shearing animations on 2D maps, and can be paired with existing interactions such as pan and zoom. We examine explicit, integrated, and hybrid interactions for triggering relief shearing and present a version that uses device tilt to control depth effects. Our evaluation shows that these interactive techniques improve depth perception when compared to standard 2D and perspective views.

Reference

Abstract

This thesis explores novel uses of physicality and cross-device interaction with augmented reality through three different research prototypes. (1) Augmented Math leverages the physical world as a basis for generating augmentations through an AI-assisted method that allows educators to create interactive math content from static diagrams. (2) HoloTouch enables users to physically interact with holograms by repurposing personal devices like smartphones as tangible input tools. (3) HoloDevice explores how AR can transform remote collaboration by simulating co-located interactions through holographic representations of users and their devices. These systems were developed through iterative prototyping, collaborative design discussions, user studies, and interviews. Together, they demonstrate how novel interaction techniques grounded in physical and cross-device affordances can enhance usability and engagement in AR environments.

Reference

Abstract

While technology plays a vital role in human communication, there still remain many significant challenges when using them in everyday life. Modern computing technologies, such as smartphones, offer convenient and swift access to information, facilitating tasks like reading documents or communicating with friends. However, these tools frequently lack adaptability, become distracting, consume excessive time, and impede interactions with people and contextual information. Furthermore, they often require numerous steps and significant time investment to gather pertinent information. We want to explore an efficient process of contextual information gathering for mixed reality (MR) interfaces that provide information directly in the user’s view. This approach allows for a seamless and flexible transition between language and subsequent contextual references, without disrupting the flow of communication. ’Augmented Language’ can be defined as the integration of language and communication with mixed reality to enhance, transform, or manipulate language-related aspects and various forms of linguistic augmentations (such as annotation/referencing, aiding social interactions, translation, localization, etc.). In this thesis, our broad objective is to explore mixed reality interfaces and their potential to enhance augmented language, particularly in the domains of speech and text. Our aim is to create interfaces that offer a more natural, generalizable, on-demand, and real-time experience of accessing contextually relevant information and providing adaptive interactions. To better address this broader objective, we systematically break it down to focus on two instances of augmented language. First, enhancing augmented conversation to support on-the-fly, co-located in-person conversations using embedded references. And second, enhancing digital and physical documents using MR to provide on-demand reading support in the form of different summarization techniques. To examine the effectiveness of these speech and text interfaces, we conducted two studies in which we asked the participants to evaluate our system prototype in different use cases. The exploratory usability study for the first exploration confirms that our system decreases distraction and friction in conversation compared to smartphone search while providing highly useful and relevant information. For the second project, we conducted an exploratory design workshop to identify categories of document enhancements. We later conducted a user study with a mixed-reality prototype to highlight five board themes to discuss the benefits of MR document enhancement.

Reference

Abstract

In this thesis, we explore the design of demographically diverse anthropographics from demographic data. Anthropographics are visualizations that often use generic human-shaped symbols as abstract representations of humans. These visualizations are frequently used to convey the human importance of data related to people’s experiences, usually focusing on demographic data such as age, gender, race among others. However, most current anthropographics employ generic human shapes to represent data about distinct demographic groups, which can hide important demographic and physical differences between these groups. The use of generic human shapes in current anthropographics highlights the lack of inclusive approaches for representing human physical diversity in data visualizations. In response,we explore the creation of demographically diverse anthropographics that communicate the visible physical diversity of demographically-distinct populations. Our contributions stem from a set of critical design explorations for visualizing demographic data with a focus on representing human physical diversity and a study exploring how viewers perceive visual representations of diversity in anthropographics. We make three contributions in this work. First, we describe critical design explorations from two prototypes for representing racial demographic data as physical characteristics of diversity (such as skin tones) in diverse anthropographics. Second, we explore how viewers may perceive visual representations of demographic diversity in anthropographics through an interview study on contemporary examples of homogeneous anthropographics from popular news media and our own set of diverse anthropographics. Finally, we identify a set of social and technical challenges in the creation of anthropographics and contribute a collection of forward-looking opportunities for advancing this line of research on equitable visual representations of diversity through demographically diverse anthropographics.

Reference

Abstract

Autonomous Vehicle (AV) technology promises to revolutionize human life. The promise of AVs includes reduced highway congestion, more efficient energy usage, and cheaper goods and services. However, without careful design, removing human drivers from vehicles will eliminate the natural communication channels which enable pedestrians to navigate safely. This thesis aims to design, present, and study anthropomorphic interfaces for autonomous vehicles, with the objective of enabling AVs to communicate with pedestrians through non-verbal cues. Non-verbal human communication is vital in human relationships. People use non-verbal communication when speech is impractical, such as when interacting with vehicles. When looking into ways in which AVs can use non-verbal communication to interact with pedestrians, we were inspired by the prospect of using anthropomorphic interfaces. This concept is well explored in Human-Robot Interaction (HRI) but has not been investigated in the context of AVs. For this thesis, we explored the design of anthropomorphic interfaces for autonomous vehicles. First, we proposed three types of anthropomorphic interfaces for AVs: facial expressions, hand gestures, and humanoid torsos. We developed a design space for each category using sketches and a low-fi prototype. Then, to research the benefits and limitations of anthropomorphic AVs, we implemented our AV interfaces in a Virtual Reality (VR) environment and developed two testbeds to evaluate their feasibility and scalability. Finally, we conducted two studies using the two testbeds. We investigated the study results using immersive analytics alongside traditional methods and revealed that anthropomorphic AVs could be helpful in AV-pedestrian interaction when designed by specific guidelines. Since we studied anthropomorphic AVs in VR, we were interested in the possibilities of analyzing the data of our study in an immersive environment. We designed a VR prototype specifically to analyze the data collected from the anthropomorphic AV study. The prototype provided basic immersive analytics features for the AV study data. We conducted an expert session with two domain experts to evaluate our immersive analytics prototype. The study contributed insights into the opportunities and challenges of utilizing immersive analytics to analyze AV studies.

Reference

Abstract

Our work seeks to augment new information visualization research with strategies and workflows from the fields of design and architecture. To this end, this research explores how to adopt tools and methods that can integrate the best of physical and digital modalities to multiple contexts and scales in HCI and data visualization. Designing information visualization systems creates a need for a design approach that addresses and ties together two main threads – 1) how we as humans interact with and make sense of our environment and 2) how we as designers create meaning through geometry, form, and material encodings. While the research community within data visualization has primarily focused on screen-based data visualizations, there is now an opportunity to study how we can create insight with hybrid physical and digital representations of data through the lens of architectural practice. My colleagues and I have conducted this research at the intersection of model building, diagrams, and generative design, applying this knowledge to the design of multifaceted digital environments, from micro to macro scale, in two- and three- dimensional worlds. To develop this research, we first observe and characterize the architectural methods of model making and their potential to facilitate the design process of interactive systems. Next, we describe how physical hand-crafted and digitally fabricated models of different types assist in various stages of the design process. To illustrate how model building could support fluid exploration of multiple data sets, we built a 3D interactive campus model visualizing multiple layers of building-specific data. The system uses physical models as tangible tokens on an interactive touch surface, visualizing energy use and weather data daily over a two-year period. As an extension of our design, we developed a conceptual framework from this project to highlight the potential of physical models for supporting embodied exploration of spatial and non-spatial visualizations through fluid interaction. We then examine the use of diagrams in architecture and develop a conceptual framework based on the concept of data tectonics to organize and structure the design process of physical and immersive data systems. To further study the use of diagrams and generative design for data visualization, I collaborated with researchers at Tableau Software to develop a patented Tableau extension that self-generates and evolves up to thirty different design permutations at a time. The system randomly assigns a pre-specified palette of mark types to a chosen dataset giving designers the option of adding or deleting options that they deem promising. As a final project for this research, we brought the three principles of model making, diagramming, and generative design together to create a large-scale physical and immersive data visualization. In collaboration with the Department of Social Work at the University of Calgary, the project uses diagrams and generative design to prototype a series of three-dimensional encodings visualizing Global Gender Gap statistics from the World Economic Forum. The tent-like forms evoke sheltering structures that can be registered, experienced, and measured with the whole body. For this project, we applied the diagrammatic approach used in parametric design to traditional information visualization design principles and identified workflows that support rapid exploration and fabrication of multiple data design alternatives. There is no doubt that data and digital technologies, including machine learning and AI, will be part of our human fabric in the future, but what that looks like and how it is structured is still up to us. We need artists, and more diversity in general, in order to do this to the best of our potential as humans. In determining which practices encourage the creation of rich data-driven environments, this research underscores the fundamental need of humans to make sense of the world, inspiring designers to develop new spatial constructs that integrate both the art and science of the built environment.

Reference

Abstract

In this thesis, we examine the intersection between personal informatics and situated visualization. Personal Informatics systems aim to help people collect and utilize their own data. Situated visualizations aim to decentralize data consumption and support people in making data-driven decisions in-situ. We present I/O Bits, a prototype personal informatics system that explores the potential for situated self-tracking. With simple tactile inputs and small e-paper visualizations, I/O Bits are dedicated physical devices that allow individuals to track and visualize different kinds of personal activities in-situ. This is in contrast to most self-tracking systems, which automate data collection, centralize information displays, or integrate into multi-purpose devices like smartwatches or mobile phones. Our contributions stem from a set of situated ideation workshops, an e-paper visualization workshop, the development of I/O Bits, and a prototype deployment where participants constructed their own I/O Bits and used them to track a range of personal data. We make three contributions with this work. First, we report on methodologies from seven design workshops that used ideation and sketching activities to prototype new situated visualizations. Based on our diverse set of workshops, we identify challenges and opportunities for sketching and ideating situated visualizations and highlight promising methods for both designers and researchers. Second, we use our design workshop results to design our novel situated self-tracking system, I/O Bits. We discuss the tensions experienced during our iterative design and development process and explore the design space of small situated visualizations on e-paper displays. Finally, we examine our findings from the situated ideation workshops, e-paper visualization workshop, development process, and prototype deployment. Using sketches, photos, hardware, audio recordings, and transcripts, we distill a set of insights and opportunities for future research on situated self-tracking.

Reference

Abstract

We present an exploration of two classes of navigation techniques designed for representations of real-world terrain. The first introduces look-from camera controls, a new style of camera control for touch devices designed with representations of real world-terrain in mind and provides an evaluation of three different implementations of this style of control. The second looks to virtual reality and compares the effectiveness of four existing and common camera control techniques within the context of a representations of real world-terrain. Effective camera controls greatly increase a user’s ability to engage with a virtual environment, and virtual map environments are no different. However, current camera controls are difficult to use within map-like environments, requiring burdensome sequences of interactions or performing poorly within ragged terrain. To examine the effectiveness of different camera controls in this space we conducted two studies in which we asked participants to perform map reading and interaction tasks. In both studies the camera control technique greatly influenced participant engagement and enjoyment within a scene. The first study highlights the effectiveness of look-from camera controls as light-weight additions to direct manipulation controls and provides design guidelines for the construction of look-from camera controls. The second study highlights which existing common navigation techniques are most appropriate within a map-like environment presented in immersive virtual reality and how combinations of these controls can combine the strengths of the controls to cover for the weaknesses of others.

Reference

Abstract

Watching foreign language videos is a popular and convenient strategy used by many people for learning a new language. However, traditional video players, such as the YouTube player, are not designed to support language learning. We created two video players to explore and to address the issues of using traditional players as a language learning tool. Our players specifically target casual language learners. After evaluating the first player, we found that a traditional player makes it difficult for learners to (1) adjust the level of difficulty, (2) recover missed information, and (3) assess learning progress. We then created the second player to address these issues. The results of the evaluation of the second player demonstrate that people found the player to be helpful for language learning. We also found common usage patterns in the results and opportunities for future improvement.

Reference

Abstract

For millennia, people have constructed physicalizations---physical representations of information---by hand. Recent studies have shown that physicalizations can be more efficient for transmitting information than on-screen visualizations. In addition, innovations like shape-changing interfaces and digital fabrication now make it possible to create physicalizations with little manual effort. Yet many physicalizations are still constructed by hand. In this thesis, we explore how manual construction of physicalizations influences the way people approach and comprehend data, through two studies.
One study compares bar chart authoring through physical construction to authoring using template-based chart creation software. A second study compares participant behaviour when constructing physicalizations to that when exploring previously-built physicalizations. Through comparison of these processes, we derive implications for the design of visualization authoring tools, and for the exploration of data.

Reference