Abstract

Astral is a prototyping tool for authoring mobile and smart object interactive behaviours. It mirrors selected display contents of desktop applications onto mobile devices (smartphones and smartwatches), and streams/remaps mobile sensor data to desktop input events (mouse or keyboard) to manipulate selected desktop contents. This allows designers to use familiar desktop applications (e.g. PowerPoint, AfterEffects) to prototype rich interactive behaviours. Astral combines and integrates display mirroring, sensor streaming and input remapping, where designers can exploit familiar desktop applications to prototype, explore and fine-tune dynamic interactive behaviours. With Astral, designers can visually author rules to test real-time behaviours while interactions take place, as well as after the interaction has occurred. We demonstrate Astral's applicability, workflow and expressiveness within the interaction design process through both new examples and replication of prior approaches that illustrate how various familiar desktop applications are leveraged and repurposed.

Reference

Abstract

Toolkit research plays an important role in the field of HCI, as it can heavily influence both the design and implementation of interactive systems. For publication, the HCI community typically expects toolkit research to include an evaluation component. The problem is that toolkit evaluation is challenging, as it is often unclear what 'evaluating' a toolkit means and what methods are appropriate. To address this problem, we analyzed 68 published toolkit papers. From our analysis, we provide an overview of, reflection on, and discussion of evaluation methods for toolkit contributions. We identify and discuss the value of four toolkit evaluation strategies, including the associated techniques that each employs. We offer a categorization of evaluation strategies for toolkit researchers, along with a discussion of the value, potential limitations, and trade-offs associated with each strategy.

Reference

Talk

Abstract

Interactive, smart objects – customized to individuals and uses – are central to many movements, such as tangibles, the internet of things (IoT), and ubiquitous computing. Yet, rapid prototyping both the form and function of these custom objects can be problematic, particularly for those with limited electronics or programming experience. Designers often need to embed custom circuitry; program its workings; and create a form factor that not only reflects the desired user experience but can also house the required circuitry and electronics. To mitigate this, we created Pineal, a design tool that lets end-users: (1) modify 3D models to include a smart watch or phone as its heart; (2) specify high-level interactive behaviours through visual programming; and (3) have the phone or watch act out such behaviours as the objects' "smarts". Furthermore, a series of prototypes show how Pineal exploits mobile sensing and output, and automatically generates 3D printed form-factors for rich, interactive, objects.

Reference

Talk

Abstract

Remote controls facilitate interactions at-a-distance with appliances. However, the complexity, diversity, and increasing number of digital appliances in ubiquitous computing ecologies make it increasingly difficult to: (1) discover which appliances are controllable; (2) select a particular appliance from the large number available; (3) view information about its status; and (4) control the appliance in a pertinent manner. To mitigate these problems we contribute proxemic-aware controls, which exploit the spatial relationships between a person's handheld device and all surrounding appliances to create a dynamic appliance control interface. Specifically, a person can discover and select an appliance by the way one orients a mobile device around the room, and then progressively view the appliance's status and control its features in increasing detail by simply moving towards it. We illustrate proxemic-aware controls of assorted appliances through various scenarios. We then provide a generalized conceptual framework that informs future designs of proxemic-aware controls.

Reference

Abstract

In this thesis, I propose methods for repurposing existing hardware and software to enable designers to create live interactive prototypes for smart interactive objects without the need to write code or create custom circuitry. The advent of ubiquitous computing brought the promise of interactive artifacts that integrate into our everyday lives. While this has led to a myriad of “smart objects”, the problem is that it is difficult for interaction designers to devise interactive behaviours for such objects. For example, how might an interaction designer prototype behaviours for a smart speaker? How can they go beyond voice responses and, for instance, animate lights to show that the speaker is listening, or searching for an answer on the web? Designers today face three challenges: (1) needing multiple expertise of designing behaviour, form, circuitry, and programming the functionality; (2) lacking software tools to author fine-tuned dynamic behaviours; and (3) needing closer-to-product representations to physically manipulate the prototype. I overcome this gap through a method and two interactive systems. I propose a design metaphor: Soul–Body Prototyping, which suggests leveraging off-the-shelf mobile phones and watches to create smart object prototypes. By enclosing the mobile device (“soul”) into a physical enclosure (“body”), the designer can exploit the mobile device’s rich sensing, outputs, and internet connectivity. I then operationalize Soul–Body Prototyping through two proof-of-concept prototyping tools. Pineal features trigger-action behaviours which automatically generate 3D models for physical forms. These forms fit a mobile device and expose the necessary inputs and outputs. Astral is a tool where designers can mirror a portion of the desktop’s screen onto a mobile device, and create mappings that convert live mobile sensor data into mouse or keyboard events. Thus, the mobile device remote controls (and repurposes) familiar desktop applications for dynamic behaviour prototyping. Overall, my work contributes an alternative way to prototype smart interactive objects, which informs the design of future prototyping tools. Moreover, I investigate fundamental questions such as the meaning of interactive behaviour, as well as evaluation methods for prototyping tools and toolkits in HCI research.

Reference

Abstract

Transparent displays are ‘see-through’ screens: a person can simultaneously view both the graphics on the screen and real-world content visible through the screen. Interactive transparent displays can serve as an important medium supporting face-to-face collaboration, where people interact with both sides of the display and work together. Such displays enhance workspace awareness, which smooths collaboration: when a person is working on one side of a transparent display, the person on the other side can see the other's hand gestures, gaze, and what s/he is currently manipulating on the shared screen. Even so, we argue that in order to provide effective support for collaboration, designing such transparent displays must go beyond current offerings. We propose using two-sided transparent displays, which can present different content on both sides. The displays should also accept interactive input on both sides and visually augment users’ actions when display transparency is reduced. We operationalized these design requirements with our two-sided transparent display prototype, FACINGBOARD-II, and devised a palette of supportive interaction techniques. Through empirical studies, we found that the workspace awareness provided by transparent displays is compromised with degrading display transparency, and that visually enhancing user actions can compensate for this awareness loss.

Reference