Abstract

This paper examined how rotary force feedback in knobs can enhance control over musical techniques, focusing on both performance and user experience. To support our study, we developed the Bend-aid system, a web-based sequencer with pre-designed haptic modes for pitch modulation, integrated with TorqueTuner, a rotary haptic device that controls pitch through programmable haptic effects. Then, twenty musically trained participants evaluated three haptic modes (No-force feedback (No-FF), Spring, and Detent) by performing a vibrato mimicry task, rating their experience on a Likert scale, and providing qualitative feedback in post-experiment interviews. The study assessed objective performance metrics (Pitch Error and Pitch Deviation) and subjective user experience ratings (Comfort, Ease of Control, and Helpfulness) of each haptic mode. User experience results showed that participants found force feedback helpful. Performance results showed that the Detent mode significantly improved pitch accuracy and vibrato stability compared to No-FF, while the Spring mode did not show a similar improvement. Post-experiment interviews showed that preferences for Spring and Detent modes varied, and the applicants provided suggestions for future knob designs. These findings suggest that force feedback may enhance both control and the experience of control in rotary knobs, with potential applications for more nuanced control in DMIs.

Reference

Materials

Abstract

A growing body of work on musical haptics focuses on vibrotactile feedback, while musical applications of force feedback, though more than four decades old, are sparser. This paper reviews related work combining music and haptics, focusing on force feedback. We then discuss the limitations of these works and elicit the main challenges in current applications of force feedback and music (FF&M), which are as follows: modularity; replicability; affordability; and usability. We call for the following opportunities in future research works on FF&M: embedding audio and haptic software into hardware modules, networking multiple modules with distributed control, and authoring with audio-inspired and audio-coupled tools. We illustrate our review with recent efforts to develop an affordable, open-source and self-contained 1-Degree-of-Freedom (DoF) rotary force-feedback device for musical applications, i.e., the TorqueTuner, and to embed audio and haptic processing and authoring in module firmware, with ForceHost, and examine their advantages and drawbacks in light of the opportunities presented in the text.

Reference

Abstract

This hands-on class will allow artists to use open-source tools to create interactive and immersive experiences. These tools have been created and incubated at the Society for Arts and Technology (SAT), a unique non-profit organization in Canada whose mission is to democratize technologies to enable people to experience and author multisensory immersions. During the class we invite participants to use their favorite software on platforms they are already familiar with, to interface with our tools. The toolset will include transmission protocols, video mapping tools, sound spatialization software, and gestural control using pose detection. The class will be organized in two parts: a presentation of the tools and context involving the development and applications, and a hands-on session with an ephemeral immersive space. This event is designed for art researchers, artists, designers, content creators, and other creatives interested in creating immersive spaces using research-developed tools. Participants will learn how to employ open-source tools for different artistic tasks so that they will be able to deploy their own immersive spaces after the class.

Reference

Materials

Abstract

We present LivePose: an open-source (GPL license) tool that democratizes pose detection for multimedia arts and telepresence applications, optimized for and distributed on open edge devices. We designed the architecture of LivePose with a 5-stage pipeline (frame capture, pose estimation, dimension mapping, filtering, output) sharing streams of data flow, distributable on networked nodes. We distribute LivePose and dependencies packages and filesystem images optimized for edge devices (NVIDIA Jetson). We showcase multimedia arts and telepresence applications enabled by LivePose.

Reference

Materials

Abstract

ForceHost is an opensource toolchain for generating firmware that hosts authoring and rendering of force-feedback and audio signals and that communicates through I2C with guest motor and sensor boards. With ForceHost, the stability of audio and haptic loops is no longer delegated to and dependent on operating systems and drivers, and devices remain discoverable beyond planned obsolescence. We modified Faust, a highlevel language and compiler for real-time audio digital signal processing, to support haptics. Our toolchain compiles audio-haptic firmware applications with Faust and embeds web-based UIs exposing their parameters. We validate our toolchain by example applications and modifications of integrated development environments: script-based programming examples of haptic firmware applications with our haptic1D Faust library, visual programming by mapping input and output signals between audio and haptic devices in Webmapper, visual programming with physically-inspired massinteraction models in Synth-a-Modeler Designer. We distribute the documentation and source code of ForceHost and all of its components and forks.

Reference

Materials

Abstract

New technologies for printing sensors and actuators combine the flexibility of interface layouts of touchscreens with localized vibrotactile feedback, but their fabrication still requires industrial-grade facilities. Until these technologies become easily replicable, interaction designers need material for ideation. We propose an open-source hardware and software toolbox providing maker-grade tools for iterative design of vibrotactile widgets with industrial-grade printed sensors and actuators. Our hardware toolbox provides a mechanical structure to clamp and stretch printed sheets, and electronic boards to drive sensors and actuators. Our software toolbox expands the design space of haptic interaction techniques by reusing the wide palette of available audio processing algorithms to generate real-time vibrotactile signals. We validate our toolbox with the implementation of three exemplar interface elements with tactile feedback: buttons, sliders, touchpads.

Reference

Materials

Talk

Abstract

TorqueTuner is an embedded module that allows Digital Musical Instrument (DMI) designers to map sensors to parameters of haptic effects and dynamically modify rotary force feedback in real-time. We embedded inside TorqueTuner a collection of haptic effects (Wall, Magnet, Detents, Spring, Friction, Spin, Free) and a bi-directional interface through libmapper, a software library for making connections between data signals on a shared network. To increase affordability and portability of force-feedback implementations in DMI design, we designed our platform to be wire-less, self-contained and built from commercially available components. To provide examples of modularity and portability, we integrated TorqueTuner into a standalone haptic knob and into an existing DMI, the T-Stick. We implemented 3 musical applications (Pitch wheel, Turntable and Exciter), by mapping sensors to sound synthesis in audio programming environment SuperCollider. While the original goal was to simulate the haptic feedback associated with turning a knob, we found that the platform allows for further expanding interaction possibilities in application scenarios where rotary control is familiar.

Reference

Materials

Talk

Abstract

Complex data visualization design projects often entail collaboration between people with different visualization-related skills. For example, many teams include both designers who create new visualization designs and developers who implement the resulting visualization software. We identify gaps between data characterization tools, visualization design tools, and development platforms that pose challenges for designer-developer teams working to create new data visualizations. While it is common for commercial interaction design tools to support collaboration between designers and developers, creating data visualizations poses several unique challenges that are not supported by current tools. In particular, visualization designers must characterize and build an understanding of the underlying data, then specify layouts, data encodings, and other data-driven parameters that will be robust across many different data values. In larger teams, designers must also clearly communicate these mappings and their dependencies to developers, clients, and other collaborators. We report observations and reflections from five large multidisciplinary visualization design projects and highlight six data-specific visualization challenges for design specification and handoff. These challenges include adapting to changing data, anticipating edge cases in data, understanding technical challenges, articulating data-dependent interactions, communicating data mappings, and preserving the integrity of data mappings across iterations. Based on these observations, we identify opportunities for future tools for prototyping, testing, and communicating data-driven designs, which might contribute to more successful and collaborative data visualization design.

Reference

Talk