Abstract

The addition of haptic feedback in digital musical instruments (DMIs) can improve performance control and expressivity. The TorqueTuner is a family of embedded rotary force-feedback modules that enable DMI designers to map haptic effects to musical parameters, facilitating straightforward haptic integration. In this demonstration, we present 3 TorqueTuner iterations based on different motors and drivers: 1) the newly redesigned Mechaduino (forked from an obsolete open-source servo platform by Tropical Labs), 2) the Mjbots Moteus (a high-power industrial motor), and the Makerfabs MaTouch SmartKnob (a self-contained device with a display). Attendees will interact with and compare each iteration, experiencing rotary haptic effects applied to a live musical context. Ultimately, we aim for this demonstration to lead to greater and easier integration of haptics into DMIs

Reference

Materials

Abstract

Creative coding is the use of programming as a medium for creative expression, often to produce interactive, generative, or audiovisual works. This 1.5-hour special session will bring together panelists with experience teaching creative coding courses at the university or equivalent level. The format will include approximately 60 minutes of lightning talks by 4–6 panelists from multidisciplinary backgrounds, including art, computer science, and engineering, followed by a combined discussion and networking segment to foster exchange among all participants

Reference

Abstract

Haptics has been a NIME topic right from the start: word stem “haptic“ was mentioned in NIME’01 workshop papers, and then part of a publication title for the first time at NIME’02, the first NIME conference], while NIME keynote presenters focused or discussed HMA on several occasions (e.g. 2003, 2013, 2015 & 2019). There are 49 occurrences of the word “haptic” in titles and 84 in abstracts for NIME paper proceedings up to 2025, as searchable in the github repository versioning the NIME bibliography.
Though common in NIME, Haptic Musical Artifacts (HMA) can only be experienced in person. With the exception of a few sub-categories of haptic hardware that are widely deployed, for instance, vibrotactile haptics in mobile phones, most HMA remain confined to research labs. Force-Feedback devices, on the other hand, are much less available to the NIME community. They tend to be expensive and demand hardware and software specifications which are not straightforward to implement. Furthermore, these devices become rapidly obsolete as computer hardware and software evolve, making them even less accessible. Finally, the replicability of such devices is not evident due to proprietary protocols (firmware, drivers, etc.) and the unavailability of original electronic circuits. All these issues have a decisive negative impact on the sustainability of haptic research in NIME and elsewhere, with most projects having a lifespan of a few years (typically the duration of a PhD thesis).
This workshop aims to present a retrospective review of HMA, specifically, force-feedback devices used in musical applications, accompanied by hands-on demonstrations, to provide NIME attendees with an opportunity to revisit and experience research on haptics and music. Furthermore, a general discussion will be held on ways to create a consortium of researchers interested in the replicability and sustainability of HMA within the NIME community.

Reference

Abstract

This paper examined how rotary force feedback in knobs can enhance control over musical techniques, focusing on both performance and user experience. To support our study, we developed the Bend-aid system, a web-based sequencer with pre-designed haptic modes for pitch modulation, integrated with TorqueTuner, a rotary haptic device that controls pitch through programmable haptic effects. Then, twenty musically trained participants evaluated three haptic modes (No-force feedback (No-FF), Spring, and Detent) by performing a vibrato mimicry task, rating their experience on a Likert scale, and providing qualitative feedback in post-experiment interviews. The study assessed objective performance metrics (Pitch Error and Pitch Deviation) and subjective user experience ratings (Comfort, Ease of Control, and Helpfulness) of each haptic mode. User experience results showed that participants found force feedback helpful. Performance results showed that the Detent mode significantly improved pitch accuracy and vibrato stability compared to No-FF, while the Spring mode did not show a similar improvement. Post-experiment interviews showed that preferences for Spring and Detent modes varied, and the applicants provided suggestions for future knob designs. These findings suggest that force feedback may enhance both control and the experience of control in rotary knobs, with potential applications for more nuanced control in DMIs.

Reference

Materials

Abstract

A growing body of work on musical haptics focuses on vibrotactile feedback, while musical applications of force feedback, though more than four decades old, are sparser. This paper reviews related work combining music and haptics, focusing on force feedback. We then discuss the limitations of these works and elicit the main challenges in current applications of force feedback and music (FF&M), which are as follows: modularity; replicability; affordability; and usability. We call for the following opportunities in future research works on FF&M: embedding audio and haptic software into hardware modules, networking multiple modules with distributed control, and authoring with audio-inspired and audio-coupled tools. We illustrate our review with recent efforts to develop an affordable, open-source and self-contained 1-Degree-of-Freedom (DoF) rotary force-feedback device for musical applications, i.e., the TorqueTuner, and to embed audio and haptic processing and authoring in module firmware, with ForceHost, and examine their advantages and drawbacks in light of the opportunities presented in the text.

Reference

Abstract

This hands-on class will allow artists to use open-source tools to create interactive and immersive experiences. These tools have been created and incubated at the Society for Arts and Technology (SAT), a unique non-profit organization in Canada whose mission is to democratize technologies to enable people to experience and author multisensory immersions. During the class we invite participants to use their favorite software on platforms they are already familiar with, to interface with our tools. The toolset will include transmission protocols, video mapping tools, sound spatialization software, and gestural control using pose detection. The class will be organized in two parts: a presentation of the tools and context involving the development and applications, and a hands-on session with an ephemeral immersive space. This event is designed for art researchers, artists, designers, content creators, and other creatives interested in creating immersive spaces using research-developed tools. Participants will learn how to employ open-source tools for different artistic tasks so that they will be able to deploy their own immersive spaces after the class.

Reference

Materials

Abstract

We present LivePose: an open-source (GPL license) tool that democratizes pose detection for multimedia arts and telepresence applications, optimized for and distributed on open edge devices. We designed the architecture of LivePose with a 5-stage pipeline (frame capture, pose estimation, dimension mapping, filtering, output) sharing streams of data flow, distributable on networked nodes. We distribute LivePose and dependencies packages and filesystem images optimized for edge devices (NVIDIA Jetson). We showcase multimedia arts and telepresence applications enabled by LivePose.

Reference

Materials

Abstract

ForceHost is an opensource toolchain for generating firmware that hosts authoring and rendering of force-feedback and audio signals and that communicates through I2C with guest motor and sensor boards. With ForceHost, the stability of audio and haptic loops is no longer delegated to and dependent on operating systems and drivers, and devices remain discoverable beyond planned obsolescence. We modified Faust, a highlevel language and compiler for real-time audio digital signal processing, to support haptics. Our toolchain compiles audio-haptic firmware applications with Faust and embeds web-based UIs exposing their parameters. We validate our toolchain by example applications and modifications of integrated development environments: script-based programming examples of haptic firmware applications with our haptic1D Faust library, visual programming by mapping input and output signals between audio and haptic devices in Webmapper, visual programming with physically-inspired massinteraction models in Synth-a-Modeler Designer. We distribute the documentation and source code of ForceHost and all of its components and forks.

Reference

Materials

Abstract

New technologies for printing sensors and actuators combine the flexibility of interface layouts of touchscreens with localized vibrotactile feedback, but their fabrication still requires industrial-grade facilities. Until these technologies become easily replicable, interaction designers need material for ideation. We propose an open-source hardware and software toolbox providing maker-grade tools for iterative design of vibrotactile widgets with industrial-grade printed sensors and actuators. Our hardware toolbox provides a mechanical structure to clamp and stretch printed sheets, and electronic boards to drive sensors and actuators. Our software toolbox expands the design space of haptic interaction techniques by reusing the wide palette of available audio processing algorithms to generate real-time vibrotactile signals. We validate our toolbox with the implementation of three exemplar interface elements with tactile feedback: buttons, sliders, touchpads.

Reference

Materials

Talk

Abstract

TorqueTuner is an embedded module that allows Digital Musical Instrument (DMI) designers to map sensors to parameters of haptic effects and dynamically modify rotary force feedback in real-time. We embedded inside TorqueTuner a collection of haptic effects (Wall, Magnet, Detents, Spring, Friction, Spin, Free) and a bi-directional interface through libmapper, a software library for making connections between data signals on a shared network. To increase affordability and portability of force-feedback implementations in DMI design, we designed our platform to be wire-less, self-contained and built from commercially available components. To provide examples of modularity and portability, we integrated TorqueTuner into a standalone haptic knob and into an existing DMI, the T-Stick. We implemented 3 musical applications (Pitch wheel, Turntable and Exciter), by mapping sensors to sound synthesis in audio programming environment SuperCollider. While the original goal was to simulate the haptic feedback associated with turning a knob, we found that the platform allows for further expanding interaction possibilities in application scenarios where rotary control is familiar.

Reference

Materials

Talk

Abstract

Complex data visualization design projects often entail collaboration between people with different visualization-related skills. For example, many teams include both designers who create new visualization designs and developers who implement the resulting visualization software. We identify gaps between data characterization tools, visualization design tools, and development platforms that pose challenges for designer-developer teams working to create new data visualizations. While it is common for commercial interaction design tools to support collaboration between designers and developers, creating data visualizations poses several unique challenges that are not supported by current tools. In particular, visualization designers must characterize and build an understanding of the underlying data, then specify layouts, data encodings, and other data-driven parameters that will be robust across many different data values. In larger teams, designers must also clearly communicate these mappings and their dependencies to developers, clients, and other collaborators. We report observations and reflections from five large multidisciplinary visualization design projects and highlight six data-specific visualization challenges for design specification and handoff. These challenges include adapting to changing data, anticipating edge cases in data, understanding technical challenges, articulating data-dependent interactions, communicating data mappings, and preserving the integrity of data mappings across iterations. Based on these observations, we identify opportunities for future tools for prototyping, testing, and communicating data-driven designs, which might contribute to more successful and collaborative data visualization design.

Reference

Talk

Reference

Reference

Reference

Reference

Reference

Reference

Abstract

We have documented a kit for students to design vibrotactile haptic feedback using embedded digital audio/sensor signal processing.

Reference

Materials

Abstract

UCXR are designing and manufacturing an augmented reality headset. Elara is their student-designed Augmented Reality headset developed to visualise human anatomy and other complex systems; advancing innovation in medicine, education, engineering design and manufacturing. The ELARA headset is an AR device for real-time 3D visualization and interaction, designed to make technical education more immersive, accessible, and engaging for students and professionals in medicine, engineering, manufacturing, and beyond.

Reference

Materials

Abstract

At WayBionic, we strive to revolutionize remote surgery by developing a bionic arm inspired by the challenges of performing medical procedures in space. Our goal is to create innovative, reliable, and precise tools that empower both astronauts and doctors, whether in orbit or on Earth.

Reference

Materials