Room Play
Year
2022
Fields
Interaction Design
Ludic Design
Educational Design
Overview
Exploring Soundscape through Tangibility
RoomPlay was a five-week collaborative project that aimed to merge play and technology in a unique way, helping participants not just to understand, but to physically engage with the concept of soundscape. The project centered around creating an embodied experience, where participants could interact with sound in a tangible manner. By positioning artifacts within a metaphorical room, the project delved into the complex relationship between sound, space, time, and human experience.
This approach allowed users to explore how sound interacts with its environment, how it can shape and be shaped by the space it occupies, and how these interactions can influence human perception and experience. The project highlighted the importance of spatial sound in everyday life, offering a hands-on exploration that made the abstract concept of soundscapes accessible and engaging.
Problem Statment
Soundscape as a Medium
Soundscapes are the auditory environments that surround us, encompassing all sounds in a particular space, from natural noises to human activities. These soundscapes significantly shape our perception of spaces, influencing our emotions, behaviors, and interactions. Despite their impact, the spatial qualities of sound are often underappreciated, leading to environments that may not fully support the well-being and activities of their users.
Investigating soundscapes is crucial for designing spaces that are acoustically balanced and enhance user experience. Through RoomPlay, we sought to make the intangible aspects of sound tangible, allowing users to engage with and understand the complexities of sound in a more embodied and interactive way.
Design Process
Background
Inspired by other published projects such as The Music Room and TuneTable, which use tangible interfaces to facilitate collaborative music-making, we hope to apply similar principles to promote understanding of spatial sound qualities. Our concept, RoomPlay, provides users with a fun and collaborative experience in which tangible objects manipulate sound within a metaphorical room. We envisioned RoomPlay to be something similar and use it both as an introduction for beginners and a "professional" entertainment tool. By engaging users in interactive experiences, we aimed to foster curiosity and exploration while enhancing practical knowledge of spatial sound concepts.
“Change the acoustic nature of a room, and make a small room into a cathedral”
- Jens (Teacher)
Prototyping & Development
In the prototyping phase, we explored various materials like cardboard, styrofoam, and MDF to assess their suitability. We created low-fidelity prototypes and used laser-cut cubes for optimal camera tracking with TensorFlow. We tested various shapes from the TensorFlow library, finding the clock, banana, and stop sign to be the best for tracking. We also evaluated different cameras, ultimately choosing an integrated laptop camera for its optimal focal length.
Programming
To achieve the desired project modifications, we utilized the Web Audio API to manipulate and synthesize audio in real-time, enhancing the user experience. We also integrated TensorFlow for advanced object detection and recognition.
A key feature was using tracking data to create a spatial audio environment. Positioning data allowed dynamic modulation of audio: panning along the x-axis for lateral movement, altering playback speed along the y-axis for vertical movement, and using a cutoff filter along the z-axis to simulate distance.
Functionality
In RoomPlay, users can adjust the sound environment using cubes in various ways. Moving a cube left to right shifts the sound's direction across the space, helping users experience how sound can be directed. Raising or lowering a cube alters the speed of the sound, allowing for changes in tempo and pitch. Additionally, the distance of the cube from the center affects the clarity of the sound—bringing it closer sharpens the audio, while moving it away makes it softer, simulating how sound changes with distance. This interactive setup provides a hands-on way to explore how sound behaves in a space.
User tests
During the playtesting phase, we engaged participants in interactive sessions to evaluate the RoomPlay prototype's usability and effectiveness. Participants were given minimal guidance and encouraged to explore the prototype freely. Observations from these sessions provided valuable insights into user behavior, preferences, and comprehension of the spatial qualities of sound. We observed instances of collaboration among participants, which validated our design goal of fostering a collaborative and playful experience. Feedback from the playtesting sessions informed iterative improvements to the prototype, ensuring that it effectively conveyed the intended concepts and provided an engaging user experience.
Conclusion & Reflection
RoomPlay combines tangible interaction with spatial sound exploration, creating an immersive experience where users manipulate sound through physical objects. Our prototype, built with TensorFlow.js, allows users to move wooden cubes with symbols to modulate sound in a virtual space, encouraging exploration and discovery.
Design elements like symbol choice and space arrangement were carefully considered for engagement and intuitive use. Though RoomPlay was less advanced than initially planned, it successfully enabled users to explore sound spatiality in a playful and collaborative environment. The project served as a proof of concept, revealing further design opportunities to enhance the experience.
