Tag Archives: sound

Stretch | semester project

Stretch is an interactive art installation which invites participants to interrupt and manipulate a stretch of time through hand gestures. It distorts sound and movement, mainly through granular synthesis, ultimately creating a frenzied yet stimulating interactive experience.

Participants wear gloves and play with an unfolding cube to control the screen projection and sound in the space. The wooden cube is the main point of interaction and guides the participants’ gestures. The setup includes a base with a pressure sensor to detect if the cube is lifted off the base and in use; this starts up the sound and visuals. Each glove has a gyroscope attached to capture the gestures made. Additional data from the right hand is input into MuBu, a machine learning system, in order to detect when participants shake the cube. The left hand also has a bending sensor to measure grip.

Part I

The screen projection is made out of several familiar head actions such as a rotating stretch (typically used for exercise warm-ups) and nodding. The head movements correspond to and act as feedback to the gestures and sound. The visuals were inspired by Modell 5 by art duo Granular Synthesis, as well as the warped portraits of Francis Bacon. I found the face and head to be a suitable subject as we are immediately drawn to it and it catches our gaze.

However, on hindsight, I would have built a physical setup consisting of objects such as pendulums and weights, and filmed clips of it instead of the face. Although the face has its advantages and recording footage of it is much easier as it does not require extra time to build, I do feel that it possibly adds another layer of narrative. This may divert the participants’ attention away from the motions and sounds, to the identity or purpose of the person.

Part II explains the technical elements behind the work:

Screenshots of Max patch:

Screen Shot 2016-04-21 at 6.57.04 PM Screen Shot 2016-04-21 at 6.56.34 PM Screen Shot 2016-04-22 at 1.37.52 PM
Screen Shot 2016-04-22 at 1.36.35 PM Screen Shot 2016-04-22 at 1.37.00 PM

Pan n’ Tilt 2.0

Documentation for second iteration of pan and tilt, now including lights to enhance the overall experience. Inspired by the movement and sound of water.

The four corners of the gyroscope correspond to the 4 lights and speakers. Tilting the gyroscope downwards in one corner intensifies the blue and brightness of the light, and increases the volume of the corresponding speaker. This decreases the volume of the speaker in the opposite corner, and drains the blue and brightness from the light, making it appear whiter and dimmer.

Improved the sound increment from the first iteration of pan & tilt by using the table object. The volume transition from when the gyroscope is leveled to being tilted is now smoother and more gradual.

Note: Second half of video has no sound but will fix ASAP! Edit: sound fixed

Playin’ around with surround sound

Documentation for pan n’ tilt prototype. This setup allows the participant to control the movement and dispersal of sound in the room using the gyroscope. Each of the four corners of the gyroscope correspond to each of the four speakers in the room. The effect is immediate; for example, tilting the top right corner of the gyroscope downwards will increase the volume of the ‘front-right’ speaker (position when the participant faces the screen projection).

The accompanying graphics on the screen are a visual representation of the sound dispersal. The free-flowing polygon has 4 corners which correspond to the 4 speakers in each corner of the room. Tilting the gyroscope in one corner will alter the shape of the polygon and stretch it furthest in the same direction of the speaker. Conversely, keeping the gyroscope leveled will produce equal volume in all 4 speakers and the polygon will become a rectangle shape.

3D graphics and a more balanced sound increment (using the table object) could be applied for the next iteration of this prototype to enhance the experience.