Sweet and Savoury is an interactive installation that uses foley to create a surreal experience for the participants. The installation demonstrates how sounds can be associated to more than one object, and when combined to create specific settings, can create a totally different atmosphere and story that plays with the participant’s imagination.

 

Sweet and Savoury

Videos:

 

 

{Aftermath}

 

 

For my final project, I decided to stick to working with foley, but this time with a plot twist. After the mid-sem presentation and some consultations and discussions, here’s the project idea I’ve decided to work on:

Sweet and Savoury

Main goal: A sense of discovery in a surreal experience

  • Sound association: Creating sounds with objects and immediately associating them to each other respectively.
  • Surreal experience: The uncanny feeling when the sound first associated with an object is now being associated to an entirely different object/action.

Sound Research

There are many objects that can be used to make sounds similar to other objects/actions. However, I had to make sure the objects I chose can come together to create a logical scene (e.g: cooking, gardening, etc.), especially since I was planning to portray two different scenes in my installation.

Here are some references and examples I found for foley sounds:

What Was That? The Top 15 Foley SFX From Everyday Household Objects

 

Cracking bones; snapping celery/carrot

Flapping wings; waving leather gloves

Body punches; hitting a thick book

Fire crackling; cellophane, chips bag, steel wool

Stack of cash; deck of cards

metal blinds; measuring tape

Grass footsteps; shaking shredded newspaper in plastic bag

Egg cracking; breaking ice cream cone

Water showering; dropping rice

 

The foley sound effects I found were really interesting, but I was having trouble creating a scene where the items make sense together. It was easier in my mid-sem project, The Study Room, where the objects on the foley side did not need to make sense together and only needed to create sounds that resemble those in the study room side.

At this point, I found that food had a lot of potential to make interesting variety of foley sounds, so I decided to go with the theme of cooking. This is when I began listening to cooking ASMR videos. I would play them but only listen to the video, allowing my imagination to help me associate the sounds to different objects. I also started testing out sounds myself with different fruits and food items and found that they can be rather gore sounding. This is when I thought of the ‘Sweet and Savoury’ concept; one scene will be making an innocent, sweet snack, while the other will be the preparation of a savoury meal with a plot twist (for shock factor).

 

After researching and running several tests, these were the objects I decided to put together to form each scene:

{Sweet}

  • pomegranate (tearing apart and removing seeds = ripping fingers off hand)
  • grapefruit (tearing apart and juicing by hand = squishy meat/guts sounds)
  • chocolate bar/wafer (snapping in half = breaking thin finger bones)
  • pop rocks (fizzing and popping = sizzling of oil in pan when meat is added)

{Savoury}

  • (fake) human hand
  • minced meat/patty
  • oil in pan

 

Setup

Equipment:

  • Audio recorder
  • Speakers
  • Foley objects
  • Props for scenes
  • table
  • Cloth divider

 

Layout sketch:

 

Setup planning:

 

Preparation of actual setup:

Sweet (foley side).

 

Savoury (observation side).

 

Speakers and recorder setup (hidden inside the cloth divider).

 

Side view of speakers and recorder setup (the cloth at the side of the divider can part, allowing me to reach through to control the equipment inside).

 

Flow of events:

  • Participant is seated in front of the sweet setup (foley side) and briefed on the ‘cooking’ instructions
  • Once they are ready to begin ‘cooking’, the recording will start and will only end when they are done with the entire foley process.
  • Participants will then move over to sit in front of the savoury setup (observation side), where the recorded audio will be played back for them to hear as they observe the new setting before them.

Intended result:

  • Participants will follow the ‘cooking’ instructions given and create the foley sounds.
  • The sounds they make will be strongly associated to what they see as they go about the foley process.
  • Upon hearing the recording play back when they are seated at the savoury side, they will have a surreal experience as they start associating the sounds they once made to entirely different objects.
  • Through pairing the familiar audio with a new visual setup, the participant will be somewhat tricked into feel as though they were the one who made the gore scene before them, creating a surreal experience.

 

Here’s some mid-sem project documentation for The Study Room. Overall, I’m glad that both the audience and the participants had fun. The flow was just about how I imagined it to be; the participant on the foley side makes the proper sound effects, while the other participant makes (almost) all the associations to the actual objects.

 

The main issue would be that it was hard to hear some of the softer sounds, especially since there were louder sounds coming from other parts of the room. Other than that, I’m glad the instructions were clear enough on the foley side, specially since other sounds can be made that might be harder to associate them to the objects.

 

Setup

Study room setup

Foley Setup

Instructions

Foley keyboard

Foley notebook

Foley keys

 

 

How It Works

Presenting the (willing) participants, Fizah and Bryan. [Thanks, guys. Very cool B)]

In the first video, we can see Fizah having to lean in a little to hear the sounds better. In the second project, I think I can add speakers to amplify the sounds. Watching the second part where they switched sections showed that sense of discovery for the both of them. It was mentioned that hearing the sounds somewhat entices them to do the action, so that;s what happened in the second video.

 

 

 

 

/Possible Ideas for Digital Project/

Main goal: A Surreal Experience

  • Entice: make the audience want to do a certain action using sounds; influence / control their actions through sound association
  • Confuse: resulting sounds that do not match the anticipated one.
  • Enhance: bring attention to everyday sounds.

 

Equipment:

  • Photo-resistor
  • Motion sensor
  • Pressure sensor
  • Tilt sensor
  • Speaker
  • Actual props
  • Foley items

 

Set-up #1: Sound Tak Tally

 

Flow of events:

  • Approaching/Passing by motion sensor at respective item triggers sfx to play (e.g: glass shattering)
  • Participant may interact with the item to replicate the sound (e.g: pushing the glass off the edge)
  • Sensor detects movement of item from interaction and triggers wrong sfx (e.g: squeaking instead of shattering)
  • Repeat with other items

Intended outcome:

  • For the participant to feel enticed to carry out an action (irresistible temptation to match sound to an action)
  • Taking away that satisfaction by playing a sound that does not match their action
  • Experiencing something surreal and confusing

 

Set-up #2: Sense in Sound

 

Flow of events:

  • Approaching/Passing by motion sensor at respective item triggers distorted sfx to play (e.g: distorted mug stirring sound)
  • Participant may interact with the item (e.g: picking it up, using it in a common manner)
  • Sensor detects movement of item from interaction and triggers correct sfx to play (e.g: sound playing is no longer distorted)
  • Putting the item back resets the sound back to being distorted
  • Repeat with other items

Intended outcome:

  • For the participant to be curious about the item and sound not matching
  • Interaction with the item allows them to uncover the true sound it makes
  • Experiencing something surreal and confusing
  • Bringing attention to everyday sounds

 

My initial idea of what critical vehicle represents was a mode of communication or medium that relays an important message. It was kinda close. The reading describes ‘critical’ as something judgmental, calling out the shortcomings or problems. As for ‘vehicle’, it is described as a carrier; ‘a person or a thing’ that is used as a medium to express an idea or an emotion. Given this, a critical vehicle refers to a medium that calls attention to negative issues that need to be addressed or rectified. By bringing the issue to light, it opens an opportunity to turn the situation around on a phycho-social terrain.

In the reading, Krzysztof Wodiczko’s also talks about interrogative designs, which refers to designs that calls out and question the issues and specific conditions of life exist in the world. It serves not only to give voice and comfort to those affected by the issue, but also to increase ethical alertness towards the issues that they face.

 

“Instead of deconstructing itself, design should deconstruct life.”

Without any sugarcoating, interrogative designs serve to expose all the harsh truths and pain experienced, from personal or societal, or even political. It is a segue of sorts, bringing an issue to light and inciting conversation about it. I really like how the artist creates these interrogative designs to his works, like Speaking through Monuments, using projections and existing monuments. These monuments hold important meaning, reflecting past issues and experiences throughout history. Yet, over time, we lose sight of their meaning, and they become symbols of the past that do not relate to the present; a decoration piece like any other structure in the city. Through the use of projections, the artist is able to use those monuments as his critical vehicle, projecting images that represent issues, such as homelessness, segregation, and isolation within the nation, onto their facade. Through those projections, he turns the monuments into sites of critical evaluation, bringing about debate and communication, inciting people to see and recognise these ongoing issues, and finally, to take action in solving them.

 

“Not speaking through city monuments is to abandon them and to abandon ourselves, losing both a sense of history and present.”

Homeless Vehicle Project was another work I found really interesting. It really brings to light the flaws within services that cater to the homeless. What was supposed to be physically and socially support turned into abuse and dehumanisation of homeless individuals; the security that was supposed to guard them abused them, and the shelters that were supposed to help them segregated them further from society. The homeless vehicles created were a response to this issue. They were designed base on the routine and needs of the homeless, including features such as having lock and alarm systems to protect their valuables, being able to transform into a vendor’s cart in order to sell items, and even forming an encampment that protects them from police harassment . It was a form of transport and shelter for the homeless, providing them aid while also creating empathy between the public and the homeless. By incorporating something we commonly see in life, such as a vendor’s cart, the observer will see the homeless as more than just some poor outcast, but someone who is part of the society.

 

Overall, there are a lot of issues out there that some society has become numb to, or refuse to acknowledge, like environmental issues, for example. Shouting “save the turtles” is one way to bring people’s attention to it, but have a voice much louder than that. Through interrogative designs, we can create critical vehicles that speak volumes about the issue, exposing and challenging it, and eventually inciting change for the better.

 

Main goal

  • Defamiliarize an everyday experience
  • Mixing up sight and sound: same motion, different sound; same sound, different object
  • Confusing the senses and creating a surreal experience

 

Set-up

[Two tables, representing the rooms, placed in a reflected position.]

 

Study room #1

A table with various materials that create the same sound as items that can be found on a study desk.

Participants can:

  • Touch and interact with everything on the table (e.g: dragging, picking up and putting down, pressing, ripping, etc.)

Intended result:

  • To confuse the participant with sound; they know it is a study room, but the sight does not match with the look of a desk that they are familiar with
  • Through interaction, the objects will create a sound that the participant can link to an object commonly found on a desk

 

Study room #2

A table with the actual objects we find on a study table, each item corresponding with the items from room #1.

Participants can:

  • Sit in the chair and observe (e.g: listen to the sounds created from room #1)

Intended result:

  • Giving the participant a surreal experience; the hear the sounds, but nothing is physically moving from their point of view
  • Feel strange, as though they are having an out of body experience

 

 

 

When I go to an exhibition or museum, I’m usually just immersed in the visuals alone. But this exhibition allowed us to touch the installations. Needless to say, I was even more enthralled. What caught my eye most was the sheltered reading room with a single tale and chair inside. As I got closer to it, I noticed the materials that were used: wooden planks with rustic colours, common textured metal sheets, and even this green plastic shelter. Seeing all these materials together evoked a warm and calm feeling from within me. It was as though these structures were trying to replicate my childhood memories, presenting them as a physical architecture.

 

 

I remember the tour guide mentioning how the structures can make us feel a sense of comfort, yet puts us slightly on edge through hostile looking elements such as the up right pencils that resemble an army or spikes, and how the chair is not sift like a couch, making you shift around in it. But I didn’t get a single hostile vibe from them. Instead, I actually found the arrangement of pencils cute, playful even, and the chairs were actually very comfortable, possibly because wooden chairs and furniture was what I grew up using.

 

 

 

The guide also explained how the rules of setting up the installations were that they have to be placed at 90 degree angles in relation to one another. This was another element that was supposed to add stiffness to the otherwise inviting reading room. However, for me, I guess the combination of materials really overpowered these hostile elements. There’s just something about the little wooden reading room and how the light streams in through the gaps of the shelter, giving the wooden table inside a soft warm glow. All of that just made it such a cozy place. Even the two other installations with just chairs and tables also didn’t seen rigid, perhaps due to the fact that they were more asymmetrical (and also mainly wooden). The feeling and memories evoked by the materials used made me think about the reading on atmospheres in architecture; how the looks, sounds, and textures of different material can pull up various memories from the audience, and linking those memories and feelings to the architecture gives it its atmosphere.

 

 

Another part of the fieldtrip that I enjoyed was the short films screening, my favourite two being Rotating Line, To Perceive 10,000 Different Squares in 6 Minutes and 55 Seconds. Rotating Line was interesting in that it chose a unique angle to show the rotation of a line. What seemed to be a line getting shorter and shorter, before extending again, was actually a side view of the rotating line, implying that it was 3 dimensional. And for the second film, I’ve never spent so much time looking at a single square, or rather, 10,000 squares.  Each frame shows a square that is smaller than the previous one, but the differences were so minute I was fairly certain it wasn’t actually getting any smaller. So, naturally, I decided to cheat and took two pictures of the square on screen, each about 3000 frames apart. By comparison, I did see a very slight difference in size, and hopefully that’s not just my mind playing tricks on me. Regardless, all the films showed how Siah Armajani used technology to create the illusion of three-dimensional space and time.

 

In one of the previous readings titled Space and Place, it describes how we perceive space in relation to our body. By placing it at the centre, we are able to divide space into different segments and directions.

In Illuminating Embodiment, it also talks about the body in relation to space, but more so on how it is interconnected with architecture. The reading shows this through interactive works done by the artist Rafael Lozano-Hemmer. Basically, the works mentioned show that despite being separate entities, bodies and architectures are receptive to alteration; they change in response to one another.

 

Displaced Emperors, Relational Architecture 2

 

I used to think of architecture and buildings as being fixed; they were made for a particular purpose and to only serve as such, like a monument, for example. However, In the artist’s relational architectures, he shows how these seemingly fixed narratives can be altered through the use of technology, “superimposing audiovisual elements to affect it, effect it and recontextualise it.” I find it interesting, how technology is able to breach this gap between separate beings or objects, allowing them to interact with one another, while not necessarily having them physically touch one another. For example, in Displaced Emperors, Relational Architecture 2, the participants can virtually wipe away the exterior surface of a building, changing it and exposing its internal layout. Or how portraits projected on a building in Body Movies: Relational Architecture 6 can be interacted with through the shadows cast by participants. On any other occasion, these architectures and people coexist without any noticeable interaction; its just a space that a body passes through. But through the artist’s unique use of technology, the experience within the exact same space is changed drastically, with participants taking in a foreign narrative of a previously familiar place. I supposed this is what Lozano-Hemmer meant by creating “anti-monuments for dissimulation”.

 

Body Movies: Relational Architecture 6

 

“The performativity of the participant as well as of the technology ensures that both play a part in their own remaking.”

This quote, to me, is pretty much what the reading is mainly about. It’s how both the audience and the artwork play a part in creating and achieving unique results and experiences together. Overall, I’ve learnt that, although separate, there is a connection between space and body, how they interact, and how they constantly affect each other.

This is a long read, but the writing style was pleasantly personal and slightly humors, which I really appreciate. It kept me interested despite my short attention span when it comes to absorbing information from huge chunks of text.

 

The reading describes the different aspects of a space or building that change its atmosphere. It states that a quality architecture is something that can move people. To move people, they have to be able to feel something within a space, an atmosphere. The atmosphere can be altered simply by changing any aspect of it, creating various types of vibes that you can feel when you enter a certain space. The initial atmosphere of a space determines if you would even consider entering it or not. Every element contributes to the overall atmosphere of a space; objects, people, materials, sounds, temperature, light. Even the positioning and proportion of objects and structures are factored in. By controlling these, a building or space can either blend in with its surroundings or stand out.

Like the anatomy of a body, the space has different components that forms it and makes it whole; one difference in material can change its looks and atmosphere completely. Compatibility of materials also matters, it results from seeing how different materials react with one another. Do they clash or compliment one another? The same material alone can be treated differently, each variation giving a different feel. Peter Zumthor talked about how he couldn’t use soft cedar in a space made of concrete because didn’t look balanced. He needed to replace it with something like ebony; something darker, dense enough to counter the weight of the concrete. But then things didn’t work out as planned, materials kept changing until the look and feel were just right. To me, this just shows that there is really no sure way to know unless it’s been tested. And even then, the atmosphere can change as time passes, or if new elements are introduced in the surroundings.

The part about sound was probably the most interesting part for me in this reading. Zumthor explained how sound within a space could come from the materials, as well as from echoes and amplifications. Some of these sounds may sound familiar to us. A sound that makes us feel nostalgic, bringing up memories that we now associate with the space. He goes on to task us with taking sound away from a space. His question: is there still sound? I guess not? But I guess the only way I would describe something as loud or soft in a silent room would be depending on the sounds and volumes that I relate to color. For example, the sound of blue would be soft and melodic, whereas red would be loud or sinister. Combined with the types of materials and their textures, I suppose I can imagine their sounds through what I expect them to feel like; rough textures creating grainy noises, and smooth textures creating soft ones. I can see how all these can alter the atmosphere of a space, especially when we relate them to our own past experiences.

Temperature in space can also affect the overall atmosphere of a space, and I honestly never really thought about how a building can extract warmth from us without direct contact. Even surrounding objects like props, decorations, plain clutter, or anything that we have a deep connection with, can trigger a sense of comfort or discomfort within a space, depending on our experiences.

 

Overall, I understand that the elements that make up a space contribute greatly in making it appealing and inviting, or off-putting and repulsive. Every aspect plays a part, and although the atmosphere may seem pleasant to some, others may not feel the same due to relating differently to the same elements.

When I think of cyborgs, an image of a person with a mechanical limb or two, something way into the future where we have advanced technology. However, I never though of us as cyborgs with the tech we have now. The cyborgs I imagine have super abilities, like keen hearing or sight. But then again, the tech we have now allows us to do just that. It allows us to sense so much more, and the information it feeds us can alter the way we perceive the world around us. It can make you feel things that are not there.

In the reading, it mentions and talks about Opera for a Small Room, an installation created by the artists Janet Cardiff and George Bures Miller, who use tech to immerse their audience in the mechanical theatrics that tells a story of a man called R. Dennehy. However, the technology used serve to deceive the audience; it manipulates their senses into seeing things, to fill in gaps where they think should be filled. This gap would be Dennehy himself, who is not on set, but is perceived to be through recordings of his voice and shadows that mimic his silhouette. His narrations and every other prop meticulously put together on set each played a huge role in manipulating the audience’s experience. It was to an extent that they themselves feel as though the memories of Dennehy were their own. They were able to capture the presence of a body without one physically present in the room.

This power that technology holds in manipulating our senses, it is not exclusive to electronic devices. Something as simple as a book can do the same. As a speaker plays sounds that can cause us to imagine things, so can text of a story in a book; it is all up to how we process the information given. The text from the reading that described Opera for a Small Room was enough to make me imagine hearing and picturing the installation, so much so that I just had  to do some research and look it up to see if what I imagined was correct (I found a video documentation and it was amazing and better than I expected).

Overall, I do agree that technology can have great control over our perceptions over what is actually happening in reality. And yes, it can be dangerous, but there is so much more it has to offer that makes it worth the risk exploring.

The concept of direction in space is something that we apply every day. In relation to our body, we divide our surrounding space according to different directions: up, down, left, right, front, back, and so on. In this reading, it points out that to be able to tell what space is in what direction, there has to be a body, a centrepoint to which we can relate every other direction in space to. It’s interesting to think that everyone uses the same concept of segmenting spaces, such as left and right, however, perception of direction is purely individualistic. For instance, there’s that problem I’m sure we all have faced where can tell someone to “go left”, but then they turn the wrong way and you have to follow up with a “no, not your left, my left”. In a way, neither of you were wrong, your perceptions of left are the same, but there is no true fixed position, because everyone is their own centrepoint.

episode 12 flight GIF

This reading also notes how direction and the space within can be related to time and abstract thoughts. For example, the space in front can represent the future, while the space behind represents the past. Additionally, what’s above represents heaven, and below, hell. Given this concept, I could say, “step into the future”, and your first thought would be to step forward. However, as with the perception of directions, each individual can have a different idea or value linked to each direction.

Lastly, the perception of space in relation to our body helps us gauge distance. But more importantly, or personally, the proximity of something can change how we feel about it. An example would be if you saw a cockroach flying five meters away, versus seeing one flying right in front of your face. Chances are, the one flying near your face would freak you out more. This is due to the cockroach invading your personal space. There is no line or fence to indicate this space, but it’s something we mentally put in place to guard ourselves; the closer something is, the more vulnerable we are to it. Being allowed within someone’s personal space can also communicate how close or comfortable we are with something without explicitly saying it.

Profile Patriots GIF

Regardless of the difference in perception, the relation we have between our body and space is a unique and personalised tool that we use to navigate the space around us, as well as affect our emotions and perceptions . Without it, the world would seem very disorienting. The values and meanings we link to each direction and space also allows us to express certain abstract ideas, expanding our mode of communication through relation.