IM showcase video

IM showcase gallery
Project Description 

Sensorium questions the interstices between our senses that we often take for granted through the sensory phenomenon known as “Synesthesia” – a condition where a stimulation of one sensory or cognitive pathway leads to automatic and involuntary experiences by other senses. In short, an experience of a single sense is now simultaneously perceived by other senses resulting in confusing outcomes for the individual with this condition.

In this installation, we aimed to make this gap more aware by recreating this condition by allowing visitors to experience a similar disassociation between their senses by creating a sense of unfamiliarity with objects that they are familiar with interacting. This “sensory overload” is created through the unfamiliar and different sounds heard and coloured “ink-drops” seen dripping into the tank when interacting with the objects on the stools.

Concept – artist references

We were also inspired with artworks that engaged different senses and emitted sounds. Through interaction some of our references are as follows:

Lenses by Hush

A installation that converts light sensors and refractions to sound.

Synesthesia Installation

Another installation that plays on the idea of engaging different senses in the form of Synesthesia.

In our initial ideas, we were focused on using lights and sounds to create the experience of Synesthesia but a closer study into the condition, and consultation with Lei, we decided to go with something more tactile and allow the installation to be more object-driven. Settling on the sense of touch, sight and sound.

Objects used and sounds that corresponded:

  • Box with Lid (with flowers on the inside) – Thunder
  • Chair – Cat meowing
  • Light switch – Toilet flushing
As for the sounds we chose, we followed what Lei suggested about considering the “textures of the sound”. To further elaborate, it was like how for the box with lid (with flowers inside), the expected sound would be something very soft and soothing.  Whereas for that we used the sound of thunder, which sound texture wise was very loud and harsh to further juxtapose for dramatic effect. For the sitting on the chair, it is expected to be associated with feelings comfort, yet we used a cat meowing sound which sound texture wise was very sharp, to throw the audience off and bring a certain element of shock and discomfort to them. As for the light switch and the toilet flushing, we felt that the “click, click” sound of the light switch turning on and off felt very rhythmic and repetitive, as such threw in the sound of toilet flushing as the texture of the sound felt very random and the swooshing sounds felt like a good contrast.
Characteristics of interface

As Sensorium is ultimately a participatory-driven installation, the viewers are in integral aspect to this installation. Thus on the continuums of interactivity, we would place “Sensorium” close to the zone or High Interactivity, where the viewer’s actions and feedback will ultimately determine the outcome of their experience with the installation.

Also, based on the characteristics of interface, we would view Sensorium as having its interface “parallel real world experience”. The nature of Sensorium plays on the disassociation between what viewers think they know and what is actually presented to them. As such our interface would be one that is very much based on real world items and objects. For example, in Sensorium a box with a lid, a chair and a light switch are selected as our main objects that viewers would interact with. These are common everyday objects that the viewers are familiar with its function and working. We then based the activation of the feedback loop based on these interactions. Thus, the sounds and ink drop will only appear as the viewer opens the lid, sits on the chair and flicks on the light switch . Thus we feel that Sensorium’s interface is one that parallels the real world in order for viewers to be intuitive in the way they interact with this installation.
We were also intentional to do our best to hide the LDR systems within the interface so as to not lose the “magic” created for the viewers.

For each object, we had to instruct our code differently based on the values read from the LDR (SensorValue). For instance, for the sitting on the chair, our requirement was that only if SensorValue <= 50 would the condition be true and hence turn the solenoid and sounds on.  This meant that only when someone sits on our chair and the light value dips below 50 will the condition be true. For the other two like opening the box and flipping the light switch, our condition was if SensorValue >= 100 because only if the LDR senses light, i.e box opening and also light turning on, will the condition be true, and the solenoid and sounds turn on.

We used what Lei thought us about Serial.println( ) in our code! Each object had different integers for Serial.println(_), as a form of communication between Processing and Arduino. Basically if for an object sensorValue>=100, Serial.println(L) is true. This integer “L” is then sent over to Processing and plays the chosen sound. We used Processing instead of an MP3 Shield because Processing could simultaneously take in three codes from Arduino and yet play all three different sounds at the same time. We made use of the minim library on Processing to play the sounds easily!

For Arduino:

For Processing:

some challenges and how we overcame them

To add on, we also ended up choosing objects that were more dynamic (opening box and flicking light switch), rather than just purely the action of picking up and putting down.



Jonathan – Head of Setup, Logistics and Concept

I think there weren’t huge challenges faced in this project but rather many small glitches and problems that occured throughout the process such as the technical aspect of getting the sounds to work. I was initially supposed to handle that area and we resolved to using a MP3 shield to play the sounds provided for the installation. However, the MP3 shield did not work alas due to some faults in the hardware and software. We decided to use processing to solve the issue in the end as Nasya had found a method to utilise it for our project.

There were many hiccups in the set ups as well. From the parts of getting the LDR to work during the set ups and how the droppers would actually run out of ink quite often.. but generally I think we were able to work around the limitations and created a very interesting and fun experience for our viewers. We were cracking our heads to come up with a strong disassociation between the objects and sounds but realised that an association could always be created regardless and that is actually a human condition as well – the tendency to draw connections and create associations. The experience created by the objects and sounds added a dimension of humour that we didn’t think it would bring and I thought that was quite interesting. 🙂

Daryl – Head Hardware, Arduino Technician, Aesthetic Advisor

When we first started the project, we were bent on created a big and extraordinary auditory visual experience in relation to synesthesia. However as we progressed through the project, we learn that synesthesia is more of a day to day experience which synesthetes have. Thus we worked towards the idea of giving everyday objects a different response in dissonance to the objects in question which we finally chose, the chair, a box and a light switch.

We encountered many little hiccups during the conceptualisation of Sensorium. Problems such as circuitry issues (we almost fried Nasya’s Macbook), programming issues, and also a lot of debugging be it in the software (Arduino, processing) or hardware (droppers, solenoids, mechanisms, we had to find the correct inks to use too).

We completed the project in the nick of time, and we were so happy that it all came together at the end. When our audience were testing and playing with Sensorium, creating the sounds and Ink clouds simultaneously it almost blew our minds. It actually turned out better than we expected. I feel that Sensorium has fulfilled its purpose: to create dissonance in everyday objects and their expected responses and thus portraying what a synesthete could potentially experience in his or her daily life.

Nasya – Head Programming, Processing Maestro, Arduino Extraordinaire

Overall the project felt like one very smooth journey! Each member owned their role and as such Sensorium was pieced together very nicely. I was quite amazed at how far 13 weeks got us, from knowing nothing about Arduino to being able to code according to what our project required. I remember initially it was very hard to code stuff due to just unfamiliarity, but as the weeks passed, it was easy to grow more accustomed to the coding language and be able to get Arduino and even Processing done. Here’s some work-in-progress! Could really see the improvements coding wise 🙂 A lot of the final codes were adapted, improvised to suit our needs and based on earlier codes that we learnt from class and from the Arduino Project Book!

Started off very simply with just the LDR being the input and having the buzzer as the output. (and we were very excited at that point that it was working)


Here’s our code from the initial buzzer and LDR adapted to fit the solenoid!

Here’s towards the end when we realised we needed a way to play the sound together with the solenoid movement thereby replacing the buzzer. Managed to get it to work with processing and we were ultra excited!

It was great that there was a progression, a growth toward our code, in that we did not suddenly write out a code overnight but rather it was based on looking through our code weekly and tweaking them to suit our project needs. Overall because of consistent work we managed to do the project well!

Lastly as a bonus, here’s the behind-the-scenes/ inside-the-box of Sensorium.

Thank you for reading 🙂

Pang gang lo~