Sensorium.

SENSORIUM

IM showcase video

IM showcase gallery

Project Description

Sensorium questions the interstices between our senses that we often take for granted through the sensory phenomenon known as “Synesthesia” – a condition where a stimulation of one sensory or cognitive pathway leads to automatic and involuntary experiences by other senses. In short, an experience of a single sense is now simultaneously perceived by other senses resulting in confusing outcomes for the individual with this condition.

In this installation, we aimed to make this gap more aware by recreating this condition by allowing visitors to experience a similar disassociation between their senses by creating a sense of unfamiliarity with objects that they are familiar with interacting. This “sensory overload” is created through the unfamiliar and different sounds heard and coloured “ink-drops” seen dripping into the tank when interacting with the objects on the stools.

Concept – artist references

We were also inspired with artworks that engaged different senses and emitted sounds. Through interaction some of our references are as follows:

Lenses by Hush

A installation that converts light sensors and refractions to sound.

Synesthesia Installation

Another installation that plays on the idea of engaging different senses in the form of Synesthesia.

In our initial ideas, we were focused on using lights and sounds to create the experience of Synesthesia but a closer study into the condition, and consultation with Lei, we decided to go with something more tactile and allow the installation to be more object-driven. Settling on the sense of touch, sight and sound.

Objects used and sounds that corresponded:

Box with Lid (with flowers on the inside) – Thunder
Chair – Cat meowing
Light switch – Toilet flushing

As for the sounds we chose, we followed what Lei suggested about considering the “textures of the sound”. To further elaborate, it was like how for the box with lid (with flowers inside), the expected sound would be something very soft and soothing. Whereas for that we used the sound of thunder, which sound texture wise was very loud and harsh to further juxtapose for dramatic effect. For the sitting on the chair, it is expected to be associated with feelings comfort, yet we used a cat meowing sound which sound texture wise was very sharp, to throw the audience off and bring a certain element of shock and discomfort to them. As for the light switch and the toilet flushing, we felt that the “click, click” sound of the light switch turning on and off felt very rhythmic and repetitive, as such threw in the sound of toilet flushing as the texture of the sound felt very random and the swooshing sounds felt like a good contrast.

Characteristics of interface

As Sensorium is ultimately a participatory-driven installation, the viewers are in integral aspect to this installation. Thus on the continuums of interactivity, we would place “Sensorium” close to the zone or High Interactivity, where the viewer’s actions and feedback will ultimately determine the outcome of their experience with the installation.

Also, based on the characteristics of interface, we would view Sensorium as having its interface “parallel real world experience”. The nature of Sensorium plays on the disassociation between what viewers think they know and what is actually presented to them. As such our interface would be one that is very much based on real world items and objects. For example, in Sensorium a box with a lid, a chair and a light switch are selected as our main objects that viewers would interact with. These are common everyday objects that the viewers are familiar with its function and working. We then based the activation of the feedback loop based on these interactions. Thus, the sounds and ink drop will only appear as the viewer opens the lid, sits on the chair and flicks on the light switch . Thus we feel that Sensorium’s interface is one that parallels the real world in order for viewers to be intuitive in the way they interact with this installation.

We were also intentional to do our best to hide the LDR systems within the interface so as to not lose the “magic” created for the viewers.

TECHNICAL DESCRIPTIONS

For each object, we had to instruct our code differently based on the values read from the LDR (SensorValue). For instance, for the sitting on the chair, our requirement was that only if SensorValue <= 50 would the condition be true and hence turn the solenoid and sounds on. This meant that only when someone sits on our chair and the light value dips below 50 will the condition be true. For the other two like opening the box and flipping the light switch, our condition was if SensorValue >= 100 because only if the LDR senses light, i.e box opening and also light turning on, will the condition be true, and the solenoid and sounds turn on.

We used what Lei thought us about Serial.println( ) in our code! Each object had different integers for Serial.println(_), as a form of communication between Processing and Arduino. Basically if for an object sensorValue>=100, Serial.println(L) is true. This integer “L” is then sent over to Processing and plays the chosen sound. We used Processing instead of an MP3 Shield because Processing could simultaneously take in three codes from Arduino and yet play all three different sounds at the same time. We made use of the minim library on Processing to play the sounds easily!

For Arduino:

For Processing:

some challenges and how we overcame them

To add on, we also ended up choosing objects that were more dynamic (opening box and flicking light switch), rather than just purely the action of picking up and putting down.

INDIVIDUAL ROLES AND REFLECTIONS

Jonathan – Head of Setup, Logistics and Concept

I think there weren’t huge challenges faced in this project but rather many small glitches and problems that occured throughout the process such as the technical aspect of getting the sounds to work. I was initially supposed to handle that area and we resolved to using a MP3 shield to play the sounds provided for the installation. However, the MP3 shield did not work alas due to some faults in the hardware and software. We decided to use processing to solve the issue in the end as Nasya had found a method to utilise it for our project.

There were many hiccups in the set ups as well. From the parts of getting the LDR to work during the set ups and how the droppers would actually run out of ink quite often.. but generally I think we were able to work around the limitations and created a very interesting and fun experience for our viewers. We were cracking our heads to come up with a strong disassociation between the objects and sounds but realised that an association could always be created regardless and that is actually a human condition as well – the tendency to draw connections and create associations. The experience created by the objects and sounds added a dimension of humour that we didn’t think it would bring and I thought that was quite interesting. 🙂

Daryl – Head Hardware, Arduino Technician, Aesthetic Advisor

When we first started the project, we were bent on created a big and extraordinary auditory visual experience in relation to synesthesia. However as we progressed through the project, we learn that synesthesia is more of a day to day experience which synesthetes have. Thus we worked towards the idea of giving everyday objects a different response in dissonance to the objects in question which we finally chose, the chair, a box and a light switch.

We encountered many little hiccups during the conceptualisation of Sensorium. Problems such as circuitry issues (we almost fried Nasya’s Macbook), programming issues, and also a lot of debugging be it in the software (Arduino, processing) or hardware (droppers, solenoids, mechanisms, we had to find the correct inks to use too).

We completed the project in the nick of time, and we were so happy that it all came together at the end. When our audience were testing and playing with Sensorium, creating the sounds and Ink clouds simultaneously it almost blew our minds. It actually turned out better than we expected. I feel that Sensorium has fulfilled its purpose: to create dissonance in everyday objects and their expected responses and thus portraying what a synesthete could potentially experience in his or her daily life.

Nasya – Head Programming, Processing Maestro, Arduino Extraordinaire

Overall the project felt like one very smooth journey! Each member owned their role and as such Sensorium was pieced together very nicely. I was quite amazed at how far 13 weeks got us, from knowing nothing about Arduino to being able to code according to what our project required. I remember initially it was very hard to code stuff due to just unfamiliarity, but as the weeks passed, it was easy to grow more accustomed to the coding language and be able to get Arduino and even Processing done. Here’s some work-in-progress! Could really see the improvements coding wise 🙂 A lot of the final codes were adapted, improvised to suit our needs and based on earlier codes that we learnt from class and from the Arduino Project Book!

Started off very simply with just the LDR being the input and having the buzzer as the output. (and we were very excited at that point that it was working)

Here’s our code from the initial buzzer and LDR adapted to fit the solenoid!

Here’s towards the end when we realised we needed a way to play the sound together with the solenoid movement thereby replacing the buzzer. Managed to get it to work with processing and we were ultra excited!

It was great that there was a progression, a growth toward our code, in that we did not suddenly write out a code overnight but rather it was based on looking through our code weekly and tweaking them to suit our project needs. Overall because of consistent work we managed to do the project well!

Lastly as a bonus, here’s the behind-the-scenes/ inside-the-box of Sensorium.

Thank you for reading 🙂

Pang gang lo~

October 23, 2018

Making Sense of Sensorium. — Principles of New Media.

In his book “The Language of New Media”, Lev Manovich attempts to contextualise and analyse the new media revolution and the “shift of all culture to computer-mediated forms of production, distribution, and communication.” Which has had an immense impact on all stages of communication, and it causes us to rethink of our definitions and values when comprehending the media of today as compared to that of old.

Under the chapter of “Principles of New Media” he reduces all principles of new media to these fundamental to five — numerical representation, modularity, automation, variability and cultural transcoding. As so aptly summarised, these principles “should be considered not as absolute laws but rather as general tendencies of a culture undergoing computerization”

Manovich’s principles offer us some handles in helping us analyse the function and systems of our projects and how it relates to the topic of new media, which I will elaborate further on. But before that, I would like to elaborate more on the concept and mechanism of Sensorium.

Sensorium explores the sensoral phenomenon known as “Synesthesia” in which a stimulation of one sensory or cognitive pathway leads to automatic, involuntary experiences in a second sensory or cognitive pathway. In short, when an individual’s particular sense is stimulated, the individual may experience multiple or an alternate sensory experience. For example, indivuals may sometimes “see” a certain colour in response to a certain letter or word.

This is achieved by allowing viewers to experience a “sensory” overload and a disconnection with common objects and behaviour much similar to Synesthetes. Viewers would hear certain sounds, and see ink-drops (of a particular colour) when they interact with the work. This is done through setting up light-sensors beneath the object that would respond to the changes in the object placement and provide a feedback accordingly.

Now back to Manovich’s principles and how does it relate to Sensorium?

1. Numerical Representation

Manovich describes that all new media objects are composed of digital code and thus can be subject to algorithmic manipulation. In short, media becomes programmable.

Similarly, Sensorium utilises Arduino system that are programmable as the brain of the art work. In addition, the use of LDR sensors also mean these readings are being actively translated into binary data script for the Arduino system to process.

2. Modularity

New media objects are ‘object-oriented’ composed of parts made up of smaller parts reminiscent of a “fractal structure”. These elements are assembled into a larger scale object but continue to maintain their separate identities. This also allows for augmentation of the smaller areas and portion of independent parts.

The different mechanism of Sensorium is modular and is made up of smaller parts. This includes the code, the software, the hardware or even the way the installation is experienced.
For example, in terms of hardware, the receiver is modular in that the use of an LDR results in a particular experience for the viewer. (The selection of this component by us as the main receiver) By changing the receiver, such as using a different sensor it would alter and augment the set up of the work.
However, what is unique about our project is that this would not ultimate affect the outcome of the experience as the output (the colour dropper and sensor) remains the same.

The modularity allows us to control and decide on the type of experience and objects that we want to create for our viewers.

3. Automation

Another characteristic of New Media is its ability for automation as a result of the modular structure and numerical coding where the computer now takes over the role of operations.

Sensorium uses an Arduino system to allow for Automatic feedback and responses to be generated for the user. This allows for the system to run with the absence of a human to physically change the system accordingly.

This automation is limited however, due to certain contraints in the system in Sensorium created. Such as, the need to top up ink fluid in the dropper and also replace the water in the tank once it becomes too murky and cloudy.

4. Variability

Manovich mentions how “a new media object is not fixed once and for all, but something that can exist in different, potentially infinite versions”

In this principle, Sensorium allows for a diverse range of variability despite its simple output. The system of Sensorium is fairly direct. In short, it invites viewers to move and interact with certain objects. For example, should a viewer lift a box, this would trigger the light sensor and cause the ink dropper to release ink drops (coloured) into the tank. A sound would also be heard. Regardless of how high you lift up the object, or interact with, the output remains the same. However, certain factors that would affect the variability in the experience of the work includes:

– Duration of the object being held. The longer it is lifted up, the more times the ink drop would occur and the longer the corresponding sound would be heard.

– Prior participants. The presence of other viewers would mean that the ink in the tank would be visible to the viewer and thus affect the experience of the viewer.

– An amalgamation. Due to the nature of the system, all 3 inputs can be happening at the same time and would allow for trigger of multiple drops to happen at once. The way in which the ink drop clouds drops and forms also is another variability of the project.

5. Transcoding

A digitisation of what we define as ‘culture’. Manovich talks about the interplay between both the “computer layer” and the “cultural layer” of this digitisation. And the combination of the two results in new experiences created.

When computerised, the experience is changed when the reaction creates a disassociation, of what is known and replacing it with something that detaches them from the space.

Sensorium plays on these experience, through a similar disassociation. There is a disconnection experienced (between the object seen, touched as well as sound heard and the colour ink seen). The physical is being digitised through the sensors and is being read as data and the subsequent result or feedback is converted again to a physical experience to be felt. As such, what we come to know or think, through a disassociation, creates new meanings.

Other Works that inspired Sensorium:

Lenses by Hush

A installation that converts light sensors and refractions to sound.

A Synesthesia Installation

A tactile installation that uses motion and touch to create light, colour and sounds as outputs.

August 28, 2018August 28, 2018

Singapore Night Festival Installations

Orbit by LiteWerkz X 3M

This installation occupies a fairly large area, with transparent orbs/spheres lit up arranged around a fairly larger sphere in the center. The idea is to liken the space to that of the solar system in space hence the name ‘Orbit’. The main form of interaction would be that audiences are invited to touch/move/spin the spheres. The spheres/planets would respond to those movements and touch by changing its light colour or flashing in different light patterns.

The installation offers audiences an immersive experience due to its set-up and scale. Just entering the space and looking at a field of lit up, semi-floating spheres was a beautiful sight. While visiting the installation, there were many others (mostly kids) playing with the installation and from a macro perspective, the lights flashing all at different times and spinning at different speeds creating a diverse imagery for audiences to behold. In a sense, while audiences are interacting with just a single sphere, there are also changing how the installation is being perceived as a whole as well.

I think the intention created for the audiences was to invoke that sense of wonder and awe at the unchartered beyond of being in Space. Likening the experience to that of orbiting and playing amongst the shining stars.

Upon a closer study of each sphere, I was able to identify that each sphere was mostly made up of LED strips attached within and I would think a motion sensor that picks up on the movement made to the sphere. The electrical circuits and wiring are hidden beneath the sphere in the rod that holds it in place which connects the sphere to the electrical plug and supply.

Graffiti Alive by Arup

So, having seen the critiques made by others in the class, I decided to choose an installation less spoken of and also an installation I felt was underwhelming compared to the other exhibits at Night Lights. Maybe from here I could draw some learning opportunities of some points to consider in my final project.

Situated at Armenien Street (beside the Substation), Graffiti Alive is a location specific installation that makes use of the graffiti art works along Armenien street and through motion sensors and lights, allow audiences to experience these works differently and see it in a new light (pun intended).

Image from: http://theunpublishables.com/wp-content/uploads/2018/08/SNF-Graffiti-Alive-1©-Gwen-Sin-1024×683.jpg

As I was visiting the installation, I saw a whole bunch of bulky equipment at the start and end of the street, accompanied by bright lights being cast on the graffiti wall. I did not quite understand the exhibit or the workings of it until I spoke to one of the volunteers present that I was suppose to work through the street and as I walked, the lights on the wall would then change colours and respond. I tried walking back and forth the street but there were minimal changes to the lighting. (It could have been due to the fact that there were others visiting the exhibit and could have messed up with the motion sensors)

I think the intention of the artist was to allow a personal experience for the audience to view and observe each graffiti art work in a new way and also provide some form of exposure to the street art scene in Singapore as Art like graffiti could only be enjoyed in the day due to lighting. By illuminating the space, the Artist wanted these works to be viewed and enjoyed in the night as well.

The set-up of the space was fairly elaborate with two huge balloon-shaped equipment (which I suppose is part of motion sensors) at the start and end of the street. As well as RGB spot lights set up opposite the wall. There were other smaller motion sensor devices placed along the wall as well.

Overall, I felt this installation was underwhelming as I thought that the artist would attempt to use some form of projection mapping on the wall to make the graffiti work come ‘alive’. I think that would allow me to experience the graffiti work differently instead of just different RGB lights cast on the artworks at different times as I walk across. Another would be how to best make use of motion sensors. I think it would be best to consider how people would and could they interact with the artwork and ensure that the system (the processes) would still work regardless. I wasn’t sure how the motion sensors would respond to the influx of a higher number of people and how the light cast on the wall would change as a result. But I suppose the bright lights did draw my attention over to this exhibit, but overall I still felt that it missed the mark for me compared to the other exhibits.

August 14, 2018

Introduction to Interactivity

GS Caltex Pavilion

by: Atelier Brückner studio

http://www.atelier-brueckner.com/en/projects/gs-caltex-pavilion

Created for Expo Korea, the GS Caltex Pavilion was designed to take the form of an over-sized rice paddy fields with tall-illuminated glass blades swaying like grass in the wind. Users are invited to walk through the larger than life installation and interact with the grass by touching the blades and watching it change colours.

The size of the installation adds to a sense of wonder as guests wonder into this larger than life exhibit and experience a new perspective. I think the colour, lights and space of the installation plays into the intention of creating a sense of awe and wonder in the concrete jungle.

Out of Bounds

by: Chris O’shea

Out of Bounds is an interactive installation that allows its viewers to peer through walls to explore hidden areas, engaging in ‘behind-the-scenes’ experiences in the Museum using an “X-ray Torch”. Participants simply shine their Torch around the walls and they are able to view the scenes behind.

The intention of the exhibit plays on our inner child-life ability to want to be able to have super-powers like X-ray vision. Coupled with the set-up of the exhibit, it allows the viewers to tap onto their curiosity and explore with the X-ray torch.