“Are you trying to understand?”

When we listen to stories, we only listen to one side of the story. But often we fail to recognize this, and taking everything face value, thinking that what we heard is the fact instead of just a side to the story. Even worse – often we realize that, yet we don’t try to find out the truth. We make ourselves content with what we know. We overlook that lack of understanding, although we know it is there.

Based on the theme of “interstice of understanding”, we created project Door.

The Door

Group members: Dan Ning and Vania

Materials / Components

Inside of the cabinet

  • Cabinet (pre-bought and self-arranged)
  • Laptop
  • Power bank
  • Bluetooth speaker
  • Earphone
  • Arduino UNO
  • Breadboard
  • LED strips
  • Photoresistors

 

About the Narrative

Bluetooth speaker and earphones

 

Recordings were played using bluetooth speaker and earphones at two different points.

Since the theme that we explored revolves around “understanding”, we thought it’s apt to convey that using a narrative. We wrote down dialogues and split them into several parts of recordings after settling down on a base plot and characters.

In a nutshell, our story is a reverse Cinderella story – we used a base story that is familiar with most people. We all know that in the Disney version of the story, Cinderella was abused by her stepmother and her two stepsisters. Eventually, with the help of a fairy godmother, she managed to get a life she finally deserved.

In our narrative, we simply ask a question: What if Cinderella was never abused, but it’s just what people thought happened?

Through the dialogue, we conveyed the idea that Cinderella actually helped out due to her own will, to the point that people suspected she was being forced to work. In actuality, Cinderella got along with her stepmother and stepsisters well. We gave the stepsisters more personality by giving them two different perspective; one of them was very fond of Cinderella, yet scared to step out and actually tell the people off; another one didn’t exactly hate Cinderella, but not fond of her because she felt that Cinderella’s presence made their family suffer. She also grew to be upset towards her own sister (the other stepsister) as her sister didn’t say anything against the people.

If you listened to the recordings, you will realize the similarities and differences of the perspectives among the characters – who strongly opposed the people’s accusation, who hated the people, who hated Cinderella, who blamed themselves.

A final recording revealed that actually the people’s accusation came from Cinderella’s grandmother. She bore hatred towards the stepmother as she felt that she took everything away – her son (Cinderella’s father) and Cinderella. She didn’t spread the words just to abuse the stepmother, but rather because she truly believed that the stepmother was in the wrong. She thought she was saving Cinderella by doing so – which, as the other recordings suggested, was not the case.

Basically, the whole narrative showed multiple perspectives. Through this, we wanted to compel people to think critically of everything they heard.

List of the recordings:

 

Behind the Scenes

At first, we planned to use 2 Arduinos: one for the main components (photoresistors and LED strips) and one for the background components (PIR and background LED strips).

How they work

The main components will allow the LED strips to be turned off when someone comes near it, and light up under normal condition. It is to attract people to come to the right spots (near the speaker and earphone) so people don’t need to search the source of sound blindly. We used photoresistors as the sensor, so when light is blocked by the presence of someone, the lights will go off.

Photoresistor (the small dot on the door) and the LED strip to attract attention to that spot

 

The background components will allow motion to be detected using the PIR. When someone comes close to the cabinet, the background light will go off to focus attention to the holes where the LED strips for the spots are located. When someone opens the cabinet door, a final recording will play, stopping all the subsequent recordings (which will be played at random between the two sources of sound).

LED strip taped to the upper side of the cabinet as the background light

 

Change of plans

The PIR worked at first; however when we were coding and trying things out, the PIR suddenly just wouldn’t detect motions, although it was fine before. Since it was nearing the deadline, we decided to just scrap the PIR out and used external power supply (power bank) to light up the background LED strip. Because of this, we can’t let people open the cabinet, so to indicate that the cabinet is not to be opened, we tied a string on the handle. But conceptually, it can be a good analogy: how much effort are you going to put to find out the truth? Is a simple string going to stop you?

String tying the handles together

 

How it goes

Since our project is contained within a cabinet, it was easy to move it around.

After we installed our components into the cabinet, however, suddenly one of the photoresistors refuse to work. So we had to present with one of them not working, but afterwards, we solved the problem; we put a wrong value for the brightness setting in the code because the conditions are slightly different. Since the photoresistors were not working when we presented, it was hard to determine where the sources of sound were.

We constantly played the recordings on loop, but not loud enough for people to hear. It compelled people to come closer and put their ears right by the cabinet in order to be able to hear, so in a sense, it can only be used by one person at a moment. However I wouldn’t say that this is an individual experience, since although you “interact” with it individually, you can ask around on what other people hear and combine your stories to find out the truth. That, again, refers to the idea of how much effort are you going to put?

 

 

Characteristics

Level of interactivity

I think the feedback is more interactive since as soon as you approach the cabinet, there is a clear and immediate action that happened, which is the LED strips lights going out due to the change detected by photoresistors. It is visible even to people who are not interacting with it at that time.

The automation in our project is also not very high; the automated feedback that is set is only the photoresistor-LED light. We’re supposed to have the PIR and background light as well, but it didn’t work and because of that, I think the automation falls more to the passive rather than interactive side. There is still a certain amount of audience control as we used LED lights to guide them to where the speakers are exactly, navigating them to approach those spots instead of wandering around.

I also think there is an indirect collaboration to this project, in the sense that you can “exchange” the recordings that you heard with other people to reveal the actual story. But then again, this collaboration depends heavily on the audience, and may not necessarily happen.

About the adaptivity, I think our project doesn’t really possess automated adaptivity. It doesn’t possess memory that can provide feedback through continuous interactions, which will allow different actions to happen to counter whatever weakness was encountered throughout previous interactions.

The communications in this project is also very one-sided, since people can only listen to the recordings without responding back. This project doesn’t really have an identity or personalization, but rather more just like a platform that is different from usual, which allows people to “overhear” a conversation.

 

Characteristics of the interface

Firstly, it is kind of separated from real world experience since in order to interact with the project fully (that is, to listen carefully to the recordings), you will have to concentrate fully to the cabinet at that time or you will miss the sounds from the recordings since they are very soft. However, even after that the experience still may continue by exchanging the recordings with other audience, and that one will involve more real-world experience. That was our intention; to mimic the story to a real-world story so people can discuss about it outside the actual physical interaction with the cabinet, bringing it even to the real world.

The interaction is also intuitive, as the recordings would be played in a continuous loop. People will subconsciously be compelled to find out what the sounds are about, and they would go to the spots where the LED lights are turned on, where they can hear the sounds clearer. They can also tell intuitively that they’re not supposed to open since the handle is tied with strings. There is also a certain degree of subtlety as all the components is being contained in the cabinet where people can’t see them.

(That one refers to our final project that was presented, which scrapped the idea of revealing of final recording after opening the cabinet. But in case we did it and the cabinet could be opened, we would cover the components with wood anyway.)

However, the experience wasn’t monitored and used to provide feedback in the future; the installation has no memory. The experience is going to be the same no matter how many times people come to it; in a sense, the audience acts more like an observer. There is no selection at all and the experience is mostly linear; the only randomness comes from how we randomize the order of the recordings played, but in the end, they’re all the same recordings.

 

Conclusion

Individual parts

Dan Ning: main coding, research for material, doing the recordings, troubleshooter, physical installation

Vania: helping with the coding, main writer for the narrative/dialogue, editing the recordings, physical installation

 

Reflection

I think one biggest weakness that we didn’t solve is how to attract people enough to stay and listen to all the recordings. Moreover, we looped them at random, which means after the final recording, the other recordings will just start playing again immediately – there is nothing that distinguish the most important final recording from the others.

At first, the connections between the wires weren’t very secure since we used crocodile clips and taped them using masking tape, but then we switched to jumper wires.

I think we’re trying too hard to achieve something that is conceptually interesting instead of getting our basics down first, which resulted in a lot of problem in the coding process since both of us are not that well-versed in programming to begin with.

One of the biggest problem Dan Ning encountered was the setting of the sources of sound. We initially wanted to use more sources, but there were problems with the coding. Moreover, the USB speakers were not working, so we had to use bluetooth speaker and earphones – the experience when listening using both of them are vastly different, and she had to program the volume every time we set it up.

We were also kind of worried about the recordings since we asked our friends to do those, and they might not be able to convey the proper emotions (they’re not actresses, after all), but I think it turned out fine.

I think another problem that we have is that we don’t have backup plans; a lot of our things failed at the last minute, and we didn’t have spare components or backups, so we have to settle for something else (i.e. the bluetooth speaker and earphones instead of USB speakers, not using PIR since it didn’t work)

All in all, we encountered a lot of troubles. For me, this personally reminds me not to do things last minute (I do that a lot). We also learned to always have spares and back up plans, no matter how minute the problem may seem. Overall, besides learning more about coding and installation, we also learn more about group work and the importance of setting up specific deadlines for each person, so there won’t be any last minute things.

Based on Lev Manovich, there are five principles of New Media: numerical representation, modularity, automation, variability, and transcoding. Now I will talk about how they can be applied to my group’s project, the Door.

 

Numerical Representation

Basically, a principle of new media that distinguish it from old media is that new media is programmable. New media objects are essentially composed from digital code, and hence can be manipulated via algorithms. In my group’s project, the numerical representation comes from the codes that are embedded into photoresistors. We set a certain numerical value based on the level of brightness that is required to “trigger” the photoresistors, which will result in recordings being played.

 

Modularity

Modularity reflects the presence of separate elements or parts, which can be modified or used independently, that create a whole new media object. In my group’s project, I think it can be seen from the separate recordings that we are going to play; we can edit the contents of the recordings without affecting the main code, for example. Moreover, in the code that we made, there are also separate elements—for example, we can edit the timing delays or the brightness trigger for one photoresistor without affecting the other photoresistors.

 

Automation

Automation involves the computer programme to produce a response using template or algorithms by itself, without direct human interference. An example is web-based search engine. For my group’s project, I think there’s not much automated response as an audience has to come and trigger the photoresistors; in a sense, interaction from audience is constantly required for something to happen.

 

Variability

Another principle that differentiates new from old media is variability, which means something that is not fixed or possibilities that can be explored. In a sense, I think my group’s project doesn’t possess a great variability; the final outcome is fixed, no matter how many times an audience interacts with it, or who interacts with it. We do give the audience the flexibility of interpreting the story itself before revealing the actual story, but the storyline is more or less fixed which doesn’t allow broad interpretations; it’s more of a selection rather than variability, since the possibilities are finite.

 

Transcoding

Technically, transcoding refers to the translation of an object from one format to another, or from one platform to another. I think it could be related to my project in a sense that the movement of the audience will be “translated” into the recordings that will be played due to the trigger from the audience, but I’m not really sure because it’s not exactly a translation like a change from text to image.

 

In conclusion, I do think my group’s project possesses some of the principles of new media, but some of them are not very strongly reflected as there are limitations to how much they can be applied to our project.

This time, we attached the circuit into our cardboard prototype.

We did write the codes, but they didn’t work the way we wanted to, so for the presentation, we just tested out the sensitivity of the photoresistors. At first I was a bit worried that the photoresistors wouldn’t be a very good sensor, but they turned out to be more sensitive than I expected.

Feedback received from the presentation:

  • Use backlight instead of LEDs on the gap so it won’t interfere with the photoresistors
  • Add ambience noise (e.g. whispers) to attract audience
  • Play around with the volume

So now, the things that we need to focus on are:

  • Think of the materials we need and the exact amount of each of those
  • Start making the story: think of how many recordings to make and the length for each one
  • Settle the code

Video of the class presentation:

 

After that, I went back and tried to fix the code. We need to make a circuit where when the light that falls on the photoresistors is on the minimum (darkest), the sound will start playing. Since we don’t have ethernet shield to read SD card (is that what we need?), for now the sounds are just from the buzzer. Here’s the working circuit.

However, I’m still not sure whether we need the sounds to overlap each other when both of them are triggered or not, but for now there will only be one sound played regardless of the number of sensors activated.

As for now, I think we’ll need to just try the different things first and make sure the electrical component works.

 

Group: Vania, Dan Ning

Our idea is to portray a gap in people’s understanding by portraying them using physical gaps on a door.

Our prototype was kind of a miniature of what the actual thing is going to be. Our whole idea is revolved around a story that would be revealed after “exploring” around the door, but we didn’t have a story yet. We put earphones behind the cardboard door and played some e-book as substitute.

For the set of actions we required the participant to do, we only told the participant to approach the door and just try to listen carefully. At first she looked unsure of what to do since the sound from the earphone was very soft, but then she could still catch the sound. She began exploring different spots on the door and I tried to match that by playing different recordings as she moved, but I couldn’t keep up and she didn’t get what the recordings were about either, so it just confused her. But she did think that she felt like eavesdropping (which is kind of our idea) due to the low volume, so that’s good. She also said that the concept is interesting, and if the gaps do play different recordings or sounds, she would be interested to test all of them to find out. She also added that at first she wasn’t sure of what to do because the door was a plain piece of cardboard with no handle or keyhole, but if there is a handle and keyhole, she would know naturally that the participants are expected to open the door.

Feedback we received:

  • The door may not have to be real-life size, it can just be something like a dollhouse
  • The concept of revealing a story is interesting; maybe we can make the door have several screens inside, so you can open it up like pages of a book to fit in the concept
  • We can make use of the gaps between door and door frame to put in speakers, but we can also put in a lot of keyholes instead so participants will have to continuously find keys to progress with the story. Moreover, it will be visually intriguing.
  • We can also make use of different heights of people; so maybe in one big door, there are smaller doors. The different gaps will play different people’s sounds, e.g. a small door will play a child’s sound and a big door will play an adult’s door. So we can make a story but through different perspectives.
  • One-person experience would be better. If the door is placed in an empty room, people will definitely be compelled to approach the door and find out more about it.

As for now, what we have to do based on the feedback is to focus on what we want to do first. Now we received a lot of feedback because we still have a lot of things we can play around with, and we still haven’t decided which idea to use exactly. After we decided on one idea, only then we can move forward with the project.

I do think this bodystorming helped a lot because the feedback we received are really valuable. They provide objective opinions and point out the strength and weaknesses we didn’t see in the beginning. At first I was really worried that people won’t be interested to find out about the door, but the participant told me she would be interested and even gave her idea on how to improve on that. Moreover, I can see other people’s works and learn from their troubleshooting as well.

Group members: Vania, Dan Ning

Watch our bodystorming process here:

I’ve never gone to Night Light Festivals previously, so this is a new experience for me. It was fun – my friend and I walked from place to place, exploring the installations. They are very spread out, which may be good since people have more places to explore, but it makes it difficult for people who want to go through everything in one go. I visited several of the installations, and I think they look magnificent.

 

Hyperbands by KopI/O

Pulse by Galina Mihaleva, Hedren Sum, Pat Pataranutaporn, Kathrin Albers, Audrey Ng

Graffiti Alive by Arup

Ember Rain by Starlight Alchemy

That being said, I have chosen two installations for me to talk about.

Orbit

By LiteWerkz X 3M

This is the first installation that I went to. From afar, I could see a group of spheres – some of them lighting up, some of them don’t.

As I walked closer, I could see the see-through spheres, with quirky geometric shapes inside. The spheres were lined with lights so that we could see the insides of the spheres even in the darkness. Circular shapes were adorning the surface of the spheres. In the middle of the spheres, there is one sphere that is different from the others – it has circular objects that light up instead of geometric shapes inside the see-through sphere. A nearby sign informed us to try taking picture of the spheres with a flash.

At first, I took pictures without flash first.

After that, I tried taking picture with flash. The picture was so different from what I expected:

Upon closer inspection, I found out that the circular-shaped stickers were actually made out of 3M retroreflective materials. When given light (i.e. the camera flash), the stickers reflected light and gave the effect as if they were sources of lights (!). That’s the interactive part of the installation. Moreover, I think there’s motion sensors inside the spheres that triggered the lights inside the sphere, so the lights will only light up when you spin the sphere. (I wanted to post a video, but it’s so hard to post a video in OSS, i.e. I can’t.)

It reminds me of a galaxy (and apparently that was the intention of the installation, based on the description put in the National Heritage Board website), not in the appearance only, but also because it gave me a sense of wonder when I tried to find out how the thing works.

I think the installation is interesting for that reason, and also that the design of the geometric shapes are all unique; there’s also a center piece which is a bigger sphere that attracts attention although it actually works the same way as the others. However the fact that everything works in the same way may dull people’s interest quickly, and people may not take their time to see the spheres one by one; they may just check one or two and leave (that’s what I did). In addition, it might not look very good when no one is interacting with it.

 

The Leap of Faith

By Teng Kai Wei

Even from a distance, I could see the lights from the installation. At once, it captured my attention. My first thought is that it looks beautiful, and it reminded me a bit of hopscotch. From the explanation board, I found out that the intention of the installation is to encourage people to take a “leap of faith”, to trust in yourself while going on a journey to reach your destination.

The anatomy looks simple; it seems that the plates are equipped with pressure sensors, so when someone steps on it, the color will change. Moreover, the original color of the plates will change constantly; hence if you step on it after some time, the color change will be different. I found that interesting, and may compel people to re-do the experience. In addition, the color change is unique as well; when you step on it, the color effect will resemble a ripple.

There’s also some kind of sculpture in the middle with geometric-shaped lights. I am not sure what the intention is, but it probably acts as a center, which could also be used to represent the “destination”.

Anyway, it’s really interesting to see different people interacting with it; I could see how different people can be. Little kids jumped from plate to plate excitedly, exploring every single plate although the resulting light would be the same; adults stepped on each plate carefully, looking around, deciding where to step next. Some people travelled back and forth; some people just walked across nonchalantly, even stepping down on the ground when they needed to make a jump across the plates.

I enjoyed playing around with that installation. I do think it would be better placed over water to actually force people to step on the plates, but I realize that may be dangerous and more difficult to set up. The lights were also constantly changing, which made the installation looked beautiful without anyone interacting with it, and noticeable even from a distance.

 

All in all, I found the installations really intriguing. I could see how even with instructions, people interact differently with the installations. I could also see how important it is to compel people to interact with the installations, and that depending on how they could interact, people could easily get bored by the installations or may not even be interested to interact at all. I realize I’m not familiar with the anatomy of the installations, which left me guessing a lot of times. However, I really enjoyed looking at those; definitely would go there again.

To be honest I don’t know a lot of interactive media projects. The ones that I know the most are video games, so one of the projects that I’ll bring into light here is a video game titled Detroit: Become Human. It’s quite a recent PS4 adventure game in which the choices of the players will shape the storyline and the ending of the game. You will play three different characters interchangeably, which will expose you to different issues and situations.

I think the game is really intriguing because of the vast possibilities it offers. It’s not a simple life-or-death choices, but there are possibilities of things in-between as well, which make the experience of playing the game more wholesome because you can see how each of your choices impact the story. Besides the experience itself, I also think that this game is interesting because it addresses real-life issues and makes you think more critically.

 

Here is the trailer for the game:

 

The second project that I’ll be talking about is musical stairs. I think musical stairs is not a very rare idea, since it has been implemented at different places all over the world. You simply have to step on the stairs to produce different tones. However, that is what makes it interesting; it can be enjoyed by anyone. Moreover, it also promotes a healthier life by encouraging people to use stairs rather than elevators.

 

Here is an example: