Principles of New Media

In Lev Manovich’s ‘The Language of New Media’, Manovich identified 5 different principles of new media. They are:

Numerical Representation
Modularity
Automation
Variability
Transcoding

According to Manovich, works that uses new media would follow these principles. Would our artwork, Inter-macy (WIP name), be considered a new media work?

Inter-macy converts analog, continuous data like heart rates and light intensity into digital / discrete data, digitising a supposed spectrum of inputs into a sample and quantified into a value from 0 to 1024. The computer is able to run these discrete data in the digital code in the Arduino software, which uses algorithms and mathematical functions to run. Through this analog-to-digital conversion and programmability of Arduino, the work is considered to have numerical representation.

Inter-macy contains parts that are Modular, in terms of the different circuit paths that connects to different components that collects discrete data, which can always be removed or added. This is also true for the code that is used to run the components, where sections or lines of codes can be edited. Individual components, wirings, resistors, and objects like gloves or the LED strip can be replaced whenever needed, and it does not have to be a 1 for 1 replacement, meaning that a flex sensor can be replaced directly with a photocell sensor. At a larger scale, all the components lead to an output that gives feedback to the participant as they interact with the artwork and with other participants. The participants are also considered to be part of the modularity of the work. All these modular pieces work individually, but on a macro scale, works like a fractal structure that performs a bigger function as a whole.

Wires and components that are just plugged in and can be changed any time

Inter-macy also use many low-level Automation, mostly in terms of how the code runs in a loop with different settings when different input is present. The LED light is coded to glow on its own when the artwork is idle, and then does a variety of things when there is an input. The detection of pulse and light is also automatic through the component itself, as they are created to do so. Through the code, the inputs and outputs run on its own, and that is where human interaction is enhanced as the automation makes it easy for the participants to focus on the interaction. Another form of automation in the work is media access, which is in the code itself as they are taken from examples in Arduino thanks to the internet.

Inter-macy has Variability as the modular parts can be recreated and changed into an infinite number of possibilities through adding or subtracting of components to scale it up or down, modifying the code to do different functions from a tiny colour change to making the entire LED light up differently. Even, through the different kinds of interaction that different people will have with the artwork or the different locations, spaces, and uses for the artwork, in which its interfaces will vary even from the same sets of data.

video: As can be seen here, there are a few variabilities here with the components. This is still a prototype, so there are still many possibilities.

Variability also reminded me of John Cage’s Variation series where he sets up a stage made of motion sensors that a dancer can move around in, creating music composed by the dancer’s body as the dancer move about.

I can also draw many similarities from this example, particularly how modular it is and how variable it can be, in the way that the parts can be disassembled, reassembled, and moved from place to place.

Finally, Inter-macy has some elements of Transcoding, as the Arduino code file is written in computer language and its information is organised in its own ways, represented in the what ‘human culture’ could understand through the code itself which we can read and write, and through tangible outputs on the LED. The code can possibly be transferred into another format, like if it were to be on another program and component like Python and Raspberry Pi, although I don’t know how to do so. This principle works because in the computer culture, these codes have similar usage and can be understood in the different programs.

Thus, with all these principles applied to our artwork, Inter-macy can be considered as a new media work.

Edit: after presenting, I realised how I misunderstood variability, as I had interpreted it as that there is always potential for variability versus what variability actually means which is variability that is achievable within the work itself. As such, I would say, there is still variability in my project as there are different heart rates and different kinds of interactions between the two participants, creating different outputs.

I was quite enlightened when I was told that the book was written 20+ years ago. Back then, these principles were probably something so new to many people, yet now we take them for granted. Although the reading feels outdated, there’s still some kind of respect I could have given to the book while reading it and not just dismiss it as ‘common sense’.

21 March, 201922 March, 2019

Mid Terms

Inter-macy

Project Description

Our project on Interstices was initially about the physical interstice between the human body, both between parts of a human body and between two human being. We wanted the participants to be aware of the hidden areas around their body, as well as the kind of spaces one can find in another person’s body. This leads to the idea of intimacy, which branched off into another idea of letting people become aware of the interstice between human relationships. The non-physical space between two humans is invisible, but can be felt by the emotions and body’s natural reactions like heartbeat or sweat. We wanted to look at such interstices and perhaps use our work to not just show them, but also allow the participants to learn about themselves, and maybe start a new relationship with another human being. We wanted this work to not just talk about interstices, but also let the participants learn something meaningful about themselves and others. This is why we approached the project with the idea of having two people interacting with one another. In terms of interactivity, we would say our work is somewhere in between reactive and interactive, as the work reacts to the inputs of the participants, but the participants are free to interact with one another, through this work.

We also wanted it to be a multi-sensory installation, so we are thinking of not just touch, but also sound and lights.

development sketches

development circuits

development codes

[if needed we will upload!]

Mid Term Review

During the mid term review, we only managed to create the circuit as we were still trying to figure out the code to allow for the function we desire. As a result, our testing were not optimal, though, we gave the participants some heads up so they are able to interact with it better.

Here’s our notes:

Experience
– Participants try to increase and decrease heart rate to see feedback (tried to hold breath, jump around)
– Intrigued by a green led on the finger and also pulsing of blue and red LEDs
– People started moving fingers to generate feedback

Participants feedback
– What they think it means: Heartbeat, the relationship between 2 users, pressure, vulnerability because making something invisible visible, something very private
– Keeyong’s experience tainted because can see wires n techs, felt a bit like medical checkup
– Lei thinks that context is very important, difficult to communicate idea
– Lei intrigued by idea of touching hands and aggregated heartbeat/ interstices between 2 people
– Audience reacts better with wows to led strip beating to pulse
– Interstices between two people and two heartbeats: Lei “combine 2 interstices instead of highlighting” “collective light and sound sculpture that is contributed by interstices and heartbeat between 2 people”
– Celine and Alvin did not know intention at first and wriggled fingers because she didn’t know it was pulse influenced
– Different ppl have diff size, could affect contact point for each person: design problem
– How do people know how to use it? People have to take the first step, and to take that leap to participate in it (bc for the longest time we have been staring at paintings), how else could you facilitate the intuitiveness
– Couples therapy hahaha
– Different parts that combine into one whole system instead of one whole glove to fix fitting problem
– The more the two parties interact with each other the more the sculpture will blossom
– “Clap my handsssss very harddddd” lma0
– Maybe hi 5s? Different kind of interaction? As opposed to strangers and friends with different relations.
– Progressive interaction? Start w high 5 then progress into touching palms
– How to guide participants? Maybe retractable string & 2 same hands, maybe place them on stands
– How do u even motivate people to do the hi 5? How to encourage strangers to do it
– Put them in a box, outsiders dk what is happening inside it, and they r already in the box so they have to touch it, and it creates a reaction and a sound

changes to ideas

The biggest issue discussed was how do we get people to come together to interact with one another. What was observed during the mid-term review was that our participants interacted with the glove on their own and did not attempt to interact with one another. Thus, Melo suggested this idea where we can have the interaction happen underneath a box, which puts participants in a position where they have to interact with one another. The participants then can interact whichever way they wish with their hands in the box, and would be able to see the sculpture light up and react according to the interaction. This makes the artwork easier to approach and may make it more meaningful for participants, to have a connection hidden in a box while creating art with the sculpture.

We like this idea a lot as it solves our problem, which is how much freedom should we give to the participants. It also adds some depth to the installation in the way the participants interact with each other, as they are now able to focus their attention in interaction using just their hands. In a way, adding control helped us make the installation more interesting.

Conclusion

With this new idea, we decided that we will remove the flex sensor as the wrist movement wasn’t moved as much when in a box. We wanted to have the flex sensors on the fingers instead but flex sensors were too expensive, so we decided to replace them with photocell sensors instead, where the hand interaction will cover the sensors, creating different inputs.

15 February, 201915 February, 2019

Body Storming

Group: Bryan, Fizah

A recap on our concept:

Wings that expresses itself through LED lights, motor movements, and sound
Heart rate detector on gloves that changes the LED light
Movement and interaction with other people will make heart rate change, altering the LED outcomes
Ultimately, interlocking fingers will cause the a different set of reactions on the wings

Last week, we spent 25mins building a quick prototype using tapes, wires, paper, and some random materials lying around. With the limited time we have, we only could make 1 set of wings for 1 person, where we initially wanted to have 2 interacting people.

We had Syahrul as our tester. Here is our instructions:

Here is the video of what happened:

So Syahrul was hesitant to wear the prototype due to how fragile it looks. When he do so, he started interacting based on the feedback he is receiving from the beeping and the shaking of the wings, despite there being lights too. This proves that wings are not the greatest idea as the tester could not see its effects.

We also found that Syahrul was interacting with other non-participants and objects around him. This problem could have been solved if we made another set of wings and gloves for him to interact with. Interestingly, he only interacted with people with the hand that had the gloves on.

Other concerns are related to the safety of the artwork and people around, where wearable artworks can easily be taken home by accident or damaged by wearing.

Here are the feedbacks from the notes taker:

What did we learn:

We missed out the effect of external factors that could impact the actions of the participant. During the Body Storming exercise, Syahrul started interacting with other objects in the surrounding.

What we need to ensure for our final project is that the audience should be able to identify what they should be interacting with.

What surprised us:

It became natural for Syahrul to only interact with his his surroundings using his gloved hands.
Syahrul also expected instant feedback from his actions through constantly tapping his gloved hands. He also tried every possible interactions he could do on his surroundings

How we can apply to designing our installation:

We learnt that wings are not very effective as the participants cannot see the effects. We can use a cloak or blanket instead. We need to make 2 wearable pieces for 2 people to interact with each other, if not they will interact with others. We have to make the 2 of them interact with each other and only with each other.

31 January, 2019

Research Critique II

I Light Singapore 2019

I went to I Light during its opening on Monday with Fizah. It was like our first I Light experience and we started walking from the beginning of the trail. There are many interesting installations that were really good looking, but lack interactivity. Amongst all the works, I picked out these 2:

Work #1: Facey Thing by Uji Studios (New Zealand)

Image taken from https://www.ilightsingapore.sg/Discover/Installations/Facey-Thing

Curious audiences getting their faces enlarged while others have created ‘marks’ that are left behind due to their interaction

Link of work: https://www.ilightsingapore.sg/Discover/Installations/Facey-Thing

This artwork is a giant LCD screen that has a camera facing in the direction opposite the screen creating some kind of digital mirror. It was created as a ‘fun and satirical celebration of … selfie culture and universal surveillance…’ commenting on how facial recognition has become such a norm that we use it for our own enjoyment rather than worrying about how a computer is smart enough to detect our presence and as such is allowed to spy on us wherever we are.

The artwork allows for some lower form of interaction where the audience only gets to see themselves on the screen and react to their face being blown up in size. Audiences can also paint the screen with their face, as their blown up face will freeze in frame after a while. The audience thus gets to either react negatively by pulling back, or engage more with the artwork to see what they can do with it.

Audiences have to come to terms with their face being exposed and enlarged in open public, allowing the audience to consider or weigh their own privacy over the sake of art-making. It also discusses the narcissistic nature of our current society, how we are not afraid to show our faces to the world.

Video of Fizah and my interaction with the artwork:

Work #2: Key of Light by Mr. Beam (Netherlands)

Image taken from https://www.ilightsingapore.sg/Discover/Installations/Keys-of-Light

This work is a piano that sits under a lit pavilion, facing a projection-mapped water tower. Audiences get to play the piano, which in turn causes the light in the pavilion to dim, as though it is a concert, and a projection will appear on the water tower in front of the piano in response the the note that is being played. The trees will light up in response as well.

The audiences get to interact with the artwork by first playing the piano. What is interesting is that here, the audience automatically becomes a performer, and is part of the artwork and affects the artwork directly. The piano gives not just audio feedback, but also visual feedback in the form of visuals on the water tower and lights. By combining different notes, the visual feedback changes, and this prompts the performer to try out different ways to play the piano. This creates a feedback loop between the artwork and the performer. Other audiences can watch the performance.

The interaction seems to be intended to allow audiences to visualise music in a way that make them feel powerful, that their actions, no matter how poorly played, still can result in something so powerful and beautiful. The artwork also allow audiences to become a performer rather than just an audience, which also allows them to experience a glorious moment as if they are a concert pianist.

Video of Fizah interacting with the artwork:

Takeaway

I like that most of the artworks are of an impressive scale. However, some are not interactive at all and have potential in being so, which was quite disappointing. It was overall a good experience, although a tiring one (my legs and stomach died cos town food is expensive!)

25 January, 201925 January, 2019

Research Critique I

Artwork 1

Image taken from https://www.smoothware.com/danny/woodenmirrormuseum.jpg

Daniel Rozin – ‘Wooden Mirror’, 1999
830 square pieces of wood, 830 servo motors, control electronics, video camera, computer, wood frame.
170cm , 203cm, 25cm

The Wooden Mirror is an interactive installation made of 830 wood pieces and motors that moves according to an image captured by the camera which tilts the wood pieces in a certain angle, creating the illusion of depth and therefore the illusion of a ‘reflection’.

“Mechanical mirrors are a platform in which Rozin investigates the borderline and contrasts between digital and analog worlds, virtual and physical experience, or order versus chaos. The first of this series, Rozin’s Wooden Mirror explores the inner workings of image creation and human visual perception.”

Q: Why do you find this artwork or project intriguing?
A: I stumbled upon this artwork on Facebook around last year and was fascinated by how the artist managed to show depth using just wood plates and the shadows they cast, which seems pretty impossible. I find it really interesting as I tried to figure out how technology allows us to make what appears to be impossible, which is to make a mirror out of a non reflective material. I was even more fascinated after thinking through the process of making this installation work the way it does.

Q: What is the situation or interaction created for the viewer?
A: The Wooden Mirror appears counter-intuitive to viewers at first, when a reflection casts on a non-reflective ‘mirror’ is made. Through instilling curiosity within viewers, the viewers would be made to look into the Wooden Mirror and interact with it by moving about and observing how the wooden plates move in relation to their own body movement.

Q: What is the intention of this interaction?
A: Other than the testing and application of a crazy but successful idea, the artwork has allowed people to question the potentials of materials, such as using a non reflective surface to create properties of a reflective surface. It also suggests to us the wonders of technology, in how each plate can be intricately programmed to display a certain shade to create a big image as a whole.

Q: What is the role of the viewer?
A: The role of the viewer is perhaps just an observer of the artwork’s effects, despite being actively engaged in the artwork when a viewer happen to walk in front of it. The viewer interacts through the act of just moving around and looking at the artwork. This generates a feedback to the artwork, allowing the artwork to keep changing.

Q: Who has control over the outcome of the artwork or project? Is it the creator / artist or the viewer/audience?
A: The creator has the primary control, ultimately, in terms of how he set the stage to make the wooden plates move in response to viewers (eg. he could have made it hard for viewers to discern their own image). However, the creator decided to make the artwork as it is now, and as such, the audience has more control over the outcome in term of how they directly affect the images portrayed on the Wooden Mirror.

Sources:
https://www.smoothware.com/danny/woodenmirror.html

Artwork 2

Image taken from https://www.marinabaysands.com/museum/exhibition-archive/human-plus/life-edges.html

Institute for Media Innovation (IMI) Nanyang Technological University (NTU) Singapore – ‘Nadine’, 2013

Nadine is a social robot modelled after Professor Nadia Magnenat Thalman. She is considered the ‘most realistic female humanoid social robots in the world’ at that time. She was created to assist people with special needs, and she can read stories, show images, and communicate with her user. She is also able to answer questions in different languages, simulate emotions in her gestures and her expressions, make eye contact with users, and can remember all the conversations she had with people.

Q: Why do you find this artwork or project intriguing?
A: My first encounter with Nadine was during the Human+ exhibition some years ago and was deeply intrigued by it. It was fascinating to me how technology is so advanced for a robot to exhibit (or simulate) social behaviours.

Q: What is the situation or interaction created for the viewer?
A: The viewers can ask Nadine questions and respond to it, creating a flow of conversation. This back to back conversations creates feedback loop between Nadine and viewers. The viewers can talk about many different things like the weather, ask about Nadine, etc.

Q: What is the intention of this interaction?
A: The main purpose of the interactivity with Nadine is to assist people in different ways. She can be a receptionist, or an assistance to people with special needs. But within the Human+ exhibition, the interaction is perhaps to awe viewers in how smart a robot can be and imagine the future where quality and advancedness of robotics can be useful to humanity.

Q: What is the role of the viewer?
A: The role is to just ask Nadine questions and talk to Nadine and reply. The viewer is also perhaps made to think about what to say, and think about pushing the limits of the robot.

Q: Who has control over the outcome of the artwork or project? Is it the creator / artist or the viewer/audience?
A: In this case, I’m quite uncertain as Nadine is programmed to learn. As such I think perhaps both the project (Nadine) and the viewers have control over the outcome of the artwork, which is their conversations, while the creator only have the parameters for the interaction set beforehand (like gestures, things Nadine will talk about, etc).

Sources:
https://www.telegraph.co.uk/science/2016/03/12/meet-nadine-the-worlds-most-human-like-robot/
https://www.marinabaysands.com/museum/exhibition-archive/human-plus/life-edges.html
https://www.channelnewsasia.com/news/singapore/this-born-in-singapore-robot-could-soon-be-your-receptionist-7968842

Q: Come up with 2 thoughtful questions in your essay that will benefit the class with regards to this week’s topic on interactivity.

Interactive art can be something that can respond to our actions. In this case, would you think that an interactive artwork could one day kill someone? Do we need regulations in interactive art in future?
.
Imagine a future where robots co-exist with us. Would an interaction between a viewer and an advanced robot similar to Nadine be considered as a performance? A normal conversation? Or something else?