Interactive Spaces – Semester Project (Part 2 – Digital)


Under Score is an interactive installation that challenges two players to play a game of soccer confined within the space under the table, relying only on their sense of hearing and touch. Players can hear the position of the ball through the headphones connected to a color tracking patch.

By Joan & Tiffany


Inspired by two branches of the sport – table soccer and blind soccer (played during Paralympics).


From our previous giant hanging foosball idea, our idea evolved towards digitizing the experience to change the conventional gameplay of soccer. Instead of tracking the ball with the sense of sight, we decided to translate what we normally see into sound. This means that players rely on their sense of hearing and touch to locate the ball. The location of the ball is indicated by the pitch and directional sound from both sides of a player’s headphones.

To add on to the aesthetic element of the space, we decided to project a visualization of what they hear onto the table. It also allows non-playing viewers a glimpse of the movement going on under the table.

Spatial layout & diagram:

Materials needed:
External camera (webcam or phone cam), attachable wide-angle lens, table (with four legs at the corners and no obstructions underneath), wooden stools, large black cloth, some cardboard panels, projector, colored soccer ball


MAX Patch processes:

1. Get feedback from the camera (interchangeable between computer cam and external cam)
2. Color tracking with min and max values for the color(s) picked, the range of color values can be adjusted via the ‘tolerance’ bar slider at the side
3. Extract the top left x and y coordinate values from the camera output
4. For the x coordinate, scale (1 300 18 58) and input the values to the keyboard to control the pitch via the colored ball’s x position
5. For the y coordinate, scale ( 1 220 1 158 ) and ( 1 220 158 1 ) and input these values into the left and right gains~ of the audio output respectively. This creates directional hearing based on the ball’s y position

Screenshot of our colour tracking MAX patch, tracked via camera
Screenshot of our MAX patch for the production of synthesized sounds/visuals – part 1
Screenshot of our MAX patch for the production of synthesized sounds/visuals – part 2

For the construction of the table, we bought 6 planks of wood, measured them and sawed them accordingly. We joined the planks together by hammering in nails (the hardware uncle told us to use nails and hammer), but discovered that the process could have been made easier/hastened by using a drill and screws instead.

Hammering processing to building our wooden table frame from wooden planks
Hammering processing to building our wooden table frame from wooden planks
How each table corner was joined and reinforced
Blocking two sides of the table using cardboard and covering the whole table with a layer of black cloth
Torch lights attached at each corner of the table frame to illuminate the playing area
Phone camera attached under the table, directly above the playing area. Camera signals are sent over to the laptop via wifi and DroidCam
Our colored ball in the lit up playing area
Presentation in class


feedback & Improvements
  1. Audiovisual feedback was fairly slow, players will have to play slowly and kick gently for the position of the ball to be updated in sync.
  2. The audio feedback was too hardcore. Perhaps the sound feedback could have less noise and smoother sounding.
  3. Lighting underneath the table could have been stronger and evenly lit, to prevent blind spots that allow the ball to hide in the dark.

FYP Pitch

Google Drive link:

Addition Research (After feedback)

Istanbul: City of Cats

‘Kedi’, directed by Ceyda Torun, is an extraordinary portrait of a city, its cats and the community that loves and cares for them

For the past few centuries, hundreds and thousands of strays live among the denizens of Istanbul, and they have become an inseparable part of the community. They are often seen coexisting with humans, lounging in cafes, shops, or perched on rooftops.

“Istanbulites often place bowls of food and water on the sidewalk in a communal effort that lets cats roam free. The money in the tip box at one restaurant goes toward the veterinary bill for sick or injured animals; at a fisherman’s stand, cats get to sample the tiny anchovies plucked from the Bosporus. People often feel duty-bound to care for the animals that hang around their home or workplace, despite the cool nonchalance of the cats themselves.”
from The Economist

Tashirojima, Japan

Image result for Tashirojima cat island

Tashirojima (aka “Cat Island”) is a small island in Ishinomaki, Miyagi Perfecture, Japan. The  feral cat population outnumbers the human population 6 to 1 on this island.

How did it come about? – During the late Edo Period (江戸時代, 1603-1868) silkworms for making silk were raised on the island. The residents kept cats to chase mice away from their silkworms.

As the cat population increased over the years, residents continued to take care and feed them. Cats eventually became a symbol of good luck and good fortune in the Japanese culture.


In Singapore, stray cat feeders are occasionally seen under HDB void decks refilling bowls of water or laying out portions of pet food. These feeders (usually elderly women) often return to the same spots regularly to feed and care for the stray cats in the area.

While the feeding of stray animals is not illegal in Singapore, littering is. Cat food left out in the open and uneatened (or half-eatened) for too long may be deemed as litter and feeders may be fined.

Unlike Istanbul and Japan, cats and dogs are not culturally appreciated here in Singapore and perhaps this may be one of the reasons that active communal care for these strays are not commonly exercised.

Below is a heartwarming article about a particularly dedicated feeder at Bedok North Ave 1:



Tashirojima – Japan’s Awesome Cat Island

A heart for stray cats in Singapore

Emergent Visions – Reflection on Krzysztof Wodiczko

Krzysztof Wodiczko is a Polish artist renowned for his large-scale slide and video projections on architectural facades and monuments. Since 1980, he has created more than 70 large-scale video projections on monumental architecture worldwide, and focuses of the ways in which these monuments reflect the collective memories of the communities and history, and are often politically charged. He uses monumental buildings as a symbol of victory, through which he gives voice to the concerns of marginalized and silent citizens who live in the monuments’ shadows.

1. The Investigators


“The Investigators” is an interactive public video projection installation which took place originally in Weimar, Germany. Live images and voices of refugees are projected onto the statues of Schiller and Goethe, facing the assembled public standing on a raised platform. The act of talking back to a monument in real time allowed the refugees and general public to open up and share their historical experiences, and as a result was a political achievement.

“The statue represents a past that cannot be changed” – Wodiczko

In this work, Wodiczko presents an opportunity of the public to voice themselves, and make changes despite the fact that the public space is barricaded by monuments. As European cities are often packed with war memorials and monuments symbolic of war victories and defeat, Wodiczko believes that the whole city is in fact one big memorial to war, and that the communities of people are all war memorials inside of themselves.

Why Schiller and Goethe? – Symbolized a remarkable friendship and collaboration between two of the most well-known figures in German literature. Goethe sheltered Schiller as a refugee.

Images courtesy Krzysztof Wodiczko Studio.
Participants getting into position in the studio, behind the scenes.

Live projection mapping by technicians behind the scenes.

2. Homeless Vehicle

Image result for homeless vehicle krzysztof wodiczko

Wodiczko teaches interrogative design at MIT. While the homeless vehicle is seen as a design solution to homelessness, 10 000 homeless vehicles cannot be made. In a way, the homeless vehicle provides emergency help, a metaphorical bandage, but has further implications of the conditions and problems behind the wound.

Image result for homeless vehicle krzysztof wodiczko

3. Hirshhorn Museum, Washington, D.C.

Image result for Hirshhorn Museum, Washington, D.C krzysztof wodiczko

The work displays a very powerful image of a hand holding a candle on the left, another on the right with a gun and microphones at the center of the building.

Concerns over the restaging of his projection following the Parkland Florida school shooting. A very relevant insight I got from this work was that there is a world behind the facade of museums. Museums operate in the same way that monuments hide behind the ideological smokescreen of victory and selectively memorable parts of the location’s history.,

The board of trustees(consisting of governmental figures and billionaires) told him to restage the work prior to the tragedy and the decision was made to postpone the work’s display as a gesture of respect. The imagery of the gun might have triggered someone who was affected by the shooting. The media plays a huge part in a work and every work is done with the expectation to be hijacked.The fact that the work opens itself to discourse is seen as an outcome that far outweighs its risks of being hijacked.


Narratives for Interaction: Her Story Game Review

Her Story is an interactive video game produced and directed by Sam Barlow in 2015, and has won several game publication awards since. The game is set in a detective desktop computer, with access to video footage of police interrogation of a woman (Viva Seifert) on several separate occasions. The page starts off with a few snippets of the interview, revealing the crime – the murder of a missing man named Simon.

The woman proceeds to introduce herself as Hannah Smith – Simon’s wife. The keyword search function in the corner prompts us to further research on the case, by searching notable names and phrases mentioned in each interview. The game leads us on to search for video after video, digging deeper and deeper into the case.

As the pieces come together, a bigger picture is gradually revealed. At some point, the woman starts to refer to Hannah as a third person, indicating the existence of a twin sister. Eve and Hannah are twin sisters and she, in the interview, is the former. The two identities, Hannah and Eve, seem to merge into a single person as one’s alibi is used as an excuse for the other’s murder of Simon. The video footage leads us endlessly – from her singing videos to her troubled relationships.

I was intrigued by how the game was so engaging, despite its simple design and 90s themed graphics. Although I didn’t play the game till the end, I watched its gameplays on YouTube, which easily lasted for an hour. The game brings us through countless twists and turns, and many surprising revelations. Important clues are scattered throughout the gameplay – such as an occasionally flickering of the office light and the faint reflection of a woman with striking resemblance to Hannah/Eve. One of the events that struck me was a video of the woman casually performing on her guitar, singing about the rain and making a bow out of hair and bones. The eventual realisation that it was actually a song about her murder of her twin sister was extremely creepy.

The game ends without any confirmed accusations or a definite conclusion – a chat window asking if the player is ‘finished’. It is then revealed that the player (you) is actually Sarah, Eve’s daughter, who came to seek the truth behind her mother and Hannah (hence the reflection on the desktop).

Her Story opens up our imagination and gives us the exciting role of a desktop detective. The concept is not only meticulously planned out, but the attention paid to the little details was also exceptional – from important hidden clues to mindless comments like Simon’s preference for blondes. It is one of the most creative and deep storylines I’ve ever read and is certainly an inspiration to me.