PEN-ing ma thoughts on Principles of New Media ;)

PEN is a sound sculpture that responds to users approaching. The clicking of pens changes from a state of chaos to a calm, unified one as the user moves towards it.

sculpture blueprint

Numerical Representation

Sampling and converting data received from kinect/ultrasonic sensor into digital data is the backbone to how PEN will work. The data collected will directly translate into how the sound sculpture will react to the user. This binary translation into values to be tracked triggers different states of the scores, which would also affect the state of chaos/calmness of the sounds.



Hardware – The pens could be swapped out and replaced by another object,changing how the entire sculpture would sound like.

Software – The scores work like different pieces of sheet music and the pens, the instruments. We could change what “songs” the pens are playing by changing or modifying the scores.



The ability for the sets of scores to be triggered based on pre-determined distances between the user and the sensor in the code gives the sculpture a life of its own. The code gives PEN a set of logic to be followed and actions to execute when certain requirements were met.



PEN is completely reliant on the interaction between the users and the sculpture. The scores the pens are clicking to will not change without the sensor detecting an object approaching.

The scores selected to be used will also be a variable due to the use of a randomiser. This would make the sounds made by the sculpture appear to be more interesting but still maintaining the same concept of chaos/calmness that we want.



The concept behind PEN plays on the idea of the interstice of sound. With the use of technology, we are able to emphasize the gaps between sounds by exaggerating and enlarging these sounds. Through this change of sounds between chaos and calmness in a short time by something we would normally not pay attention to, we hope to highlight the gaps/silence between sounds to the users.

200227.3 // Mini Solenoid Mockup


sculpture blueprint
  • Solenoid gets really hot quickly, duration between clicks to be increased to prevent overheating



Loaned out the kinect to test the depth sensor.

sample base code

Kinect outputs a double image as seen on the right. This is because there is a  physical offset between the IR camera and the projector.

Thankfully, this isn’t an issue as all we need is the depth data and not precise location information.


Next, we tried to colour code objects based on how far they are from the kinect.

weird glitch

The sample code we found online did not include the function to purely draw the raw depth distance data.

Look for corey for some help and…

raw depth data

200227.2 // Mid Term Project Review


We focused on the code and tried testing out the scores by using the relay first.



  • Decided to change the ultrasonic sensor to kinect as the ultrasonic sensor could only detect distance linearly. Using the depth function of the kinect would increase the AoE and accuracy of incoming humans.
  • We will retrieve the depth values from the kinect in processing and hook it up to arduino to make the solenoids click.