Interactive 2 Veejay (Bridgel, Viena, Mavis)

For this Veejay assignment Mavis Viena and I have worked together to optimise the patch which LP has posted on dropbox!

In the main patch, we changed the threshold value to allow the contrast between the background and the trigger object to be more prominent hence the camera can now detect when the object is moved across the boxes

This is in the pPlaySound patch and what we have change is that instead of putting all the sound into a list like in the original patch we have separated them out so that the sounds can overlap one another and play more than 1 sound at a time when more than 1 box is triggered at any moment

This is the send sequence patch which sends the steps to the main patch so that the boxes coordinates can be controlled

The final change we made was to the p Pix patch which we tried to add 8 different pictures to be triggered by different boxes, but for some reason (perhaps the pictures are too small) it just shows a black box so we decided not to include it in our video

Interactive 2 Brad Pitt

Am using Nicki’s face for this assignment because who doesn’t want blonde and pink bangs amirite?

Here is the end result! The image of Nicki was pretty blur at first so I lowered the mode number on jit.fastblur and it helps a little.

Made some small changes to the step 1 patch in order to fix the issue of the face size constantly changing. I added a jit.matrix in order to restrain the image of Nicki within a fixed size, and it looked a lot better after that (:

Progress!

As per what we (myself, Jiaman and Viena) have discussed with LP during consultation, we have decided to stick to our concept of “Migration” for this project. It will still be about bird migration, however focusing more on the science behind it and with a more educational purpose to our installation.

Migration

This concept aims to allow the audience to learn about bird migration, such as migration paths and geographical locations that birds visit. By projecting the birds onto a table, the audience can interact by placing certain object (“seeds”) on the table. Each “seed” will produce a different island on the table at the location the person has placed it on, and the birds animation will flock towards the island. This works through colour detection; each “seed” is coloured a different colour and a camera will detect the colour of the “seed” and produce the corresponding island on the projection.

Colour tracking on Max MSP

Initially, we wanted to use colour tracking on Max MSP. Our idea was to have a video content projected onto a clear table, and when certain physical objects or “seeds” are placed on the table over the video, it will play a certain animation according to the object that is placed. So for example, since our project is on bird migration, when a “seed” from Africa is placed on the table, then an animation of Africa appears out of the seed on the projection.

We consulted this idea with LP and he suggested that instead of colour tracking, we can make use of something called AR fiducials to allow Max MSP to recognise specific objects. How it works is that we can make use of fiducial markers, which are available online, and we can laser cut it to paste on our “seeds”. By installing a camera under the clear table and linking it to Max MSP, we can allow the program to recognise which markers are picked up by the camera and play the video animation accordingly.

AR Fiducial Markers preparation for laser cutting

Above are a few fiducial markers which we downloaded and prepared on Illustrator for laser cutting. We decided to try them out in different sizes, as well as with and without the square border background, and with regular and negative image in order to see which one is best recognised by the webcam.

Next step was to figure out how to get the Max MSP software to recognise the markers. LP introduced us to a software called reacTIVision/TUIO which is able to be linked into Max and it allows the webcam to recognise the fiducial markers. After laser cutting the markers we realised that the negative one with square border works best due to the higher contrast between the edges and the marker!

Above is how we used Max along with the TUIO client in order to trigger specific videos to play when the specific fiducial marker is seen on the webcam. At first we were unable to make the video play outside of the Max MSP interface, then after that it was able to play outside but not simultaneously/not at the location of the fiducial marker. After consulting with LP and referring to our old Brad Pitt code, we were able to fix those issues!

Next step for us would be to come up with our video content/animation and figure how to get videos to play in the background of the fiducial-triggered videos, as well as to decide on physical aspects of our project like what to use as the “seed” and how we want to arrange the camera/projector!

Max MSP Mirror Project

Done by: Mavis & Bridgel

‘Problems’ identified:

  1. Jumpy transition of brightness decrease and increase
  2. Full-screen
  3. Reverse Image (to emulate mirror)
  4. Max/min limit of brightness depending on face distance
  5. Auto startup

Solutions in Code:

1. Jumpy transition of brightness decrease and increase

Under p area2brightness patcher, the object ‘line 0 50’ and its connected message ‘$1 100’ is edited in order to make the transition of decrease/increase in brightness smoother. The function of the object line is to generate a timed ramp. The initial value of ‘100’ is in milliseconds, which means that each ramp is generated taking the value of face area from the face recognition and passed every 100 milliseconds. By increasing this value to 1000 milliseconds, the ramp is generated every second, allowing for a smoother transition.

 

 

2. Full-screen

The patcher p-spacebar fullscreen contains the object ‘key’ followed by ‘select 32’. The value 32 is associated on the keyboard as the spacebar. It is followed by toggle and ‘fullscreen $1’, where $1 takes the value of the previous toggle object and releases the output to Jit.window. This patcher combined with Jit-window allows the program to toggle between full-screen and windowed states of the final video output when the spacebar is used.

 

3. Reverse Image

The object jit.dimmap is added, connected to ‘pak invert 0 0’ connected to two toggle buttons. The toggle button on the left is to inverse the output horizontally while the toggle button on the right inverses the output vertically. Hence, only the left button has to be toggled for the image to inverse to emulate the mirror effect.

4. Max-min limit of brightness depending on face distance

In order to optimise the user experience (based on the space available, for example), the sensitivity related to face size can be adjusted under ‘p area2brightness’. The line ‘scale 0 12345 0. 1.’ can be edited based on testing for this. By checking the approximate area output of the face at the distance you would like the program to start decreasing the brightness of the video output (i.e. furthest distance – check value output of ‘expr (($i3-$i1)*($i4-$i2))’ where you would like the brightness to start decreasing), enter this approximate value into the first value after ‘scale’. In rough terms, the 4 values after ‘scale’ corresponds to minimum input, maximum input, minimum output and maximum output. Hence, take the next approximate value of when you would like the screen to completely blackout (i.e. face close to the camera) and input it as the second value after ‘scale’. This can be easily edited based on the space. Based on our testing, our current ideal minimum and maximum was 500 and 8000. Hence, our final code states ‘scale 500 8000 0. 1.’

The scale function is necessary as the following object after it only takes values from 0 to 1.

 

5. Auto startup

In order for the program to start automatically in fullscreen mode when the code is opened from the folder, a total of two toggle buttons have to be activated (‘p video in’ and ‘pak invert 0’). Hence, the ‘loadbang’ command was inserted directly above ‘pak invert 0’ to ensure that the output video starts inversed, like a mirror. In order to give the program some time to inverse the output video first, the patcher ‘p initial delay’ is written with ‘loadbang’ followed by ‘delay 2000’ which delays the video input by 2 seconds.

 

Screenshot:

Video: