As per what we (myself, Jiaman and Viena) have discussed with LP during consultation, we have decided to stick to our concept of “Migration” for this project. It will still be about bird migration, however focusing more on the science behind it and with a more educational purpose to our installation.
This concept aims to allow the audience to learn about bird migration, such as migration paths and geographical locations that birds visit. By projecting the birds onto a table, the audience can interact by placing certain object (“seeds”) on the table. Each “seed” will produce a different island on the table at the location the person has placed it on, and the birds animation will flock towards the island. This works through colour detection; each “seed” is coloured a different colour and a camera will detect the colour of the “seed” and produce the corresponding island on the projection.
Colour tracking on Max MSP
Initially, we wanted to use colour tracking on Max MSP. Our idea was to have a video content projected onto a clear table, and when certain physical objects or “seeds” are placed on the table over the video, it will play a certain animation according to the object that is placed. So for example, since our project is on bird migration, when a “seed” from Africa is placed on the table, then an animation of Africa appears out of the seed on the projection.
We consulted this idea with LP and he suggested that instead of colour tracking, we can make use of something called AR fiducials to allow Max MSP to recognise specific objects. How it works is that we can make use of fiducial markers, which are available online, and we can laser cut it to paste on our “seeds”. By installing a camera under the clear table and linking it to Max MSP, we can allow the program to recognise which markers are picked up by the camera and play the video animation accordingly.
AR Fiducial Markers preparation for laser cutting
Above are a few fiducial markers which we downloaded and prepared on Illustrator for laser cutting. We decided to try them out in different sizes, as well as with and without the square border background, and with regular and negative image in order to see which one is best recognised by the webcam.
Next step was to figure out how to get the Max MSP software to recognise the markers. LP introduced us to a software called reacTIVision/TUIO which is able to be linked into Max and it allows the webcam to recognise the fiducial markers. After laser cutting the markers we realised that the negative one with square border works best due to the higher contrast between the edges and the marker!
Above is how we used Max along with the TUIO client in order to trigger specific videos to play when the specific fiducial marker is seen on the webcam. At first we were unable to make the video play outside of the Max MSP interface, then after that it was able to play outside but not simultaneously/not at the location of the fiducial marker. After consulting with LP and referring to our old Brad Pitt code, we were able to fix those issues!
Next step for us would be to come up with our video content/animation and figure how to get videos to play in the background of the fiducial-triggered videos, as well as to decide on physical aspects of our project like what to use as the “seed” and how we want to arrange the camera/projector!