Final Video

Initial Proposal

 

Project Idea & Premise

Premise: Now as most of us know, it is common for ADM students to stay in school working on their projects late into the night long after the lights and the air conditioning has switched off. Sometimes we stay till the light of the early morning sun peaks through our tall windows, for days on end (especially during hell weeks or a few weeks before final submissions). During short moments of respite, when we have dinner, with food cobbled together from what we can find from 7-11 usually, after midnight, we would talk around the campfire of sorts. What I have noticed is that these conversations though sometimes silly and comical are actually quite profound. My theory is that this is because when we are tired, the subconscious filters we place upon our speech are somewhat lifted and this adds some truth to the words said during this strange “witching hour”. The next thing I have noticed is that recently, these conversations end up being about purpose. Maybe it is because of a combination of being at the midpoint of our university lives, having the innocence and wonder of first entering NTU and ADM somewhat dissipate by now and also the thought of our FYPs looming ever so nearer, my friends from year 2 find themselves questioning  reasons why we are in ADM and reasons why we are doing so and so projects.

 

Purpose of “Purpose” : I really like these conversations we have and thus in reference to Krysztof Wodiczko’s Critical Vehicles, where he imagines monuments which are ever watching,reflecting the voice of the hidden and the forgotten, I instead imagine spaces of interstice, small nooks and corner, being silent watchers, that absorb and remembers the feelings that radiate from these conversations and thus that is why I decided to do my projection mapping at the corner of the ADM lounge.

As for the message that I intend to convey with this project, I intend “Purpose” to be a response to these conversations that have been happening around school and a manifesto of sorts for my fellow colleagues in ADM. Reflecting about our purpose and existence is important and it can instill us with new perspectives and drive. However sometimes if we are not careful, this could go into a nihilistic direction. Especially if we come to the conclusion that we have no agency in the environment that we are in. I see my colleagues starting to move towards this direction as there are some seeds of resentment for the system. Yes, the system isn’t perfect and it has bitten us, but at the same time it vexes me when my friends complain about the system without realising that they themselves are a cog within it, perpetuating the very flaws and problems they complain about when they stay passive.  This manifesto will hopefully invoke them to think critically, have hope and bulk up at the challenges ahead. No matter how bad the system gets, my opinion is that we will always have the power to invoke change. This is especially so for us as artists. The beauty and verisimilitude of our works is largely dependent on us as creators.

Changes

In my firs try at projection mapping, the idea was to have something move about in an image of the space and project that image onto the space itself to create the illusion that there was actually something move about there. However this proved to be problematic as the projection just looked strange. This was mainly because the simulated space I had created within Unity did not really match the space in real life.

In my original project proposal, I wanted to create an animation that was more figurative where we have a character that is a representation of ADM students moving about the space that I was projecting on. However after reassessing the amount of time left I had for this project, I decided to work instead mainly with aniconic representations and forms as to have something figurative look like they are moving about the physical space I would have required much more time to figure out how to accurately recreate the physical space within the digital space. Thus with this, I also decided to work with AfterEffects instead of Unity to create the animation as I was much more familiar with the particle effects system within AfterEffects.

Tools

Aftereffects:

After effects is a powerful tool and useful for creating many different compositions. It is especially easy to create aniconic and abstract forms using the particle effects tools available. Creating figurative vector based animations is also relatively simple as After Effects has compatibility with Adobe Illustrator. It has also proved to be useful when trying to match the animation with the music as you are able to view the waveforms in a tab while editting in After Effects.

 

VPT8:

Since I was not working on the Chinese Heritage Center Projection Brief (another project option for this class) I had to figure out another way to work with projection mapping. And it was a nice surprise to find free open sourced tool that was already out there. Now VPT8 is definitely not as powerful as Green Hippo, but it is still decent and perfectly suitable for crating smaller scale projection mappings. What is interesting is that VPT8 also has compatibility with arduino and processing, and this makes interactivity an option for future projection mapping projects. VPT8 also has an easy to follow documentation and youtube tutorial which makes it an easy software for beginners to pick up. however, it does come with some restrictions.

Restrictions

Unlike Green Hippo where you can add a million and one cross hair points to scale and stretch the video to fit the surface, VPT8 has only 4. Suitable enough for creating projection mappings on square-ish surfaces but I imagine this will be much more difficult for more complex surfaces like the facades of buildings. This however can be circumvented somewhat by using masks.Furthermore I still have not figured out how the mesh tools works in VPT8 to project on curved surfaces.

VPT8 also only allows for for one screen output unlike Green Hippo, this only one projector can be used.

VPT8 like Green Hippo also can’t play audio and thus audio has to be played from a separate platform.

VPT8 also takes up quite a bit of processing power and thus you have to be careful of the sizes of your videos lest you will lag the system. This means that your videos have to be of a lower quality. This is however not as bad as it sounds as videos at 720p can be played and still looks very crisp and clear when used for projection mapping.

 

Vivitek Projector:

Simple enough to use projector that does what you need for a small scale personal projection mapping. Propped up with a Kokocrunch box.

Process

 

1) Finding the Surface.

Now that I knew what I wanted to do, the first step was to figure out which surface I wanted to project on. The where was decided quite early on. Since this semester I was spending most of my time in the lounge I thought it was suitable to project there. Furthermore, I think the idea of projecting on an interstice space hidden in a space of congregation fits with my them and idea quite well.

I then decided to use the surfaces already found within the lounge and not add to it as I thought that would fit better with the idea of the natural space “absorbing” thoughts and emotions. I however rearranged some items so that I can create more interesting projection mapping. For example an angled pedestal is more interesting to project on rather than a forward facing one as then it would have just been a flat surface.

 

 

2) What was I going to Project

I knew that I was going to create an abstract form moving about the space with each surface giving the illusion as if they were extensions of one another. How this surface would move about the space however was something I only figured out once I had the music. Being inspired by games such like “Everything” I knew that I wanted my project to have a somber and serious, but at the same time up uplifting and inspirational tone. So I took quite a while to find the best music and at last settled on Whitasand’s Piano Music- Salvation

 

Matching the animation with the music also proved simple enough as you can see the waveforms while editing in After Effect. Since peaks in the waveform corresponds to significant beats in the music, you can match your keyframes to those peaks.

 

3) How to Project

Now that I knew where and what to project, I needed to figure out how. I knew that to make this projection mapping interesting, I had to create the illusion of added depth within the surfaces that I was working on. For example, with the pedestal, I wanted to create the illusion that you are looking into the pedestal. I spent quite sometime trying to figure this out. First and foremost, I knew that I had to simulate to a degree the physical space within the digital world and thus I measured to dimensions of the surfaces I was going to project on and converted cm dimensions to pixels. The videos did not have to match the dimensions 1-1 as then the videos would be needlessly large, but what was important was that the proportions needed to be correct so that when you stretch and warp your videos, it would not look strange.

 

At first I figured adding frames inside the animation to give the illusion of depth and creating the animation from different angles corresponding to different faces of the pedestal would be enough. But upon testing, I realised that it wasn’t as easy as I thought it would be.

I realised that this looked off because from the angle where I wanted viewers to see the animation, you would not see it in that way. My solution to this was either to rotate the particle world in After Effects to match the angle of the pedestal in real life, or to treat the side surface and the front surface as extensions of one another within after effects.

In my second try I rotated the angle of the particle world and also added more lines to give the illusion of depth and experimented with a shake effect.

With there second iteration there was still a lot of problems. The angle was still not right as from the viewer’s angle you were not suppose to see what was happening in the side profile in that way. Secondly, rotating the particle world in after effects did not add much to the final view as it still looked like a flat particle effect and lastly the shake effect made the videos lag in VPT8 for some reason causing them to go out of sync.

From this try, I realised that it would be best to treat both surfaces as extensions of one another. I was also perplexed as to why the particles looked like they were moving about a 3D space more when in after effects as compared to when I was projecting them in real life. Upon reflection I realised that the grid ground within After Effects is the element that tricks the brain into seeing the space as a 3D space.

This reminded me on how may projection mappings on buildings use stroke effects to trick viewers into seeing the facade a certain way.

Essentially, the human brain is a flawed interface that can be tricked. These stroke effects initialises the brain to see the building in the way you want it to be seen and this paves the way for other illusions you want to create afterwards. I realised also that I have to take into consideration the angle from which the viewers are watching my projection from.Thus I had to keep in mind that my projection is below eye level and thus the gaze will come from a higher angle.

Keeping all of this in mind, adding stroke generation effects and working both surfaces as one screen  my third try surprisingly well. I also played with animating the size of the particles, making it bigger when it comes “nearer” to you and smaller when it moves “further away”to give the illusion of depth.

From here my work flow was to create the animation for each surface. For it to play seamlessly, each separate animation is 2 min long (the length of the whole video) so that when all 5 animations were played at the same time from the start, all 5 animations will be in sync.

 

4) The feels

The last part of the project was to add the manifesto that was meant to inspire. For the script I took inspiration from the movie “Cloud Atlas” and Like Stories of Old’s (a youtube channel) analysis of the movie. Within this analysis the author talks about how “Beauty” can save the world. I interpreted  beauty as as art and applied those ideas to finding purpose within an ADM setting

Manifesto

Remember when we first came to this place? Remember how full of wonder and excitement we were. Do you think it’s still the same place? … Or are we the ones who have changed? The ones who are no longer recognizable.

Isn’t it comical. You would think existing is one of the easiest things a human could do. And yet it is one of the hardest questions for us to solve. What is our purpose here? Creating beauty? … But I ask you then , when in bloodthirsty history has beauty ever saved anyone. Ennobled, uplifted yes, .. but whom has it saved.

And in the face of such inconsequentiality, how could we not buckle and fall … and in such a state suddenly, all the flaws this place holds become ever so prominent. Do we then simply prostrate and accept ? … this fate that has been put in place seemingly by design. Do we rage? Against the system or the unseen forces that are never in our favour.

But oh if only it were all that simple. That there were evil people somewhere insidiously working against us and the solution is but to stop them. … It should occur to you gradually that these problems that are ever so preset not because that they are insistent on following us, … but because these insecurities, they are constants in every human heart.

I say to you now, that we do have agency and the power to make a difference. But the onus lies on us to assume it. Art can motivate, art can enthuse, art is the only substitute for experiences that we have not lived through. It is unclear what this beauty is, but it has the power to transcend the conventional boundaries between our hearts only we if we have the courage to define it.

It is frightening to be a drop in the vast limitless ocean that is the world. But the existence of this drop is not a lost cause,… it is all that matters. After all, what is a ocean but a multitude of drops.

End —

 

Some parts of the speech had to be redacted as I wanted it to still fit into the 2.min time frame. I am also glad that I got my friend Tania (Tania Matilda Liang Qi Ying) to read out this manifesto for me as her performance was spectacular and she managed to deliver it in the way that I wanted.

Next steps

Of course this project can be improved on and many things could be added to it. For one thing, I want to explore the potential of making this project interactive since VPT8 has compatibility with Processing and Arduino. The most basic form of interaction would be for the projection to play only when there is a person within the space. Taking this a step further would be to have the projection change based on the action and movements of the viewer in real time.

Also, currently the illusions in my project only works from a specific angle and height ( that is kneeling down on a bean bag as that was the angle I was working from). Bringing this further, I should tweak the animations such that they work when viewed from an average human height and also I should do more experimentation to make this projection mapping work when viewed from different angles (lets say someone decides to view the animation from the side instead of from the front). It would also be interesting to design the animation in such a way that when viewed from a different angle, the viewers would see a different narrative and story play out even though it is the same animation.Of course the animation that I currently have could definitely be animated better. I should also strive to try and figure out VPT8’s mesh tool as that could be used for the animation to look nicer when played on a curved surface. Currently it is treating the curved pillar as any old flat surface.

Final Thoughts

When I first entered this module I was worried and really insecure at the thought of being in the same class as the Yearr3s. I was worried that I wouldn’t be able to keep up and carry my own weight especially after seeing how talented and smart all of them were. I was also questioned my decision to do my own project instead of an animation for the CHC constantly as I was extremely worried that I wasn’t going to have it all figured out in time. I However I do not regret one bit staying in this module as it was such an amazing experience to learn from people outside my circle, that is, the amazing guest lecturers during the Urban Media Art Academy and to learn from the Year3s. And I am also ultimately really proud for challenging myself with this module and my project. I hope the others in class had just an amazing experience and I hope future classes would be able to do the same. Thank you for taking time to read this post and I wish you an absolutely wonderful day.