Month: February 2017

PROCESS 4: Game Mechanics

This is further into the development we have got going for the interaction.

I have managed a few things here:

  1. Rotating objects on mouse over
  2. Making objects move out and move back on mouse clicks
  3. Slider that makes objects appear and disappear according to the  position of the slider handle
  4. More to come…

Don’t mind the fog, I was just messing around with the particle effects for fun.

So this has been an interesting development because like I mentioned in earlier posts, our game has to show the passing of time. The timeslider element we have now will allow players to control that. If you notice above the slider I put the year in which this is happening. So at a certain year the thief would have obtained a certain object and then the object will go away after some time. So visually it creates this sense of power for the player in the controlling of time. But however they will need to click around to discover objects, listen to the stories and piece the entire world of the game together through that sort of scrutiny.

Which is what we really will need to sort out in the recess week: What is the overarching story here that we want to tell?

SHARING 7: Generating Interactive Stories

Barber, Heather, and Daniel Kudenko. “Dynamic Generation of Dilemma-based Interactive Narratives.” AIIDE 7 (2007): 2-7.


Read this paper which talks about mechanisms for Interactive Stories. And how we might use the decision making aspect that we design for the user to drive  and shape how we tell the stories.

They presented in the paper many avenues to consider and I will share some of the more interesting points.

In speaking about types of dilemmas, they categorized it into 5 distinct types: Betrayal, Sacrifice, Greater Good, Take Down, Favour. And all these I found really closely related to the sort of moral standards and systems prevalent in society. For example in the Betrayal Dilemma, it is about the moral virtue of loyalty and should the player then go against his faction or team in order to save himself? And for the Greater Good, it is about utilitarianism and maybe the player will decide that he has to leave people behind or avoid a fight in order to preserve and protect the majority.

The other thing that the paper brought up was this diagram to help structure these dilemmas into a scene or a story component in the interactive narrative:

So it is quite a good read and really gets you thinking about what we want the player to feel and think about when interacting with the story. And I find that thinking about it this way accomplishes two things, both the story plot (situation leading towards the dilemma) and the interactive story mechanism (choice and scenarios presented to the player).


To see other Assignments:

Assignment 1: Magic Mirror  |  Assignment 2: Face Tracking  |  Assignment 3: Seflie Instructor  |   Assignment 4: Alpha Blending   |   Assignment 5: Pixelation Mapping

For this third assignment, we are using MAX  and MSP to create a Selfie Instructor. And just to note, I did not add any music or additional voice overlays in the video, so what you hear is only the tracks invoked by MSP as well as my own reactions.

It basically detects where your face is on the screen and gives you instructions. So end result is to get you facing forward looking at the camera with your face in the center before it says “OKAY” and allow you to take the picture! Which you can take simply by making a sound. KA-Cha!


  • Coordinate info from cvjit.faces matrix used to calculate the position of face with respect to the screen
  • cvjit.faces coordinate info is then used to determine if the face is in the LEFT, RIGHT, UP, DOWN quadrant of the screen, or if there are no faces at all
  • MSP can detect if sounds are made  through the Microphone


  • Relative position of the face (LEFT, RIGHT, UP, DOWN quadrant) from Sensing is then used to:
    • Invoke audio playback of respective tracks on a playlist in MSP
    • Display instructions on the screen corresponding to the audio playback
    • So basically telling the user to go in the directions towards center of screen depending on which quadrant the face is in  and to also alert them that the app is running and to look at the screen should they be looking away
  • When user is in the centralized position, the screen will display “OKAY”, and the MSP sound detection through Microphone mentioned earlier on will be activated. User can then tap or say something at the screen to have their photo taken.


It was great fun doing this project and I kind of wanted to have a bit of humor as you can see, because I like the idea of machines scolding people and making them do stuff. Also it was good to finally be able to attempt combining MAX and MSP together in the same patcher. Having a proper layout was also essential in helping the process along so that I could have great visualization of the dataflow.

Of course, there are still some issues. Like displaying the images there will be this glitchy effect because the bangs are being outputted at a steady rate when user goes into any of the quadrants. Although in this case it kind of worked to the assignment’s advantage as it creates a sense of urgency to follow the instructions the user is being yelled at.

However would be good to know if there are elegant methods of controlling these flows. I know that the onebang can be used as a stopper valve for this and I actually used that for the center quadrant. However for the side quadrants you tend to not want too long a delay for the onebang or else it does not reset itself quick enough to react to the next position the user gets into. Whereas for the center it is fine because they will stay still to have their picture taken. So that is kind of the reflection on this assignment and I hope to learn more subsequently!

Process video log is below! Cheers 🙂

SHARING 6: Playing Lynch

Hey all, this is really a shout-out to those who are fans of David Lynch and have seen most of his filmography! Twin Peaks, the TV series that sparked so much interest and controversy is back for the third season, this coming May.

Along with many teasers for season 3, the Lynch Foundation also released this “Playing Lynch” interactive website where we can see the full length director’s cut for Psychogenic Fugue. Hosted by Squarespace the website creator, and directed by Sandro Miller.

Featuring  American actor, director, and producer, John Malkovich, who has appeared in 70 over films. And in this interactive film website we get to see him play 7 of the most iconic or intriguing Lynch characters (John Merrick, Henry Spencer, Lady in the Radiator, Mystery Man, Frank Booth, Dale Cooper and the Log Lady). Featuring also, the music of David Lynch, as performed by artists like The Flaming Lips and Angelo Badalamenti. The interactive film certainly pays homage to the Lynchian film! It was a great immersive experience going through the roughly 20mins of content.

If you are reading this and Lynchian films are your thing then don’t miss out! Click here now! You can see the trailer I linked up below also.

But if you have not watched any Lynch film, or you don’t even know who is Lynch to the point that the silhouette in the poster below does not bear recognition, then PLEASE WATCH HIS FILMS!!! You are missing out a great deal of great compelling story-telling in an often non-linear, well-crafted sequencing of pace, concept and emotion.

Meanwhile, I am still counting down to Twin Peaks Season 3! After 25 years.. whoo!!


Psychogenic Fugue starring John Malkovich

PROCESS 3: Isometric view in Unity

This was what we had in the video I uploaded the previous week. Top view is game editor, bottom view is the screen during gameplay.

It is isometric view but as you can see, formed out of these small images which you see in the image viewer i opened up on the right that we have to piece together in that angled fashion in the game editor. Which is quite a hassle. In fact the video I uploaded had maybe had 200 of these images??

So I found an alternative that achieves the same effect.

So now we can use 3D models to give us a accurate sense of space and really have the flexibility to arrange things the way we want. And the camera will be able to render them in the  isometric view. Hence we can look to having more intricately designed spaces and have objects placed in many different ways.

More to come…

PROCESS 2: Intro to Unity Game Engine

In sourcing the technical aspects of the Interactive Narrative project my team has embarked on, I am decidedly looking towards Unity Game Engine and what it can offer as a platform to shape and tell our story.

So our story is currently about this thief who has a house full of objects he has stolen and each object has a story to tell. These branched out narratives will eventually coalesce and lead to the capture of the thief. So in terms of plot for the game itself, the players start off at the time when the thief is already caught, so they kind of are looking into the past to see what and how these things all come together.

We decided on an isometric view for the game. Because of the players ability to ‘time jump’ which we will give them, I feel that it is appropriate as the isometric view has this sort of god-like presence to it and it is very much like in the Sims games. So will be sticking with that for now.

And here is a preliminary tryout at the game engine. I have created an isometric view by locking the camera vantage point to a certain angle. But the images are like these 2D images of small blocks that I need to piece together which is quite a lot of work. But because of the angles of the placement you will see it as an isometric space.

We can already see the potential of placement of objects, how they might be revealed or hidden due to the isometric restrictions for the player’s line of sight. And here I am only just interested in trying out the simple interaction of mouse over to make the objects rotate. Which I can then further extend into click actions, button actions, etc to maybe play audio, highlight objects. So that will depend on how we move on from this point.

PROCESS 1: Class Presentation for Interactive Narrative Idea

So just to recap the presentation that my team (Bao, Winzaw and me) made during class. We are very much inspired by this drawing by John Kenn. About the multitude of objects and how they are laid out for the viewer to see and just allow their eyes to travel around and form associations and connections to the objects. Which made us think about our story which is about a thief who has a this huge collection of stolen items.

My role in this project will be to handle more of the programming/coding aspects. So currently I am looking at various options (1) a webflow design where the user scrolls down to reveal various levels and more objects (2) a 2D point and click interaction in maybe Processing Java or Javascript so we put something simmilar to the John Kenn drawing but with additional display of story information (3) Unity game engine, using C# to power the interaction (but I will need to go for a crash course!)

We will continue to discuss and frame the plot of the story. Hopefully by next week we might figure out which platform we might use.

REFLECTION 2: On Fanthoming The Garden of Forking Paths

This short story is certainly captivating, although for the most part of the first 3 pages I was really trying my best to get a bearing over the situation as it unfolded.

The very first paragraph is a short preamble to kind of straighten things out where I understand that Dr. Yu Tsun is writing the statement that is to follow. And then I am told that the first two pages of this are missing. Thrown into the deep blue. I encounter with much intrigue a multitude of facts and clues to find out gradually what this story is about. I learn that it is about espionage, about Dr. Yu trying to evade the capture of Capt. Richard Madden until he could accomplish the mission of informing Germany of an eminent attack on the city of Albert. We are also told that Dr. Yu already knows that he will die at the end of this story. So there is already these sort of framework and foretelling where the narrative has set expectations for the reader or even rules for itself.

Yet there are many sort of ‘dualities’ in the story. So all these events, characters are both this and that at the same time and there are then these many possibilities that the story could have gone. Yu Tsun appreciates Albert’s respect for his ancestor’s literature and yet he has to kill Albert. Yu Tsun and Albert are both friends and non-friends.Capt. Madden has won the battle because he manages to apprehend Yu Tsun. However Yu Tsun has won the battle too, as he manages to alert Germany with Albert’s death published in the British Newspapers. Yu Tsun was both a spy working for Germany and determined to carry out the mission yet he is not truly loyal.

Yu Tsun mentioned that he took ten minutes to plan this elaborate scheme. Yet how much of this story could he have foretold at that point in time? There is a lot of descriptive detail going on and for the most part rather mundane and unimportant to the events of the story. Like what was in Yu Tsun’s inventory, how much cash he had left or the people he happened to see at the train station. So I really get the feeling like all these thoughts are still rather floaty and Dr. Yu is really trying to recall the events transpired here.  Coupled with that are his imaginative spells. I am both following along and getting lost in his stream of consciousness. This is the charming thing about the story that kind of intriguing haziness yet there is still a logic to it.

Also parallel to that story is the myth of Dr. Yu’s great grandfather Ts’ui Pen. Dr. Yu and Albert discuss about the conceptualization and interpretation of Ts’ui Pen’s literary labyrinth. Center to the narrative is this ideal maze of character, events, concepts and narratives that are explained to be a writing about time not space. This is contrasted with two non-fictitious great works of literature – 红楼梦 and 1001 Arabian Nights. These 2 works although vast in their amassing of  narrative are not infinite and all-encompassing. However Ts’ui Pen’s work is a true work of the labyrinth of narrative and time – “various future times but not to all”. Simultaneous outcomes, solutions, points of departure for the continuation and discontinuation of the narrative, a network of dominoes if you will. And I found that that was the true mystery of the short story. Can something like that even be written?


The Garden of Forking Paths (1941) by Jorge Louis Borges is a great read and it kind of attempts to write with much efficiency a taste of that sort of complex, conflated narrative like the ideal literary labyrinth of Ts’ui Pen’s. This is a great challenge to the traditional linear literary structures and I would say that The Garden of Forking Paths is a very conceptual piece that has and will continue to inspire many to the possibilities it has open and invites.

ASSIGNMENT 2: MAX Face Tracking

To see other Assignments:

Assignment 1: Magic Mirror  |  Assignment 2: Face Tracking  |  Assignment 3: Seflie Instructor  |   Assignment 4: Alpha Blending   |   Assignment 5: Pixelation Mapping

Hey! So this second assignment I am calling Gun in Your Face!

So to sum up the assignment first:


  • Coordinate info from cvjit.faces matrix used to calculate the position of face laterally across the screen
  • cvjit.faces detects when the face is at a certain proximity from the screen
  • cvjit.faces also detects when the face is off-screen or there is no more faces


  • Value for face position used to determine which frame from the video of gun moving left and right will play
  • When face is close to screen, playback of the gun firing video is triggered
  • When no faces are detected, video of me spinning the gun is set on loop like a ‘screen saver’
  • All videos have a 1.5seconds crossfading transition


I am rather satisfied with the outcome and more in-depth presentation of the process is in the video below. Here I really wish to express that apart from learning the stuff in MAX, what was really important to me was the asset creation in the assignment. What I have attempted was to use the assets to my advantage, the angle of the shot, the composition of the videos, thinking about transitions. And I really spent quite a bit of time dwelling on those aspects. So this played out really well as for most of the time in interacting with this piece, the transitions are as smooth as I would have them.

Also, this time I attempted to organize things a little more, showing only the essential stuff and visual information in the main patcher while the complex computations are put into the hidden ones.

What could be improved for this assignment is maybe thinking of ways to track vertically as well.