The following is a step by step procedure for installing and performing Touch, a new telematic dance collaboration between Nanyang Technological University in Singapore and Arizona State University. This document is a work-in-progress that will evolve over the next few weeks until the workshop at Brunel University, April 9th, and the performance at ASU and NTU on April 21/20.
The setup combines Adobe Connect web-conferencing with Max/MSP/Jitter to capture the sources and enable video processing, audio, and interactivity. The camera sources are networked, but the resulting video manipulating and audio output must be carried out independently at each location that is participating.
The online audience will be viewing via streaming broadcast, which is not detailed in this manual.
Hardware Setup
- Connect the computer directly to Ethernet and check the network preferences to be sure it is working. Note in the figure below that the LAN connection is on top:
- Connect an external video camera or Webcam. If using an external video camera via HDMI, be sure the camera is outputting without any display information and connected to a Thunderbolt or USB video convertor, preferably Blackmagic to insure smooth operation with Adobe Connect.
NOTE: If you are using a thunderbolt video convertor, you will run out of thunderbolt ports, in which case I recommend using either a USB video convertor or USB ethernet adapter, the latter available at an Apple Store or online.
- Connect a second monitor via the mini-display port. This is used for the Max patch (see below).
- Connect a third monitor via the HDMI port or second mini-display port to the projector. This is used for presentation to the audience
- If you are using a non-retina capable computer, use 1280 x 720 for Monitor 1 resolution.
- If you are using a retina capable computer for Monitor 1, set the resolution to the left of the default setting (see figure below). This is necessary for precise sizing of the desktop capture in Max to be the right size, dimension, and aspect ratio.
Adobe Connect
- Go through the Adobe Connection test if you haven’t done so already to be sure you have the latest version of Flash, and the Connect add-in, essential for the setup. BE SURE YOU RUN THIS TEST IN FIREFOX, otherwise you will have to do it all over again. This will take a minute or two. Be sure that all four steps are checked off:
- Launch Adobe Connect with the following URL (http://bit.ly/1trzhtK) and ONLY USE FIREFOX as your browser.
- Login as a GUEST and type ‘Arizona’ or ‘London’ depending on the location.
- Click on ENTER ROOM.
- The Adobe Connect Add-in Application will then launch (if the add-in was previously installed correctly). The add-in is a stand-alone application that launches separately from the browser. Connect SHOULD NOT OPEN in a browser tab, if it does, you need to be sure you have installed the add-in in FIREFOX correctly and start over again.
- Click on the microphone icon at the top of the screen so it turns green to be able talk.
NOTE: Be careful about feedback when audio is broadcast to a PA system. Mostly likely you can use the built-in mic and built-in output on the Mac. Be careful to adjust the audio levels of the speaker volume (drop down menu to the right of the ‘speaker’ icon) and the mic volume to avoid feedback. If necessary you may need to turn the mic off whenever sound is playing.
- With the microphone icon turned on, access the drop down menu to the right of the mic icon and set the level to about 50% so that the level is not too high.
- To the right of the mic icon is the camera icon. Before you click on it, first select the drop down menu to the right of the icon and choose the camera you are using. If you are using an external camera via a video convertor, such as Blackmagic, choose the resolution 1080i60 8 bit. If not, you can use the built in camera on the computer or external Webcam, which will be listed in the menu.
- Once selected, click on the camera icon to activate the camera. With both the mic and camera icons turned on, as well as the speaker for audio output, it looks like the following in the video pod window:
- Click on the four arrow icon (see above) in the upper right hand corner of the video pod in order to go full screen. It should now look like below when there are two people logged in. Note you do not see the menubar and the video pod is now fullscreen.
- You need to download the image in the figure above to use as a template for testing Max in the steps below.
FOR THE NEXT SECTION: COPY THE DROPBOX FOLDER ENTITLED ‘PERFORMANCE FILES’ (SENT BY INVITATION). RUN THE MAX PATCH FROM THIS FOLDER FOR THE SOUNDS TO PLAY.
Max/MSP/Jitter
- (Optional) From the ‘Performance Files’ folder copy the folder entitled ‘cv.jit-1.8.0-alpha” to the folder: Documents/Max 7/Library. This will allow you to use the motion sensor.
- Open the MAX patch and place it on Monitor 2.
- Be sure the patch is in Presentation View. You can type option-command-E to cycle between Edit and Presentation View as in the figure below. Note there is a floating ‘Touch’ window as in the lower right hand corner of the figure below (it may be behind the patch), find it (you can select it in the Max Window menu if you don’t see it) and drag it to Monitor 3, which is the projection screen. (If you are not using projection, you can leave it on Monitor 2).
NOTE: Be sure that you don’t place anything on top of Adobe Connect on Monitor 1because Max is capturing the desktop, splitting it into two separate images, compositing the two, and then outputting to the projector.
Each function in the Max patch is numbered, here is an explanation:
- (1) Turn the patch on by clicking on the toggle in the upper left hand corner.
NOTE: Your graphics card will slow down significantly when the patch is turned on and capturing the desktop. If you need to use the computer for anything other than running the patch, it is advised to turn the toggle off.
- (2) To go full screen with the ‘Touch’ window, click on the escape key. Be sure the ‘Touch’ window has been placed on Monitor 3, which is output to the projector. Also, be sure Max is the active application.
- (3) Click on the escape key again to return the ‘Touch’ window to its original size.
- (4) To scrub between ‘juxtaposition’ (side by side) and ‘blend’ (composited) configurations, move the slider between ‘juxt’ and ‘blend.’
- (5) To switch blend modes, click on the numbers. Note that they are color coded for types of blends. We will probably not use the grey ones.
- (6) If you need to display the full Adobe Connect screen, such as when there is only one video connection, click on ‘1.’ The ‘0’ mode is default and is used in regular operation.
- (7) Click the toggle to enable communication between the wearable and the computer.
- (8) Click the toggle to enable two motion sensors for location 1 (left side) and location 2 (right side).
- (9) To adjust the brightness of the left and right sources, click on 1.4 (bright) or 1 (normal).
Sound
- (10) Click on the toggle to turn on audio. The red fader raises the volume for the Master stereo. The audio is set to go out through the built-in sound system, if you are using an audio interface, please ask for direction.
- (11) Each of the six characters has their own soundfile. ‘Queen’ is Angeline in Arizona, and the rest are the students in Singapore. With the Master audio turned on and the level ready, click on any of the solo toggles to play the sound for that character from the beginning to the end without looping. The volume will automatically jump so be careful your system is not too loud.
- (11) Clicking on the Ensemble toggles starts the playback of each soundfile associated with the character in loop mode, requiring control of the mixer faders to balance the levels according to the needs of the performance.
Stage and Performance Instructions
- The computer and Monitor 2 are offstage but in a position where the operator can see the performer(s).
- The camera is positioned stage right or left, slightly angled from downstage, so as not to capture the projections.
- The performer will reference the projection.
- Use focused light on the performer, who will need to be close to the camera and brightly lit in order to fill the screen and be seen in the Max processing.
- It is possible to have an on stage camera operator changing position and zoom, but not necessary. This is also up to the performer.
- When the work is performed, an operator in each location will be communicating cues to change settings via Adobe Connect in between each section of the performance.
If you have any questions, please contact Randall Packer (rpacker@ntu.edu.sg).