Final Sem Project: BALLance

The goal of BALLance is to balance a ball for as long as possible without having to physically touch it. By redefining the idea of physical toys, BALLance is made to exemplify the idea of telepresence. It gives participants a façade of being present at a place other than their true location. Being able to play BALLance remotely, this system renders distance meaningless.

General Flow of setup:

Leap motion takes the:
– angle of HandPitch which translates to the front and back movement of Servo motor 1
– x position of the middle finger which translates to the left and right movement of Servo motor 2

In order to release a ball:
– When a pinch is detected, Servo motor 3 is activated.

Tracking High Score:
– When an orange colour pixel is detected, the timer starts. When an orange pixel is no longer detected, the timer stops.

Process:

Structure

From where I left off, the low fidelity prototype, here are the updates to the structure:

The main issue with this structure is that I totally forgot the body of the motor needs to be mounted on the structure so that only the rod turns. Thats why in the photo it is held on by rubber bands. After dealing with the mounting, the corners were in the way and needed to be shaved.

Also the snout of the motor was SO tiny the rod barely held on so I needed to find another way to attach it.

Changes made:

The other motor for front and back.

Calibrating the board

From the video above, the next alteration from advice of LP is to add a cloth and thats what I eventually settled with. Works great!

Highscore

Basically works on: looking for a pixel closest/is orange and starts timer when that pixel is found aka ball rolls in. Timer starts. Timer stops when pixel is gone aka when ball rolls away. Compare this time to previous high score time. If it is longer, becomes new high score.

Initially had some problems with setting up a camera, but problem solved using Processing 4. Also, managed to set up DSLR with Camera Live and Camtwist through Syphon to set up a virtual webcam.

Final thoughts:
This project was so SO much trial and error. Because there was not one fixed result I was looking for, sometimes I was not too sure if the ball is rolling off because the ramp is at the wrong angle? Or was it the board is small? Or simply because I was lousy at the game and lack hand eye coordination. I had to try many various calibrations, keep picking up balls to try my best in finding a good in between. I also had to brainstorm a way to get the ball onto the device without having to physically touch it, since I felt that could vary the gameplay depending on how the ball was placed onto the device.

Overall, if given more time, having the feedback from user testing, I believe if I could some how calibrate the board toward the Z-axis, this game could be a-lot more convincing.

27 Oct Update

just a heads up im stuck

Right now, the goal is to have a camera to detect if the orange ball is still on the plane- if yes, timer runs/ if not, timer stops.

I have tried a few methods to achieve this but every one of it doesn’t work.

Processing + Macbook Facetime HD Camera
Used the ‘Video’ library of Processing
Problem as per screenshot:
Unfortunately, I am on a macOS Catalina ver 10.15.5https://github.com/processing/processing-video/issues/134
Tried the solution as suggested by ihaveaccount on this above forum but the string doesn’t return as they said it wouldwithout the video library, this eliminates the use of external webcams also (i tried with my DSLR as a webcam)
Processing + IP Camera
Since I was hoping for the webcam to be portable anyway, I tried to use an IP Camera.
Problem: On my processing sketch, nothing shows up, no error, no video feed show- nothing. IP camera doesn’t show it is being connected to anything either.

Not sure if its because im accessing processing via a Macbook and im using an android IP Camera app?
TouchDesigner + IP Camera
Used the Video Stream in, and with RSTP network protocol, while yes there is a video feed, it is SOOO choppy and pretty much only refreshes when I “pulse” the node
TouchDesigner + Web Camera
https://forum.derivative.ca/t/video-device-in-webcam-doesnt-work/12208/13
Once again seems like everyone else with Mac ver after 10.14, webcam doesnt seem to work
I did try with the latest beta ver of Touchdesigner however, it only works when I first start a timeline, once I save the file and close it and then reopen, its just a black screen again.

So now Im stuck. Im thinking maybe a Raspberry PI(since it runs Windows) with a webcam, or a camera module and maybe something along the line of python w OpenCV library, or it will be great if i could have processing on Raspberry PI. But this is a whole new set of problems because first I don’t have a Raspberry PI and just learning to work it is like 😵🤯😩.

Low-fidelity prototype

Front Back tracking
Had to lift my whole hand to change the angle of the servo
Couldn’t have my wrist in place and just adjust my hand

After inserting both motors to track both front-and-back and also 360 (in which this case still 180 because using servo), the motors are going CRAZZYYY until I use just 1 finger. Likely because its taking coordinates of all 5 fingers. But, still an issue was even though just 1 finger was stuck out, the leap motion is still trying to its best to get readings of all fingers.

Kinda works
Looks very rigid and tried using a board and a bead to simulate the game for now,
realised UX might be abit confusing. because the bead starts off being still on the board so no point in moving your hand if the goal of the game is to balance the ball on the plane for as long as possible

After consultation:

DC motor might not be the most practical because cant control angle of rotation, only controls the count of rotation- in which needs sensors to send feedback also
Build a more stable structure, have the motors screwed on!
Adjust the code such that if the movement is too small, the values wont be sent over to arduino- to clean up the jerky-ness
Webcam to detect where the ball is , to start and stop timer

Final Pitch: BALLance

Previous pitch: https://oss.adm.ntu.edu.sg/ho0011an/final-project-pitch-build-me-up/
Short recap Basically, I was too fixated on the idea of using weighing scale as the input just because I wanted that specific aspect to tie the project to the theme ‘Distant Body‘. But, overall, the initial pitch didn’t really make sense and I was left off with the question- what more can you do with ‘reflection’? & advised to do experiments first.

So- that’s what I did.

Using TouchDesigner, I tried to move a red line to where there is motion.

Trying to replicate Daniel Rozin’s trash mirror but on a digital screen. This led to a couple of random ideas.

Brief explanations on some ideas
Idea 2 Thinking along the lines of trying to create a tangible user interface perhaps where the physical ball is able to move according to the projected graphics or vice versa where the projected graphics changes according to the placement of the ball.
Idea 3 Maybe a screen based device and user can manipulate the graphics in ‘3D’ space?
Could be in a form of 2 screens or maybe 1 screen but with acrylic to give a hologram look

What Harvey Moon said: “you don't need disguise or unreal or stype to do the virtual production stuff. you can do it…

Posted by TouchDesigner on Saturday, 26 September 2020

Idea 4 Individual modules will move up to replicate the shapes of what is detect on the webcam. EG pin needle toy

With all these ideas, I was further challenged to do something with the ball. To perhaps make it into a game.

So after deliberating more, my final sketch idea is an expansion of Idea 4.

It going to be a game and the only goal is: control the plane, remotely with your palm, such that the ball stays on the plane for as long as possible. Time starts when the hand is detected and stops once the ball drops off the plane. Longest time = Highest on the score board.