For the semester project, I wanted to make use of what I have learnt in the previous remakes. And given that the time frame is not very long for the semester project, I also decided to keep things simple. Hence, I thought of this project which makes use of motion tracking translating into servo movements from Remake 4 and Wekinator from Remake 3.
Ms Manners is a project derived from the idea of reciprocating others when they are courteous and greet you, inspired from the courteous culture in Japan. In this project, a single servo with a doll attached to the needle will mirror a bow when the camera sense a bow from the person.
Code Flow
The project flow starts with the sensing side. I used MediaPipe Holistic to sense the whole body movement because I wanted to register a particular body position moving, not just a single point moving. Also if I used only a single point, it would be the same as Remake 4 and I wanted to try something different, hence I used MediaPipe Holistic. MediaPipe Holistic consist of 543 points which includes the general body position points, the detailed hand points and detailed facial points. For the purpose of this project, I only needed the body position points, so I only took into account the 33 body points.
The points are processed in Wekinator. I set the algorithm in Wekinator to register 1 output with 2 values, 1 and 0. When the body is upright, The value will be 0. When the body is in a bowing position, the value is 1. The script is programmed such that when 0 is received, a value of 70 degrees is sent to the raspberry pi to move the servo motor. When 1 is received, a value of 160 degrees is sent over. For some reason, when 170 degrees or 180 degrees was sent over, the servo motor would not respond and it did not move.
Problems Faced
The first problem faced was the impact of indentation. In the first image, the indentation for the message loop was not done properly. As a result, only 2 inputs were sent over to Wekinator when there was supposed to be 64 points sent to Wekinator. After doing the proper indention, the Wekinator was finally receiving the correct number of input from the sensing script and the algorithm was ready to run.
Another problem was the degrees of the servo motor as mentioned. I tried with 180 degrees at first but the servo motor does not seem to recognise this angle and it didn’t move. The same thing happened when I tried with 170 degrees. The motor only started moving when I tried with 160. Since the original value was 90 at upright and 180 when bowing, I changed the final values to 70 at upright and 160 when bowing to get the 90 degrees bow effect.
After solving that problem, the rest of the steps were not as difficult. Below is the final video for Ms Manners.