Category: Experimental Interaction – G5 (Thursday)

“Pat-Pat”
By Ayesha and Man Wei

PROJECT DESCRIPTION

 

Our product, the Pat-Pat, is a dark object that explores feelings of loss, loneliness and nostalgia.

On the surface, we are creating a product that mimics and potentially replaces a mother’s patting her child to sleep. The Pat-Pat is catered to adults who long for this comforting ritual that they experienced during childhood, or perhaps to those who never experienced it because they lacked a maternal figure. With the Pat-Pat, one could easily relive these feelings of comfort and childhood nostalgia by purchasing a bedside appliance.

However, our aim is to subvert this idea by proving that it is impossible to replace or recreate the warmth and comfort of your actual mother’s patting, and to evoke a sense of discomfort at the idea of trying to replace your mother with a mechanical appliance.

 

USER TESTS: OBSERVATIONAL DOCUMENTATION 

TEST 1: Bodystorming Exercise

(Refer to Bodystorming OSS post)

TEST 2: IN-CLASS PLAYTEST

Details on stage of development:

• Environment – blanket laid on floor, pillow with sensor
• Pat-Pat Mom – made of cardboard, hand-drawn details, chopstick arm
• Interaction – motor hand activated upon resting on pillow, adjustable speed, no sound

NOTES:

- Make arm longer
- Suggestion to change position to face Mom, instead of back-facing
- Nice to customise speed
- Clear instructions
- Unsure if slider on phone worked
- Ashley didn’t lie down completely so it didn’t work properly
- Customise the speed, some people lie it fast
- Make the hand bigger
- Eugene shifted his body to adjust to the Mom after seeing Wen De fail
- If the presentation/user experience setting is framed like a sales pitch product test, classmates will be apprehensive about approaching set-up: awkward
- Users prefer experiencing product on their own (instead of guided): more personal being left with just instructions 
- “Greet Mom” instruction a bit awkward 

FInal critique user test

Details on final deliverables:

• Environment – room simulation with dark room and bedlight, mattress, pillow and blanket
• Pat-Pat Mom – made of white mounting board, vector-printed designs, extended arm
• Interaction – motor hand activated upon resting on pillow, adjustable speed, no sound
• Secondary elements/presentation – online website and shop, user guide video, Blynk app with designed interface, physical user manual, packaging with customisable clothes

Online website & store: https://manweileong.wixsite.com/website

NOTES:

1 Hamimah 
- Really liked the soft and hard force changes, responsive 
- Think it’s funny 
- Harder patting quite shocking but fun
2 Emma
- Like customizable elements: face and clothes
- Tried to adjust to harder force but unable to 
- Got scared by it tapping on her butt
- Feels embarrassed in front of everyone
3 Wende
- Very comfortable didn't even remember to test out the strength
- Thought of grandmother who passed away and used to pat him
- Cloth that they used reminds him of grandparents
4 Lei
- Mechanical sound reminds her of 'sarong spring' cradles used for babies
- Lighting is effective in setting up mood
- Likes choice of fabric coinciding with mother's clothes  

How do testers feel with and without music (silence)?
- Wende: reminds him of the baby and swing
- Hamimah: too engrossed doesn’t really matter to her 
- Music can give a sense of what to feel but also be taken differently

 

DESIGN PROCESS DOCUMENTATION 

{ Product Design

1 Initial idea

• Product = Cardboard standy Mom + baby mattress + sheet covers/blanket (with fabric button)

2 After playtest

• Product = Cardboard standy Mom + pillow

3 Final

• Product = Cardboard standy Mom only

We refined the decision to have the pillow framed as part of the bedroom environment, instead of being part of the marketed Pat-Pat product. The idea was to have a portable Mom that could be brought anywhere, set-up by any bed, (contrast with physical distance between mother and grown-up children) so a pillow would be bulky. We didn’t need to include the fabric button in the marketed product– we needed only to suggest that the product worked so and so (VS the functioning actual product supporting this narrative).

}

{ Aesthetic Considerations: Unpolished VS Refined

This was an important design consideration we had to constantly return to and mediate between creating an unpolished and refined product. The finish and aesthetics of Pat-Pat directly translates how genuine/insincere we are in trying to create a product that is able to fulfil its postulated functions (replacing the actual patting experience and serving as a substitute mother). A balance of both approaches was crucial to communicate our work’s intent.

}

{ User-friendliness/ Functionality

We had to conduct a few tests to figure out a suitable:

• Height for Pat-Pat Mom (corresponding with mattress set-up), such that she could pat people of different heights
• Length of arm, such that user could be a comfortable distance away from the Mom and not risk knocking her over by shifting their bodies
• Position for Pat-Pat marker sticker

}

{ Input/Sensor: Fabric Button

1 Initial idea

• Small circular pressure button hidden under mattress sheet at butt position

2 Final idea

• Pillow-size pressure button hidden in pillow

Before running the playtest, we tried the patting ourselves and discovered that having the pressure button positioned at the butt started the patting motion before the user could properly lie down and position his/her body. This was problematic because the patting hand would get stuck/obstructed by the body that was still shifting and not yet in the optimal position. To resolve this, we shifted the button to the pillow, since it is the last part of the body to come into contact with the bed set-up while the user tries to position him/herself. For this new set-up, we had to design a larger pressure button to cover a greater area of the pillow. This also worked better in making the product more sensitive compared to having a small spherical circle that users were more likely to miss.

}

{ Music

1 Initial idea

• Have polyphonic mechanic instrumental soundtrack of lullaby (via Processing/buzzer speaker)

//BODYSTORMING: used unedited soothing instrumental lullaby for simulation//

2 After bodystorming

• Have robotic/mechanical voice singing lyrics to lullaby

The music made a big difference in inducing the mood of a comforting experience. This wasn’t quite our intention. At least in terms of music, we wanted to emphasise the machine nature of the Mom and cause unease or discomfort.

We decided also that instead of an instrumental soundtrack, we wanted to play an actual voice singing lyrics to a lullaby instead to evoke the idea of a Mom singing. The bodystorming music sounded a lot like just music playing in the backdrop.

//PLAYTEST: experienced without sound (did not manage to record a polyphonic singing voice and program it with the rest of the arduino circuit)//

3 After playtest

• Remove music completely

Running the product without music, we became aware of the mechanic sound of the motor itself, previously concealed by the lullaby. It made a rhythmic sort of white noise that was comforting almost, which we felt was appropriate as a lullaby from a machine. More importantly, the sound allowed users to be aware of the presence and functioning state of Pat-Pat. Where they could not feel the patting on their butt because they were not in an optimum position, the sound of the motor rotating hinted to them that the machine was working rather than malfunctioning. The sounds also helped signal changes to speed/force of the patting by adding an auditory dimension where you could hear the rhythmic differences. For the onlooking non-testers, this helped them better appreciate how the speed/force changed since they were not feeling the patting themselves. Without the sound to signal the changes, they could only rely on seeing the hand change its angle of patting.

Additionally, after watching how the motor started and stopped almost like it was glitching when testers shifted their bodies, pressing and missing the fabric button repeatedly, we imagined how the abrupt starting and stopping of a soundtrack the same way might be distracting. A broken soundtrack disrupted by the input state would make the product sound like its glitching and probably scary even, distracting from the patting action itself too.

4 After tech consult

• Remove music completely

We had planned to nonetheless continue the coding with processing that we started in order to play the lullaby, and test ourselves how that affected the experience of the product. After consulting Lei however, we found out that we could not connect all 3 programs (arduino, Blynk and Processing) at the same time because of serial port issues. So that cemented our decision to remove music entirely and work with the “music” of the motor.

}

{ The Hand

We tried different ways of attaching the arm to the motor, considering the durability and aesthetics each material allowed. Materials tried:

• Rubber band
• Cloth/duct tape
• Masking tape

Initially we used only a wooden chopstick for the arm but after the user tests we got feedback that the arm was too short. Although we conceptualised our design to be user “unfriendly” (requiring the user to adjust and fit him/herself to the product instead of the reverse), we still wanted the design to have a “minimum” threshold of functionality or user-friendliness. So we decided to extend the length with a dowel instead.

However the dowel was too heavy, together with the momentum created by weight of the hand attached at the end. We tried using two lighter chopsticks joined together as well but the same problem persisted with the resultant momentum. We ended up overworking our servo motor as it slowly became unable to support the weight and made sounds like its gears were going to break.

We resolved that we had 2 solutions:

1. Change the rubber hand to a lighter material like a cut-out cardboard hand
2. Buy a stronger servomotor able to withstand the weight and momentum of the rubber hand

We ended up going with 2: we bought servo motor MG995(180) with a higher stall torque in order to support the hand on the dowel. After changing the motor, we had to make adjustments to our code for a new suitable angle. Even with this new motor however, we faced problems. This motor was somehow more affected by obstructions to its motion, i.e. when its arm contacted with the butt and could not complete the full angle of rotation. The obstruction (needed to create the effect of a hand being held longer for the pat), together with the weight and torque produced, caused the screw to loosen over time, making the motor unable to lift the arm when this happened. We had no full-proof solution to this and could only make sure to screw and rescrew the motor arms to ensure the patting could last at least 3 user tests. We also adjusted the slider angle limit for Blynk to a smaller angle (from 120 to 90 to 80 to 60) in order to reduce the duration of each time the motor was obstructed/stopped by the butt, so that there was a lower risk of the motor spoiling/screw coming loose. We resolved that to prevent the scenario of the hand stopping too early and not coming into contact with the butt of someone who was shorter, we would change the limit of the Blynk slider accordingly to better fit the user. (Higher range for shorter people capped at 90, and lower range for taller people around 60).

}

{ Installation Environment & Context

1 Initial idea

• Mattress – baby mattress reduced scale, kiddy prints for bedsheet

For the audience, the visual composition of the user curled in foetal position on a tiny baby mattress that they have clearly outgrown would help communicate our concept (how the mother-child bond has changed since they used to be small enough to fit the mattress, how the machine and set-up cannot measure up and replicate the actual pat-pat experience).

However this approach did not fit with our product narrative of it being for set-up in buyers’ own bedrooms. Since we could communicate our concept through other means, we ditched the kiddy aesthetics to simulate the bedroom of a grown-up person instead.

2 After playtest

• Mattress – smaller-than-single size, plain prints for bedsheet (grown-up)
• Table lamp and dark lighting to simulate room environment at night

From the playtest, we received feedback that framing the product in the context of a sales product test walkthrough/demonstration would feel awkward or intimidating for users, and also rushed. They preferred going through the instructions on their own and felt that this allowed their experience of the product to be more personal.

Hence for final critique, we decided to create an environment that embodied both contexts. On one hand, we tried to simulate a setting that allowed users to imagine their own bedroom environment, to allow them to experience the product on their own without our guidance/sales directives. But we also maintained and made explicit the idea that the Pat-Pat is a product that they have bought, through the display of a shopping bag, laid out user manual, and instructions that start off with “after setting up, to get started”. The instructions were meant to imply to users that they had already set up the Pat-Pat themselves in their own room (that we created for them).

}

{ Instructions Design

We considered a few alternatives for delivering instructions in the context established above:

• Written instructions only VS
• Pre-recorded narration step-by-step + written instructions

a. Broadcasted over speaker: single audio with instructions strung together (own coordination, without any coding)

b. Broadcasted over bluetooth speaker connected to processing, activated sequentially by user who sends sensor/switch input after performing each step (e.g. arduino button/button pin on Blynk that users press each time to display/hear the next instruction)

c. Narrated by Blynk app itself as an audio user guide

PROS:

We thought how narrations made it less awkward for users to do the unnatural reading aloud of instructions themselves for the class to hear, and could also help with pacing the experience. In the playtest, testers rushed through all the instructions at one-shot from the start before executing the steps, instead of following through step-by-step. This meant that they paid less attention to registering each step, especially the instruction disclaimer for them to adjust their body accordingly to fit the Pat-Pat– which was important in conveying the idea of our product as user unfriendly to a certain extent, and helping users get the most of the experience.

CONS:

User might find the narrated instructions scary/awkward. They might prefer to have instructions to hold and figure out themselves, at their own pace (VS narration pacing). Possibly more intimate experience. Narration could also be distracting

We decided to stick to written instructions only after gathering feedback from classmates. To make the product narrative more cohesive, we hosted the instructions on the Blynk app itself together with the slider and offered a hardcopy user guide as well.

}

{ Business Model Ecosystem

We constructed a comprehensive business model ecosystem with an online website and store, and physical product deliverables (paper bag with logo, zine user manual) to “complete the product narrative”. We came up with a colour scheme and typeface for our brand identity and researched on existing online shops and user manuals (Apple Store and Fitbit manual) on how they framed their products and wrote their documents.

We paid particular attention to the aesthetics of the Blynk interface as well since the motor adjustment was the highlight of our product. We added tabs as well as images to make the Blynk app seem like an app we created ourselves called the Pat-Pat Assistant.

CODING DEVELOPMENT DOCUMENTATION

At the start of the project, after brainstorming and narrowing down our choice to building a product that mimicked a mother’s patting, we consulted Lei.

Our original idea included:

Input

• Photocell OR pressure sensor OR fabric button
• Blynk virtual slider

Output

• Servomotor
• Buzzer OR speaker for sound playback

Lei steered us to the Arduino Blynk IFTTT tutorial in Week 9’s Design Noir post, as well as the Servo > Sweep example found in Arduino. The IFTTT code allows the servomotor in our set-up to be controlled wirelessly by a virtual slider on Blynk, changing the direction that the blades are facing according to the value on the slider, while the Servo Sweep code turns the servomotor blades back and forth on a loop and the speed can be changed within the code.

1. Code for Arduino Blynk IFTTT

2. Code for Servo > Sweep

Both codes worked successfully on their own, but we needed to combine them so that we could change the direction via Blynk slider while also adjusting speed. We also needed to add in an analog sensor as the trigger. In this case we used a photocell. However, when we tried to mix them together, our code was incorrect so it did not work.

3. FAILED CODE: Mash-up of IFTTT + Sweep + Photocell

We also tried a code we found online (https://community.blynk.cc/t/c-blynk-code-examples-for-basic-tasks-work-in-progress/22596/6)  that used a button + slider on Blynk to control a servomotor. The original code used a wifi connection, so we changed those parts to serial port language.

4. CODE for blynk + timer + Servo

This code worked, however, the button used here was a virtual one in Blynk. This was a problem as our Pat-Pat machine was supposed to be triggered by an analog sensor that responded when the user laid down on the mattress.

Another problem was the speed of the servomotor. The blade turned very slowly in steps and we tried to adjust it but couldn’t figure out how to tweak the code to change the speed.

We abandoned this code as we felt that the previous ones served our needs better if we managed to figure them out. However, in trying out this code, we learned something crucial: how to write the variables for Blynk and incorporate them into our original code.

5. CODE for MASH-up of ifttt + sweep + photocell reworked

Observing the code in #4, we applied what we saw to change code #3 so that the while the servo was moving at a constant set speed, the slider on Blynk would change the angle that the servo blade would return to. This meant it wouldn’t change the speed directly, but rather indirectly by either lessening or extending the range of motion of the motor.

What we learned and applied from code #4:

1. We need to declare:
int servoPos;
2. CHANGE FROM:
BLYNK_WRITE(V3)
{
myservo.write(param.asInt());
}
TO:
BLYNK_WRITE(V3)
{
 servoPos = param.asInt();
} 
3. WRONG: 
(1)
pos +/- = servoPos
servo.write(pos)

(2)
pos + = number
servo.write(servoPos);

Pos - = number
servo.write(pos)
<Reason: return angle and position different>

CORRECT:
(1)
pos + = number
servo.write(pos);

Pos - = number
servo.write(servoPos)
5 - 10 - 7/8

After the excitement of managing to complete the first part of our code, which was to use an analog light sensor to turn on a servomotor and remotely control the speed of the servomotor via Blynk, we moved on to the hard part. Our original idea included a lullaby that would play at the same time as the servomotor turned. Lei advised us to use Processing, which can be connected to Arduino via the Serial Port USB.

6. failed code for #5 + processing code

import processing.serial.*;
import ddf.minim.*;

Minim minim;
AudioPlayer player;

Serial myPort; // The serial port
int val = 0;

void setup() {
 myPort = new Serial(this, Serial.list()[0], 9600);
 minim = new Minim(this);
 player = minim.loadFile("rock_a_bye_baby.mp3"); 
}

void draw() {
println(val);
if (1<myPort.available()){
  val = myPort.read();
}
if (val==1)
player.play();
}

We tried many different codes that we found online and tried to adapt to our project.

https://learn.sparkfun.com/tutorials/connecting-arduino-to-processing/all

http://forum.arduino.cc/index.php?topic=19584.0

https://bgsu.instructure.com/courses/1157282/pages/tutorial-sound-arduino-to-processing

http://cmuems.com/2013/b/sounds-with-arduino-and-processing/

However, none of them were able to play the music and this was the error we encountered over and over again:

Error message on Processing

However, when we tried the code without connecting to Blynk, we found that the music player worked after all (although we still had a problem with getting the music to pause when the sensor wasn’t triggered). This meant that Blynk and Processing could not work together at the same time. We tried to confirm this online but found nothing, and only when we consulted Lei again did we learn that it was not possible to connect two applications to Arduino via the same Serial Port, so we still kept trying until we neared the end of the project.

For our circuit, we also created a fabric button that we could easily conceal within a blanket or pillow case. We DIY-ed it with aluminium foil and some foam, and then we had to change the code. Since the fabric button basically had two states, we changed our code to reflect that.

7. code for new fabric button

Then, we tried Processing again (this was before Lei told us), hoping that since the fabric button would only be sending HIGH or LOW to Processing, it would be easier to figure out our problem, but it didn’t work, unless we disconnected Blynk by commenting out all the parts of the code for it.

8. processing + fabric button

import processing.serial.*;

import ddf.minim.*;


Minim minim;
AudioPlayer player;

Serial myPort; // The serial port
int val = 0;

void setup() {
 myPort = new Serial(this, Serial.list()[0], 9600);
 minim = new Minim(this);
 player = minim.loadFile("rock_a_bye_baby.mp3"); 
}

void draw() {
if (0<myPort.available()){
  val = myPort.read();

if (val==1){
player.play();
println(val);
}
else{
player.close();
}}
}

void stop()
{
  player.close();
  minim.stop();
}

We still couldn’t find a way to make it pause, even though we tried adding player.close(); commands.

We were hoping to get Lei’s help to fix our whole Processing problem during consultation in Week 13. But it turns out it wasn’t possible to incorporate Processing at all as we already used Blynk, and both apps were trying to communicate with Arduino via the same serial port. We did, however, get her help to make the music pause when the button was not pressed.

9. final code

In the end, this was the code that we needed:

#define BLYNK_PRINT SwSerial//


#include <SoftwareSerial.h>
SoftwareSerial SwSerial(10, 11); // RX, TX
#include <BlynkSimpleStream.h>
#include <Servo.h>
char auth[] = "a9e72ed499b144d5a97cd7d2fb1db43f";
int servoPin = 9;
Servo servo;
int pos = 0;
int servoPos; 
const int buttonPin = 2;
int buttonState = 0;

void setup() {
  pinMode(buttonPin, INPUT);
  SwSerial.begin(9600);
  Serial.begin(9600);
  Blynk.begin(Serial, auth);
}

void loop()
{ 
  Blynk.run();
  buttonState = digitalRead(buttonPin);

  if (buttonState == HIGH) {
    servo.attach(servoPin);
    for (pos = 0; pos <= 180; pos += 6) { // goes from 0 degrees to 180 degrees
      // in steps of 1 degree
    servo.write(pos);              // tell servo to go to position in variable 'pos'
    delay(15);  // waits 15ms for the servo to reach the position
  }
  for (pos = 180; pos >= 0; pos -= 6) { // goes from 180 degrees to 0 degrees
    servo.write(servoPos);              // tell servo to go to position in variable 'pos'
    delay(15);                       // waits 15ms for the servo to reach the position
  }
  } else {
    servo.detach();
    servo.write (0);
  }
  delay(100); 
}

 BLYNK_WRITE(V2)
{
  servoPos = param.asInt();
}

 

INSTRUCTABLES PROCESS DOCUMENT

 

INDIVIDUAL REFLECTIONS

I wonder if I got carried away with the whole polishing of set-up and business ecosystem thing. The module was about interaction so the highlight/star of the show should have been the product with its interactive mechanisms. I felt slightly like I placed more emphasis on the website and fictional business narratives I had built, than the product itself. That is not to say that I did not pay great attention and consideration to the design of the product, together with Ayesha. Admittedly our code is relatively simple and we had more elaborate ideas we explored, but after user tests and discussions among ourselves, the final code and interaction we produced was what we felt best suited our intentions and concept.

Going forward however, I hope to create works that can be more effective and powerful in communicating their intent on their own. Definitely I recognise the importance of having a polished work “holistically”: interaction, design, presentation, concept etc. It was clear from works like Kaitlyn and Alvin’s HIDEBEAST hat how the “secondary elements” like packaging and (very engaging) promo video could really make a difference to the user experience. After viewing everyone’s works however, I thought it was also possible to make highly powerful products on their own. I felt this about June and Jia Xi’s teddy bear. They didn’t have elaborate “secondary elements” to compliment their product, nor a very polished box for the bear, but the bear itself was sewn together beautifully and the interaction it provided was so carefully curated with the strings of text and light sequences. I wonder what viewers might take away differently from just experiencing our Pat-Pat alone, if it hadn’t been set-up in the elaborate business model we constructed (most perhaps felt it was completely unnecessary/excessive).

Final critique session aside, the whole development process has been a trying one that I’ve gained and learnt a lot from. Compared to our Disobedient Objects project, this final project was met with a lot more challenges. There was a lot more trial and error and trying of different codes from forums to figure out how to play sound on processing, connect processing to arduino successfully the way we want it, and figuring out how to vary the speed/force of the patting. There was also the headache of trying to connect all 3 programs (arduino, processing and blynk) that we tried for days before finding out there was no solution for it after consult with Lei.

In terms of hardware we also had a lot of setbacks with the motor. Attaching the rubber hand to the end of a stick, to the motor created a lot of weight and torque that caused our servo motor to malfunction. The screw with the motor arms came loose, the gears got forcefully turned by the weight, the angle of rotation changed several times… Eventually our micro servo motor not only made cracking noises but also became unable to lift the hand at all completely. We bought a stronger motor after but even that faced some of the same problems and we had to try our best to reduce the weight and torque, and adjust our codes accordingly for the angles. Overall, I’m quite proud of myself and Ayesha for accomplishing what we have done 🙂 (yay hi-5 gurl)

I think the final project really allowed me to combine all the things I learnt over the whole Interactive module. In my own design of Pat-Pat, I applied lessons from the Design Noir artists on unpopular and critical design. Meywa Denki that I researched on for Researcher of the Week was a big influence in the fictional business narrative and unpopular design direction. As a tester for Gwen and Deb’s Silent Pillow, I felt firsthand the thrill of waiting for my turn (and how onlookers were participants of an art installation this way too) that Wen De explained in his Uncomfortable Interactions presentation on “Breathless”. In the creation process itself, I thoroughly appreciated the open-source culture of arduino, processing and blynk. This final project didn’t have our micro-project brief of being an open-source/crowd-sourced work, but its very creation is based entirely on help Ayesha and I found on online forums. Our final code is a mix mash of suggestions and codes plucked from different people and places. Together with the feedback gathered from our classmates, we consolidated and refined these with our own artistic direction.