Ideation

I thought of an idea that involves using closed captions. I had never used closed captions on Zoom or on any other platform before this so I did some research. I found several options for closed captions on video call. Firstly, there was the inbuilt closed captions in Zoom where somebody would manually transcribe the captions as the call was happening. The second option I found was a third-party software you could connect to Zoom, called Otter.ai which would transcribe live from listening to the audio in the call. My idea came when I found out that there was a closed caption function on Google Slides when one was presenting the slideshow, which would take audio from the person’s microphone and transcribe it live as subtitles below the slide. 

When I saw this I instantly thought of a karaoke video that had lyrics at the bottom. With this in mind I sketched out my idea and wrote down the flow of the performance. During the class in Week 2 we also shared our ideas on Zoom and I got to see everyone else’s ideas too!

My final idea was as follows:

• I share screen on Zoom with the performer, sharing Google slides with closed captions.
• I read out the lyrics to the song as the sound starts, but I am on mute so the other person cannot hear what I’m reading, but can only read the closed captions on Google.
• The other person has to try and sing along to the backing track with the “lyrics” given by Google’s CC.

In short, I am part of the performance reading out the lyrics, which goes through a computer transcription into closed captions for the second performer to interpret into song.

Process

The setup for the performance was fairly simple, I created a slideshow on Google Slides, with the first slide being a buffer to the real start of the performance. I decided to make it look like an old Karaoke start screen. The song I chose was Knowing Me, Knowing You by ABBA, which had a pretty clear beat drop into the chorus.

After that I added a karaoke backing track to the song as an audio clip to the second slide, which would automatically play as I clicked onto the next slide. I also included some stars and animated them to fill up the duration of the instrumental beginning of the song. In the middle of the slide I added a space for the “singer” and I requested for them to set their virtual backgrounds to black so they would blend in with the slide.

I asked Daryl and Amanda to help me with the performance. Daryl has listened to the song before but Amanda has not, so I thought it would be interesting to see the differences between their reactions. Amanda told me that there was actually a lag between when I spoke and when the captions appeared on her shared screen, so that also added to the mismatch and out-of-sync experience. Daryl was a little more apprehensive actually, as he was trying to read the lyrics but also remember the lyrics he had in mind from what he knew of the song. Either way it was a really funny performance to try and see their interpretations (and struggle) in reading the lyrics in tune with the music!

Here are the performances: