Continued Reading of See This Sound: Audiovisuology

Here are some remarkable scientists/artists as I walk through the history of sound and image making.

On Hearing Eyes and Seeing Ears:

A Media Aesthetics of Relationships between Sound and Image

by Birgit Schneider

Ernst Chladni

Sound Figures (1787)

-Chladni created all these patterns simply by stroking a violin bow along the edge of a glass or metal plate that’s been sprinkled with fine sand.

-Chladni interpreted the patterns as “knot lines” and “knot circles”. Each sound has its own patterns.

-He carried out the experiment for 20 years with various shaped plates.

-An important part point of reference for the history of color music and synesthesia. His sound figures are considered the first systematic attempt to visualize sound as images.

Maximilian Plessner

Die Zukunft des elektrischen Fernsehens (The Future of Electric Television, 1892)

-In the text, Plessner conceived applications for phenomena involving electrical transformation, ranging from the artistic to the aesthetic-analytical to the practical domains.

selenium cells

-Much of the text devoted to the use of a selenium cell to transform sounds into images or to listen to images as sounds. (Which will be known today as sonification and visualization.)

-Proposed transforming sounds into optical phenomena using Alexander Graham Bell’s Photophone and creating an instrument called an “Optophone”

-Hopes to create a “unity of beauty” between sound art and spatial visual art.

Fritz Wilhelm Winckel

Sound/Image Transformations by Means of Television (1930)

Generation of sound patterns of classical music on the screen of a Nipkow television system (1930)

-Winckel believed that it is possible to represent acoustic impulses in optical form.

via la-radiovision.fr

–

via http://www.provideocoalition.com/mechanical-television/

-Winckel attached a radio to the image line of the TV as these two media process electrical oscillations within a similar spectrum.

-The end result is a symphony of jagged contours and pianissimo generated indistinct, cloudy figures.

-Harmony from harmouniously constructed, two dimensional mathematical curves.

Raoul Hausmann

Optophone (1922)

First Optophone created in the 1913’s

*Definition by Oxford Dictionary: An instrument designed to enable a blind person to read, in which a photoelectric cell is used to scan a text and produce electric signals that are converted into audible ones corresponding to the different characters.

A little backstory here in the video

-Hausmann created his very own iteration in the 1919

-The idea was to create sound and visual at the same time with the press of a button

-Heavy Dadaist influencee

-Consists of quartz and glassp rism and a neon lamp and slenium cell

-Pressing keys send spectral colors and bands of lines into the optical system. Photoelectric cell transmits electrical impulses into loudpseakers.

-Visuals: abstract rainbow patterns refracted in a crystalline manner by the quartz and glass.

– Sound: Crackles and technical sounds of various pitches.

Note: There were no actual picture and video documentations of the Optophone created by Hausmann, therefore some speculated that it was never actually built back then.

Raoul Hausmann revisited, Peter Keene, 2004

An Interesting video that I came across depicting how an Optophone reads type and traslates it into sound of various tones.

Really really useful stuff.

Research for week 7: On Modularity

Last week’s consultation directed me to zero in on the focus of making something more modular. Placement of bits and pieces and visuals and crafts can lead to different end results in creating sound.

In my first prototype( in progress) outlined in my previous post, my process shows that the assemblage could be in a way a tribute to the birth of analog music. Having a rotary system in my work reading a rotating visual of perforated holes or patterns feels pretty much like a phonograph.

Ina directed me depart more from that and go into creating a more modular construct which can create sets of different results when different modules are being removed or added.

Interesting Note: Leafcutter John, the guy giving the tutorial is one fifth of the Experimental Jazz band Polar Bear. I’ve been running through his youtube page for some interesting experimental sound tutorials that really helped me.

The following are some interesting case studies that I came across last week that triggered my idea for the second prototype:

Via Video Description

” This is a demo video of ‘SoundLightSound’. ‘SoundLightSound’ is an audiovisual performance exploring feedback between sound and light. The amplitude of the sound is controlling the density of the light. An inductor is placed on each light, transforming the flowing current into sound, feeding it to the computer. It explores the transformation of data and the feedback that can be created between different media, often resembling the distortion of information (e.g. the news) and the vicious circles created by them.”

Via Arduino Blog

“Developed by media artist Bojana Petkovic, Swamp Orchestra is an interactive sound installation that mimics the natural chorus of swamp creatures. The project is comprised of 16 light-sensitive sound modules, with each one producing noises from insects, frogs, amphibians, birds and other organisms. Each module responds to a flashlight, and the sound varies based on the amount of the light.”

Skimming through these interesting cases that made use of electrical current sounds and feedbacks, I felt that these are once again some intangible processes being brought to light.

So the next step to me naturally became: what if I can make electrical currents, in their rawest form, sing and communicate a symphony for us?

I immediately drafted a plan for multiple inductive pickups on a single board reading multiple electronic devices.

In this simple sketch, the idea of sending multiple signals to a single amp might not be feasible. Therefore, I’m thinking that it would be the best if I used multiple individual amplifiers for each pickup so that circuit noise wouldn’t drown out the feedbacks that I want from the pickups.

The English Surgeon: My Thoughts

Right at the start of the film, we get treated to some exposition shots that helped us understand the environment of the story. The film helped us to establish the relationship that Henry Marsh had with the Ukrainian doctor Igor from the start and slowly guided us through the daily routines that they have to face in the medically deprived country.

Using graphic imagery, the film is able to present us with the dire situation in the country as well as the truth and consequences of neurosurgery.

As we progressed on in the first act, the patient Marian is introduced and presented us with a story. Marian has a benign brain tumor and getting rid of it entirely could present him with a repercussion that could be life-changing. That’s definitely something that got us hooked onto the film. As we get on further, he was told that he has to do his surgery lucid really upped the stakes for all the emotionally invested audiences.

As we move on with the doctors and their daily encounters, we were presented with a failure coming back to haunt Henry Marsh emotionally. Which makes all the current stakes so much higher.

Another thing that I felt really worked well for the film will be the use of the polarity of healthcare systems. One interesting scene that struck me will be Henry Marsh explaining that the bone perforator tool that he uses home gets discarded with each use, thus showing how well funded the healthcare and medical facilities back in The UK is. On the other hand, we have Igor who shops for equipment from the local hardware store and used his perforator for five years straight. This display simply tugs at our emotions as we can’t help but feel sorry for the people that are living in Ukraine. So not only is the film successful in nailing down the nitty gritty parts of a medical documentary, it provides a huge social commentary on developing countries and how things were run there.

At the end of the film, we leaped over the hurdle of Marian and this time looking back to the failure of Tanya which came back to haunt Marsh. Similar to what was discussed in class, I felt that the ending would have sufficed after the operation of Marian. I do understand the theme of “some things will never go away and keep haunting you”, but I felt that the ending could be done to excess with the visit to deceased Tanya’s mother. It felt inorganic and slightly manipulative.

On the whole, I felt that the film successfully got me to invest my emotions in it and it’s all because of drama, risks, stakes and grief

Process: Week 6 and Audiovisuology

For a start, some of the keywords that I rooted for wanted to be a huge part of my FYP.

From the keywords, I branched out to several more words for the quality that I want in my final work.

*3 keyword definitions from dictionary.com. The rest annotations came from me.

Since starting my research, I’ve found some great DIY videos on the internet that helped me with experimentation. I came across one that taught me how to build a laser pickup. The solar cell I’ve clipped onto the cable produces certain electrical pulses that triggered sound in the amplifier. I found that this is something very fundamental in creating sound way back in the days.

I started out wanting to create a rotary system (similar to that of a phonograph/vinyl player). The rotary system will read the analog visuals that I’ve created. An iteration that I’ve thought of so far is a tube of cardboard roll with perforated holes and markings that the laser can pass through and create a percussive series of impulses that gets input into the amplifier output.

Then I went on to find out that my method is actually how the first recording worked.

From Audio Engineering Society

1877 – Edison made the first recording of a human voice (“Mary had a little lamb”) on the first tinfoil cylinder phonograph Dec. 6 (the word “Halloo” may have been recorded in July on an early paper model derived from his 1876 telegraph repeater) and filed for an American patent Dec. 24.

So I went on to read See This Sound: Audiovisuology and came to learn more about the history of music recording, corresponding relationships between sight and sound and so on.

Some of the notes I’ve picked out from reading the segment called :

On Hearing Eyes and Seeing Ears: A Media Aesthetics of Relationships between Sound and Image

Birgit Schneider

“Around the year 1900, currents and waves were considered the universal currency of hearing and seeing, in the 1990s, this function was taken over by the digital code, which seems to fuse genres in the “universal machine” of the computer.”
There are too many ways to connect hearing and seeing to each other. This amount of artists and artworks out there testify to this.
Subjective results in the perception and audio and visual relationship.
Sonification vs Visualization: 1929: Fritz Winckel carried out the thought experiment of Du Bois-Reymond who asked what will happen if the separate modes of sensory perception can be exchanged? He didn’t do this to his own senses of course. He used Mihaly’s television system, which was partially mechanical and broke down images with the means of perforated Nipkow disk . The result was appealing moire like images that altered its appearance to the rhythm of the music.
By contrast, Winckel was not as enthusiastic about the changing images into sounds. He felt that the sound of an image could only reveal whether it is a photograph, a black and white drawing, a manuscript, or a fingerprint. Further test did not allow for much differentiation as well. “synchronization beats at each line drowned everything out” due to the requirement of the technology.
Patterns from music are harmoniously constructed whereas sound from image created the sound of interference.
Dadaist artist Raoul Hausmann came up with an Optophone that could control not only sounds but also images at the same time. His apparatus produced sound and image simultaneously. The player can choose to engage with the output of either sense at one time or both at the same time for their artistic improvisation.

Week 6 Progress:

Initially, I figured that the digital output and processing would be important. After consulting this week, Ina pointed me to a direction that shifts more of the focus on the analog and input aspects and also to experiment with modularity. Will update as I go along.

Research : A Compilation of Audio-Visual That Got Me Started

Here’s a list of videos and subjects that got me started on the project.

I’ve had this obsession with sound from way back and I’m ready to connect back with it once again through this FYP.

Aphex Twins Spectrogram

The Aphex twins Spectrogram is something that I came across when I was crazily invested in rock music back then in 2010. Apart from the crazy huge calculators that I used in school to plot sin graphs, this spectrogram shows me how simple it is to link everything together using math.

Analog Vinyl Sampling

Before people started editing in digital where erasing mistakes are just one button away, people trimmed tapes, films etc. It never occurred to me that people would actually do this on the vinyl. Intrigued me.

It Might Get Loud : Jack White Makes a Guitar

Apart from being extremely cool, the film showed me that all you needed to get a decent analog sound input is a pickup, a string and a body.

Koka Beat Machine no.1 and no.2

These are rulers, springs, and forks, not your common well-polished parts in your classical instruments and yet they work wonders. Makes me think about the possibilities of uncovering sound potentials in the objects around us.

Case Study : Littlebit x Korg Synth Kit

Came across this on ARS Electronica

Besides the interesting presentation by Reggie Watts, what I found interesting about this synthesizer is that it’s quite bare and the modular possibilities feel endless to me. Visually, seeing this mini analog synth put together and played really brought out a certain very experimental and primitive aspect of sound and technology that I want in my work. Of course, I won’t expect my work to look perfectly polished and the visual aspects will be focused on more.

Mainstream Articles : Clever Attack Uses the Sound of a Computer’s Fan to Steal Data

An article that makes me think more about tangible things that can be transformed into data in our everyday lives.

Clever Attack Uses the Sound of a Computer’s Fan to Steal Data

A hacking method that uses the sound of a computer to steal data which just gives us an idea of how crazy data extraction can get. Nothing around you is safe, everything can be sampled and turned into data.