Month: September 2020

Posted on

Week 7: Generative Study

SOUND OF STONES

Generative Study:
Real-time sound generation using depth data from kinect

Over Week 7, I experimented with SuperCollider, a platform for real-time audio synthesis and algorithmic composition, which supports live coding, visualisation, external connections to software/hardware, etc. On the real-time audio server, I wanted to experiment with unit generators (UGens) for sound analysis, synthesis and processing to study the components of sound through the programming language and visually (Wavetable). The physical modelling functions (plot, frequency analyser, stethoscope, s.meter, etc) would allow me to explore the visual components of sound.

Goals 

  1. Connecting Kinect to processing to obtain visual data (x, y, z/b)
  2. Create and experiment with virtual synthesizers (SynthDef function) in SuperCollider + visualisation
  3. Connect Processing and SuperCollider, send data from Kinect to generate sound (using Open Sound Control OSC)

Initially, I was going to work with Python to generate data flow into SuperCollider but Processing would be more suitable for smaller sets of data.

SuperCollider

SynthDef() is an object which allows us to create a function (design of a sound), and the sound generation has to been run using the Synth.new() or .play line eg. x = Synth.new(\pulseTest);. This allows us to create different types of sound in the SynthDef(/name) function and the server allows us to play and combine the different sounds live (live-coding).

Interesting functions:
Within SuperCollider, there are interesting variables that can be used to generate sound other than the .play function. This would be relevant to my project where I would like to generate the sounds or the design of the sound using external data from the kinect. The functions I worked with include MouseX/Y (where the sound varies based on the position of the mouse), Wavetable synthesis and Wave Shaper (where the input signals are shaped using wave functions) and Open Sound Control (OSCFunc which allows SuperCollider to receive data from external NetAddr.).

MouseX

Multi-Wave Shaper (Wavetable Synthesis)

SuperCollider Tutorials (really amazing ones) by Eli Fieldsteel
Link: https://youtu.be/yRzsOOiJ_p4

https://funprogramming.org/138-Processing-talks-to-SuperCollider-via-OSC.html

Processing + SuperCollider 

My previous experimentation involves obtaining depth data (x, y, z/b) from the kinect and processing the data into three-dimensional greyscale visuals on Processing. The depth value z is used to generate a brightness value, b, from white to black (0 to 255) which is reflected in each square of every 20 pixels (skip value). To experiment with using real-time data to generate sound, I thought the brightness data, b, generated in the processing sketch would make a good data point.

Connecting Processing to SuperCollider 

Using the OscP5 library in Processing, we input data into a NetAdress that will be redirected into SuperCollider.

Using the brightness value, when the b value is more than 100, a message ‘/brightness’ is sent to OSC SuperCollider.

When the OSCdef() function is running in SuperCollider, and you receive ‘-> OSCdef(brightness, /brightness, nil, nil, nil)’ in the Post window, it means that it is open to receiving data from Processing. After running the processing sketch, whenever the message ‘/brightness’ is received, the Synth(‘stone1’) will be played.

Generative Sketch – Connecting depth data with sound

For the purpose of the generative sketch, I am working with using data as a trigger for the sound that has been pre-determined in the SynthDef function. Different SynthDef functions can be coded for different sounds. So far, the interaction between kinect and the sound generation is time-based, where movement generates the beats of a single sound. For larger range of sounds specific to the captured visual data, and thus textures, I would have to consider using the received data values within the design of each sound synthesizer.

Improvements

I see the generative sketch as a means to the end, so by no means does it serve as a final iteration. It was a good experiment for me to explore the SuperCollider platform which is new to me, and I was able to understand the workings of audio a little better. I would have to work more on the specifics of the sound design, playing with its component, making it more specific to the material.

Further direction and Application 

Further experiments would be to use more data values (x, y, z/b) beyond the sound generation (Synth();) but be used in the design of the sound (Synthdef();). A possible development is to use Wave Shaper function to generate sounds specific to the Wavetable generated using functions that are manipulated or transformed using the real-time data from kinect.

Topography – 3D scanning of forms (stones)
Developing a soundscape using WaveTable synthesis

 

Generative Sound and Soundscape Design

I would like to use the pure depth data of three-dimensional forms to map the individual soundscape (synthesizer) of each sound, so that the sound generated would be specific to the material. This relates to my concept of translating the materiality into a sound, where the textures of the stone correspond to a certain sound. So, if the stone is unmoved under the camera, an unchanging loop of a specific sound will be generated. When different stones are placed under the camera, the sounds would be layered to create a composition.

In terms of instrumentation and interaction, I can also use time-based data (motion, distance, etc) to change different aspects of sound (frequency, amplitude, rhythm, etc). The soundscape would then change when the stones are moved.

Steps for Generative Study:

I have yet to establish a threshold on the kinect to isolate smaller objects and get more data specific to the visual textures of materials. I might have to explore more 3D scanning programs that would allow me to extract information specific to three-dimensionality.

My next step would be to connect more data points from processing to Supercollider and try to create more specific arguments in SynthDef(). After which, I would connect my Pointcloud sketch to Supercollider where I might be able create more detailed sound generation specific to 3D space.


Link to Performance and Interaction:

Proposal for Performance and Interaction class:
https://drive.google.com/file/d/1U5J0XajPlCrGfuhPQEI6J1zQDqRu2tJL/view?usp=sharing

Lee Ufan, Relatum- L’ombre des étoiles, 2014

 

Audio Set up for Mac (SuperCollider):

Audio MIDI Setup

 

Posted on

Semester Project Proposal – Stone

Proposal for Performance & Interaction Semester Project
https://drive.google.com/file/d/1U5J0XajPlCrGfuhPQEI6J1zQDqRu2tJL/view?usp=sharing

My definition of ‘ING A THING’
‘Materialising an experience and Experiencing a material’

Feedback:
Look deeper into ‘What it means to be a stone?” and Extracting the essence of the material
Go beyond the representational, see what abstract connections can be revealed between the material and experience

Generative Art Project:
Proposal:
https://oss.adm.ntu.edu.sg/a170141/generative-study-generating-sound-using-materiality/
Experimentation/ Execution: https://oss.adm.ntu.edu.sg/a170141/generative-sketch-and-study-updates/

Posted on

Generative Art Reading 2

Amplifying The Uncanny

Analysing the methodology and applications of Machine Learning and Generative Adversarial Networks (GAN) framework.

Computational tools and techniques, such as machine learning and GAN, are definitive to the applications of such technology for a generative purpose. The paper explores the exploitations of these deep generative models in the production of artificial images of human faces (deepfakes) and in turn invert its “objective function” and turn the process of creating human likeness to that of human unlikeness. The author highlighted the concept of “The Uncanny Valley”, introduced by roboticist and researcher Masahiro Mori, which theorises the dip in feelings of familiarity or comfort when increasing human likeness of artificial forms reaches a certain point. Using the idea of “the uncanny”, Being foiled maximises human unlikelihood by programming the optimisation towards producing images based on what the machine predicts are fake.

Methodology 

Machine Learning uses the process of optimisation (the best outcome) to solve a pre-defined objective function. The algorithms used to process data produce parameters that categorise what can be generated (by the choice of function). In producing deepfakes through the GAN framework, the generator serve to produce random samples and the discriminator is optimised to classify real data as being real and generated data as being fake, where the generator is trained to fool the discriminator.

Being foiled uses the parameters generated by the discriminator which predicts signs that the image is fake to change the highly realistic samples produced by the generator. It reverses the process of generating likelihood to pin-point at which point do we cognitively recognise a human face to be unreal, which relates to a visceral feeling of dissonance (the uncanny valley). When the system generates abstraction, where images cannot be cognitively recognised, I would imagine that the feelings of discomfort dissipates. In a way, Being Foiled studies the “unexplainable” phenomena of human understanding and feelings.

Applications

As a study, I feel that the generative piece serves its purpose of introspective visual representations of uncanniness. However, the work should exist as more than  “aesthetic outcomes” and the learning can be applied to various  fields, such as AI and human robotics, that develop and explore human likeness and machines.

The Artificial Intelligence field is quite advanced in the development of intelligent technology and computers that mimic human behaviour and thinking, threading the fine line of what is living and what is machine. Considering the analogue forms of art, Hyperrealism saw artists and sculptors, such as Duane Hanson and Ron Mueck, recreating human forms in such detail that it is hard to differentiate which is real and unreal visually. When it comes to robotics and artificial intelligence, what defines it to be “human” is the responses that are produced by the human mind and body. By studying the data collected on “normal” human behaviour, the AI systems generate responses trained to be human-like. “The Uncanny Valley” explores the threshold of human tolerance for non-human forms, where imitation no longer feels like imitation, which is often referenced in the field. With Being Foiled,  the point where uncanniness starts to develop visually can be tracked and the information can be used when developing these non-human forms.

Geminoid HI by Hiroshi Ishiguro Photo: Osaka University/ATR/Kokoro

Where “Being Foiled” can be applied

When I was in KTH in Stockholm, I was introduced and had the experience of using and interacting with an artifical intelligence robot developed by the university. Furhat (https://furhatrobotics.com/) is a “social robot with human-like expressions and advanced conversational artificial intelligence (AI) capabilities.” He/ She is able to communicate with us humans as we do with each other – by speaking, listening, showing emotions and maintaining eye contact. The computer interfaces combines a three-dimensional screen to project human-like faces, which can be swapped according to the robot’s identity and intended function. Furhat constantly monitors the faces (their position and expressions) of people in front of it, making it responsive to the environment or the people it is talking to.

Article on Furhat:
https://newatlas.com/furhat-robotics-social-communication-robot/57118/
“The system seems to avoid slipping into uncanny valley territory by not trying to explicitly resemble the physical texture of a human face. Instead, it can offer an interesting simulacrum of a face that interacts in real-time with humans. This offers an interesting middle-ground between alien robot faces and clunky attempts to resemble human heads using latex and mechanical servos.”
When interacting with the Furhat humanoid, personally I did not experience any feelings of discomfort and it seemed to have escape the phenomenon of “the uncanny valley”. It even seemed friendly and have a personality.

It is interesting to think that a machine could have a “personality” and the concept of ‘the uncanny valley’ was brought up when I was learning about the system. What came to my mind was at which point of  likeness to human intelligence would the system reach the uncanny valley (discomfort) beyond just our response to the visuals of human likeness. Can we use the machine learning technique that predicts what is fake or what is real on images (facial expressions) for actual human behaviours (which is connected to facial behaviour in the Furhat system)? -> how I would apply the algorithm/ technique explored in the paper

An interesting idea:
Projecting the “distorted” faces on the humanoid to explore the feelings of dissonance when interacting with the AI system

The many faces of Furhat. Image from: Furhat Robotics

Conclusion

While generative art cleverly makes use of machine learning techniques to generate outcomes that serve objective functions, the produced outcomes are very introspective in nature. The outcomes should go beyond the aesthetic, where the concept can be applied in very interesting ways with artificial intelligence and what it means to be human.

Posted on

Generative Sketch and Study Updates

SOUND OF STONES

Project: Creating an instrumental system that uses visual data of textures (stones) to generate sound in real time

For the project, there are two parts:
1. Converting three-dimensional forms into visual data
2. Connecting the data with audio for real-time generation

Generative Sketch

Over week 5, I experimented with the  depth image and raw depth data from kinect to processing to see what kind of visual data (colour/brightness, depth, etc) can be obtained from a camera. The kinect has three key components, infrared projector, infrared camera and RGB camera, from which the captured three-dimensional visual data can be sent to processing.

Using ‘depth image’ 

With the ‘depth image’ sketch, the data values, x (horizontal position) and y (vertical position) of the pixel from the kinect are mapped on recorded image. The sketch involves looking at each pixel (x, y), looking at the index of the depth image and obtaining the colour/ brightness (b – single value between 0 and 255). A rectangle of a fixed size is mapped using z value (depth in 3D space) according to their brightness value b,  where things that are dark to appear close and things that are bright to be farther away.

The purpose of this sketch is to see what data values can be obtained from the kinect and see if I use the data as input for audio generation. From this sketch, the depth data from a kinect that can be obtained are x, y, z, b values, that I think can be used to as input data to map textures of three-dimensional forms.

Using ‘Raw depth data’ to map forms on a point cloud 

For scanning three dimensional forms, raw depth data (kinect1: 0 to 2048 and kinect2: 0 to 4500) from the kinect might be more useful to generate information about textural surfaces in 3D space. The point cloud can be used to map all the points the kinect is obtaining in a 3D space (from the infrared projector).

By giving each pixel an offset value (= x + y * kinect.width), we get a raw depth value d (= depth[offset]). Each PVector point (x, y, d) on the point cloud can be pushed into the three dimensional plane to map the object the kinect is seeing. For smaller objects (I have yet to try this out), a minimum and maximum depth threshold can be used to look only a particular set of pixels, to isolate an object that is close to the kinect.

Tutorial sources:
https://www.youtube.com/watch?v=FBmxc4EyVjs
https://www.youtube.com/watch?v=E1eIg54clGo

 

For sound generation 

Initially, I looked into virtual modular synthesizer program (VCV Rack) to generate the sounds and if they could be coded in real time. However, the programme exists only as a modular synthesizer, a good one, to develop complex audio.

I am interested in sending real-time data from the kinect/ camera/ sensor into an audio-generating software. Referencing Aire CDMX (2016) by Interspecifics, I found out that I could use Python (data access) and SuperCollider (virtual synthesizers) to connect data flow to sounds that I design.

Aire CDMX (2016) by Interspecifics
http://interspecifics.cc/wocon
https://muac.unam.mx/exposicion/aire?lang=en

Aire is a generative sound piece that uses data that environmental sensors pick up in real-time in Mexico City. Using a software written in Python to access the real-time data of pollutants, the data flow is used to animate an assembly set of virtual synthesizers programmed in Supercollider. “In the piece, each one of the pollutants has its own sound identity and the fluctuation of the information modulates all its characteristics.”

From this work, I can study their code as a reference to find a way to map data to designed sounds on Supercollider. As it is my first time working with Python, I might need some help writing the code that specifically works for my project.

Source Code: https://github.com/interspecifics/Aire

For Week 7 Generative Sketch 

For the next two weeks, I will be working on connecting the raw data values from the kinect to virtual synthesizers that I will develop on SuperCollider. My aim is to see what sounds can be generated when a three-dimensional object is captured using a kinect.

Direction for Week 7:
1. Connecting kinect data to Python
Some references: on Linux with pylibfreenect2 https://stackoverflow.com/questions/41241236/vectorizing-the-kinect-real-world-coordinate-processing-algorithm-for-speed
2. Experiment with SuperCollider to create virtual synthesizers
3. Connect Python and SuperCollider to generate sound using data flow

For Final Project Generative Study

For the final project, my goal is to map visual textures of materials, in particularly stones, to generate an auditory perception to the material. Rather than using raw depth data, I would like to obtain more data specific to three-dimensional forms.

Ideation – Using Topography data
https://arsandbox.ucdavis.edu/#sidr

Topography, in geography, is the study of the arrangement of the natural and artificial physical features of an area. When looking at topographic sketches, I wonder if it can be scaled down to map three-dimensional forms by the circumferences formed by intersecting horizontal planes. I would have to research into 3D scanning software and the type of data that can be obtained. How I imagine it would be convert each layer of visual shapes into a corresponding audio feedback (maybe in terms of how the sound wave is shaped/developed).

https://theconversation.com/us/topics/topography-6950

 

Ideation sketch – Converting Visual texture into Audio Visually?

AR Sandbox by UC Davis

The AR Sandbox is an interactive tool combined with 3D visualisation applications to create real-time generation of an elevation color map, topographic contour lines and simulated water when the sand is augmented. I think that the 3D software used to track the changes in forms can be applied to my project, where the contour lines generated by the stones can serve as data or input for sound. I would have to research more into this after I complete the experimentation for the generative study.

I would consider using the sensors that were suggested (RGB+ Clear and IR (facture)) to use for capturing data. I would first work with the kinect, but if the data generated is insufficient or not specific, I would consider other options. I would have to think about where to position the kinect and also use the threshold from the kinect raw depth data tutorial to isolate the captured object.

Other References:
To study texture:
Vija Celmins – photo-realistic nature

Concept development for interaction

Just a possibility:
If the code for capturing real-time visual data is developed enough, I would have the participants collect stones from a walk/ on their path to create generative pieces specific to a journey.
Or it could just exist as a instrumental tool to play around with the sound textures.

Connecting to Performance and Interaction:
I would like to use the developed system on a bigger scale for a performance piece for the semester project for Performance and Interaction class. It would involve capturing human-sized objects in a space on a bigger scale, which would change the threshold of the captured space.

 

Posted on

FYP 20/21 Week 5 Project Updates

For Week 5 and 6, I intend to start on experimentation for my project. My first step for crafting the sensory experiments would be to determine what visual qualities of light I would like and what kinds of light can be used as tools for exploration.

Readings:
Light
The Lighting Art: The Aesthetics of Stage Lighting Design by Richard H. Palmer
– Chapter 2: Psychophysical Considerations: Light, the Eye, the Brain and Brightness
– Chapter 3: Psychophysical Considerations: Color
– Chapter 4: Psychophysical Considerations: Space and Form Perception

Handbook of Experimental Phenomenology: Visual Perception of Shape, Space and Appearance by Liliana Albertazzi
– Chapter 6: Surface Shape, the Science and the Looks
– Chapter 8: Spatial and Form-Giving qualities of light

Material Perception (Visual and Haptic)

Neural Mechanisms of Material Perception: Quest on Shitsukan / Hidehiko Komatsu  and Naokazu Goda

Visual perception of materials and their properties/ Roland W. Fleming

Visual and Haptic Representations of Material Properties  / Elisabeth Baumgartner, Christiane B. Wiebel and Karl R. Gegenfurtner

Multisensory Texture Perception / Roberta L. Klatzky

Effects of Visual Expectation on Perceived Tactile Perception: An Evaluation Method of Surface Texture with Expectation Effect / Hideyoshi Yanagisawa, Kenji Takatsuji

DEVELOPMENT 

The main goal of my sensory experiments would be to see if perceived tactile perception can be induced when visually perceiving or interacting with light. My experiments would center around connecting visual qualities of light to those of materials.

I would have to do further studies into visual and textile of materials through the readings, and try to formulate more concrete experiments with light using those that were conducted with materials. I plan to conduct at least two simple experiments in the following week before the presentation on Friday (week 6).

Some concepts/ ideas I am working with:
– Television static and the sensation of uncomfortable tingling or prickling, ‘pins and needles’
– Diffused (soft) vs. Sharp (hard) light, associating a tactile quality to light quality
> If a ray of hard light (laser, projection, etc) comes towards you, would you avoid it with your body?
– Studying motion/ rhythm of light (black and white) and sensory and neural conditions (epilepsy, hypnosis, vertigo, dizziness)
> Replacing objects with light?
– Optical illusions with light? Translating 2D to 3D with projected/ physical light?
> Akiyoshi Kitaoka http://www.ritsumei.ac.jp/~akitaoka/index-e.html
– Squinting at light > diffraction effect

UPDATES (Sensory experiments)
– Will update the type and specifications of the small sensory experiments that I intend to do over the week here

Posted on

InstructionArt – Choreographic Light

Choreographic Light 

Choreographic Light serves as a choreographic system, consisting of a wearable to reflect light directed by laser and geometric ‘scores’, to generate movement.

Using the behaviours of reflected light, the performers are instructed to draw using the point of light reflected from a point on their bodies. Wearing the reflective objects (as costume), the body becomes a tool of interaction with light (directed by a laser). Given geometric ‘scores’ that corresponds to the music, the performers move to create visualised forms with the wearable tool. The movement of the performer is derivative from the interactions with light and body, as well as their own interpretations of the geometric forms.

Wearable as Choreographic Object

To me, performance has a element of orchestration or script, which comes in many forms from basic instruction, physical and spatial settings to social context. A choreography is a form of instruction to generate movement, loosely defined as the sequence of staged steps and movement. A costume or wearable serves as an extension of the body, which affects or controls the way we move. I would like to explore how a wearable system that manipulates an intangible material, such as light, in space can be used to generate movement.

Initial sketches

Using the “costume” to define movement, I intend to explore the body as the tool for interaction. My idea was to re-interpret the space occupied by our body (physical or virtual in the form of light), and see how our body readapts to the restriction and what kind of movements are controlled or generated.

Bauhaus Costumes – Das Triadische Ballet

Costumes by Oskar Schlemmer (Bauhaus) for the Triadic Ballet, at Metropol Theater in Berlin. Photo: Ernst Schneider, 1926. (Apic/Getty Images)

Schlemmer, the choreographer of Bauhaus ballet, intended for the dancers, adorned with geometric costumes, to explore the reinvented silhouettes of their bodies in the avant-garde performance. The costumes serve as metaphysical forms of expression – removing the fluid fabrics and movements, archetypal of ballet, and replace them with structural forms and their interpreted gestures.

 

Sketch – Small prototypes of reflective costumes

For the InstructionArt performance, I wanted to test small prototypes of reflective object, instead of a costume, on different parts of the body to study how the tool would be used (through the restriction of motion). The wearable objects are designed to be easily worn and in different sizes, with the reflective surfaces that can be interchanged.

Reflective objects that can be worn on different parts of the body
Different shapes can be detached and attached

A laser will be directed to the reflective surface, resulting in a point of light on the wall

Using geometric scores as instruction for movement

As light is an intangible material, the fluid interaction of light and movement is affected by many factors, creating unpredictability in the outcomes. As part of the system, I would like to choreograph the interaction (drawing with the body) using simple lines and geometric forms, such as circles, square and triangles, that can be easily understood while using the tool. I would like to keep the instruction as minimal as possible so that it will not be so distracting. It would serve as a directive for the manipulated light, while the interpretation (the size, orientation and speed) would be up to the dancer.

A geometric score of different lines and shapes is composed for the performance

William Forsythe – Improvisation Technologies 

Improvisation Technologies – Forsythe created video segments about his approach to improvisation for modern dance, to train his company’s dancer. His categorisation of different classes of movement (lines, curves, shapes, etc) can be “analysed as geometrically inscriptive – a formal drawing with the body in space”. I was inspired by his systematic approach of providing techniques/ instruction for improvisation (usually with no choreography), the choreography then becomes a combination or sequences of set/ designed motions.


‘Improvisation’ Free-Moving state: Moving according to Sound

During the process of “drawing with light” with our bodies, I wanted to see how the body would respond when there is no geometric instruction. Through the experience of drawing with the body, I realised that the light drawn are restricted to lines or curves due to the small point of contact of the reflective objects. Under no instruction or ‘free-style’,  I had the performers move according to the way they like or with the music. The performers would then using the light to interact with each other, creating patterns and moving with the other with the music, which created a collaborative visual experience.

Future direction – Semester project?

Moving on, I would like to experiment with bigger forms of reflective objects, so that bigger movements can be generated. I would also fixed different points of light instead of having one for one object/ performer, so that a spatial performance or spectacle could be generated.

Feedback:
– Using movement to generate music – Collaborative performance with different roles for each performer
– Different parts of the body – more dynamic movements

Posted on

Generative Study – Generating sound using materiality

CONCEPT – Converting texture of three-dimensional forms into sound

My idea is to create a ‘instrumental’ system to generate sound using data from physical materials. To me, instruments are generative systems, while composed music are outcomes. Intrigued by our material perception, I would like to map the patterns of stones (primary medium of interest) and convert the visual data into sound waves/ audio signals in real-time.  I am interested in exploring the perceptual commonalities of our senses of sight, touch and hearing. When we touch a ‘rough’ material and we hear a ‘rough’ sound, how do we cognitively recognise or associate a tactile quality of ‘roughness’ to what we sense? My project is an attempt to connect material textures (through sight and touch) to sound textures.

Perimeters/ Structure of system:

Connecting textural qualities (visual) to auditory qualities, the sound quality will vary according to the type of stone/ texture used. I would define a space when the stones can be placed in the range of the camera lens (kinect). When the stone is stationary, the three dimensional data captured by the camera will be mapped as two-dimensional lines to be converted into sound waves (Need for a software that connects visual data to audio). I will also experiment with other forms of data (numerical based on volume/ texture density) that can be converted into audio data. 

“Oh that sounds like a rough rock!” “That sounds like a smooth stone!”

Interaction design:

Each stone when placed under camera will have a sound loop related to their material qualities. The audience can move the stone to change the sound, using movement feedback as a ‘synthesizer’. Stones of different textures can be switched and combined, to ‘play’ the instrument to generate music in real time. 

Inspirations

  • Ambient Techno
    Connecting atmospheric textures of ambient music with the melodic and rhythmic elements of techno and electro.
  • “Play the Barcoder” by ELECTRONICOS FANTASTICOS, Ei Wada
    https://www.youtube.com/watch?v=bOfpQt4KFCc– Converting patterns into sounds
  • “The sound of the earth” Yuri Suzuki
    http://yurisuzuki.com/artist/the-sound-of-the-earth
    A record player uses textures/ grooves (sound waves directed into the diaphragm) on a record to create vibrations when in contact with a needle. The sound is played through an amplifier. This is an analogue method of generating sound through textures on a “predetermined” record, similar to a conventional artwork.  My projects deals with a digital/ computational system to generate sound as an instrument, as a generative system.
  • “Years” Bartholomäus Traubeck
    http://traubeck.com/works/years

TECHNICAL EXPLORATIONS
– Kinect and processing/ Touchdesigner
– VCV Rack (software modular synthesizer)
– Research into visual-feedback audio software

Posted on

Reading Reflection 1

A system, composed of parts working together as a whole, is our way of understanding structures and phenomena in the world. One of the reasons why we are so in awe by nature alludes to the power of creation, subjective to the limitations of the human hands. Nature “evolved from the harmony of the myriad of chances and necessity” and is a constellation of systems that generate forms and phenomena. It is in human culture to break down phenomena into systems, for example, formulating scientific principles to explain how the material world works.

We can take inspiration from the organic “processes found in nature” to condition unpredictability into systems that can generate outcomes infinitely. Generative systems are mechanisms used to create structures that “could not be made by human hands”, using computational models that combine encoded  “organic behaviour and spontaneous irregularities” with logic. These systems come in the form of computer language and digital tools, which are malleable in nature. One small change in the code/ algorithm can lead to a variety of outcomes, or different combinations of written code/ conditional factors can be used to achieve the same or similar outcomes.

The process of using computational systems to generate aesthetic processes involves translating individual components into a series of decisions that can serve as building blocks for the desired result. These decisions have to be interpreted as computer code to satisfy each functional element of the system and the programming process involves a lot of reverse-engineering and ‘trial and error’ to work around the tendencies of the computer algorithms used. The art of coding is the clever manipulation of these generative systems – “choosing computational strategies and appropriate parameters in a combination of technical skill and aesthetic intuition.”

It is interesting to think of Generative Art as works categorised by the strategy of adopting a “certain methodology”, as compared to art movements which are paradigms of works defined by the characteristics or ideology. The aesthetic application of rules and systems is key in generative art works, where the artist designs a closed system by setting conditions and perimeters, in which  the unpredictable outcomes exist. Not all generative art has to be code-based, but the art is in the system instead of the results. This allows the work to be timeless, where as long as the conditions of the system are available, the outcomes can be generated infinitely.

The next step for Generative Art:

Watz states that while computer code algorithms are essentially immaterial, they display material properties. It would be interesting to see the material properties translated into sensory ones, where computer code algorithms can be used to generate sensory experiences. Sensory perception is specific to the person, and what one perceive would a subjective experience of the generated sensory activation, which can be seen as a form of unpredictability and generativity.