Movement with Neurons

A quick recap of my sketch, I wanted to depict the movement of the non-linear and unpredictability element of fish movements. To make my idea possible, I used processing as my medium.

Research

I started by exploring the various behaviours and movements of the fishes influenced by their neurons’ connectivity. I felt that it is essential first to acquire the understanding of the fish mechanisms and the nature that comes along with it before I could start to digitalise it using processing. Besides that, I also explored various artist’s works to be inspired, especially on artwork that was created digitally.

Guiding Reference

Mandelbrot Fractal (1979) by Benoit Mandelbrot

One of the artworks that inspired me was Mandelbrot Fractal (1979) by Benoit Mandelbrot, which was one of the first computer graphics to display fractal geometric images using a computer algorithm. What it does is using a mathematical formula to create endless repeating patterns of aesthetic properties. This was something similar to my sketch idea, so I always referred to his methodology as guiding points throughout my experimentation with processing.

First iteration (Basic Mechanism)

I started with the basic mechanics that simulate the patterns of a flocking fish. The first iteration was solely focused on the basic mechanism of flocking behaviours using particles simulation. As this was the foundational block of the sketch, I felt that the rules have to be written in a simple manner such that my future self will be able to understand and make changes to the parameters easily.

The basic rules/behaviour of the algorithm was created simplistically with only three actions.

• B1. Alignment: follow the heading of a local particle
• B2. Cohesion: Move towards the average position of a local particle
• B3. Separation: Avoid crowding local particle

Critic

While the basic mechanism was achieved, I felt that it was too rigid. It did not have this perception of chaos and order. My rules were constraining the particle, such that there was less room for a mistake to occur, and to me, that was not generative enough.

Second Iteration (Order & Disorder)

For my second iteration, I focused on blending order and disorder to mimic the patterns similar to the biological life of fishes in the ocean. I looked upon some examples made using Unity and found a work that I could relate on back to my sketch.

https://blogs.unity3d.com/2020/03/09/crossing-into-the-4th-dimension-how-carl-emil-carlsen-creates-generative-art-with-unity/

Constrains

I tried to create something similar; however, I was constrained with the number of particles I can produce due to hardware limitations. Nonetheless, I still managed to tweak the particle behaviour to reflect something like the example.

Further Tweaks

Additionally, I took Professor Dejan suggestion and made further tweaks to the properties of the particles, to portray a more reactive and natural form of particle behaviour. Now the particles are less constrained, and it creates room for the particles to make mistakes. Furthermore, instead of one group of flocking fish, now there are multiple groups of fishes. However, it still lacks that visual aesthetic touch and immersive experience.

Third Iteration (Aesthetics & Immersive)

My last iteration of the sketch was focused more on the visual aesthetics and immersive enhancement. One of the artworks that intrigued me was Artist Refik Anadol. I felt that his artwork was mesmerizing and something that I am inspired to achieve.

Melting Memories

Further Tweaks

I explored the different variation of colours that can be used to complement the sketch, and I decided to implement a system that could change the colour of the particles over time. Furthermore, I have also ramped up the particle count to 2,500 (it was quite demanding for my computer). To add a final touch, I switched my sketch from 2D to 3D.

Strong & Weak Aspects of The Finished Work

Weak Aspects

There were two stumbling blocks that I felt restricted in my explorative sketch. First, I was limited by my computer processing capabilities. As much as I want to render 100,000 particles simulation to create a visually appealing artwork like Artist Refik Anadol, I couldn’t expand my sketch beyond what my computer is capable of producing. Perhaps is the lack of understanding of how processing works behind the scene, such as how the algorithm is processed.

Next, I couldn’t achieve some suggestion by Prof, as I was limited to the library functions in Processing. One of the suggestions was to create a “predator” particle that could influence the dynamic of the particle’s behaviour.

However, I could not create the “predator” particle due to frequent crash when I tried to generate two separate particle systems (One for normal fish, Another for Predator).

Strong Aspects

The finalised work was less rigid and dull as compared to the first iteration, in terms of aesthetics and randomness. There was an evolution in terms of the behaviour of the particles; the algorithm is now optimised for particles to make mistakes and in a sense, make the particle appear alive and natural. Also, the system is built such that countless possible variations can be produced with just a tweak to the parameters of the code.

What I have learned in the process.

Ironically, I learnt that generative art is neither programming nor art. Generative art is the meeting point between the two disciplines; it’s the discipline of taking logical, systematic and rigid processes and subverting them into illogical, unpredictable and expressive results.

Therefore, I have learnt to remove this logical thinking that I had previously, to be more flexible with my ideas and exploration. Throughout the process, I also learnt to explore with various artworks, and just be inspired and open-minded to different works on the internet. Most importantly, incorporate the different ideas by the various artists into my sketch.

Everything else that was important.

During the guest talk by Professor Vladimir Todorovic, I raised the question with regards to finding the fine line between abstract and concrete, and he mentioned that there is ‘no perfect fine line between abstract and concrete’. I tried to relate this to my exploratory sketch process to understand what he meant.

Indeed, the process has shown me that order and chaos, simplicity and complexity, abstract and concrete aren’t necessarily the opposite ends of a spectrum. They can be easily intertwined. However, at the same time, we have to be careful not to stray too far into one spectrum or the other. Thus, I find that it is crucial to find this balance when we are creating generative pieces.

A Possible Generative Sketch Idea

Moving forward, I would like to explore further the use of particle movement to create abstract drawings, similar to William Anastasi’s subway drawings.

Reflection Essay – Broad, Leymarie & Grierson; Amplifying The Uncanny.

The article, Amplifying The Uncanny, by Broad, Leymarie and Grierson offered an insightful knowledge of how machine learning algorithms can be optimised and used to produce uncanny results.

Machine Learning & Generative Adversarial Network

Machine learning is a system where pattern data are used to predict future data or other outcomes of interest. It is understood as an automated process of optimisation, where an algorithm processes data and finds a set of parameters that best ‘solve’ a pre-defined objective function. DeepFake, a software that produces images that can trick the human eye into believing that they are real; it utilises machine learning, in particular, the Generative Adversarial Networks (GAN) framework to produce a sample of realistic images.

Next Great Movement in Art or Zombie Art?

Generative Adversarial Networks have then been widely used in the production of art and is seen as the next great movement in art. However, it has also received several critiques. Firstly, the production of new artworks by the dataset of existing artworks is seen as dull and obvious as Hassine and Ziv called it the “Zombie art”. Another criticism is that generative artwork relies too heavily on deep neural networks to produce mesmerising samples, without any meaningful framing of the works being presented by the artist.

Argument

Perhaps, the reason why Hassine and Ziv label generative art as “zombie art” is the problem of how it is very difficult to value generativity. One can’t see at a glance whether it took time or skill to create, similar to photography, one may think is just a press of a button to produce art as compared to paintings or sculptures which are seen to require more work. Nonetheless, from the experience in making my first sketch, generative art is definitely not just a press of a button and it does require creativity in the process.

Next, even though machines are heavily relied upon to automate some aspect of the generativity, it does not mean that it is not significant or meaningful as compared to other fields of art. Rather, I argue that it is actually a new form of creative outlet that artist could utilise in producing art pieces.

“Being Foiled” & the Uncanny experience.

Expanding on GAN, one of the explored artworks in this paper was “Being foiled”, which reverses the outcome of DeepFake algorithms. Instead of producing images that look imperceptibly realistic, “Being foiled” invert this objective, by optimising the system to generate what it sees as being fake and turning the image into complete abstraction. As a result, it amplifies the uncanny nature of these machine hallucinations.

The ‘Uncanny‘ is referred to as a psychological or aesthetic experience that can be characterised as observing something familiar that is encountered in an unsettling way. The uncanny effect is usually seen in objects that have a likeness of human qualities, whereby it is close to human-like but not human; as a result, it becomes creepy. For instance, a robot that looks very similar to a human, which invokes familiarity yet an uncomfortable feeling.

The author illustrated his point of uncanniness by picking an iteration of the model whereby the uncanniness is potentially most amplified. Indeed, just by viewing the sample, it does provoke a feeling of uncanniness. It is familiar, yet it gives a sense of unease. Relating this to generative art criticism as dull or monotonous repetitions, “Being foiled” actually shows us how generativity can invoke human perceptions and feelings. In this context, it provokes a sense of uncanniness to the audience.

Hence, the question is, how can we incorporate intangible emotions into our sketch that can be understood visually and consequently make generative art less banal?

Sketch Exploration/Changes

Reactive Trails

I have modified the particle trails to be more reactive based on its speed. For example, particles that are moving slower will have a shorter trail; particles that are moving fast will have a longer trail.

Initial Appearance of the Sketch

The initial sequence of the sketch is modified, whereby particles will “explode” with some particles leaving outside the sketch window, and slowly transit to the various school of fishes. I aim to create a perception of chaos at the start, whereby it slowly transits to groups of fishes. The live demo will better illustrate my point.

Different Species

I have included different variations of particle behaviours. The particles create their group, but there are also instances whereby the particle moves alone. Particles may also eventually leave their group to join another group or proceed as an individual particle.

Small changes

• Particles now can leave outside the sketch window
• Particles have different generated behaviour based on the rules of the algorithm
• E.g. Speed, groups, separation, etc.

Challenge

Introducing a “predator” to a school of single species to influence the shape dynamics.

• I tried to introduce a “predator” as suggested by Prof Dejan, but somehow my code always crashes whenever I introduce a new particle system with a distinct set of behaviours. I am assuming that the library of the code only allows one particle system, and I may have to find other alternatives to create the “predator”.

Next Iteration

• Will be exploring on incorporating colours into the particles
• Add a background music
• Will explore 3D Geometrics for the particles (hopefully it doesn’t crash)

Particle Prototype

Quick Exploration

To begin understanding how to move particles,  I did a quick research on the 3 properties that are commonly used to control particles simulation.

• Alignment: steer toward the average heading of local flockmates
• Cohesion: steer to move toward the average position of local flockmates
• Separation: steer to avoid crowding local flockmates

The intention is to simulate the rhythm and harmonic movement of a fish, to give my viewers a sense of oneness (unity) in their mind.

Prototyping & Testing

With the basic knowledge of the 3 properties, I used a physic library in processing to formulate the behaviour of the particles. In the beginning, there were some errors with the physics system whereby the particles are not constrained (anchor) to each other. This was quickly solved by constraining the particle into a “sphere”.

Further tweaks

After the particle mechanism is much more stable, I introduced more particles into the system. I have also added “trails” behind the particle, for aesthetic purpose. Every time the code is executed, a different result will be displayed. Perhaps, further, improvement will be to add colours? Flashing lights? Calming background music using minim?

Closed Systems: Generative Art and Software Abstraction

By Marius Watz, 2010

With the rise of computational technology, art is going through an evolving process with the combination of the principles of unpredictability and the purity of logic as Marius Watz describes it.

In his paper, Watz touches on the notion of exploiting the complexity produced by a computation that could not be made by human hands. He elucidates that generativity is a useful strategy that allows the author to harness the power of a computer to generate an infinite series of possible outcomes. However, to perform such a process, the author requires technical skills to be able to understand computer code and perform reverse engineering to achieve a desirable outcome. Therefore, as Watz contends, generative art requires both technical skills and aesthetic intuition for the artist to be able to express their art through the manipulation of computation. This resonates with me, as someone who sees through the lens of both art and programming, as both are vital ingredients to produce an interesting computational generative artwork. Moreover, to a certain extent, computational skill is an art because it requires critical thinking and ingenuity to create objects of beauty.

On the other hand, computational aesthetics also has its constraint. As pointed out by Watz, simulation is spontaneous and organic, which can only be replicated by computers with the explicit encoding of how the data is processed and even the most experienced programmer will encounter unexpected results or errors. Nonetheless, programming error can lead to discoveries and new learnings. Indeed, as a programmer that is still learning, more often than not, I gained valuable information from errors. For that reason, I see mistakes and errors as a beneficial process of discovery, learning and improvement.

Besides that, Watz also made a comparison with interactive art and generative art. He elaborates by drawing a distinct difference between generative art and interactive art. Generative art operates based on an autonomous system with infinite outcomes based on the algorithms, while interactive art utilises a feedback loop of interaction between a system and its users. Based on this definition, I do see the distinct differences and mutual exclusiveness in their respective art forms. The way I see it, both arts require controllability through a set of rules. In generative art, the algorithmic rules create art, and for interactive art, the rules of human interaction create art.

Beyond that, Watz has also postulated that generative art has reached a level apart from computation means. While Watz has embraced the complexity of computer systems, he does agree that generative art is not about the computer itself and would be a mistake to think that generativity is only capable of being expressed in pixels. Thus, generative art can come in many possible outputs, and I certainly do agree and believe we should expand and explore our creativity of generativity in many various possible outputs, and not just limit our observations of generativity to solely one medium.

Video Examples

https://youtu.be/Y-5ffl5_7AI?t=75

https://youtu.be/dkP8NUwB2io?t=244

Inspiration

Essentially, these two videos depict the fish movement, but more precisely, it is based on an invisible path that is created from the connection of multiple fish neurons.

“Soon, so many neurons are interacting in so many different ways at once, that the system becomes chaotic.”

Generative systems often include chaotic behaviour. Like the fish, while the neurons of the fish is a simplistic system following a strict sequence of cause and effect, there is a dynamic nonlinear and unpredictability element.

Generative Sketch

Therefore, for my chosen medium, I will be using Processing whereby I will be exploring the feasibility of particles to simulate the movement of the fish.

To do so, I will be learning ways to create particle simulation and on how to move particles in a unison manner. The unpredictability element comes from the procedural coding,

1. whereby the manipulation of the number of particles,
2. the characteristics of the behaviours of the particle (Separation, Alignment, Cohesion, Speed)
3. and the size of the boundary will offer a new result.

One challenge I foresee will be the manipulation of the characteristics of the particle to achieve a desirable outcome, such as moving the particles in a unison manner. However, that is part of the learning journey and I look forward to the challenges that lie ahead.