Final Project — Initial Ideation

INtial Exploration

Strengths:

Exploration of Emotions via Text allows for us to dissect and observe the change in one’s emotional state over a period of days. We do not need to personally know someone but rather just have access to the information on their devices, to have a perception of their intimate self.

Weakness: 

Not visually powerful enough. The use of circles signify closure and was not ideal in exploring multiple relationships.

Idea 1: (Sculpture)

Topographical Map,
2-Dimensional to 3-Dimensional
Web-Like Structure with both Peaks  And Valleys.

(Areas of Concentration of both Positive and Negative Emotions)

Centre, Neutral line representing the timeline or days.

Digital/Physical Visualisation.

 

 

Inspiration

 

Idea 2 : Mixed Media

Simpler but possibly more visually powerful

Creating a Visual Map:
By using Identified Texts —> Assigning Emotion to it —> Looking at Date/Day Text was sent —> Look at Phone Gallery/Social Media to find Image/Tweet saved/liked on Corresponding Date.

To try and visualise if there is any correlation between the intimate interactions we have online vs the more subtle interactions we make with social media in itself. See if there is any pattern in the type of image that is generated or comes up for any particular emotion.

Do the emotions we feel and show via our texts influence the content we interact with on social media or vice versa?

16 October 

Emotion: Vulnerability

Text: ‘Please keep it to yourself’

Tweet Liked:

Image Saved on Gallery:

Outcome:

Creating a detailed visual database of the relationships we involve in, in the virtual world. Proof of Concept that our Devices provide access to our most intimate and private thoughts/feelings and that these information can be publicly accessed. Archiving information over a period of days, and then connecting the days with similar imagery to see if there is any correlation between emotion and type of social media ‘trace’ within the same day.

Exploratory Generative Study — Update

Contacts:

I will be working with 5 contacts (the few I mainly communicate and have some sort of emotionally intimacy with in this current period). It is significant to note that these dynamics might change over time and completely disappear too — this however will be part of tracking the transient role we play in each other’s life through our phones. If I happen to break communication with 1 out of 5 of the people, the data visualisation representing that respective person will be empty for that period of time. There will not be any additional or new contacts added into the system designed around the initial 5 subjects.

Time Period:

Will take place over a period of 1.5 to 2 weeks so as to have enough diversity in the visualised outcomes yet have some overarching structure/parameter.

Data:

Initially I only thought of recording the connections we made between each other based on similar keywords indicative of the same sort of emotion, and  the ‘background’ third party connections between the other users themselves make (via me as an intermediary). However I have decided to include the types of emotions within the graphs (categorising of them as either positive or negative feelings and taking note how these map out through the days).

The initial idea was to use discrete/isolated and progressive grids that represented each respective days. However I felt like the structure seemed very rigid and would be counter intuitive to the way they were meant to portray the very organic and salient connections of our digital relationships and intimacy. Hence I will be using circles instead (the centre being Myself, and 5 Equidistant points on the Circumference to represent the contacts).

 

 

Instead of organising them in a linear manner, I will be organising the circles themselves as part of a larger circular clock like interface or some sort of spiral — this will allow us to view the entirety of the data as one cohesive pattern/piece that can be taken apart slowly instead of a blatantly broken down grid structure. This improved iteration will require slightly more ‘effort’ from the viewer, increasing engagement and stimulation. Having no discrete start or end in a circular loop also reinforces the concept of transience and fluidity of digital intimacy.

 

Reflection — Amplifying The Uncanny

Overarching Thoughts (on GAN , ‘ The Uncanny’ in contemporary society)

In this article, the authors explores how Generative Adversarial Networks (GAN) and Machine Learning have intensified the phenomena of the ‘uncanny’ — described as a certain level of discomfort or unease one feels when a machine mimicking organic human behaviour comes seemingly close to, yet disjointed and ‘faulty’ in its portrayal. It’s the simultaneous humane semblance and recognisability in the machine that plays with our emotional perception, yet our awareness and consciousness that it is a visibly artificial machine-construction that evokes this sensation.

It is provocative, provoking, unsettling and ultimately a unique reflection of our own perception of ourselves. The dysfunction in the machine and its faults are not rationalised in the way we would observe, clinically, the faults of a system like a washing machine or a vehicle (also machines). It is the machine-learned human nuances these machines have and their anthropomorphic form that seduce us into making some sort of ‘involuntary’ psychological and mental connection to them. Their form, appearance and behaviour (though crude, a sufficient enough representation of the human) creates an intuitive intimacy and we project our own feelings, experiences and learned world views onto our observation of this machine. This at the same time is inextricably linked to our obsession, fear and intrigue with technology/artificial intelligence.

It is not just a random discomfort but one that deeply is rooted in some way to our subconscious feelings towards technology’s presence and co-existence in the contemporary world. For example, if a primitive human like a Neanderthal who has no known knowledge or experience with technology sees a work like Ken Feingold’s ‘If, Then’ 2001 , it will probably be shocked and intrigued in a very banal and unintelligent manner.

For us humans who have some preconceived perception and familiar intimacy with technology and are constantly adapting to live with it, as augmentations of our body/mind, a representation like that is even more uncomfortable and uncanny. It plays on this innate, (irrational?) fear (one that has been propagated by novels, movies and media) of artificial intelligence eventually becoming intelligent enough to adopt and appropriate what we hold on to as our ‘humanity’.

Essentially this concept is what we believe distinguishes us from machines. Though crude in its superficial portrayal, any machine that is mildly capable of replicating some sort of human learned behaviour like speech immediately destabilises our position of supremacy, reminding us that the lines between the real and ‘unreal’ is becoming increasingly blurred. It also plays on this fear of automation and artificial intelligence replacing the role of humans.

However this is not something that is insignificant. In fact, I personally feel that this concept in relation to GAN in this context then opens up the possibilities for a very contemporary personification or reflection of the human condition. The outcomes of these GANs mirror the change in perceptions and emotions the amalgamation of technology has had on us. They mirror the fears, attachment and feelings we have grown to possess with the increasing presence of these artificial ‘bodies’. After all, this is an extremely valid, relevant and powerful aspect of the contemporary human condition that cannot be ignored. It is this then, that I believe, plays a significant role in plunging us into the ‘Uncanny Valley’ (first introduced by Masahiro Mori in 1970). It is only when the ‘humanoid’s’ anthropomorphic form is transitioned into being fully indiscernible from that of a normal human that we find ourselves out of this disconcerting state of Uncanny.

http://www.umich.edu/~uncanny/

Personal experience of ‘the Uncanny’ 

This music video by one of my favourite artists, Bjork came to mind with regards to ‘The Uncanny’. Though it is a music video and I am aware that it is graphically modelled and not real, the portrayed interaction of these anthropomorphic humanoids (the way sexual intercourse and ‘love’ is expressed via turning of gears in the figures, the leaking of fuel liquid etc) evokes an uneasy yet seductive feeling. The song being about ‘Love’ — a feeling that is very humane and far from machine, being portrayed by humanoids then makes us for a moment re consider if ‘Love’ is exclusive to us and if the ‘Love’ we feel/know of is just one possible perception limited by the human experience.

FURTHER THOUGHTS (On Being Foiled)

The article describes the Generative Process of the work ‘Being Foiled’, as a Positive Feedback Loop. By using the initialised system that is pre-trained to produced Unreal, Deep Fake Portraits, (based on the dynamics of the generator and discriminator) and then subverting the function by ‘fine tuning’ the system, the machine goes through a 3 Part Process (Divergence, Convergence and Collapse) which leaves an abstract ‘trail’ or track of the deep fake construction and configuration process. The outcome is essentially a gradual and increasingly intense breakdown from imperceptibly realistic portraits to discrete geometric shapes, bold colours and full abstraction.

Source: xCoAx 2020: Proceedings of the Eighth Conference on Computation, Communication, Aesthetics & X

As the portraits breakdown in this feedback loop, we see prominent blue lines marking the face shape, eye contours and silhouette of the hair that intensify and retain giving us some sort of ‘clue’ on how these remaining parameters could possibly be the basis for a Deep Fake face construction. In’Being Foiled’, all the portraits lead up to one identical abstracted image. This suggests that the system in its deep learning process of being fed countless faces, (constant reaction to discriminator by the generator and vice versa), might have determined/learned a standard and arbitrary ‘blueprint’ as a basis to construct convincing Deep Fakes that ‘dupe’ the discriminator into recognising it as a real face.

On Generativity

The unpredictability aspect of the Generative Methodology in the GAN system, is most noticeable in what is described as the ‘peak uncanny’ in the iteration of the early stages of the feedback loop process.

Source: xCoAx 2020: Proceedings of the Eighth Conference on Computation, Communication, Aesthetics & X

In this image, the bold red hue pervades through the face and the background, while thin blue lines exaggerating the contours of the eyes, face and hair are present. The eyes are misaligned, the hair highly regularised and simplified as crosshatch lines and the wrinkles around the eyes unrealistically pronounced.

What the authors’ describe as ‘fault lines’ of the system ‘indicative’ of its generativity, these very perceivable and consistent marks in each iteration points to the relative points of the system where it is at it’s ‘weakest’ (in this context, with regards to which features of the face it faces the most struggle in replicating the organic human face). The word ‘weakness’ is used here relative to system’s objective and prescribed function. In fact, this very weak ‘points’ I describe, are probably the most optimal points from which the unpredictability potential of the system radiates from.  In this concentrated areas of ‘fault lines’ where the system breaks down and exposes itself most apparently, there is potential for the designer or artist to then target and expand on propagation in these areas, perhaps by by fine tuning the weights of the generator or by some other relevant manipulation.

However part of generativity and in the GAN system is the lack of control we have over the system. We are able to trigger and adjust the weights of either the generators or the discriminators but the circular loop in which the two entities communication and react to each other, is constant and immutable. The information that is communicated between the two changes depending on how we fine tune the system but the fundamental process remains the same. There is no fixed end state and the systems constantly evolve over the training process.

Concluding Thoughts

How generativity is successfully harnessed in this work and exploration is by fundamentally subverting a relevant and powerful machine constructed system (GANs & Deep Fake). Instead of constructing a generative system from scratch, an existing Generative System with an innate objective function is manipulated in such a way that it’s underlying unpredictabilities cause a feedback loop that deconstructs and reverses the Deep Fake Process into Abstraction.

Essentially a system that is trained to produce Images that we are unable to distinguish from real images, is manipulated into accomplishing the exact opposite. What is then significant from this outcome is the fact that a generative system is capable of manifesting itself in this extreme duality — two very different iterations that diverge from a common process. It is then our role as Artists to trigger or explore the divergent and variant outcomes these ‘closed systems’ can produce, by first analysing and then manipulating the interactions between the objects of the system and, our interaction with the system in its design process.

We also need to be conscious in the way the ‘Artistic’ outcomes of these Generative Process are presented: “while initial mesmerising and transfixing, can quickly become banal, monotonous repetitions for the sake of overwhelming the viewer with the “sublime of algorithmic productivity” (Zylinska 2019).”

Though it is argued in the article that this very algorithmic representation is powerful, we have to consider the affect and over saturation these type of repetitive output can have on our audience — we do not want them to be immune or ‘numb’ in their intuitive reaction to the works. Hence it is important to possibly explore how these iterations can be displayed — perhaps via projection or by intersecting the physical and virtual planes. One question we can ask ourselves as Generative Artist is if it possible to enhance the ‘experience’ of the dialogue with a closed system without taking away its essence?

Exploratory Generative Study

Themes

Shared Emotional Space, Virtuality, Latent Influence/Connections, Indirect Affect, Intimacy.

Concept

Exploring how our screens and phones have become augmentations of our psyche and emotions. Our phones are vessels that hold all the information about our lives — private and public.

We leave both digital and physical traces on our phones.

Physical: Fingerprints, the pressure, speed and direction in which we text on our phones is relative to our mood and state of mind – when we are arguing we tend to text faster. Some days we barely touch our phones or reply conversations when we’re feeling existential or disassociated from our world.

Digital: Text Messages are often intimate and representative of our state of mind. We engage with different people via text, creating a shared space with each one of them. We also leave traces on Social Media via Text or Visuals.

Aim 

To create a Visual Map of our emotions by identifying descriptive words associated with emotion in our text messages. While  taking note of the emotion expressed by the person we are texting. Our mood constantly changes and is influenced by everyone around us. Two people who are detached from each other can influence each other indirectly if they both communicate via an intermediary entity. For example if person A texts me and I am affected by his emotional state of mind, this then will have an influence on how I respond to person C. This is ever more so apparent in the contemporary world today where we are psychologically connected (voluntarily and involuntarily) due to the veracity of the digital/virtual world.

We are always multitasking and replying or engaging with multiple people at the same time unlike in real life where we do not have the ability to have multiple intimate dialogues simultaneously. Essentially my idea is to attempt to visualise a map that connects specific people via their expressed emotions and at the same time visualise how they play a part in influencing my mental state. Our phones have become augmentations of us and we barely pay attention to the way in which we engage with it and how this engagement has affect on our cognitive and emotional state.

Part of the exploration in this study is to take a step back and attempt to deconstruct the relationships we form in the virtual world (specifically text conversations) and sequence a set of representations that express this complex phenomena in a clearer way. By forming a sequence of simple ‘data graphs’ over a specific period of time, we can then track and process how  the people we text play a role in our lives.

Constant Variables

Size of Parameter
Positions of Point
Relative distance between Points
People

Outcomes:

Number of lines
Area within Boundary
Number of connections between me and specific person
Number of connections between people

INITIAL Visual Sketch

For each corresponding emotion or word between any two people, a line will be drawn to connect the respective points. The red line represents a connection between two isolated people while the black line represents a connection between a specific person and myself.

Reflection — Marius Waltz, Closed Systems: Generative art and Software Abstraction

Overarching Thoughts 

Waltz mainly discusses how generative art in itself though highly precise and ‘controlled’ in the way the system which produces it is designed, its aim is to reproduce an outcome that is as close to nature as possible — organic and spontaneous phenomena which carry the inherent characteristic of ‘life’ itself — mutability and hence unpredictability. As artists and designers we want to create experiences that replicate the poetic nuances of what constitutes ‘living’. Yet we are acutely and admittedly aware that we are inextricably limited by our logic and human perception. Though we are able to ‘feel’ various intangibles such as emotion, we mostly attach these ‘feelings’ to objects or people to give them some sort of physical association and meaning. Science explains that chemicals such as dopamine and serotonin are responsible for these ‘feelings’ yet there is a somewhat spiritual and noetic aspect to these emotions that we simply can’t fathom — concepts of the soul etc.

Aware that we are only limited by our own human condition and perception, and that we simply do not have the power to ‘play god’ and detach ourselves from this schema or worldly experience to deconstruct everything from a completely unbiased, pure and distilled approach, we then turn to the use of software systems and algorithms. In a way these systems operate out of the parameters that confine ‘life’. They lack intuition and perception and their ‘learning’ depends on what information we choose to feed them. However this then renders these highly precise systems with  the ‘non existent human-objectivity’ to assist us in deconstructing our experience as a detached entity. What we have inbuilt in us, that these computerised binary systems lack is the very thing that limits us from fully taking apart (and hence recreating) the condition of ‘to be living’. We cannot put together something we do not even know how to take apart.

This then brings us to the systems we rely on. Though these systems are inanimate and devoid of any sort of ‘lively’ essence, they too produce unpredictabilities. A ‘glitch’ in the matrix or a ‘bug’ that comes up once in awhile, derailing the system from producing its supposedly premeditated outcome, resulting in moments of deviance and dysfunctionality. This capability and propensity for highly objective and parametric systems to produce unpredictabilities while lacking life, forms the key fundamental basis for generativity to exist as a significant philosophy or methodology in answering the questions we have always sought out.

The implications of a set algorithms having some sort of possible innate biases that ‘mimic’ or rather come close to mirroring the similar unpredictabilities seen in nature, while being tangible and adjustable to our touch, is highly significant. Essentially we now have in our hands a modifiable, parametric ‘replica’ that comes as close as possible to ‘life’- something we have zero control over. Whether the way it manifests is indeed controlled by a ‘higher being’ or simply operates on some sort of universal code, or balance, it is beyond us as humans as ‘life’ in itself is not in our power. We are executioners of life in the way we carry it in our physical bodies, yet when we age it is ‘taken’ away from us. We reproduce but we cannot ‘ inject ‘life’ into something. ‘Life’ in itself is not exclusive to our race. Yet given our highly developed cognitive functions and emotional perception, we are inevitably unable to simply dismiss it — existentialism, nihilism, solipsism, absurdism are all simply manifestations of the human acute awareness and frustration of our inability to understand or comprehend fully the greater world we live in.

As designers we have to apply this intuition with suitable technical skills to create a complex and interesting generative system. Waltz explains that ‘reverse engineering’ is an essential approach towards generativity — we try to ‘plan’ the intended outcome (together with its possible ‘set of unpredictabilities’) and decide what type of software, input algorithms or data set we can utilise to produce this result.

Further thoughts

In the  ‘void loop()’ section, Waltz talks about the rise of the net and how advancements in technology, together with our fascination of it, led to early exploratory works of generativity. Though they were relatively ‘crude’ and rather simple (far from the likes of veracity and complexity with associate with technology today), they fundamentally operated as valid generative closed systems. Waltz suggests that because of this closed system characteristic, and the lack of need for interactivity, coupled with the focus on the infrastructure and nature of the system itself, generative art is closer to fine art than it is to media art.

This then contextualises the approach he is suggesting for us as designers to adopt. Rather than creating an experiential piece that capitalises on user-system interaction, we should analyse the materiality of the system, it’s patterns and behaviour and modify it to our desire. It is then a process that requires a sufficient level of ‘stringency’ and discipline. We have to be precise about how we piece the elements of the system together the same way a painter precisely places brush strokes onto a canvas. It is only with this precision that we will be able to create a system that is able to function to produce profound yet meaningful outcomes.

The point of generativity is not to directly intervene/alter in the system’s process but rather to first deconstruct it and understand the potential underlying unpredictabilities it is capable of producing on its own. We then modify or add elements to it to ‘capture’ or record these unpredictabilities. We are hence required to be careful in the way we construct the system — whether digital or analogue, that each of the element is calibrated and not artificially biased to ‘rig’ the outcomes. The beauty of the outcome is not in its visual appearance or how close it is to the artist’s ideal but rather it being a meaningful and conceptual representation/manifestation of the non visible nuances and characteristics that are inherent in the designed system.

However by designing the system with some sort of discretion, we will then able to predetermine one that is able to also produce visually/mentally stimulating outcomes. Here our intuition and understanding of world views come into consideration as we make deliberate choices. Choosing to add Element A instead of B can and will result in a very different set of outcomes but which would be a more effective representation of the system? ‘Effective’ in a sense where it targets and attacks or destabilises our worldview. Does it satiate our craving to want to process intangible phenomena or offer any sort of visible, identifiable respite to deepen/expand our understanding of the relationship we have with these non discrete, poetic entities.

Data Scultpures

In the final section, Waltz fantasises about the idea of generative art completely moving off the screen (he is careful to remind us that it is indeed not limited to just pixels displayed on a screen). He believes that with technology enabling us to “extrude virtual forms” onto three dimensional space itself, this opens up the possibility for generativity’s exploration to expand even further in the contemporary world. Waltz also mentions the experimentation of data sculptures, ‘representing normally intangible information flows as physical manifestations.’

Refik Anadol, Engram : Data Sculpture for Melting Memories

The above work is an example of data sculpture I personally find fascinating.

‘Anadol gathers data on the neural mechanisms of cognitive control from an EEG (electroencephalogram) that measures changes in brain wave activity and provides evidence of how the brain functions over time. These data sets constitute the building blocks for the unique algorithms that the artist needs for the multi-dimensional visual structures on display.’ 

Earlier I mentioned how we as humans are constantly looking for ways to grasp and portray the intangible processes that we experience in our brains and body. ‘Engram” successfully harnesses and measures the empirical aspect of our cognition to then use it as input data to essentially produce an organic visual ‘life-like’ representation of our memories. The outcome is profound and mesmerising — through Generative Methodology we are able to appreciate the sheer wonderment of our brain and its functions via an almost spiritual experience. Though there is no direct interactivity and it is a closed system, there is psychological and emotional dialogue between the work and viewer intuitively. This dialogue then helps us process and internalise the intangible phenomenology surrounding our memories.

Generative Art Example — Subway Drawings by William Anastasi

Subway Drawings by William Anastasi

Thoughts

The most interesting or rather central aspect of this work is its simplicity and sublimity. In this series of graphite drawings on paper, the artist sat on the subway, allowing the motion of the train determine the resulting ‘artwork’. The train’s motion – it’s halts, acceleration, sudden bumps, manoeuvres and direction determine the resultant line’s size, direction and weight. It also controls the speed and the direction in which the pencil he holds moves. The simple act of sitting down on a train, and obtaining different generative sets of artwork is significant as it amplifies the mutability and transience of life and in this example, life in transit/journey.

No one drawing is consistent as the tracks on which the subway train operates on, may also vary in orientation, condition, temperature spending on the destination/location. Anastasi noted below on each drawing the destination he was heading towards. The drawings then become an abstract manifestation of the transient journey between two points. They are almost cartographic in the way they record the unique and discrete motions for each specific journey to the respective destination. Fundamentally the resultant drawings then become a map of different journeys to to different locations.

Most of us when we take a ride on the bus or the train, especially in today’s fast paced society, we are often fixated and obsessed with simply reaching our destination on time — to meet a friend, get to work, or get home etc. However each time we take the train, no single journey is the same — some days we might be in a bad mood, we may have a different playlist plugged into our earphones, the train might be more densely populated or the train might even break down or halt momentarily.

However these poetic nuances are intangible and we often don’t pay attention to the process even though the ride itself has more affect on our emotional and cognitive state than the reaching of the destination. (If we happen to be on a crowded train we will probably feel more exasperated by the time we reach our endpoint as opposed to if it was peaceful and empty). Anastasi’s Subway Drawings manage to capture this process or phenomenology of travelling by using the tangible and physical factors that take place simultaneously with our experience. By letting the movement of the train control and determine his automated drawings, Anastasi is able to visualise and manifest a unique identity and portrait for each of these ‘journeys’. Apart from the movement of the train, his subtle and spontaneous movements in his hands (due to muscle memory) would have also played a role in the lines’ visual outcome. These then in a sense also capture the human experience of the journey on paper.

Generative Aspect

Anastasi’s Subway Drawings is generative in the way the constant factor is the act of taking a subway train ride, with a piece of pencil and paper. The unpredictabilities which constitute the motion of the train and mildly, the spontaneous movements of the body, make the work generative. No one outcome is identical and will never be identical. Yet they all share a common visual appearance (haphazard, overlapping lines) that expresses the underlying constant function/’algorithm’ of the designed system.

Personal interests & Further Exploration

I personally am interested in analysing the human condition and the way our experiences alter the perception of the world around us — relationships, spaces, memories. As humans, most of the time we are travelling between places and destinations. While we physically travel, our emotional and mental state also ‘travel’ and wander off to psychological or virtual ‘places’. This work could be built upon to explore this intersectionality of these realms and how the experience of travelling is a significant representation of ourselves.

It is interesting to think about how a work like this might differ if it were to take place in our current Mass Rapid Transport (MRT) System. In a densely populated city like Singapore where we rarely are able to get seats on the train, will the motion of rubbing against passengers of all sorts, having to navigate the space to find a comfortable spot, and other movements caused by congestion/rushing result in visually interesting drawings? This might be something I would want to explore for Project 1 — extrapolating the idea of recording and mapping the transient nature of journeys in today’s world. It is also interesting to think about how this work could perhaps be modified to be produced on our phones or devices instead as it is the main object that accompanies and occupies us when we travel. We are either texting, watching a movie, browsing social media or listening to music. The way our fingers interact with the screen (speed, direction, pressure) all depend on these factors. Perhaps, one interesting and powerful exploration would be thinking of a way to map the motion and traces of our fingerprints on our phone for each journey, over a period of time.