Aspen Movie Map

The Aspen Movie Map, created by Michael Naimark and his team at MIT, represents one of the forefronts of interactive video computing technology that was in rapid development during the late 70s. Seen as a revolutionary example of hypermedia, the movie map was perhaps the first interactive platform of it’s kind to utilize non-linear live-action content to create an interactive experience. Many technologist and artists during this period were also actively pursuing the idea of a platform that would allow the perusal of interconnected media in a way that was non-linear and associative. The basis for this idea can be traced back to the concept of the Memex, first proposed by Vannevar Bush in his essay, As We May Think back in 1945.

In his discourse, Bush lamented the lack of a system that enabled humans to easily reference the collective knowledge of the entire race in a quick and intuitive manner. He described the laborious process of index based referencing and how it was an unnatural way for the human mind to process information. “The human mind […] operates by association. With one item in its grasp, it snaps instantly to the next that is suggested by the association of thoughts.” He envisioned a system of interlinked documents and media that would behave similarly to how humans connected thoughts, ideas and concepts in a chain-like fashion. While this concept predates hypertext, it most definitely behaves similarly on a conceptual level.

The Memex device, as imagined by Vennevar Bush

Bush also envisioned that the system would take the form of a desk, a visual metaphor for graphical computing that will still use today. While this might have been a subconscious decision, it would stand to reason that Bush would have wanted the interactions with the Memex to be as similar to what people were already doing in their day to day lives. This would have greatly helped to reduce the friction in adopting such a radically new way of working and thinking. To the more technologically astute, this rudimentary description of Bush’s would remind one of the Graphical User Interface (GUI) that was eventually developed by Alan Kay and his team over at Xerox PARC. The Memex then, can be considered as the philosophical ancestor of all of modern computing as we know it; from the way in which content is non-linear and interconnected, to the way users went about interacting with them.

In a similar vein to the Memex, the Aspen Movie Map is another example of technology that leverages hypermedia to more closely imitate real world experiences. While the Memex was interested in creating a better user experience for the perusal of documents, the Aspen Movie Map proposes a better system than traditional two-dimensional maps to make sense of  positional and spatial data. While maps have been used by human society as an aid to understand spatiality for the longest time, it still requires a certain degree of mental gymnastics before the two-dimensional map on the drawing can be re-interpreted to guide one through a three-dimensional reality such as the one we all live in.

However, what makes the Movie Map more pertinent in media history, is that instead of having users sit through an ‘on-rails’ experience where they are brought along for the ride, the Movie Map allows for people to travel through the space and wander around the city of Aspen, Colorado in a similar fashion to real life. Niamark termed it “surrogate travel”, where he served as the surrogate traveller for viewers to later go through the same journeys digitally. The usage of hypermedia allowed the team to create a richer experience for users, providing them the option to enter buildings, interact with restaurant menus, and even change between seasons. To the modern person, this would seem extremely similar to the Street View feature of Google Maps. The user experience from the Interface, coupled with the use of live-action footage, creates a sense of immersion for users of the movie map that far supersedes the experience of making sense of a traditional cartographical diagram.

The rig on the left used by the MIT team to create the Movie Map as compared to a remarkably similar set-up used by the Google Maps Street View team on the right.

The modern world is extremely indebted to hypermedia and metaphorical visual computing for laying the ground work for our present day digital existence. From creating non-linear information portals that provide us with knowledge, to visual interfaces that rely on intuition to operate these portals and devices, Vannenar Bush, Michael Niamark and Alan Kay have really pushed the human race forward such that we may truly go about our lives as we may think.

Final Hyperessay: Key Work Selection | Archive Dreaming

Archive Dreaming (2017) by Refik Anadol is a data visualization work installed at SALT Research in Istanbul, Turkey. It was presented as part of The Uses of Art: Final Exhibition with the support of the Culture Programme of the European Union. Being a public art installation, the work was conceived as an alternative method for members of the public to peruse and engage with the document collections of SALT Research. The research facility comprises a specialized library and an archive of both physical and digital sources. Their collection includes visual and textual sources on art history, the development of architecture and design in Turkey, as well as documents dealing with the transformations in society and the region from the Ottoman Empire to the Turkish Republic. Due to the significant age of most of these documents, large parts of the collection are housed in specialized, pressure and temperature controlled rooms; causing most of the physical sources to be as easily accessible.

Architectural diagram of ‘Archive Dreaming’

While other attempts at a such a project might have led to the creation of a simple photographic database, Archive Dreaming takes the concept of an interactive archive much further. The most obvious and significant aspect of this work is in its use of physical space and architecture. The installation is designed to be a spatial experience. In the words of Anadol himself, the work is described as an effort to “deconstruct the framework of an illusory space [that] will transgress the normal boundaries of the viewing experience of a library”. The work was specifically conceptualized with the notion to significantly transform the experience of accessing a knowledge repository. The work was to be a “a three dimensional kinetic and architectonic space of an archive.”

One of the browsing modes of “Archive Dreaming”

The other aspect of Archive Dreaming that sets is apart from a simple digital database is the flexibility that it provides users of the system. The work allows users browse the 1.7 million documents in the archive in a number of possible ways; each drawing different associations between the documents through the various viewing modes. This assists the viewer to draw relationships with the documents in a way that is suitable for their individual needs. In this way, the installation “intertwines history with the contemporary, and challenges immutable concepts of the archive.”

Such a system is only conceivable with the aid of Machine Learning and AI that have the capabilities of looking at, understanding and organizing all of the content found within this large set of documents. All of this processed information is then easily controlled by visitors from a touch sensitive console at the center of the installation space.

A user accessing the central console to interact with the archive.

Machine Intelligence also takes center-stage for the final aspect of the installation. Getting its name from this process, the installation enters into a standby “dream mode” when no users are interacting with the archives. The system uses the knowledge gained from studying and classifying all of the 1.7 million documents in the archive to “hallucinate” new ones that might possibly exist in the archives. In a post-truth era where “facts” are constantly challenged and called by other names, this process calls into question about how much of our shared history is “real” and how much of it is to be trusted; a challenge that historians and documentarians constant face and grapple with. It also postulates the idea of a brave new world where creativity and creation might no longer be just in the domain of humans and perhaps, machines might play just a vital role in shaping the art histories of the future.

Final Hyperessay: Artist Selection | Refik Anadol

About the Artist

Refik Anadol is a contemporary media artist that works across a large variety of digital media. Born in Istanbul, Anadol is currently based in Los Angeles, California in the United States where he is a lecturer and visiting researcher in UCLA’s Department of Design Media Arts. He holds a master of fine arts degree from University of California, Los Angeles in Media Arts, master of fine arts degree from Istanbul Bilgi University in Visual Communication Design as well as a bachelors of arts degree with summa cum laude in Photography and Video.

“Virtual Depictions – San Francisco”

Much of Anadol’s works focus on site-specificity, choosing to create digital works of public art that leverage and accentuate the surrounding architecture. He works with projections (both indoors as well as outdoors)  and large scale screen installations to create immersive environments that seek to transport audiences into an imagined space. As a media artist, designer and spatial thinker, Refik Anadol is intrigued by the ways in which the transformation of the subject of contemporary culture requires rethinking of new aesthetics, technique and dynamic perception of space. His works aim to create visually agnostic depictions that form a visual language capable of appealing to varying groups of people; despite cultural backgrounds or aesthetic preferences. Due to their public nature, Anadol also caters his works towards eliciting the reactions and interactions of passers-by  within unconventional spatial orientations.

“Wind of Boston – Data Painting”

By embedding media arts into architecture, he questions the possibility of a post-digital architectural future in which there are no more non-digital realities; a future whereby all human made surfaces are informed in some shape or form, by the digital footprint that mankind leaves behind. His works invite viewers to visualize alternative realities by presenting them the possibility of re-defining the functionalities of both interior and exterior architectural formations. Anadol’s work suggests that all spaces and facades have potentials to be utilized as the media artists’ canvases.

A large percentage of Refik Andols works and creative process is deeply influenced by and based off of technology – specifically generative data. We as modern society are profligate with our creation of new data. Just within the last two years, the human race has generated about 90% of all the data output ever made within the life-span of our species. The concept of smart cities and embedded sensors all over our built environment have contributed towards this exponential explosion of data-generation. We now live in living, evolving, self-aware cities bristling with input devices that are providing feedback of various kinds . We ourselves produce large quantities of data, be it through our self-expression via social media, or simply the communicative data we send each other. Often times, all of this data rarely sees the light of day; relegated to internal uses within the companies and government bodies that process these information.

Melting Memories: Data Processing brainwaves to create visual depictions.

Through his works, Anadol questions how our experience of space is changing now that digital objects ranging from smart phones and embedded sensors to urban screens have all but colonized our everyday lives. How have media technologies changed our conceptualizations of our everyday spaces, and how has architecture embraced these shifting conceptualizations to accommodate these changes? These are the three main questions that Anadol tackles in his works; not by simply integrating media into built forms, but by translating the logic of media technologies into spatial design itself.

Justification

Refik Anadol is a good case study to research and explore for the purposes of this hyper essay as he represents a growing trend of multimedia artists whose works are heavily dependent and influenced by data. As much as the impressionists used paint and light as their mediums, this burgeoning group of media artists to which Anadol belongs to, are using data as a medium to develop their works. The added spatial qualities of Refik Anadol’s pieces begs us to question the role spatiality plays in the experience and consumption of media art, as well as how this new phase of media art harks back to the roots of the pioneers while also carving out a new path for itself.

Agency in the Age of Social Media

For much of human history, people have understood to varying degrees that in order to live more comfortable lives, we as individuals had to give up certain personal liberties. Our prehistoric ancestors knew this when they started gathering into tribes to increase their chances of survival. As human dwelling groups got larger, individuals gave up some of their own personal freedoms to carry out tasks that contributed towards a greater purpose.

Leaders of varying levels came into the picture, becoming representatives that helped to steer parts of society so it could function as intended. Today, we understand this as politics and governance. We hold elections to have people represent us in varying levels of hierarchy such that large groups of us can live and function together with ease. We willingly give up some fraction of our personal agency in exchange for more convenient lives. That being said, we as a society are also constantly in flux when it comes to agreeing on how much agency should we concede to our representatives.

This thought forms the conceptual crux of Kidnap by Blast Theory, a performative art work that invited potential participants to be kidnapped. One of the strangest part of the work that audience had a difficult time reconciling with was the question why someone would willingly pay a £10 fee to essentially enter a raffle that might result with their kidnapping. In essence, this is a critique of modern society at a fundamental level. We pay taxes to a government (the entry fee) in hopes that they guide us the right way (the act of giving up control). We are not physically kidnapped as such, but the psychological essence is remarkably similar.

The way Kidnap pans out further echos this sentiment. In his interview with Maria Chatzichristodoulou, Matt Adams of Blast theory comments that “they were putting the audience body in a situation where one had to decide […] how to position themselves.” The audience members were “constructing an experience by filtering his/her understanding of what is going on.” The works mirror the way we naturally experience and process our surroundings because of how well they are tied to actual, in-world experiences. The audience are no longer ‘on rails’, experiencing a pre-determined narrative outcome with pre-determined visuals such as a film or painting. They become an integral part of the narrative itself, directly driving the final outcome. By leveraging the participation of their audience, Blast Theory in essence creates a scenario whereby the audience are directly responsible for the situations they place themselves in. Their agency, or lack thereof, makes the work.

Fast-forward to the 21st century and this questioning of agency becomes even more pertinent. Social Media has become a point of contention recently making Kidnap just as relevant almost 20 years on. Adams comments that it is important to understand “to what extent these technologies function as consumerist, solipsistic toys” and if they really do generate social transformation. Much of the discussion centers around our lack of agency over the data we create and how all of this data gets weaponized against us to mainly further capitalistic and political ambitions.  To many of us, these social media networks are marketed as utopian sanctuaries where people of similar interests can gather to meet new friends and reconnect with old ones. However in doing so, most of us are either unaware or unwilling to accept that we are exchanging our data and privacy to become a part of these networks. Ultimately, these networks are also businesses that have costs and must face the challenge of remaining profitable.

In our fervor to express our individuality with the rest of our social circle, we populate our streams and profiles with highly personal and identifiable information that paint extremely accurate pictures of who we are. This participatory act of giving up our data is, for all intents and purposes, exactly the same action agreeing to be kidnapped by members of Blast Theory or giving up liberties so we may be governed. It is an intrinsic part of being an inhabitant on these networks. If this is the case, then are we allowed to be dissatisfied when these services use the data they gather to fund their operations or affect decisions? Are we justified in our stance that these networks should not be allowed to use the very information that we so willing contribute? Are we justified to believe that this data is not to be used in any shape, way or form when we were so willing to give it up just moments before under the guise of connecting and communicating?

Much like it was for the audience participants of Kidnap, this issue becomes a question of “how much agency are we truly comfortable with?” Similar to the participants, we as modern humans have collectively deemed that the use of social media to connect with one another is enough of a convenience to part with our privacy and data by way of the content we publish onto these mediums. If we are comfortable doing this, we should be just as comfortable realizing how much of it is used as a means to profile and categorize us. Instead, we enter a state of mass hysteria and panic just like how the participants of Kidnap became more and more concerned about the ‘boundaries’ of their kidnapping despite agreeing to it themselves in the first place. Perhaps, the answer (and the message Blast Theory was attempting to communicate via Kidnap) is to participate in the act of actually losing all of it in the first place – such that we may learn firsthand what it truly means to retain the power of choice, agency and freedom as an individual.

The Eternal Frame

Understanding Ant Farm and their works would be impossible without first understanding American culture from the early 50s to the late 70s. The rise of the automobile’s importance in American society was one that began as the would-be members of the collective were still children. Freeways ripped through cities as the American life began to revolve more and more around the automobile. A symbol of physical, cultural and societal mobility, the car ushered in an era of motels, drive-thrus and other car-centric innovations. Continue reading “The Eternal Frame”

Osmose

Osmose by Char Davies is an example of a media art work that uses virtual reality to craft a fully immersive experience for its audience. Going beyond establishing a visual outcome, the artwork employs a multi-sensory approach to successfully transport viewers into the artist’s imagined space. The unique interaction method centered around breathing, as well as the use of a head-mounted display create a sense of tangibility that  embeds the immersants deep into an otherwise foreign and unrealistic space that feels real.

In that regard, Osmose may be seen as pioneering attempt at realizing Ivan Sutherland’s vision that was penned in his essay ‘The Ultimate Display’ thirty years prior to the creation of this work. In it, Sutherland postulated that as technology got better, humans would be more capable of designing all encompassing sensory interfaces that would appear to simulate our reality to a point where  “a computer can control the existence of matter”. While highly ambitions and visionary, the statement does align with how technology has progressed over the last 50 years since it was written. As a society, we strive to create computer systems and interfaces that remove the intermediary layer with which we manipulate media, and have tried to place the user as close to the content as possible with regard to the user experience. The ultimate goal then, is to march towards “complete immersion”.

Sutherland also discusses the idea of familiarity and how that affects the pursuit and uses of the theoretical ultimate display. He posits that being able to recreate experiences that one is already familiar with in real life would help to increase the perceived immersion in this “ultimate display”. In his essay  ‘Virtual Environments’, Scott Fisher mirrors this line of thought by explaining that “for most people, duplicating reality is an assumed […] goal” for digital imaging technology. While the display might be useful in perfectly recreating the reality we are all familiar with, the display could also be used to help us discover new alternate realities with rules that we are not familiar with. While Sutherland uses the example of possibly exploring the physical properties of quantum mechanics to further human understanding, exploiting such displays for a more poetic and artistic outcome is also reasonable. Davies uses Osmose to allow her viewers to experience a new reality that would have been other difficult to reconcile with something as static as a painting.

Historically, art has always been communicated to others on a mediated platform. A painting is a summation of an artists internal experiences and believes that are then repackaged and visually represented for the consumption of another individual. The experience becomes largely external to the original thought and intent of the artist; it is consumed second-hand. As Fischer puts it, the “abstracted second-hand knowledge is often more generalized and concentrated” but lacks the “balance and completeness of experience.” First-hand experience then, becomes the defining factor as to how complete the encounter with a work becomes. Osmose brings this to the table by having the immersants experience first-hand, the imagined space just as the artist would have in her head during the conceptualizing of this work. The artist is provided the opportunity to externalize their thoughts in a more tangible fashion and viewers are able to step into the artist’s mind and form their own conclusions from that experience.

One could argue that the crucial element that ties all of this together is the way in which locomotion is controlled within the world of Osmose. By having her immersants’ breath patterns tie directly to how they move within the world, Davies is able to successfully marry a real-world physical experience (breathing and the tangible principles of buoyancy) to an imagined new space with it’s own rules in a way that is both familiar and awe-inspiring at the same time. This was the key to the unprecedented immersion that viewers of the work felt and perhaps, the way to continue approaching immersion in the field of media art.

 

Rethinking Multimedia

It is often common for one to assume the meaning of certain words and phrases; especially if said word or phrase is common place in one’s daily life.

Navigating modern music libraries become a multimedia experience in and of themselves.

Such is the case with the term, multimedia. In this digital day and age, the word multimedia is thrown about rather flippantly. We have come to assume that all media is multimedia – no form of artistic expression really exists by itself anymore. Even something like assumably “single-media” like listening to music has become an exercise in multimedia consumption. The digital consumption of audio puts in a situation where we first interact with our digital “libraries” sorted by any number of categories from albums to playlists. We sift through the lists of words and images before arriving at our music. Even then we are likely presented with the album art of the track with all its subtext and suggestions as the music begins to play. The experience is never purely auditory; perhaps even a little distracted.

With the advent of video sharing services like YouTube, the act of music consumption if further skewed towards a multimedia experience with users likely engaging with music videos as opposed to a purely auditory soundtrack. These videos augment the experience of listening to a track by providing visuals that create a richer experience that would otherwise be unattainable with just one type of media. While some may embrace the expansion of the original intended experience of the media, others yet may lament the added distraction muddying the original intended experience.

Coldplay’s music video for “Hymn for The Weekend” illustrates the vibrant Holi festival in India, creating a stunning visual that is otherwise impossible to create using just the media of music.

No matter one’s stance, multimedia as a term then, could be defined as the totalizing sensory experience gained from combining more than one method of expression to create a sum that is arguably greater than its parts. This concept of totalizing is discussed in the essay as the Gesamtkuntswerk. Attributed to Richard Wagner, the Gesamkuntswerk or “Total Artwork” is Wagner’s idealized union of all the various aspects of creative expression; the combination and unification of all the arts to create an experience that could capture the essence of human experience.

This concept and definition was of particular interest to me as it reinforced a personal believe that I always maintained about the term multimedia. While it may have only be in prevalent use over the last 20 to 30 years and heavily tied with technology, I have always held onto the notion that multimedia was far from a recent concept. Like the caves of Lascaux, examples of multimedia experiences predate many of humankind’s technological advances. However, as the reading suggests, it is with the advent of technology (particularly the personal computer) that artists and technologist alike have been able to fully bring to fruition the Wagner’s vision. The concept of multimedia as a meta-media (a medium comprising other media) would not have been able to be as successful if not for the efforts of technologists like Vennevar Bush and Tim Berners Lee. Our modern understanding of multimedia is predicated on the works of these pioneers.

What was particularly striking to me as well was how many of the speculations made by artists and technologists in the essay have since come to be reality. For example, Weiner’s preoccupation with human-computer relationships in his study of cybernetics led him to the conclusion that “the quality of our communication with machines effects the quality of our inner lives”. Today, we have entire professions built around this notion – user experience design. The rise of social media and the significant social impact that our relationship with our devices have greatly changed how me consume media and perceive the world. These “machine-mediated” experience of social interactions and our experience of the world around us have the power to shape and affect how we think and perceive. Multimedia then, has become not just a tool for artists to create better modes of creative expressions, but it has also evolved to place greater power in the hands of creatives to influence audience reactions and the outcome of a work.

Another arresting prediction made by Scott Fischer was the vision of a immersive virtual space that would “give birth to a new form of participatory, interactive electronic theatre.” With recent advances in the field of Virtual and Augmented Reality, VR has fast become a technology that is getting more and more accessible to the general public. Platforms such as the HTC Vive have given birth to online shared spaces where users can engage in telematic interactions with others across vast distances. With full body tracking, voice chat and surround audio, people now have access to fully immersive environments where they are able to take on any likeness and interact with others as if in the same space.

VRChat and the Ugandan Knuckles meme phenomena is one such example of Fischer’s vision of a participatory electronic theatre coming to life. The platform was original envisioned as a space for users to interact with one another in an attempt to escape the mundane realities of regular life, similar to earlier concepts such as Second Life. However, the richer interactive experiences afforded to users soon saw players creating personas, characters, stories and lores in a machine-mediated multi media space. This types of ad-hoc, participatory theatre would be difficult to recreate without the aid of multimedia.

Multimedia over the years has come to represent more than just the nexus point of a host of various creative expressions. We have been able to create, richer, deeper and more all encompassing experiences that challenge both creators and audiences alike. The digital revolution has allowed for the genesis of type of media that truly allows for us to express ourselves the way our minds work. While some may argue that technology and multimedia have worked against us by making us more distracted individuals, perhaps, this was the intended result after all? As Ted Nelson philosophized, perhaps multimedia, like all human inventions, is an extension of ourselves. Perhaps the burden of responsible consumption sits of our shoulders after all.