immediacy

Critics have understandably fetishized the electronic page or digital screen as a way to understand the relationship between the algorithmic logics that drive computation and the public rhetorics of display. At the same time an evolving set of practices within electronic literature continues to be in dialogue with contemporary digital media arts practice and its move to explore the meaning of incorporating autonomous sensing and new forms of human-computer interaction in dialogic works. Considering the rhetorical position of devices such as the iPad and considering them as more than viewing apparatuses or interfaces for reading it is possible to engage differently with a whole set of binaries around camera vs. scanner, optics vs. sensors, and representation vs. registration.

This presentation focuses on three writers who are utilizing augmented reality technologies to expand the repertoire of digital poetics. Judd Morrissey has collaborated with choreographer Mark Jeffery to stage The Operature (2014), combining live performance and augmented reality multimodal poetry to highlight anatomical science and voyeuristic erotic spectacle in which the temporary tattoos worn by the work’s dancers can be read by a surveillance apparatus. In contrast, a voice of intensely personal lyricism that speaks very intimately to the listener defines Caitlin Fisher’s Circle (2011), which is an “augmented reality tabletop theatre piece” that deploys the iPad or smart phone in a much more private setting. Amaranth Borsuk’s approach to augmented reality multimedia favors an aesthetic of sleek mid-century modernism and machined characters in Between Page and Screen (2012), which investigates “the place of books as objects in an era of increasingly screen-based reading.“ The actual pages of this artist’s book contain no legible text; the reader is presented with only abstract geometric patterns and a URL leading to the Between Page and Screen website, where the book may be read by using any browser and a webcam. With a new generation of reading machines that can perceive contrast relationships in a 2D visual environment, sensors can read the “ink” of tattoos, the grain of family artifacts, and the code of a numbered artist’s book or print-at-home emulation.

These works may also spur a new kind of criticism that may require that we rethink the theoretical framework of immediacy, hypermediation, and remediation proposed by Bolter and Grusin as we reconsider our own interchanges with the sensorium of the mechanical apparatus. In responding to Galloway, Thacker, and Wark’s theses about “excommunication” and the possibility that the relation between “objects and things” problematizes the standard narrative about media, mediation, and communication, Benjamin Bratton has suggested that this could more precisely be characterized as “incommunication” around the activities of “sensing, addressing, and pricing.” Borsuk, Morrissey, and Fisher create works that dramatize device-to-device relations and their associated modes of reading.

(Source: ELO 2015 Conference Catalog)

Most new media work establishes interactivity within a curated installation space: a gallery, a festival, or an area whose purpose is to exhibit art. However, recent experiments in new media narratives have made use of the capabilities of smartphones and tablets to present experiences that are aware of the user’s position in space and
even their current behavior or object of attention.

Specifically,augmented reality works set themselves apart by re-contextualizing environments and objects encountered in everyday life, removing the fourth wall and blurring or eliminating an interactive experience's boundaries. This differs markedly from the purism of the imagination tapped by literature, and often even favors more realistic integration, in contrast to stylistic depictions and abstractions used in monitor-based works. Augmented reality’s strength and interest lies in how it embeds a story in an environment, or how it can be used to awaken new awareness of a viewer to their surroundings. This bridges the world of the reader with the diegesis of the narrative, resulting in works that react to the immediacy of the experienced space.

The major draw to augmented reality in industry has been to use this immediacy to push a product or service, and embed it within the viewer's world. As with any new platform for generating content, its commercial appropriations and implementations threaten to control the discourse of its use. It is important therefore to establish an artistic framework that conveys and critiques the narrative affordances AR makes possible.

Augmented reality is a relatively nascent medium, especially when set alongside other works of electronic literature. Our talk will start with a brief illustration that will allow audience members with smartphones to collaboratively experience an augmented reality work.

(Source: Author's abstract, 2012 ELO Conference site)