mixed reality

This presentation aims to reflect on two labels that have been used to define sets of artefacts born out of the same context but evoking different connotations. I refer to the terms “post-internet” and “post-digital”. Both terms allude to a post-stage, a leap that announces a cultural shift, perceived by artists but difficult to pinpoint and demarcate with precision, a prefix that might refer to ‘after’ (chronologically) as well as ‘beyond’ (spatially); often used to highlight that what has been superseded is the novelty and exceptionality of the internet and digital technology. Actually, these terms address the fact that digital media is no longer a form of mediation but it has become our ontology, though this new form of being is of such a diffuse, complex and assembled nature, not even Haraway could have anticipated it.Triggered by impulses of excess and overindulgence, on the one hand, or sustainability and preservation, on the other, post-internet and post-digital art emerge from a networking and tech-savvy sensibility that has altered the relation between artist, audience, and art object.In particular, I am going to focus on the work of artists that use Instagram as an art gallery for exhibiting their work (Almudena Lobera, Johanna Jaskowska, Lucy Hardcastle) or to sabotage the platform from the inside (Amalia Ulman, Joshua Citarella). Bearing a family resemblance to electronic literature, these works also explore the narrative process in the construction of an artist’s identity, the changing territories of human-machine/artist-spectator interactions and digital-analogue materializations. The art objects they produce can be classified as “phygital”, physical/digital constructs that inhabit, in a myriad of different possibilities of mediation and convergence, the physical and the digital spaces.As artists explore the platform’s potential as exhibition space, advertising site, and conversation aisle, their phygital objects reflect the tensions between a nostalgia for an analogue craftsmanship which rebels against machinic perfection and an interrogation of human creativity that propels us into the future through an ever more profound symbiosis with our technological habitus.

(Source: author's own abstract)

This dissertation explores the relationships between literacy, technology, and bodies in the emerging media of Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR). In response to the recent, rapid emergence of new media forms, questions arise as to how and why we should prepare to compose in new digital media. To interrogate the newness accorded to new media composing, I historicize the literacy practices demanded by new media by examining digital texts, such as video games and software applications, alongside analogous “antiquated” media, such as dioramas and museum exhibits. Comparative textual analysis of analogous digital and non-digital VR, AR, and MR texts reveals new media and “antiquated” media utilize common characteristics of dimensionality, layering, and absence/presence, respectively. The establishment of shared traits demonstrates how media operate on a continuum of mutually held textual practices; despite their distinctive forms, new media texts do not represent either a hierarchical or linear progression of maturing development. Such an understanding aids composing in new VR, AR, and MR media by enabling composers to make fuller use of prior knowledge in a rapidly evolving new media environment, a finding significant both for educators and communicators. As these technologies mature, we will continue to compose both traditional and new forms of texts. As such, we need literacy theory that attends to both the traditional and the new and also is comprehensive enough to encompass future acts of composing in media yet to emerge.

Augmented reality y (AR) is the term for a constellation of digital technologies that enable users to display and interact with digital information integrated into their immediate physical environment. AR is the technological counterpart of virtual reality (VR), which until recently was much better known, though not necessarily widely used (see virtual realit y). In the late 1960s and early 1970s, the digital graphics pioneer Ivan Sutherland developed the first head-worn computer displays permitting the user to see computer graphics overlaid on their visual field. fi Although Sutherland’s displays constituted the beginning of both AR and VR, interest in VR eclipsed that of AR in the following decades, as display and tracking technologies were being developed. Work on AR was revived in the 1990s by Steve Feiner, together with his graduate students Blair MacIntyre and Doree Seligmann at Columbia University, as well as at other universities and research centers. (The term augmented reality y itself was possibly coined in 1990 by a researcher at the Boeing Company.) AR and VR are often classed as examples of mixed reality (MR) on a spectrum described by Paul Milgram in 1994.

The spectrum indicates the ratio between the amount of information provided by the computer and the amount coming from the user’s visual surround. At one extreme there is no computer input (only the socalled real environment); at the other, the computer is providing all the visual information, and possibly sound as well, to constitute a complete “virtual environment” or “virtual reality.” AR lies in between these extremes, but typically far more of the user’s view is constituted by the actual visual environment and the computer is adding relatively little information. “Augmented virtuality” is a little-used term to describe the case where some elements of the physical world are integrated into a predominantly virtual environment.
The spectrum, however, obscures a fundamental distinction between AR and VR. VR cuts the user off ff from involvement in the physical and, by implication, the social world. AR acknowledges the physical world rather than eliding it. It is one of a trio of such technologies that came to prominence in the 1990s—the other two were ubiquitous computing and tangible computing. Each of these was in its own way a response to the implicit promise of VR to project the user into a disembodied cyberspace (see cyberspace, virtual bodies).

(Johns Hopkins University Press)

Starting with Homer’’s Odyssey through R. Larsen’’s The Selected Works of T.S. Spivet literature describes journeys, wanderings and world-explorations in which spatial realms provide the basic dispositives for the series of narrated events. While theories of literature from G. E. Lessing’’s Laocoon to K. Hamburger’’s Logic of Literature had conceived of literature as a particular way of perceiving time, M. M. Bakhtin’’s theory of the ““chronotope”” made space into a central constituent affecting the perception of the models in literary scholarship.

In recent years, current electronic media have prompted wide-ranging considerations on the importance of space for socio-cultural processes (the ““spatial turn””). With the application of mobile media devices such as mobile phones, GPS and PDAs and the development of mixed reality environments in museums, galleries or research labs, new combinations of physical, virtual and symbolic spaces have been realized. Metaphorically, we might even say that literature, after having passed through the needle’’s eye of book culture, seems to be reverting back to the multimodal patterns of action and the forms of antiquity, of the Middle Ages, or of the Renaissance. This, however, is taking place on a completely changed media-technological level: texts, objects, bodies and spaces combine in a largely uncharted way; electronic media take ““body language”” to a new level since more and more often the whole body is involved in the media activity. Increasingly complex sensors (integrated into vehicles, clothes and environments) ““realize””——in other words: measure——the movements of the body, its mimics and gestures. This ““multimodal”” body itself then also exchanges information with the ““products”” of this kind of technology. Such medial couplings and framings enable the cooperation of non-symbolic activities, natural language activities and algorithmic processes of computer systems.

Of special interest for the analysis of literary developments today are environmental, exterior or urban projects, the so-called ““Locative Narratives”” using the previously-mentioned locative media such as GPS-tools, PDAs or others, aestheticizing each of them in a quite unexpected turn that inverts the traditional processes of literarization from the ““head”” back to the ““feet;”” they adapt literary patterns like travel-, adventure-, love-, or detective narratives, returning their imaginary movements into real ones again. Among these are projects like Jean-Pierre Balpe’’s Fictions d’’Issy, Stefan Schemat’’s Wasser, Gabe Sawhney’’s [murmur], works of the collaborative artists’’ group ““34 North, 118 West,”” narrative online journals like the Madison Avenue Journal (http://www.madisonavenuejournal.com), and also projects like Worldwatchers by Susanne Berkenheger and Gisela Müller (http://www.worldwatchers.de/) that study the growing intensification of social control via electronic systems of observation. This contribution attempts to outline an initial overview theoretically situating these projects.

(Source: Authors' abstract for ELO_AI).

The objective of this paper is to describe the potentialities of Mobile Tagging as a tool for increasing and spreading the effects of Mixed Realities in Electronic Literature. In this sense, we will start introducing the main concepts and some examples of Mixed Realities followed by the concepts and examples of Mobile Tagging, showing that they are connected and benefit each other and can benefit eLit as well. Mixed Reality (or MR) refers to the fusion of the physical and virtual worlds to produce new environments and visualizations where physical and digital objects co-­‐exist and interact in real time. On the other hand, mobile tagging is the process of reading a 2D barcode using a mobile device camera. Allowing the encryption of URLs in the barcodes, the mobile tagging can add a digital and/or online layer to any physical object, providing so several levels of mixed realities related to that object. Although Mixed Realities technologies have already existed for decades, in the past they were very expensive. Recently, mobile devices have also become tools for mixed realities. Due their pervasiveness, their potentiality for increasing the dissemination of mixed realities is enormous and can be leveraged by mobile tagging. Since mobile tags are simple tags that can be placed on virtually any place, physical object or person, added to the fact that the cell phones with cameras have become a very inexpensive and common device, the mobile tagging process can be said as one of the easiest and simplest ways to create mixed realities. The uses of mobile tagging and the several levels of mixed realities have applications in many areas going from medicine and engineering to literature and arts. Mobile Tags work like physical links to the web, allowing so that virtually anything can be part of an expanded mixed reality environment, including printed books, e-­‐books, poetry and any kind of text. This paper/presentation will use examples in the fields of art and literature to illustrate the functionality of the mobile tagging to create mixed realities.

(Source: Author's abstract for ELO_AI)

This essay discusses the emergence of lifestyles under the paradigm of urban life, based on the results of research on interface design for mobile connections in ubiquitous computing with pervasive and sentient interfaces, which generate cybrid (cyber+hybrid) scenarios for co-located beings that act in physical and digital space. Artistic creation using software art writes programs and uses hardware that convey a sense of presence and action, with digital collage adding information about the physical scene. The digital material is pasted in layers onto the physical space, redesigning places, reconfiguring actions, and mixing realities in a cybrid manner. In other words, locative and mobile interfaces reconfigure the sense of presence by blending in the digital material that adds information to locations. Computers mix into the periphery through transparent interfaces, enabling enactions and affordances in quotidian actions in calm connections with transparent interfaces. Instants are experienced through computers, which become invisible in portable and mobile technologies: cell phones, PADs, displays, computational vision, tags, RFID, Bluetooth, wearable computers, geolocators, trackers, GPS, SMS, MMS, make us co-exist here and there. Augmented Reality (AR) and Mixed Reality (MR), social platforms, perceptual and affective computing, wearable computing, among other examples from various artists and scientists, are discussed here.