augmented reality

Spaces Speak is a panel presentation to raise awareness and enlist participation in (RE)VERB an Audio AR ‘zine for e-lit writers/artists. (RE)VERB is an audio augmented reality zine dedicated to spatially conceived electronic literature projects that explore the aesthetic possibilities of sonically delivered language engaging with the physical and corporeal experience of the environment. As a publication (RE)VERB was inspired by the Emerging Spaces for E-Lit Creations initiative to expand works that engage with popular social media spaces. 

Spaces Speak will consist of five presenters including editorial board members, the guest curator for issue one and artists who created work for the first issue. The artists and curators will discuss the challenges and rewards for producing site-specific work and the concepts driving their creative decisions. 

The panel will also feature an overview of the goals and artistic vision for the ‘zine. A discussion of the first issue, with a sneak-peak/advanced listen to excerpts from the forthcoming ‘zine to be released in June 2021. Other details discussed will include the release schedule for future issues, explanations for how listeners can access content, integration with social media platforms and how e-lit community members can participate in upcoming issues. 

In addition, Spaces Speak will highlight the partnerships with organizations the ‘zine will be pursuing to participate in its open calls and curation (Eyebeam, NEW INC, AFROTECTOPIA, the New Media Caucus, Rhizome, Harvestworks, and AudioAR.org) for E-Lit to reach new audiences in related fields like new media art, internet art, sound art and audio AR. 

The last segment of the panel discussion will be an open dialog with the ELO community to hear the directions they would like the ‘zine to pursue, what might be compelling thematic topics and locations, what other organizations should we include in our outreach and who they would suggest for international curators and artists for future issues as we continue to expand the global reach of the ‘zine. 

In relation to the conference themes, the (RE)VERB ‘zine can be seen as a case study of a third space, instead of a large corporate behemoth platform siloed from the content or individualistic artist vision, (RE)VERB is a partnership between publication and platform, working in dialog with the Gesso, an AR platform dedicated to spatially conceived projects, to create a sustainable E-Lit creative space. 

Taken as a whole, (RE)VERB enables electronic literature writers to engage with the sensorial experience of place, the granularity of the human voice and chance occurrences in the environment to provide an expansive opportunity for aesthetic experimentation and a vital co-mingling of creative communities. Spaces Speak provides an opportunity for an open exchange of ideas and an exploration for how all community members can be involved on any level of the project.

Even before worldwide quarantines added impetus, material gaming had already become increasingly enacted in virtual spaces. Rather than virtual play replacing the material, as some speculated in the early days of videogames, material play has become increasingly entangled with virtuality. These increasingly complementary modes of play offer a rich space for exploring the multifaceted embodied and conceptual activity of play, the blending of material and virtual that in many ways defines games.The three panelists encompass a wide range of perspectives, including the perspective of a game maker translating material play into the digital realm, that of a Human-Computer Interaction (HCI) scholar who researched how players interact differently with the Catan boardgame and its digital implementations, and that of a theorist reflecting on how virtual spaces remediate material affects. Together, these diverse perspectives aim to explore the paradoxical yet generative spaces where materiality and virtuality intersect in gaming.The theoretical approach looks at analog games as capable of producing the specific circumstances that foreground the affective relationships between the players and the other pieces of the assemblage. Because of the procedural nature that necessitates specific types of interactions between parts of the play assemblage, analog games amplify the social interactions between players and differently produce affective orientations as a consequence of their systems. Then examines the ways that these games are remediated and adapted to digital platforms highlighting the things that are lost or changed in the move to digital, uncovering the types of experiences that are important for each type of adaptation.

The HCI approach presents Association Mapping (AM) in HCI; called so because the formation of a network is due to objects making associations in context. By recording the associations that form a network, it is possible to understand what objects are most central within that network. . This research contributes to the next paradigm of HCI by providing a new tool to understand use that is fragmented, distributed, and invisible. AM incorporates association as its measurement. This results in passive measures of attention, hybridity, and influence in network formation of any kind. It does this by making the systemic nature of use visible and capable of evaluation at any level.And finally the design approach applies design strategies for incorporating three main types of play: Screenplay, Gameplay, and Roleplay, seeking to answer questions about how to bridge the narrative and performance aspects of digital and analog play. This is particularly applicable to classic games that are associated with transmedia narratives and characters, such as the Clue board game, where there are established cinematic traditions and character roles.During the COVID-19 pandemic, board games have become a useful medium for examining our changing relationship with physical and digital interaction. In addition to presenting our own findings, this panel also offers several methodologies for furthering research into the intersections of the analog, digital, physical, and virtual.

What Do We Call This?

Between 2019 – 2020 The University of South Wales collaborated with a consortium of creative commercial practitioners dubbed Fictioneers in a UKRI funded, Audience of the Future R&D demonstrator project designed to further develop digital storytelling within the UK Creative Industries. Using the popular Wallace and Gromit IP, the consortium drew upon their combined skills in games production, animation, creative marketing and new technology development to create a location-based experience targeting young audiences entitled Wallace & Gromit: Big Fix Up, designed to propel new and playful identities for a traditional narrative media.

Wallace & Gromit: Big Fix Up is an ambitious and complex production. Through their research and development efforts, Fictioneers sought to develop a viable production alternative to branching tree, digital story-telling structures which risk combinatorial explosion. Instead, the application delivers a rich tapestry of serialised, short media elements. Linked by a central, enhanced mobile application, the multi-platform media include YouTube videos and comics, as well as augmented reality game-play challenges. The application aims to engage new audiences and provide innovative ways for long term fans to interact with media favourites. Mimicking a variety of social newsfeed items these media elements are variable, chunked and optional to view. They are also pre-determined and closely networked via the central newsfeed. The story-flow is complex nevertheless, incorporating enhanced augmented reality story-telling, multi-platform media and mobile game-play.

The hybridity of this experience posed new challenges regarding the most definitive way to describe the experience on offer, as well as the most helpful frameworks to evaluate it. With few alternative terms on hand to describe this genre, the term experience was often used to describe the sort of hybrid encounter made possible via this complex network of media influences, but experience is still an open-ended concept that can be hard to pin-point. Alternative terms like digital story-telling may also be useful place-holders to help delineate interactive and narrativised experiences from traditional media encounters, nevertheless such terminology is still only useful to an extent. Narrative frameworks such as characterisation, pace and tone are relevant to projects like Wallace & Gromit: Big Fix Up, but they don’t capture all the elements that audiences encounter in real time. Describing the experience as a game can be equally problematic, since it can set up expectations of a very different type of challenge-driven, dramatic experience than this application delivers.

In this paper I explore what additional insights can be gained by also considering the interplay of technology and creativity within the research and development process. Technology is a defining feature of this digital storytelling experience. Augmented reality technologies, for example, offer dynamic, enhanced tracking and visualisation opportunities, whilst also demanding strict file-size constraints, comprehensive audience testing and extensive cross-disciplinary collaboration. By evaluating the creative and technical processes shaping the development of this hybrid media identity, I explore the ways in which any effective definition of this new type of distributed genre is likely to be as much about co-ordination, as new experience.

This dissertation explores the relationships between literacy, technology, and bodies in the emerging media of Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR). In response to the recent, rapid emergence of new media forms, questions arise as to how and why we should prepare to compose in new digital media. To interrogate the newness accorded to new media composing, I historicize the literacy practices demanded by new media by examining digital texts, such as video games and software applications, alongside analogous “antiquated” media, such as dioramas and museum exhibits. Comparative textual analysis of analogous digital and non-digital VR, AR, and MR texts reveals new media and “antiquated” media utilize common characteristics of dimensionality, layering, and absence/presence, respectively. The establishment of shared traits demonstrates how media operate on a continuum of mutually held textual practices; despite their distinctive forms, new media texts do not represent either a hierarchical or linear progression of maturing development. Such an understanding aids composing in new VR, AR, and MR media by enabling composers to make fuller use of prior knowledge in a rapidly evolving new media environment, a finding significant both for educators and communicators. As these technologies mature, we will continue to compose both traditional and new forms of texts. As such, we need literacy theory that attends to both the traditional and the new and also is comprehensive enough to encompass future acts of composing in media yet to emerge.

We are witnessing the emergence of a third generation of electronic literature, one that breaks with the publishing paradigms and e-literatury traditions of the past and present.

N. Katherine Hayles first historicized electronic literature by establishing 1995 as the break point between a text heavy and link driven first generation and a multimodal second generation “with a wide variety of navigation schemes and interface metaphors” (“Electronic Literature: What Is It?”). Even though Hayles has since rebranded the first wave of electronic literature as “classical,” generational demarcations are still useful, especially when enriching the first generation with pre-Web genres described by Christopher Funkhouser in ​Prehistoric Digital Poetry​ and others. My paper redefines the second generation as one aligned with Modernist poetics of innovation by creating interfaces and multimodal works in which form is invented to fit content.

Third generation electronic literature emerges with the rise of social media networks, the development of mobile, touchscreen, augmented reality (AR) and virtual reality (VR) platforms. This generation is less concerned with inventing form and more with remixing and creating work within well established platforms and their interfaces, parallel to a return to recognizable poetic forms, Romantic subjectivity, and pastiche in Postmodern poetry. This includes Instagram poetry, bots, apps, kinetic typography, lyric videos, memes, Twine games, and works that take advantage of smartphone, touchscreen, and VR technologies. This generation leaves behind book and open Web publishing paradigms and embraces new funding models, such as crowdfunding and software distribution platforms.

Even though the first generation of e-lit ended about 20 years ago, the second and third generation currently coexist, much as Modernist and Postmodernist literature do. And while second generation works are currently more sophisticated, complex, and aligned with academia, the third generation will produce the first massively successful works because they operate in platforms with large audiences that need little to no training to reading them. So while second generation works will continue to attract critical acclaim with limited audiences, it is the third generation that will produce the field’s first #1 hit.

(author abstract)

The idea of walking as the practice of narrating the city constitutes the recurrent theme of Michel de Certau’s “The Practice of Everyday Life:” the pedestrian activity is repeatedly compared to or described as “enunciation,” “enunciatory operations,” “statements” and “stories”. According to the French sociologist, “[t]he act of walking is to the urban system what the speech act is to language or to the statements uttered” (de Certeau 1988: 97). The story of spatial practices “begins on ground level, with footsteps” (de Certeau 1988: 97) and “the art of >>turning<< phrases finds an equivalent in an art of composing a path (tourner un parcours)” (de Certeau 1988: 100). It might well be said that walking in the city represents not only the very prototype of ergodic literature (Aarseth 1997) but also predates the notion of augmented reality in its technological sense. One of the practices which directly address narrative potential of moving across space is soundwalking: theorized both as the classic method of acoustic ecology (Westerkamp 1974, 2002) and its current re-examinations (Paquette & McCartney 2012, McCartney 2012). In my paper I would like to analyze the practice of soundwalking as narrating the city in “the augmented aur(e)ality” (Noll 2017), yet shifting the focus to its mobile media, touch screen-based version. My case study will be Udo Noll’s radio aporee platform and its most recent incarnation: miniatures for mobiles (a platform for sound-based, locative and spatial micro-narratives, including the phone app). Therefore, the practice of narrating the city through walking will be analyzed from the double perspective: as augmented aur(e)ality and as the embodied experience played out on two levels simultaneously (as physically moving the body through space and as interacting with touch screens of mobile media).