hci

Natural language generation (NLG) – when computers produce text-based output in readable human languages – is becoming increasingly prevalent in our modern digital age. This paper will review the ways in which an NLG system may be framed in popular and scholarly discourse: namely, as a tool or as an agent. It will consider the implications of such perspectives for general perceptions of NLG systems and computer-generated texts. Negotiating claims made by system developers and the opinions of ordinary readers amassed through empirical studies conducted for this research, this paper delves into a theoretical and philosophical exploration of questions of authorial agency related to computer-generated texts, and by considering whether NLG systems constitute tools for manifesting human intention or agents in themselves.

This paper will begin by considering NLG systems as tools for manifesting human intent, the more commonly expressed view amongst developers and readers. An NLG system arguably serves as an extension of a human self (e.g. the developer or the user). Yet one cannot ignore the increasing autonomy of such systems. At what point does an extension of the self become a distinct entity altogether?

The discussion will then shift to considering NLG systems as agents in themselves. As evidenced by the results of studies conducted for this research, ordinary readers do tend to attribute authorship to computer-generated texts. However, these readers often attribute authorship to the system rather than its developers, indicating that – in some way – the system is distinct enough from its creators to warrant the title of author. Yet conventional modern understandings of the word ‘author’ suggest that authorship at least partly presumes intentiondriven agency. Do NLG systems adhere to this expectation? Through reference to various theoretical perspectives, this paper will argue that some NLG systems may surpass the ‘tool’ title and more appropriately be deemed authorial agents. This type of agency, however, is not so characterised by the free-will intention of human writers, but by the intention to fulfil a designated objective that is respected within broader social contexts. When readers attribute authorship to the NLG system itself, that entity is permitted a place within the fluid social networks that humans populate. The NLG system becomes an algorithmic author.

For the last year or two I’ve been focusing most of my research and writing on the notion of ‘interface’ – a technology, whether book or screen, that is the intermediary layer between reader and writing. What I’ve found is that ‘interface’ gives us a wedge to approach the broad and complex question of how the reading and writing of poetry have changed in the digital age and how the digital age has in turn changed the way in which we understand what I call “bookbound” poetry. It seems to me that a discussion of digital poetry in terms of interface – a discussion whose methodology is driven by the field of Media Archaeology – could be a crucial intervention into both poetry/poetics and media studies in that it meshes these fields together to 1) make visible the Human-Computer interfaces we take for granted everyday; and 2) to frame certain works of electronic literature as instances of activist media poetics.

In part influenced by the so-called “Berlin school of media studies” that has grown out of Friedrich Kittler’s new media approach, Media Archaeology is invested in both recovering the analog ancestors of the digital and reading the digital back into the analog. And so the argument I keep trying to make is this:  nineteenth-century fascicles as much as mid-twentieth century typewriters and later-twentieth century digital computers are now slowly but surely revealing themselves not just as media but as media whose functioning depends on interfaces that fundamentally frame what can and cannot be said. I am, then, trying to move the definition of “interface” outside its conventional HCI-based usage (in which interface is usually defined as the intermediary layer between a user and a digital computer or computer program) and apply it to writing media more broadly to mean the layer between reader and any given writing medium which allows the reader to interact with the text itself. Moving the fields of HCI and literary studies closer together through a simple widening of the term “interface” does not just signal a mere shift in terminology. Instead, my sense is that a hybridizing of the two fields helps to move the study of electronic literature into the post-Marshall McLuhan, enabling us to go beyond repeatedly pointing out how the medium is the message and take up Katherine Hayles’ well-received injunction for “media-specific analysis” to get at not just particular media, but particularities such as the interface in the individual media instantiations of e-literature.

It also seems to me that an attention to interface – again, made possible through attention to certain works of e-literature – is a crucial tool in our arsenal against a receding present…by which I mean without attention to the ways in which present and past writing interfaces frame what can and cannot be said, the contemporary computing industry will only continue un-checked in its accelerating drive to achieve perfect invisibility through mulit-touch, so-called Natural User Interfaces, and ubiquitous computing devices. My sense is that the computing industry desires nothing more than to efface the interface altogether and so also efface our ability to read let alone write the interface.

(Source: Author's introduction to the essay)

Metainterface is about interface aesthetics and culture, and as an analytical strategy, it focuses on the tendency in art that reflects the contemporary interface; that is, on readings of artworks. In this sense, it presents contemporary art works, but it also reflects on the current challenges of contemporary interface culture in a situation where the computer’s interface seemingly both becomes omnipresent and invisible; where it at once is embedded in everyday objects and characterised by hidden exchanges of information between objects; or, what it conceptualizes as a metainterface. By bringing the tendency in artworks forward, the book aims to demonstrate how certain critical interfaces have an ability to reflect the deeper fissures within new technologies and the production of the work of art itself; an ability to show us an interface, after the interface has seemingly disappeared into ‘smart’ futures and new promises of anticipation, participation, and emancipation.

Critics have understandably fetishized the electronic page or digital screen as a way to understand the relationship between the algorithmic logics that drive computation and the public rhetorics of display. At the same time an evolving set of practices within electronic literature continues to be in dialogue with contemporary digital media arts practice and its move to explore the meaning of incorporating autonomous sensing and new forms of human-computer interaction in dialogic works. Considering the rhetorical position of devices such as the iPad and considering them as more than viewing apparatuses or interfaces for reading it is possible to engage differently with a whole set of binaries around camera vs. scanner, optics vs. sensors, and representation vs. registration.

This presentation focuses on three writers who are utilizing augmented reality technologies to expand the repertoire of digital poetics. Judd Morrissey has collaborated with choreographer Mark Jeffery to stage The Operature (2014), combining live performance and augmented reality multimodal poetry to highlight anatomical science and voyeuristic erotic spectacle in which the temporary tattoos worn by the work’s dancers can be read by a surveillance apparatus. In contrast, a voice of intensely personal lyricism that speaks very intimately to the listener defines Caitlin Fisher’s Circle (2011), which is an “augmented reality tabletop theatre piece” that deploys the iPad or smart phone in a much more private setting. Amaranth Borsuk’s approach to augmented reality multimedia favors an aesthetic of sleek mid-century modernism and machined characters in Between Page and Screen (2012), which investigates “the place of books as objects in an era of increasingly screen-based reading.“ The actual pages of this artist’s book contain no legible text; the reader is presented with only abstract geometric patterns and a URL leading to the Between Page and Screen website, where the book may be read by using any browser and a webcam. With a new generation of reading machines that can perceive contrast relationships in a 2D visual environment, sensors can read the “ink” of tattoos, the grain of family artifacts, and the code of a numbered artist’s book or print-at-home emulation.

These works may also spur a new kind of criticism that may require that we rethink the theoretical framework of immediacy, hypermediation, and remediation proposed by Bolter and Grusin as we reconsider our own interchanges with the sensorium of the mechanical apparatus. In responding to Galloway, Thacker, and Wark’s theses about “excommunication” and the possibility that the relation between “objects and things” problematizes the standard narrative about media, mediation, and communication, Benjamin Bratton has suggested that this could more precisely be characterized as “incommunication” around the activities of “sensing, addressing, and pricing.” Borsuk, Morrissey, and Fisher create works that dramatize device-to-device relations and their associated modes of reading.

(Source: ELO 2015 Conference Catalog)

The transformation of interface from a merely indicative tool of navigation to a suggestive element infused with metaphorical power in text-based hypertext literature, and the incorporation of hypermedia and modes of play and games into the hypertext scenario--both strains are gradually winning attention in electronic writing. Topics such as the clarification of paidia (play) and ludus (game) constituents, their formal impact on literature, and the comprehension of the aesthetic matrices projected by the symbiotic infusion of literature, play and games, have been posited, creating a new node in the network of literary studies. In order to explore these fertile new fields, this paper first assigns itself to a survey of interface design and a formal observation of play and games in samples of electronic literature. Furthermore, the paper is focused on the interlaced poetics of representation (narrative) and simulation (paidia / ludus) in literary hypertext, play and games (together to be occasionally called, cybertext or ergodic literature, both terms taken from Espen P. Aarseth). It is hoped that the paper can bring more poetical recognition to digital textualities.

Source: Author's abstract

Although human interaction with technological artifacts often involves treating them as if they are alive, the dominant discourse in our society portrays technology as the instrument of its human master. In the context of computing, our desire of absolute control over machines manifests itself both as the human computer interaction (HCI) community’s emphasis on “usability” and as popular culture’s apocalyptic imagination of the out-of-control artificial intelligence (AI) systems trying to eliminate humanity. It is revealing that, for instance, the word “robot” comes from “slave” in Czech. This paper examines the social and aesthetic limitations of this narrow instrumental view of technology. It proposes an alternative interaction model based on machine subjectivity, that is, constructing and perceiving computer systems as an independent entity in its own right.

Based on Heidegger’s theory of the “enframing” nature of modern technology (Heidegger, 1977), and experiments of modern dance (Copeland, 2004), this paper argues that perceptual and sensory habits, including our interaction with computing artifacts, are political. Both the prevailing human-leader-computer-follower interaction model and the apocalyptic literary imagination are limited by and reinforce the discourse of hierarchical control and power relationship in broader social contexts. Machine subjectivity, on the other hand, offers a playground for exploring alternative relationships between human and “the other” (i.e., computer) and consequentially provides insights to new social orders, such as the ones based on “multi-dominance” (Lewis, 2000).

In addition to political commentaries, machine subjectivity also affords novel aesthetic and meaningful interactive experiences. This paper examines the manifestation of machine subjectivity in a range of cultural artifacts in electronic literature, music, dance and visual arts, and highlights the novel aesthetics and expressions afforded by allowing computers to act independently.

Finally, the paper discusses our text-based interactive narrative system Memory, Reverie Machine. The system algorithmically generates stories about a robot character, who is controlled jointly by the user and the AI system. The tension between user and machine subjectivity, foregrounded in the struggle to gain control over the main character, is used to explore themes such as agency, resistance, and dis/empowerment.

(Source: Author's abstract for ELO_AI)