interaction

Content type
Year
Language
Publication Type
Record Status
Description (in English)

Part of Tsead Bruinja's poetry collection 'Overwoekerd.' Students of Artez created animations for the poem, for which they experimented with typography, image, sounds, interaction and typography.

Description (in original language)

Onderdeel van Tsead Bruinjas poeziebundel 'Overwoekerd.' Het gedicht heeft meerdere animaties die gemaakt zijn door studenten van Artez. Zij maakten gebruik van beeld, geluid, interactie en typografie.

Description in original language
Screen shots
Image
Image
Short description

The appearance of new technologies and their exponential growth for several decades has changed our way of understanding knowledge. Although it is already a topic that is part of the contemporary background, it is worth remembering that digital culture and the possibilities of the internet have meant a radical change, only comparable, according to Alejandro Baricco, to the printing revolution.

The incorporation of the network and transmedia resources into the literary environment is fostering new poetics; new forms of textuality that, according to Joan-Elies Adell, go beyond the book and turn the computer or any mobile device into the natural space of the work. Hypertext, interaction, video game ... The very essence of literature is changing. Writers who think of the word in conjunction with HTML code, geolocation, processing or other programming tools. With their creations they come to expel us from our areas of literary comfort.

We are talking about jobs designed for the network, that new agora. We are talking about hypermedia works that, in contrast to orality or printed tradition, investigate within what Ernesto Zapata defines as electronality. We are talking simply about literature in the post-Gutenberg era.

Description (in original language)
La aparición de nuevas tecnologías y su crecimiento exponencial desde hace varias décadas ha cambiado nuestra manera de entender el conocimiento. Aunque ya es un tema que forma parte del background contemporáneo, no está de más recordar que la cultura digital y las posibilidades de internet han supuesto un cambio radical, solo comparable, según Alejandro Baricco, a la revolución de la imprenta.

La incorporación de la red y de los recursos transmedia al entorno literario está propiciando nuevas poéticas; nuevas formas de textualidad que, según Joan-Elies Adell, desbordan el libro y convierten el ordenador o cualquier dispositivo móvil en el espacio natural de la obra. Hipertexto, interacción, videojuego… La esencia misma de la literatura está mutando. Escritores que piensan la palabra de forma conjunta al código HTML, a la geolocalización, al processing u otras herramientas de programación. Con sus creaciones vienen a expulsarnos de nuestras áreas de confort literario.

Hablamos de trabajos pensados para la red, ese nuevo ágora. Hablamos de obras hipermedia que, frente a la oralidad o la tradición impresa, investigan dentro de lo que Ernesto Zapata define como electronalidad. Hablamos, sencillamente, de literatura en la era post-Gutenberg.
Description in original language
Record Status
By Hannah Ackermans, 4 December, 2018
Language
Year
Record Status
Abstract (in English)

We are witnessing the emergence of a third generation of electronic literature, one that breaks with the publishing paradigms and e-literatury traditions of the past and present.

N. Katherine Hayles first historicized electronic literature by establishing 1995 as the break point between a text heavy and link driven first generation and a multimodal second generation “with a wide variety of navigation schemes and interface metaphors” (“Electronic Literature: What Is It?”). Even though Hayles has since rebranded the first wave of electronic literature as “classical,” generational demarcations are still useful, especially when enriching the first generation with pre-Web genres described by Christopher Funkhouser in ​Prehistoric Digital Poetry​ and others. My paper redefines the second generation as one aligned with Modernist poetics of innovation by creating interfaces and multimodal works in which form is invented to fit content.

Third generation electronic literature emerges with the rise of social media networks, the development of mobile, touchscreen, augmented reality (AR) and virtual reality (VR) platforms. This generation is less concerned with inventing form and more with remixing and creating work within well established platforms and their interfaces, parallel to a return to recognizable poetic forms, Romantic subjectivity, and pastiche in Postmodern poetry. This includes Instagram poetry, bots, apps, kinetic typography, lyric videos, memes, Twine games, and works that take advantage of smartphone, touchscreen, and VR technologies. This generation leaves behind book and open Web publishing paradigms and embraces new funding models, such as crowdfunding and software distribution platforms.

Even though the first generation of e-lit ended about 20 years ago, the second and third generation currently coexist, much as Modernist and Postmodernist literature do. And while second generation works are currently more sophisticated, complex, and aligned with academia, the third generation will produce the first massively successful works because they operate in platforms with large audiences that need little to no training to reading them. So while second generation works will continue to attract critical acclaim with limited audiences, it is the third generation that will produce the field’s first #1 hit.

(author abstract)

Multimedia
Remote video URL
Description (in English)

The ontological question of the essence of cinema, embodied by Bazin and his "what is cinema" moves today, under the pressure of new forms of consumption of images to a "relocation" of cinema, summarized in this question: "where is the cinema? ". After having investigated the code, the generative, the algorithm, the flow, we could ask ourselves also where is the numerical literature, with an exit of the screen sometimes rendering obsolete this idea of ​​a mainly electronic literature. The desire to write in digital "seems to abandon the tests and essays of digital literature for fear of seeing the possible tracks of new forms of writing to close one by one. Can we ask that this electronic literature resurface in a book in good form like stones in an open digital garden? This is what we asked with our multimedia design students from the DSAA Boulogne (France) whose four creations are detailed below in this proposal. It seems that these students, these digital natives - this strange generation whose lack of history with digital "appearance" allows them to reshape the fields of intervention and limit of digital writing - take up the question where it seems finally the most obvious: in the book, by exploring a digital writing that is written, visualized, sounded and becomes a literature of the sensory object. The four creations of these students are thus exploratory objects working the delimitations of the digital in a logic of the addition. Moved out of the central frame, digital writing is embodied in a diffuse way, sometimes off the screen, in a drift of meaning on the materiality of the digital, in the manner of our everyday, mobile and embodied in the object . Assume that the increased book allows you to climb the first steps of a future location. It is in this idea that we propose to expose these four books made by students of DSAA, today members of the research and creation workshop 3i (Images, Innovation, Interactivity), here is the description more detailed.

 

1. Insignificant noise Heloise of Almeida Insignifiés du Noise is a trans-media project conceived around the plastic and sound exploration of the noise domain. Grouped into three categories, the sounds used in this augmented edition belong to organic, mechanical and electronic fields. Noises from the first two fields, organic and mechanical, have been invested to produce the visual content of the book in its printed form, which the reader manipulates. The noises of the third field - the electronic field - were invested like material of the increase of the edition. This led to the creation of a book consisting of two booklets and a sound increase on smartphone. Thanks to an image recognition system, the reader is able to scan the pages of both booklets simultaneously, and to transcribe the visual translations into a new sound combination. Thus the sounds produced are really the sound signatures of graphic noises.

2. Rode B-6-9 / Anne Deurgroënd Lucie Plancon Rode B-6-9, Anne Deurgroënd (= Road B-6-9, Underground) is an augmented edition on the theme of everyday music, and more specifically that perceptible in transport. It is a call to travel, an abstract and offbeat poetry, composed of photographs and texts interpreted by three actors. In the unique tempo of the train, there exists a multitude of rhythms, singular and diverse, which can gradually appear absurd. This environment, perfectly realistic at the origin, gives way to the chimeras of each. The written text, like the path, is divided into three parts: B-6-9. They become forms of songs interpreted by three different actors. Each adds his sensitivity and his vision of the course to bring a radically abstract and absurd tone. These sound interpretations are readable thanks to NFC chips that must be scanned with your smartphone. In all its abstractivity, this project mixes new and old technologies in an editorial concept comprising 18 musics, 12 posters, 3 pockets and 3 voices united in the same box.

3. The invasion Marjorie Terral The invasion is an edition offering augmented reality content that mixes texts, photos and graphics. The edition evokes an idea of ​​sounds that are repeated, accumulate in different places, at different times of the day, with a search for visual representation for sound environments. The shots, more evocative than figurative, are there to stimulate the imagination, as are the motifs that unfold throughout the book as an attempt at subjective sound translation. These motifs, present in the printed edition, then come alive in the increase bringing each photograph to life, transforming each page into an animated and sonorous picture. For each heard sound corresponds a pattern that invades the photo, these sounds come to accumulate, repeat themselves, register in a rhythm more and more charged to transcribe this feeling of invasion, saturating, blurring the photo, in revealing the presence of these noises in a new way.

4. The world of Hylae Guillaume Verguin "The world of Hylae takes you into a strange universe, inspired by a distant culture ... It's up to you to capture a talisman in this amazing world where only a wild fauna and flora live and to reveal what this magic stone hides ... " On this narrative thread for youth, The world of Hylae experiments with different digital devices: - an interactive installation - an augmented edition, between paper and digital - VR experience All three question the meaning of touch, exploring a digital materiality. The edition positions a paper notebook cut out on a tablet and proposes new interactions. The reader reads the story printed on the cut out notebook, turns the pages and sees the screen of the tablet through the shapes cut out in the notebook; creating new interactions. Each double page then offers a specific interaction for the viewer. Can we talk here about interactive publishing; indeed who comes here to increase the other, paper or digital? Because the notebook physically hides the digital, while digital offers new possibilities for paper. Moreover the two mediums were thought simultaneously, why talk of increase? Perhaps to better understand this new type of object and visualize the stakes.

Source:https://sites.grenadine.uqam.ca/sites/nt2/en/elo2018/schedule/1459/O%C3…

Description (in original language)

La question ontologique de l’essence du cinéma, incarnée par Bazin et son « qu’est-ce que le cinéma » se déplace aujourd’hui, sous la poussée de nouvelles formes de consommation des images vers une « relocation » du cinéma, résumée dans cette question : « où est le cinéma ? ». Après avoir investigué le code, le génératif, l’algorithme, le flux, nous pourrions nous demander aussi où est la littérature numérique, avec une sortie de l’écran rendant parfois caduque cette idée d’une littérature principalement électronique. La velléité d“écrire en numérique” semble abandonner les tests et essais de littérature numérique par crainte de voir les pistes possibles de nouvelles formes d’écriture se fermer une à une. Peut-on poser que cette littérature électronique reprenne pied dans un livre en bonne et due forme comme des pierres dans un jardin numérique ouvert ? C’est ce que nous avons questionné avec nos étudiants en design multimédia du DSAA de Boulogne (France) dont quatre créations sont détaillées ci-dessous dans cette proposition. Il semblerait que ces étudiants, ces digital natives - cette étrange génération dont l’absence d’histoire avec “l’apparaître” numérique leur permet de remodeler les champs d’intervention et de limite de l’écriture numérique - reprennent la question là où elle semble finalement la plus évidente : dans le livre, en explorant une écriture numérique qui se graphie, se visualise, se sonorise et devient une littérature de l’objet sensitif. Les quatre créations de ces étudiants sont donc des objets exploratoires travaillant les délimitations du digital dans une logique de l’ajout. Déplacée hors du cadre central, l’écriture numérique s’incarne de façon diffuse, parfois hors de l’écran, dans un glissement de sens sur la matérialité du numérique, à la manière de nos usages quotidiens, mobiles et incarnés dans l’objet. Prenons l’hypothèse que le livre augmenté permet de grimper les premières marches d’une localisation à venir. C’est dans cette idée que nous vous proposons d’exposer ces quatre livres réalisés par des étudiants du DSAA, aujourd’hui membres de l’atelier de recherche et de création 3i (Images, Innovation, Interactivité), dont voici le descriptif plus détaillé. 1. Insignifiés du bruit Héloïse d’Almeida Insignifiés du Bruit est un projet trans media conçu autour de l’exploration plastique et sonore du domaine des bruits. Regroupés en trois catégories, les bruits utilisés dans cette édition augmentée appartiennent aux champs organiques, mécaniques et électroniques. Des bruits des deux premiers champs, organique et mécanique, ont été investis pour produire le contenu visuel du livre dans sa forme imprimée, que le lecteur manipule. Les bruits du troisième champ - le champ électronique - ont été investis comme matière de l’augmentation de l’édition. Aboutissant ainsi à la création d’un livre composé de deux livrets et d’une augmentation sonore sur smartphone. Grâce à un système de reconnaissance d’image, le lecteur est en mesure de scanner les pages des deux livrets simultanément, et de retranscrire les traductions visuelles en une nouvelle combinaison sonore. Ainsi les sons produits sont véritablement les signatures sonores des bruits graphiques. 2. Rode B-6-9 / Anne Deurgroënd Lucie Plançon Rode B-6-9, Anne Deurgroënd (= Road B-6-9, Underground) est une édition augmentée sur le thème de la musique au quotidien, et plus précisément celle perceptible dans les transports. C’est un appel au voyage, une poésie abstraite et décalée, composée de photographies et de textes interprétés par trois acteurs. Dans le tempo unique du train, existe une multitude de rythmes, singuliers et divers, qui peuvent peu à peu paraître absurdes. Cet environnement, parfaitement réaliste à l’origine, laisse place aux chimères de chacun. Le texte écrit, comme le trajet, se découpe en trois parties : B-6-9. Elles deviennent des formes de chansons interprétées par trois acteurs différents. Chacun y ajoute sa sensibilité et sa vision du parcours jusqu’à apporter un ton radicalement abstrait et absurde. Ces interprétations sonores sont lisibles grâce à des puces NFC qu’il faut scanner avec son smartphone. Dans toute son abstractivité, ce projet mélange les nouvelles et anciennes technologies dans un concept éditorial comprenant 18 musiques, 12 posters, 3 pochettes et 3 voix réunis dans au sein d’1 même coffret. 3. L’invasion Marjorie Terral L’invasion est une édition proposant du contenu en réalité augmentée qui mêle textes, photos et graphismes. L’édition évoque une idée de sons qui se répètent, s'accumulent à différents endroits, à différents moments de la journée, avec une recherche de représentation visuelle pour les ambiances sonores. Les prises de vue, plus évocatrices que figuratives, sont là pour stimuler l’imaginaire, tout comme les motifs qui se déploient sur l’ensemble de l’ouvrage comme une tentative de traduction sonore subjective. Ces motifs, présents dans l'édition imprimée, s'animent ensuite dans l'augmentation faisant prendre vie à chaque photographie, transformant chaque page en tableau animé et sonore. Pour chaque son entendu correspond un motif qui vient envahir la photo, ces sons viennent s'accumuler, se répéter, s'inscrire dans un rythme de plus en plus chargé pour transcrire cette sensation d'invasion, en saturant, brouillant la photo, en révélant la présence de ces bruits d'une nouvelle façon. 4. Le monde d’HylaeGuillaume Verguin “Le monde d’Hylae vous emmène dans un univers étrange, inspiré d’une culture lointaine … À vous de capturer un talisman dans ce monde étonnant où ne vivent qu’une faune et une flore sauvage et de dévoiler ce que cache cette pierre magique …” Sur cette trame narrative pour la jeunesse, Le monde d’Hylae expérimente différents dispositifs numériques : - une installation interactive - une édition augmentée, entre papier et numérique - une expérience VR Tous trois remettent en question le sens du toucher, explorant une matérialité numérique. L’édition positionne un cahier papier découpé sur une tablette et propose de nouvelles interactions. Le lecteur lit l’histoire imprimée sur le cahier découpé, tourne les pages et aperçoit l’écran de la tablette à travers les formes découpées dans le cahier ; créant ainsi de nouvelles interactions. Chaque double page offre alors une interaction spécifique pour le spectateur. Peut-on parler ici d’édition interactive; en effet qui vient ici augmenter l’autre, papier ou numérique? Car le cahier cache physiquement le numérique, alors que le numérique offre de nouveaux possibles au papier. De plus les deux médiums ont été pensé simultanément, pourquoi parler d’augmentation? Peut-être pour mieux comprendre ce nouveau type d’objet et visualiser les enjeux.

Source:https://sites.grenadine.uqam.ca/sites/nt2/en/elo2018/schedule/1459/O%C3…

Description in original language
Description (in English)

Sacrosanct is a parser-based work of interactive fiction set in the bare halls of MIT.Originally written for Nick Montfort’s class on Interactive Narratives, it invites players on a surreal, metaleptic quest to hand in an overdue final project. Taking playful liberties with conventional notions of narrator, narratee, and narrative, Sacrosanct explores the theme of transgression in many forms.

Content type
Year
Language
Publication Type
License
All Rights reserved
Record Status
Description (in English)

Sensitive to the idea of ​​reconnecting children to their environment using a paper book, the designers wanted to make the experience more stimulating through technology. "I first wanted to encourage my daughter to enjoy slow, contemplative reading, because I observed that this kind of reading made her able to invent stories with a richer imagination," says Jonathan Belisle, author of Wuxia the Fox and partner at SAGA. This children's story, uniting the paper book with an iPad application, features speech recognition technology that is animated by reading aloud and triggers musical patterns, sound effects and interactive scenes.

Screen shots
Image
Screen shot_1
Multimedia
Remote video URL
Description (in English)

It Must Have Been Dark By Then' is a book and audio experience that uses a mixture of evocative music, narration and field recording to bring you stories of changing environments, from the swamplands of Louisiana, to empty Latvian villages and the edge of the Tunisian Sahara. Unlike many audio guides, there is no preset route, the software builds a unique map for each person’s experience. It is up to you to choose your own path through the city, connecting the remote to the immediate, the precious to the disappearing. 

Source: https://sites.grenadine.uqam.ca/sites/nt2/en/elo2018/schedule/1465/It+M…

Screen shots
Image
Multimedia
Remote video URL
Description (in English)

"StoryFace" is a digital fiction based on the capture and recognition of facial emotions.

The user logs onto a dating website. He/she is asked to display, in front of the webcam, the emotion that seems to characterize him/her the best. After this the website proposes profiles of partners. The user can choose one and exchange with a fictional partner. The user is now expected to focus on the content of messages. However, the user's facial expressions continue to be tracked and analyzed… 

What is highlighted here is the tendency of emotion recognition devices to normalize emotions. Which emotion does the device expect? We go from the measurement of emotions to the standardization of emotions. 

StoryFace was re-published in The New River in 2018.

Multimedia
Remote video URL
Description (in English)

“You’re On” explores the relationship and particularly the gap between the types of expressions we use and understand and what technology can "read". Technology has rapidly begun to both produce human-like performances, including speech synthesis in products such as Alexa, synthetic artwork based on deep neural networks as well as reproductions of human performers trained on recorded videos.

In this work, the interactor sits in front of a simple screen and is provided instructions and interacts with the work entirely through reading the text on the screen and expressing emotions. It takes advantage of the facial recognition toolkit "OpenFace: open source facial behavior analysis toolkit" which analyzes facial action units in real time and Google's text to speech service. These are used as input into an interactive narrative built using the open source interactive narrative scripting language "Ink" by Inkle Studios. The story and role were inspired by Neal Stephenson's novel The Diamond Age, or, A Young Lady's Illustrated Primer and Terminal Time by Michael Mateas, Steffi Domike, and Paul Vanouse.