Eververse is a project which synthesises perspectives from disciplines in the humanities and sciences to develop critical and creative explorations of poetry and poetic identity in the digital age. Eververse sends biometric data from a fitness tracking device worn by the poet to its custom-built poetry generator. This generator utilises NLG techniques to output poetic text published in real time, and 24/7, on the Eververse website.
The Eververse application consists of three main modules. The first module interfaces with the Fitbit device and its data through its Application Programming Interface (API). The activity data of the poet wearing the device is then sent, in JSON form to the NLG module referred to as the 'generator.’ This generator carries out a number of steps in order to generate and return a poetic couplet based on a conceptual model of states based on the activity information contained within the passed JSON data. The number of words and the frequency of the generated couplets correlate with the heart rate of the poet, whereas the textual content of the couplet is generated from the input corpus which is fed to the generator. The input corpus currently comprises fifty nine poems on the topic of the body; all are previously published and none is composed by the Eververse poet; a separate, nocturnal corpus is deployed when sleep data is passed to the generator. In order to disassemble and reassemble the corpora for publication in EverVerse, they are arranged in a reverse ngram matrix and further shaped into a frequency lookup table by Poesy, a Markov Model-based Natural Language Poetry Generator. The lookup table is used to create verse lines and a python library is deployed to rhyme the verses. In short, our method takes a language model approach similar to Barbieri, et al. although we do exploit some semantics, specifically alignment of couplets with fitbit activity states. Future work will involve experimenting with exploiting language resources such as WordNet and SentiWordNet similar to previous work by Tobing and Oliveira.
The generator is written mainly in the Python programming language using the micro web framework, Flask. It consists of a web interface to display the generated poetry and an administrator interface that is used to define heart rate parameters for different zones and to determine the form and content of the verse that corresponds to these zones.
The public user interface created to display the generated poetry relies heavily on a number of Open Source JavaScript libraries. These libraries enable display of the generated text (Handlebars.js, Textillate.js), the retrieval of data from the web application’s API and user interface animations (jQuery), and the creation of generative background images (p5.js). The dynamic background images are created in realtime, and utilise the activity data as an input to affect their form and colour, representing a visual correlate to the generated poetry.
Multiple versions of this interface were created for deployment on the web, in a live performance environment, and for display in a standalone exhibition setting. Each interface was adapted to take into account the context in which it would be experienced, for example, differences in how, or if, user interaction was required, and addressing the differing requirements for text size, line spacing, and overall page layouts.
From a theoretical framework of cognitive semiotics, emotion history, and image science, Schiller’s scholarship focuses on the media genealogies of physiognomy, the science of facial expression, and digital biometrics. He analyzes how artists and scientists use media to interpret from the outside physiological behavior of the face the psychological phenomena inside an individual; the visual rhetoric of these methodologies; and how face images can inform display rules, social scripting, and truth claims for emotion in society. Schiller is also an internationally exhibited artist. Across the media genealogies of physiognomic science in the German-speaking countries, artists researching at the intersection of art, science, and technology have from the “form” [Greek physis] of the face “interpreted” [gnṓmōn] characteristics such as temperament and emotion. To historicize how media is the method for this grammatical view of the facial language–long called an art and today known as a science–I problematize the connections and conflictions between ‘face-reading’ in the tetradic humoural ‘theory of everything’ of the Northern Renaissance, and automatic facial expression analysis in the digital information society of the Cognitive Revolution. Using a new online Media Art Research Thesaurus, I probe the intermedial prestige of Albrecht Dürer’s The Four Apostles (1526) and its references to physiognomic ‘books on the face,’ by comparatively analyzing the semantic bridges that link the transposition of this artwork in Johann Nepomuk Strixner’s master study (1815), and combination into Julius von Bismarck’s Public Face (2008).
(Source: Author's Abstract ICDMT 2016)