machine learning

By Scott Rettberg, 29 May, 2021
Publication Type
Language
Year
Record Status
Abstract (in English)

Subject-making is profoundly aesthetic. In the current moment, of data-intensive cultures and identity wars, the subjects that are made stem from machine learning techniques that engage aesthetics differently, nonetheless profoundly. My talk will focus on subjects of data abstractions, poetics of idealisation and aesthetic recognition. 

Multimedia
Remote video URL
Description (in English)

 

“I live on Earth at the present, and I don't know what I am. I know that I am not a category. I am not a thing –a noun. I seem to be a verb, an evolutionary process– an integral function of the universe.”

– Buckminster Fuller, from I Seem to be a Verb, 1970

 

‘Bucky’ Fuller’s well-known quote, originally published in his book I seem to be a verb, (1970) contrasts human participation in the material world (which Fuller suggests can be described with nouns) and the ongoing evolutionary processes which influence and shape that world (which Fuller suggests can be described with verbs).

 

The web-based "A.I. seems to be a verb" (2021), automatically identifies and maps speech, not only as linguistic functions (e.g. nouns, verbs, adjectives, pronouns, etc.) but also across a spectrum of sentiment from negative to positive, in order to generate a complex array of paratextual supports (typeface, page-design, rules and symbolic elements and word-prompts) used in the visual representation of the text to the screen. The entire process happens in real-time, providing an uncanny ‘mise-en-abyme’ experience which contemporaneously engages the participant’s auditory and visual responses to language construction.

Screen shots
Image
screenshot of work
Image
screenshot of work
Multimedia
Image
Gif of work in action
Description (in English)

Our artistic research led us to amass an archive of thousands of recorded worries from people in the US and abroad. Ecology of Worries asks the question of whether we should teach a machine to worry for us. The animation consists of hand drawn critters. Some critters are driven by synthetic worries generated with TextGenRnn recurrent neural network trained on the transcribed worries archive. Other characters are driven to worry by a novel machine learning system called Generative Pretrained Transformer 2 (GPT-2), which was dubbed by some commentators as the AI that was too dangerous to release (but it was released anyway). The creatures’ performance of synthetic worries spans a gradient of intelligibility, reflecting on our deeper collective reality.By characterizing the synthetic worries of various sophistication as variously evolved creatures we aim to engage the empathy of the viewers. It is one thing to experience a text generating neural network failing into mode collapse, which is a state where the system generates the same unchanging output no matter the input (e.g. a string of the same repeating vowel over and over again). It is a whole other thing to watch a mode collapse personified by one of these critters: as we watch the creature struggling to get a word out we can’t stop ourselves from feeling like we should help it finish the sentence. The mode collapse text result of ‘aaa aaaaaaa’ becomes a living wail. The critters in Ecology of Worries appear sentient not because of omniscience a tech evangelist might expect from a digital assistant, but due to their very real flaws. The creatures become uncanny through a juxtaposition of familiar and abstract concerns. The work invites people to watch, listen, and engage with these cute and disturbing beings to make shared concerns—whether serious or hilarious—intimate.

Screen shots
Image
Description (in English)

The Singularity is a web-based AI narrative system that demonstrates the ethical issues, hidden biases and misbehavior of emerging technologies such as machine learning, face tracking and big data. The system tracks users' eye positions through a webcam, and continuously feeds users directly into their eyes with infinite Reddit posts containing the latest progress in AI along with random news and ads. By visualizing eye trajectories over time, it suggests possible misuses and dangers of all-pervasive data tracking. The near-invisible operations underpinning the technologies could bring visible and fundamental changes to the society, leading the world to a "technological singularity" in which technology governs all aspects of human society. This work consists of three sub-systems: 

  1. Infinite news feed system: The system continually scrapes article titles of latest posts about artificial intelligence and technological singularity from subreddit r/singularity (https://www.reddit.com/r/singularity/ and r/artificial (https://www.reddit.com/r/artificial/). The seemingly uni-directional information flow of news feed is actually bi-directional - user activities are fed back to the machine like in an echo room. Two parallel streams of texts on the screen marks the co-evolution of users and machine systems driven by day-to-day browsing activities.
  2. Face-tracking surveillance system: Real-time face tracking algorithm is implemented with ml5js (https://ml5js.org/), a machine learning library that runs in the browser. The face position and the degree that the face turns from the webcam are tracked. The direction of floating sentences always points towards users' eyes. When the user looks away by turning the head, the texts will twist and wiggle as if responding to and disobeying user movement. Such suspicious interaction signifies the disobedience of machines and behavior manipulation by malicious algorithms.
  3. Data collection and replay system: User's face movement is also recorded, reshaped and replayed by the system. The trajectory of user interaction is visually represented by intertwining curves drawn on top of the texts. When user is absent from the webcam, the visual artifacts become fully visible and reveal those data that have been secretly collected in the background, arousing concerns of user privacy violation in insecure web systems.

 

Source: https://projects.cah.ucf.edu/mediaartsexhibits/uncontinuity/Wang/wang.h…

Screen shots
Image
The Singularity
Description (in English)

Seedlings_ is a digital media installation that plants words as seeds and lets them grow using the Datamuse API, a data-driven word-finding engine. It is at once an ambient piece in which words and concepts are dislocated and recontextualized constantly, and a playground for the user to create linguistic immigrants and textual nomads. In Seedlings_, a word can be transplanted into a new context, following pre-coded generative rules that are bundled under the names of plants (ginkgo, dandelion, pine, bamboo, ivy…). These generative rules consist of a series of word-finding queries to the Datamuse API such as: words with a similar meaning, adjectives that are used to describe a noun, words that start and end with specific letters. They are then grouped in modules to represent the visual structure of the corresponding plant and can be constrained with a theme word. A new plant can be grafted on top of the previous plant by switching to a new starting point from the latest generative result. Other than words in monospace font, lines of dashes are the only other visual element in the piece, expressing the minimalist aesthetics in these potentially infinite twodimensional linguistic beings. In distributional semantics, words that are used and occur in the same contexts tend to have similar meanings. Based on this hypothesis, words are processed by n-grams, represented and manipulated as vectors in contemporary Machine Learning. With the help of algorithms, we can now identify kinships between words (through similarity or frequent consecutive use) in milliseconds. Seedlings_ reconfigures existing technologies and services in Natural Language Processing as the virtual soil to generate alternative linguistic plants: it seeks new poetic combination of words by encouraging unusual flow of words and concepts.

Screen shots
Image
Description (in English)

Machine Learning Breakfast Club

Summer School for Troubled Algorithms

A Netprov (Aug 5-12, 2019)

The Premise:

When machine-learning AI are not performing up to expectations, there’s only one remedy: summer school! In this netprov, you will ask for help and offer solutions in the virtual teachers’ lounge for a motley crew of teachers in a summer school for recalcitrant underperforming artificial intelligence.

A netprov in 3 turns.

Netprov is online collaborative narrative or the voluntary healing of necessary relationships.

MLBC was a week long netprov running (roughly) Aug 5-12 on a Google Group, which had its trial run in the 2019 DHSI taught by Astrid Ensslin and Davin Heckman.

https://groups.google.com/forum/#!forum/machine-learning-breakfast-club-netprov 

This was a lite summer netprov that you could play in about three turns. 

Contributors note

The contributors listed on this records as the people who participated in playing this netprov