While discussion of the relationship of image and word has been prominent in the discourses surrounding new media writing, the role of sound is rarely addressed in this context, even though words are sounds and sounds are a major component of multimedia. This paper explores possibilities for new theoretical frameworks in this area, drawing on musico-literary discourse and cross-cultural theory, and using ideas about semiotic and cultural exchange as a basis. It argues that words and music in new media writing create emergent structures and meanings that can facilate ideas to do with boundary crossing, transnationalism and cross-cultural exchange.
The paper will examine the different types of sound in new media writing from voicescapes to soundscapes to musical composition. Building on my previous work on affective intensities in new media writing (Smith 2007 ; Smith 2009), and the manipulation of the voice to create cultural effects such as sonic cross-dressing (Smith 1999), I will discuss the ways in which sound plays a distinctive role in new media writing. I will draw up a typology of different kinds of conjunctions between sound and words in this area (e.g. parallelism, co-ordination, semiotic exchange, algorithmic synaesthesia and heterogeneity). I will also, constructing the term musico-literary miscegenation, explore the cultural effects of these word-sound blends, and how they can interrogate ideas about gender or ethnic identity.
The paper will refer to word and sound relationships in classic electronic literature works such as John Cayley’s Translation, Young Hae Chang Heavy industries Operation Nukorea and MD Coverley’s Afterimage. It will also discuss the exploration of different types of synergies between word and sound by the Australian sound and multimedia group austraLYSIS — of which I am a member. In particular it will feature some of my own work with composer Roger Dean, and our recent collaborations with video artist Will Luers.
The paper will take the form of a talk and powerpoint presentation.
(Author's introduction)