Creative Artificial Intelligence
| Communication Acoustics and Aural
Architecture Research Laboratory
- A Creative Artificially-Intuitive and Reasoning
Agent in the Context of Ensemble Music Improvisation
was a project funded by the National Science Foundation within the CreativeIT program (NSF 1002851). The goal of the CAIRA project was to develop artificially intelligent system that is capable of improvising live music along with human performers. We aimed to achieve this goal using a multi-layer structure simulating core functions of the auditory pathway, higher cognitive functions as well as an efferent pathway to produce actual music. The main layers of the CAIRA structure were to include:
Jonas Braasch (PI), Selmer Brinsjord (Co-PI), Pauline Oliveros (Co-PI), Doug Van Nort (Post-Doc)
Nikhil Deshpande, Simon Ellis, Colin Kuebler, Anthony Parks, Naveen Sundar G., M. Torben Pastore, Joe Valerio
Jonas Braasch: Soprano
Doug Van Nort: granular-feedback expanded instrument system (GREIS)
CAIRA: realtime music improvisation agent - using audio material from Pauline Oliveros, V-Accordion, for this recording
A Robust Distributed Intelligent System for Telematic Music Applications
was a project funded by the National Science Foundation within the CreativeIT program. Within this project, we are interested to develop intelligent agents for improvisational music collaborations over the internet.
Jonas Braasch, Pauline Oliveros, Doug Van Nort
Jonas Braasch, Pauline Oliveros, Doug Van Nort
(photo: Jonathan Chen)
Complex communication for co-located performers within telepresence applications across networks is still impaired compared performers sharing one physical location. This impairment must be significantly reduced to allow the broader community to participate in complex communication scenarios. To achieve this goal, an avatar in the form of a musical conductor with forms of artificial intelligence will coordinate between co-located musicians. Improvised Contemporary Live Music of a larger ensemble, serving as a test bed, is arguably one of the most complex scenarios one could think of, because it requires engaged communication between individuals within a multiple-source sound field that also has to be considered as a whole. The results are expected to inspire solutions for other communication tasks.
The avatar system will actively coordinate co-located improvisation ensembles in a creative way. To achieve this goal, Computational Auditory Scene Analysis (CASA) systems, to allow robust feature recognition, and Evolutionary algorithms, for the creative component, will be combined, to form the first model of its kind. The research results are expected to be significant by themselves and are not bound to telematic applications. With regard to the latter, the proposed system will have a clear advantage over a human musician/conductor, while intelligent algorithms are clearly lacking behind human performance in most other applications, especially when it comes to creativity.
(Pauline Oliveros, Jonas Braasch & Doug van Nort) is
a 2009 trio
with a post-genre approach well inclined to the Deep Listening practice.
The trio work is based on the
development of the new IT tools which are developed
within their CreativeIT grant from the National Science
Foundation. The band derives its
name from the thermodynamical point in the phase diagram
where all three phases of water exist. Figuratively,
this is where the trio operates
exploring musical spaces and boundary conditions where
contrasting ideas and streams can co-exist, while
expanding the vocabulary of musical instruments
acoustically (Braasch on soprano saxophone) and
electronically (Oliveros, digital accordion and
Expanded Instrument System, EIS, Doug van Nort on laptop
and GREIS). For many decades
Pauline Oliveros has been actively expanding the voice
of her main instrument, the accordion. Given the limited
natural possibilities of this instrument with respect to
sound (fixed tuning, no pitch bends, narrow variety in
overtone spectrum), Oliveros has begun half a century
ago to alter the sound of her instrument using tape
delays and other electronic devices. Van Nort’s work is
based in digital signal processing, transforming
Oliveros’, Braasch’s an his own sounds using Granular
Synthesis, psychoacoustically-motivated sound analysis
tools and Genetic Algorithms to explore new musical
textures and timbres.
Here is another excerpt from our concert at the Emily Harvey Foundation (March 12, 2009, New York City), in which a number of new developments from the CreativeIT project were presented:
Here is a short excerpt from our telematic work
Tintinnabulate & related Courses
Students are able to participate in the project through seminars that are linked to the project. The following excerpt shows a concert at EMPAC and Second Life with the RPI ensemble Tintinnabulate.
Tintinnabulate and our CreativeIT project play an important role in our ongoing seminar on Music Composition, Improvisation and Telematic Music. The current seminar is:
"Experimental Telepresence" (ARTS 4962/6962, instructor: Pauline Oliveros, co-instructors: Jonas Braasch, Doug Van Nort) Spring, 2010
and these were the related seminars in the past:
"Composition, Improvisation & Performance" (ARTS 4964/6964, instructor: Pauline Oliveros, co-instructors: Jonas Braasch, Doug Van Nort), Fall 2009/10.
"Experimental Telepresence" (ARTS-4962/6962, 4 credits, instructor: Pauline Oliveros, co-instructor: Jonas Braasch), Spring 2009.
"Mixed Reality Seminar" (ARTS-4961/6961, instructor: Pauline Oliveros, co-instructor: Jonas Braasch), Fall 2008/09.
Here is a short clip of Tintinnabulate in a Concert at EMPAC and Second Life: