The Theater of Pattern Formation
The Theater of Pattern Formation
The Theater of Pattern Formation is nonstationary; a work in progress, largely irreversible. What follows is the view from 9 May 2003, when talking to funders ...
0. Sound samples.
I. Introduction
The project will bring the most recent concepts of complex systems to the public via a new class of all-digital theaters. To do this, we propose a strongly educational approach that is oriented toward fundamental scientific research on perceiving patterns in sensory-immersive environments and that, via a companion ITR proposal, will develop the necessary acoustic and image computing and display technologies.
II. The Theater of Pattern Formation
We will develop educational programs for exploring the patterns generated by large-scale, complex, nonlinear systems. The results will be brought to the public using sensory-immersive environments, with a particular focus on the relatively new technology of all-digital dome theaters. There are two sets of tools developed by the companion ITR proposal that we will use to produce the programs. These consist of (i) real-time sound and image synthesis algorithms optimized for clustered multiprocessors that can be used for on-line investigation, as well as interactive theater performances, and (ii) a small-scale, lost-cost multi-channel acoustic and -image presentation system that uses off-the-shelf components for research and development of wide-screen program material. The new technology applications, that will arise in the ITR project and in the program production funded by the present proposal, will be complemented by visual and aural psychophysics studies designed to understand directly how audiences perceive and discover new spatial and temporal patterns that are presented via sensory-immersive environments. This will form an important part, that we believe is new and novel, of our evaluation of the program and performance content.
We explain the basic historical and educational motivations behind the development of the Theater of Pattern Formation program material in the following sections.
II.A Dynamical Pattern Formation in Acoustic Feedback Systems
II.B Spatiotemporal Pattern Formation in Image Feedback Systems
II.C Integrating Acoustic and Visual Patterns: Psychophysics of Sensory-Immersive Environments
The scientific need to process and comprehend large data sets has converged with increasingly powerful and available digital media technologies [40,41]. This has driven the expansion of the fields of scientific visualization and sonification at a time when it has also become evident that purely mathematical analysis is not the only way for comprehending complex data [42,43]. This is particularly true when attempting to translate mathematical and scientific breakthroughs to a general audience. The techniques of computer-based visualization and sonification often optimize the highly evolved capacity of our perceptual systems to recognize patterns and comprehend relationships. While preliminary research suggests that different sensory modalities are better at perceiving certain types of information [44], further research is necessary to identify what data features are most apprehensible to particular sense modalities [45,46]. Multi-modal interactions, such as those that occur between simultaneous visual and auditory displays, are even more mysterious (and potentially useful) [47,48]. While the challenge of cross-modal interaction has been a major issue of the contemporary electronic and media arts, it has received very little scientific investigation. The question of how visual events can change the perception of a sound and vice versa remains an open and salient research frontier [49,50].
III. Program Development
The Theater of Pattern Formation will allow the public to explore these frontier questions by bridging sound and image. The result is a large-scale feedback system in which pattern information flows between both acoustic and visual modalities.
III.A Audio Input to Pattern Formation Engine
III.B Pattern Input to Acoustic Feedback System
IV. Program Dissemination and Evaluation
In addition to its research focus, the Theater of Pattern Formation will provide, to both lay audiences and the scientific community generally, an educational entry into the world of pattern formation, complexity, and chaos, one that celebrates recent major innovations in our understanding of the origins of order and disorder in nature and in mathematics. While the project will consist of a variety of mathematically driven visualizations-sonifications, these will go beyond mere didactic expositions of scientific principles. Our agenda is to create an integrated perceptual environment that focuses on leading-edge scientific results and builds an experiential understanding of fundamental principles of natural pattern formation and their mathematical descriptions.
To develop successful program material, we will consider a number of related issues, including sound localization, visual perception of novel objects and structures in space and in time, a subject's tendency to perceive patterns when there are none and to miss them when there, and so on.
Another important aspect of the Theater of Pattern Formation will be, as program developers, our own discovery of novel and interesting patterns within the vast search spaces of pattern forming systems. Similarly, there is the question of whether or not viewers will detect the patterns and structures when presented in an immersive environment. The question common to both program development and audience experience is a fundamental one: What is a pattern and can we quantitatively measure the degree of structure in temporal sequences of images and sounds? To address these questions we will adapt recent work on quantitative measures of structural complexity and randomness [35-37]). We will use the measures developed there to aid in our automatically exploring and discovering engaging patterns and behaviors. These measures will also be used to monitor audience reaction to the programs we develop. Again, we believe these efforts to determine the impact of the program material will introduce novel approaches to evaluating educational impact.
Bibliography: Coming.
Performances & presentations:
The Theater of Pattern Formation
1 December 2003, CUNY Graduate Center
The Making Of `The Theater of Pattern Formation'
1 December 2003, Working Session, CUNY Graduate Center
Apollo by Day, Dionysus by Night, Theme Camp
Burning Man, August 2004
CEAIT Festival, Center for Experiments in Art, Information and Technology
California Institute of the Arts
Roy and Edna Disney/CalArts Theater, Los Angeles, 28 January 2006
Arts Technology Center, University of New Mexico
ToPF In Sensory Immersive Environments
Chaos All The Way Down:
“Chaos All The Way Down”, Leonardo (2009) in press. Download PDF.
Preshow to Collapse (sudden falling down)
Premiered Mondavi Center Studio Theatre, 25-28 October & 1-4 November 2007
James P. Crutchfield & David D. Dunn
Art & Science Laboratory
artscilab.com