A partnership is developing lhai offers a special complementarity of relationship between musical and visual design With this new partnership we are learnin to compose a unique counterpoint by means of computer algorithms. I would define the algorithms that we use as the routines of computer instructions that generate co ordinated color design with musk through solid state opera tions. as a real-time event. These algorithms can be composed interactively on many of the fast personal com puters (PCs). Only recentb have computers acquired a musk and graphics-generating capability, wbkh often can lie installed in one computer instrument. This has become what we right fully may call the artist’s first unisersal machine: an instru ment for composing time-based audio and visual modalities. Founded upon my own vision of just such a ‘universal machine’ long before the first computers, my study ofcolor in-inotion began as a search for various germinal aesthetic piini ipals directed toward developing a fine an that might employ that instrument.
When I later gained access to computers, ms explorations of what I began to rail ‘digital harmony’ gradually revealed to me a world of pure fluid graphics I uncovered aesthetic concepts that suggested how differential functions within a wide variety of geometric algorithms generate order disordcr graphics in a world of abstract design in motion. A tspiral ordcr disordcr algo rithm might distribute points in a field that would appear to be scattered at random, while moments later (some computer displavs later), the points might evolve bs differ ential progression into an obvious pattern of order, form- ing. perhaps, an elegant rose curve. Order that resolves to disorder, or disorder that resolves to order and repose, is like a musical resolution. The similarities (hat these tem poral development* share with music struck me long ago (Color I’latc В No I). Thus my concepts for a visual har mony evolved. Computer algorithms, plotted on mam one frame displavs and copied to film, helped me clarify my ideas of orderly development. I tested and demonstrated my ideaswith a succession of complete films.
Eventually. I tame to understand how an arithmetic of resonance and ratio actually embodies the architecture of musk. The dilferential arithmetic of musicits scales, in tervals and chords—could complement a graphic differen tial geometry. Visual patterns, derived from simple periodic geometry, produce order-disorder resonances in evolving actions that sometimes mimic the consonances and dis sonanccy the tension») dynam ics and the universal emotive powers of musical figuration, rhythm and harmony. These were summary conclusions I was able to draw from com puter filmmaking. Through mv essays and film compositions. I derived broad hypotheses about the pervasive pcriodicitv of all temporal artvisual or musicalfor mv book.
Shaping a New Foundation
Note durations, the tempo of music and the timing of the sounds and silence of musical structure are overwhelmingly determined by the demands of harmony (2). Vet f hail begun to suspect that another determinant giving shape to time would soon become known. There was to be a visual structuring agent, such as the ordcr-disordcr patterns, in the audio-visual worlds of cinema and video that would shape the lime required for an action to lake place in space. Indeed, for an action to piocred from here to there (or to proceed from order to disorder to order), time must pass. Whether quick or slow, action, as well as hannonv, deter mines much of the shape of my own audio-visual work today. Action itself has an impact on emotions.
Fluid, orderly action generates or resolves tensions much in the manner that orderly sequences of resonant tonal harmonv have an impait on emotion and feeling Dancers know this. The conductor’s gestures implv something ol this sort. Creative work with computers soon demonstrated that the smallest basic, quantifiable units of construction for my computer art are the pixel point of color and the pure audio sine wave. I learned that the basic musical unit is no longer the single acoustic tone as produced bv any of the wide family of acoustic musical instruments or the human voice. Natural forces construc t tones out of sine waves. How ever. a new world of non-acoustic (i.e electronically generated and pat terned) musical tones, tonal complexes and textures also flows from complex or simple sine-wave constructs gene rated bv computer algorithms.
Nor is llie basic visual element a gestalt or visual symbol like the letters of the al phabet I found that with pixels, one constructs a liquid plasma of graphic architecture in an unprecedented tem poral domain formerly belonging ex clusively to musical architecture. Sine wave and pixel can he used to generate resonant structures, with hoth element* contributing essential constructive parts to a greater whole. The mathematical processes driving and shaping these rev onant visual constructs, as well as the intnguingly similar mathematical pro cesses shaping tonal constructs, seemed to provide the foundation for what was to become a truly audio-visual art.
Synthesis and Genesis
Assuming such a basic foundation, we may compare two terms belonging to electronic music jatgon: synthesis and genesis. My studies suggested that when composing music bv computer one should stress basic algorithmic or generative processes. The elemental* pixel and sine wave can be con structed fioin the ground up, so to speak, into dynamic visual patterns as well as melodic patterns of unique new timbres and actions, each by its own respective assorted algorithms in vented and applied by the composer. Heretofore, we viewed each of the classical acoustic musical instruments, figuratively, as its own ‘algorithm’ producing characteristic. Itardb alter able timbre. Now. as one explores the diversity of possible computer al gorithms. one finds a veritable Ein steinian conceptual universe of tonal relativity—an unprecedented universe of relative pitch, relative time and rela tive timbre Here is a fresh domain for generating continuously variable tonal pattern and texture.
I would distinguish genesis from syn- thesis with the following, perhaps too simple, generalization: usually, com posing with rpttiusis methods is ac complished on real-time kcsboards constructed as acoustic wavc-fomt svmhe sizers or samplers. These procedures of synthesis naturally induce imitations of the traditional fixed scale», timbres and tempos, etc. Automatically, composers gravitate to methods of the live per fonuance, all components being made to facilitate an ensemble performance. My own ongoing experience con firms that computer sine wave and pixel offer an exciting potential for audio visual creativity. This potential should not l«e misused. It need not be bent out of shape in older to synthesie inti late artworks associated with the art gallcrv. the concert hall or cinema. Audio-video works like mine belong elsewhere. Given that dvnamic com puter arts are still new. no doubt ittanv talents will soon carve for themselves new places in that expanding world. Outlets to the public will be on video discs and of course on television, which needs this audio-visual form quite as much as the cable television station MTS’ (Musk Television) toriav needs lively computer graphics composed and interwoven with new computer music.
This end. one hopes to employ very smart expert-svstems technology more creatively than merely to copv. imitate or plagiarize the piano or to simulate or synthesize the prcsent-dav range (from pop to chamber music) of highly devrloed improvisatinnal techniques among musical ensembles that other-wise continue to perform best simply without smart electronics. Electronic music concepts of genesis clarified mv ideas about pattern poten tials for music with art. My earliest film making had demonstrated that i lie traditions and ensembles of music, which require fixed tuning, abundant tempo markings and skilled instru mentalists. could be supplanted by inventing and using new musical algo rithms that I was free to associate with graphic algorithms, without restric tions or constraints except those that I imposed upon myself. This led to naming my new methodology ‘digital harmony’. I believed there could be a harmonic foundation for audio-visual musk within the newly revealed world of digtul music and digital graphics.
Digital Harmony
located outside instrumental and vocal traditions while retaining a valid har monic foundation, digital harmony would provide a new medium for a new kind of composer artist, like the crea tive processes of the painter or sculptor, digital harmony is direct Mv present composing program, for example, allows direct action with instant tccd ba similar tо Jac kson Pollo ac lion painting canvas on the llnor. which allows ac non-reaction as a kind of per formance (3) Intimate interaction with temporal materials, freely involving re tuneable random chance, is a fruitful mother of invention. Lively creative interactions here distinguish a com poser’s relationship to material. The resultant works arc* a sum of an artist’s countless с lioices. revisions and chance discoveries. These crucial choices once made—arc forever insulated from music’s rock-bound traditions of inter pretation through rfomiancc. My work resides in olid-atc digital memory with possibilities for instant and unlimited revision (e.g. I play hack a work-m-progress with exact audio visual detail, all of which is freely subject to any and all of my own creative revi sions). Such editing flexibility surpasses the interactive memory access (cut and paste, etc.) available to the creative writer with ordinary word processors. Music was once the most fleeting and transitory of all the am. Today, even in the hands of a non performing composer, and even with an optical graphics counterpoint, music has be come a solid-state, pliable material).
Some General Reflections
My guess is dial a powerful appeal lies wiihin die natural interlace and active coordination of eye to ear. and ear to eve. at the integrated level of digital audiovisual harmony. But who can foresee the expressive power of these relationships before dies are brought to life in many, many successful works of an? As for the established instruments and methodologies, refinement of the Baroque family of musical instruments and the standardization of musical no tation seemed to open floodgates, ena bling the production of a great lihrary of ensemble music that is still meaning ful. fresh and popular today after 200-MU years.
Cherished as this heri tage is and will likely remain, we may nonetheless expect that the perfection of real-time audio-graphics computer instrumentation (including a feasible interface with TV) will offer a develop ment in modern times possibly as fruit ful as the Classical epoch. While Renaissance muskaJ notation permitted an ensemble of musicians to play together in correct time, computer technology has provided simply total recasting ol time because o solid-state digital memory. Time, which is the essential dimension of music, which is studied and perfected throughout a performing artist’s career—this time, in digital memory, can be molded and reshaped over and over like sculptor’s clay. Oi perhaps it is more exactly like the concert pianist’s working all dav to refine a brief keyboard figuration, training his or her ‘solid-state’ memory to create the sound exactly as it is in tended to be performed in concert. The computer has expanded the op portunity for an ait of color action and music that was not widely foreseen. Before recent, fast digital systems with their rraNimc generation of both graphics and tone as well as instant replay became available in the late 1980s. the potentials were not even sub ject to exploration. Now. as if over night, a broad methodology is at hand Time or a time event—stored in digital memory is accessible to control in unique ways. One is reminded again of the concert performer training to accomplish exactness of fingering manipulations at great speed.
The per forming genius can take apatt these rapid action patients one at a time, step by step, in order to pause, reflect, then reassemble them with only the slightest modification, inching toward the ulti mate performance. Time stored in dig ital mentors becomes a malleable mate rial. perhaps, in this tense. This singular fact, above all others the new physical uactubilily ol time has significantly changed musical per spectives. This is what I sought to understand and to control. On film, lime is fixed into the silver image. But in digital computer memory’, utne bfreely alterable and permanently storable. With access to this new dimension of time, a few composers will sureb elect to set aside their heretofore cherished and essential musicianship and the en semble members with whom they per formed. Thev will join tanks with the large community of painters and sculp tors wliuse work rests in their own hands, whose wiork is subject to the most subtle personal manipulations, and whose work is final—unchallenged by any form of subsequent interpretation. Since, however, argument over the relationship ol art to newlv developing instrumentation is an ongoing subject of no slight controversy.
I can provide (hisconcluding anecdote: With two homemade devices simple sine wave pendulum arrav and an optical-printer instrument my bro ther James and I composed our first modes* international success in the rati fied avant-garde of ‘MTV’ (5). Wr were awarded firs prize at Belgium’s First International Experi mental Film Competition ‘Ihi early triumph implanted in our minds an urgent, lifelong drive to gam access to a perfected facility that would provide music and graphics capabilities unified within one instrument. All this began at least SO years before computer technology would allow us to realize this drram. In 1959 I wrote a description of our pendulum and optical-printer methods and theories of composition for Karl heinz Stockhausen’» journal of elec tronic musk Die Hfthr, Vol. 7 (61. Volume 1 of Ihr Rriht included an ar ticle by Pierre Boulez in which he raised the question. “Is a concert hall really necessary when the performing artist has been eliminated?” .As long ago as the I9MH. when electronk music was created bv spiking magnetic tape seg ments, many were skrptkal about bothering to enter a concert hall to hear music played over bland loud speakers. Boulez appropriates asked. “Is it not necessary then to find new conditions for listening or are we to contemplate the reuniting of this ‘arti ficial’ music with a ‘visual double (7) Reading Boulez’s opinions in the late 1950s.
I remember asking myself, “Why would anyone question if this clrrtronic music should have a visual double?” Today the answer is even more obvious, unless it be to prcseive traditions of black tie and tails, earnest electronic composers hardly need to reinvent either violin or concert hall. But composers who elect to deal fully and correctly with computational elec tronics simph must, in my view, invent that ‘visual double’. In the 1940s and 1950s mv brother and I were possessed of a benign obses sion to combine ‘artificial’ music with a ‘visual double’ We continued to search for new formal structures and appro priate universal instrumentation. We had conceived an indelible dream of aural-vivualiiv within a new art form. Privately. I envied Domenico Scar latti and Antonio Solar, who. by virtue of roval generosity or through a Pope’s large», seemed to have been provided tile instruments and the patronage with which to compose hundreds and hundreds of simple essays exploring a keyboard sonata form that was largely of their own invention (Scatlatii called them ‘exercises’a term that my bro ther and I adopted for our Five Abstract Film Exercises) [8] The seemingly bliss ful continuity of their lifelong creativity was exactly what my brother and I longed to emulate. Would that we hail had such a gift of insti uincntation! And yet, it is here at last. My compositions at best are in tended to point a way toward future developments in the aits. Above all. I want to demonstrate that electronic music and electronic color-in-action combine to make an inseparable whole that is much greater than its parts.
