Introduction
Contents & Program Notes
Technical Comments
Theory and schematics
Article about the Invisible Instrument

Introduction

My investigations and experiments into the area of musical uses for radar and sonar systems go back to the early seventies, when I started to build my first movement-to-sound converters. On the local flea-market I had discovered a small book on industrial and military applications of ultrasound, published in 1952, the year of my birth. It immediately gave me lots of ideas for musical applications. The problems to overcome were considerable however, since for work in this field, no standard components were available on the electronics market. Thus I had to make the basic components such as transducers and microphones myself. Only with the advent of integrated circuits working circuits became a realistic possibility.

The first equipment using ultrasonic technology that I built, was develloped between 1972 and 1982. The first real-time performance piece that we, this is, the Logos-Duo composed of Moniek Darge and myself, realized taking advantage of this technology, was 'Holosound'(1983). This highly theatrical piece we have been travelling with for a decade, has had over 200 performances all over the world. The Holosound-equipment I built for it has seen over 6 completely different and every time improved versions of the hardware. The equipment was capable of translating three-dimensional body movement into relevant sound material, thus implementing something that could be called 'sound- holograms'; hence the name 'Holosound'.

My research into the underlying technology did not come to an end with this first achievement however. In 1988, at the occasion of the International Computer Music Conference in Koeln, Germany, I presented a first midi- compatible version of the holosound hardware, one that enabled musicians to use three-dimensional body movement as a wireless multichannel midicontroller.

In april of 1992, after another extensive period of electronic research, I started work on what now became 'A Book of Moves', a full-evening multimedia production using a specially develloped version of both hardware and software of my sonar technology. I finished my doctoral dissertation on this and related topics and presented it at Ghent State University under the title 'An Invisible Instrument'.

The first edition of the 'Book of Moves' was premiered in Perth, Australia, thanks to Evos and Pica on 24th of July 1992. In the weeks after this premiere, the piece was further performed in Melbourne, Tokio, Miyoshi (Shikoku,Japan), Shangai, Beijing, Ghent (Flanders), Amsterdam, Bonn, New York. Since its premiere, it had over 250 performances all over the world.

The compositional idea behind this piece was to design a series of very different, yet invisible, musical instruments. In each instrument, specific movements would be used as controllers for specific musical parameters. The most 'simple' of these 'mappings' of movement parameters into musical ones, was set up as follows: amplitude could be controlled by the amount of moving body (moving body-mass), octave-position of a sound, by the absolute velocity of a movement, pitch by the acceleration of deceleration of the movement. Since the system operates fully 3-dimensionally, these angular parameters were used to control 3 different musical voices independently; the panning of these voices in the sound field, following the spacial set-up of the transducers. Although this simple mapping does work and is used as the base underlying some of the pieces ('Call' and 'Solo') contained in 'A book of moves', more elaborate algorithms lead to far more interesting musical results in most of the other pieces in the book.

In the piece called 'Topoi' the inputs from the transducers are used to feed a simple neural-network that attempts to map relations between 'harmonic' chords to the results of topological analysis of the moving body (the performer). In the piece 'Beat', the performer plays an invisible but very large percussion set, where all the instruments are arranged in the space and there the mapping translates acceleration into dynamics. In the piece 'Lead', the performer actually gets to conduct a full but virtual orchestra.

Contents & program notes

Chapter 1: performers: Moniek Darge or Karin De Fleyt

an ouverture based on a single repeated 8-note chord. Here the movement-information is used to gradually switch off the sound and to organize its spacial distribution.

a demonstrative and gradual build-up of the functionality of the invisible instrument. The piece starts by sounding distant shofars, at first apparently unrelated to the movements, and ends as an implementation of an invisible accordion, a set of small drums, and a celtic harp, where the sounds are closely matched to the movements.

a composition where the result of place-calculations are mapped into harmonic chordal progressions. Thus the tonal center of this piece coincides with the physical center of the imaginary tetrahedron. The further a movement deviates from the center, the further the chords sounded will be from the tonal center.

a slow upward chordal progression controlled by movement position, angular displacement and motoric acceleration.

a mysterious piece where movemental parameters are mapped in aspects of non-pitched sound materials, such as nature sounds. Continuous controll parameters are applied to the sound material.

an implementation of an invisible stringed keyboard instrument playing in a minor scale.

an alternation of at the one hand a simple movement controlled drum ('Drum') and, at the other hand ('Beat'), an implementation of a huge percussion setup requiring karate like movements from the performer.

a centripetal composition with a strong tonal centre. Velocity is translated in pitch-deviation from the tonal centre. Dynamics are a discontinuous function of movement amplitude

an implementation of a large symphonic orchestra, conducted by the performer. The 8-voice harmony as well as the orchestration is movement dependent.

a composition using nature sounds and 'primitive' musical materials such as didjeredoo, water sounds, bullroarers etc...

a movement recorder capable of recording a movemental section and playing it back as musical phrases in 3 voices according to the 3 vectors of movement the equipment is capable of capturing. It can also be seen as a sampler for movement, with musical output.

Two pieces played in alternation, wherein an invisible solo-instrument (i.e. a Nej-flute) is played. In the 'Tance' section, the solo-instrument gets automatic accompaniment from a small 4-voice band.

a percussion piece based on complex meters based on prime-numbers. Amplitudes and pitches are movement- dependent. At the end of this section the second player joins the first.

Here the movement information is used to controll the tempo and the musical information played by three musical voices.

Whenever a reversal of movemental parameters, such as acceleration/deceleration is detected, musical events are triggered on the edge of the event.

The title of this paragraph is a concatenation of the musical form 'stretto' as part of a fugue and the well known english notion of stress. In fact it is a stretto in 16 voices whereby the whole polyphonic temporal structure is controlled by different aspects of the interpreters motoric.

an implementation of a hammered cimbalon, whereby acceleration-parameters are mapped into pitch and velocity into octave position.

Two complementary small percussion pieces: a cyclic percussion piece (Lock), changing at some point by a breaking of the cycles, into Unlock, where the metrum is a function of movemental parameters. In Unlock the periodicity and duration of the sounds are movement dependent.

A finale for the Book, based on a chordal downward progression ending in a final unison. The intervals are a function of movemental parameters.

Technical comments

Although the piece might appear to people as having a lot to do with dance, it was neither conceived as a dance piece nor is it actually very appropriate to be used as a dance piece. First of all, it really is and behaves like a real musical instrument and should be played as such. Moves that are too "elegant," for instance, do not lead at all to musically interesting results. Furthermore there is the fact that dancers are trained to follow the music as it goes, whereas here things only work the other way round. The movement has to be performed as a sound producing behaviour and not as a gesture of mainly visual nature. The minimal aspects of staging (costumes and lighting) that we use in our performance of the book, are there mostly as an element in the rhetoric of the piece. The costumes -designed by Moniek Darge and made of silvery fabric- are made such as to maximally reflect ultrasound. Only wet nude skin would be technically superior.

For whoever is interested in the technical side of the hardware and software used, I could add following description of the setup:

The transducers work on ultrasound and can be tuned to frequencies between 42 and 200kHz. The operating frequency determines the frequency range of the received differential tones. The lower this frequency, however, the larger the area covered by the system. The higher the frequency, the higher the resolution will be. There is one emitter and three receivers each placed on the vertexes of an imaginary tetrahedron. The received signals (carrier plus differential tones) are fed into a high precision analog computer that calculates the Doppler-shifted differential frequencies.

A second analog computer takes care of the Fourier transforms and does the conversions from the frequency domain into the time domain. It also does all the required lin- log conversions and the square rooting needed for the power calculation. This analog computer outputs 16 channels of analog data: 3 channels with vectorial movement-amplitude information, 3 channels with vectorial velocity information, 3 channels with non-vectorial power information, 3 channels with vectorial accelleration data and 4 channels carrying the integrated sum of vectorial amplitudes and power signals as well as the momentaneous non-vectorial peak- values of velocity and acceleration.

These channels are fed into a dedicated microcomputersystem with a fast multi-channel 12-bit ADC converter. This digital computer is responsible for the conversion of the preprocessed analog data-stream into musical parameters, for the different mappings used in the 'Book of Moves' and for most of the midi-control. Although in the original (first edition of the Book of Moves, 1992) version, I used an industrial microcontroller, since the second edition I ported the software to a modified 80486DX platform. This laptop system takes at the same time care of all the midi-processing, including the real-time midi-dumps to whatever midi-gear is connected to the setup. The third edition, since 1998 involved a complete port of the code to our <GMT> programming language. The platforms on which the software now runs is Pentium (minimum 120MHz, with MMX) and the OS can be either Win98 Win-NT4 or Win2000. Also the analog computer and interfacing hardware has been completely rebuild and highly improved.

I designed the system to work with a modified Emu Proteus sound module, but I might very well change this in an upcoming version of the piece. All hardware, exception made for the laptop computer, was designed and built by the author who also wrote all of the software, without using any commercially available packages other than the programming language compilers.

In no way this piece can be considered as a form of movement controlled sequencer, since none of the sound-structures exist neither in hardware nor software prior to the performance of the piece. The algorithms -many of them making use of fuzzy-logic- residing in Eproms and on the harddisk of the computer only hold the extensive math relevant for the musical syntax of the different pieces that make up the Book.

The 'Book of Moves' is a real time algorithmic composition taking its control-inputs from the performer.

Dr.Godfried-Willem RAES

Buffalo, october 1993

First Edition:: Perth, Melbourne, Tokio, Bejing, Shangai, july 1992; New York, april 1993

Second Edition:: Buffalo, october 1993 (V5.2); Warschau, Krakau, november 1993 Leuven, 10.02.94 (V8.0); Gent, 23.04.94 (V9.0) Mills College Berkeley, San Jose SJSU, Arizona University Phoenix , 04.95 Brasil Tour, August 1995 (V6.9) including 'Songbook' Austria Tour, August 1996 (V7.2) including 'Songbook' , 09.97, Transform Festival, Herne,Germany, including 'Songbook', Roeselare, Pax, 10.97 Kinetic Festival. Indonesia Tour, 1999; Stockholm, Fylkingen, 1999. Hulste, Muizelhuis, Hiroshima, Ikeda (Japan), 2000.

Third Edition: In the fall of 2000, the author developped completely new analog computers for the invisible instrument. Now the system makes use of National Instruments data acquisition devices (PCMCIA cards on laptops, but USB DAQPad 6020 can be used as well) and runs both under Windows 98 as under Windows 2000 / NT. New versions and chapters of the Book will include realtime movement based audio synthesis on the pentium platform.

Authorized performers for <A Book of Moves>: Moniek Darge, Joachim Brackx, Karin De Fleyt, Godfried-Willem Raes.

MP3 excerpt from 'A Book of Moves'

download picture of Godfried-Willem Raes performing 'Lead', a section of 'A Book of Moves'