LIM Banner




Home Unimi

Home Lim


The Role of Computer Technology in Music and Musicology


Technology has followed a path parallel to the development of music. From early pipes with cross holes, embouchures and reeds to the fine mechanics of 19th century pianos and wind instruments, passing through medieval carillons and renaissance string instruments, music has never been shy about incorporating the latest developments of technological discoveries.

This is especially true for contemporary electronic and computing devices, because, if the computer can be seen as the ultimate instrument capable of reproducing any known sound and of generating new ones, it is also a machine for the manipulation of symbols that can assist a composer or a musicologist in practical activities and in theoretical studies. And while electronic computers are a relatively modern invention, the idea behind algorithmic procedures and programming is probably as old as mathematics and known in the ancient worlds of Egypt, Babylon, Greece.

This writing is a collection of examples to show, in a historical progression, how computer technology (like any technology) has already significantly contributed to music and musicology, and supports the thesis that modern applications of programming, artificial intelligence and the like, to music, follow a well established and ancient tradition.

Recent times have seen the development of two significant aspects of computing technology: the availability of inexpensive, good quality sound processors and synthesizers, which have bridged the gap between ancient subdivisions of computer music, one preoccupied with sound synthesis, and the other with algorithmic and musicological processes - thus allowing immediate reproduction of computer generated music; and the emergence of standards [reference IEEE CS], such as MIDI and SMDL, that allow easy encoding, reproduction and exchange of musical data. Without these, the examples described below would have hardly been possible.

Early contributions: Dice Game

The Dice Game is a method for automatic composing, attributed to Mozart [reference Mozart] or to Haydn [reference Haydn].
The user throws a dice: assume the outcome is 4, then he takes, from an initial table of measures, the fourth one to begin the piece, and thereafter he turns to table no.4. He then throws the dice again, and if he hits, say, a 1, he takes measure no.1 and turns to table no.1 for the next throw. The user continues this way until the piece is finished.

It is an aleatory method, or a variation of the MonteCarlo method, in which random numbers are generated that are then subjected to constraints. Mozart's or Haydn's skills consist therefore in constructing the set of tables or formal grammar, as it is called in computer science, namely the productions which, from an already initiated string or word, allow selection of the next string element until the complete sentence - in this case, the musical phrase - is generated.

It is remarkable that the idea of a generating grammar, one of the most important paradigms of computer science (computer languages are defined exactly this way) arose already in the 18th century in a musical context. While it is argued that Babylonian clay tablets of the 13th century B.C. describe algorithms for harmonization, and it is likely that earlier unknown manuscripts (e.g., 15th century Flemish) may contain composing algorithms, it can be stated that Mozart and Haydn were among the first "composers of computer music", as if to indicate that the desire of applying computer technology to music is probably as old as music itself.

Experimental Music and the Illiac Suite

It is generally recognized that the first piece of music actually written by an electronic computer is the Illiac Suite in 1956, so titled in honor of the Illinois Automatic Computer, built at the same university at Urbana-Champaign, one the first high-performance computers [reference Hiller].

The Illiac Suite is an experiment to test various algorithms for composition. It can be said that it does not aim to be, in a strict sense, "great" or "beautiful" music, but a clever utilization of a computing machine to see which musical effects a certain rule produces. It consists of four movements, called experiments: the first is about the generation of cantus firmi, the second generates four-voice segments with various rules, the third deals with rhythm, dynamics and playing instructions, and the fourth with various models and probabilities for generative grammars or Markoff Chains.

Consider for example Experiment 2. It consists of eight sections, subdivided in two segments each, with random dynamics. Each section introduces a new counterpoint rule, in the following order (reprinted from Hiller):


  1. Random music; no rules
  2. Skip-stepwise rule; no more than one repeated note
  3. Cantus firmus starts on C with C chord for opening; cadence on C with leading tone in one of the four voices; resolution of tritone in VII-6, e.g. F/B must resolve to E/C
  4. Octave-range rule
  5. Only consonant chords permitted except for 6-4 chords; i.e., harmonic subroutine added
  6. Parallel unisons, octaves, fifths and fourths still permitted; melodic subroutine added
  7. Parallel fourths, 6-4 chords containing tenth still permitted
  8. Best counterpoint

Listening to the piece confirms the impression of randomness generally associated with computing machinery, notoriously incapable of grasping and conceiving a whole out of local details. Nevertheless, there is some merit in how, slowly, the music converges to something resembling 17th century music. The idea must have seemed fascinating, over forty years ago, that after century of dreams a machine has become available with unlimited possibilities for testing hypotheses. And testing hypotheses, from Bach to Schönberg, is also what music, and especially musicology, is about. Thus again, the Illiac Suite, in spite of its "modernity", "computer age" features, mathematical procedures,is nothing but a continuation of a tradition to which a new technology has been added.

Harmonization of the Unfigured Bass

The technology of Artificial Intelligence, a discipline of the mid-fifties with the goal of constructing "intelligent" machines, brought about among others the language LISP, one of the first to express in an elegant and concise way operations on symbols - instead of on numbers as in FORTRAN - as well as dynamic data structures of unlimited length for searches the depth of which is not known beforehand. Symbols can be defined as entity with non-numerical quality, such as notes, chords and the like, hence operators can represent construction of intervals, key transposition and the like.

The harmonization of the unfigured bass is an example of the use of early LISP in batch mode [reference Baggi 1974]. It was inspired by SAINT, a symbolic integrator performing at the level of a student learning calculus [reference Slagle] and by an earlier work on harmony theory with programs written in SNOBOL, a language for string manipulation now defunct [reference Rothgeb]. A system of LISP programs receives as input an alphanumeric representation of a bass line of the type given to harmony class students, such as those from Pedron [reference Pedron], used in Italian conservatories. From this, the system constructs the upper three voices (soprano, alto and tenor) and passes its data to a Calcomp plotter that draws a somewhat readable score with all four voices.

The fundamental idea was that, to overcome the random feeling of typical computer music as in the Illiac Suite, a a contextual framework, at least at some level, is needed by a computing system, such as the one provided by tonal classical harmony. However, harmony is more like a language learnt by examples and rules about what-not-to-do than a procedure, hence it was felt that a formal treatment in the form of algorithms was needed - there is perhaps some parallelism with natural language processing, in spite of serious differences, which has been treated elsewhere [reference Baggi and Negrotti, 1997]. In the same path, modern interactive, window-based computers have allowed completion of projects in which harmonic constraints in symbolic form have become easy to program and to use[reference Cope].
In spite of its limitations and of the "primitive" hardware" of the early 70's on which it ran - a CDC 6400 with 128k words of memory - the system was shown to be capable of passing the exams from the booklet, at some variable level of proficiency. Tonicizations and modulations are recognized, though only in the neighboring keys and relative minor-major modes. It is an example of algorithmization of four-part harmony, at a level at which criteria such as correct or incorrect do play a more important role than esthetic ones.

The Paul Glass Method

The theory

In his study in dodecaphony, composer Paul Glass hit the series

c - b - c# - b - d - a - e - a - e - g - f - f#

which has some remarkable properties: it is invariant under retrogression sounding the same if played backwards, and it contains all decaphonic intervals from 1 to 6 (semitones), from one note to its next, exactly twice. It is therefore the candidate for a composition with the least possible reference to diatonic intervals and tonal centers.

Moreover, and this is the basis of his new system of composition, each note of the series can be seen as the tonic of a diatonic scale (e.g., a major scale with seven notes and no alterations) such that, by proper choice of the order of the notes, the sequence of these twelve 7-note keys becomes at the same time a sequence of seven 12-note dodecaphonic series.

The figure illustrates how that works, by grouping, above, the twelve notes of each series and below those of each key. Let us start with the key of C-major, which contains the notes c-d-e-f-g-a-b. To complete the dodecaphonic series, one needs the extra five notes c#, d#, f#, g# and a#, which are contained in the keys of B, C# and F#. Assume we choose the key of B to complete series 1. Then there are two extra notes, namely e and b, which spill into series 2. Hence series 2 must be made, among others, of a key which does not already contain e and b: C# is such a key. This gives nine notes for series 2, which now needs notes d, g and a to be complete: B is a key with such notes, yielding notes b, c, e and f as the first four notes for series 3. Thus each note in a dodecaphonic series determines the next key diatonic key, and the whole converges at the 7th series and 12th key - proving, if need be, that 7 x 12 is the same as 12 x 7! It is left for the keen reader to follow the picture and see how the remaining eight keys are determined from the method of above.

Paul Glass proved therefore by construction that there is at least one series of 12 keys that satisfies these constraints. Obviously its inversion, namely

c - c# - b - d - b - d# - a - e - a - f - g - g

satisfies them too and enjoys the same properties of invariance and completeness of intervals. It less obvious to see that the chromatic scale, namely

c - c# - d - d# - e - f - f# - g - g# - a - a# - b

also gives rise to a Glass sequence of 84 notes like the above (tough it contains only one interval, the semitone, throughout, from a note to the next). Hence, counting also the inversion of the chromatic scale, there exist at least four Glass sequences.

Even less obvious is the fact that, at any point where there is a choice of keys, some choices do not generate a complete sequence. For instance, in the case described in detail above, starting with C-major one could in theory choose F# as he next key, since it contains all five sharps, however the process pretty soon stops and it is necessary to go back and choose either the key of B or of C# - a process known in computer science as backtracking, which means taking another upper branch for the depth search.

Hence the question by Glass was: are there any other series of this kind? If so, which ones, and how many? And are there some with properties such as symmetry and completeness of intervals (at least once)?

To answer such questions, a program in the language LISP has been written by this author in 1985, with primitives such as key computation from given notes, knowledge of the chromatic scale, of intervals and the like (it is easy to define such operators in LISP, which then work like built-in operators for arithmetic operations in other languages, but operate on symbols, in this case musical notes, instead of on numbers). The main algorithm is a loop to find all twelve keys in a sequence that satisfy all constraints, that backtracks in case of failure, and this for all possible sequences.

The interesting, and somewhat unexpected, result is the following:


  • there are exactly 1,200 distinct Glass sequences, plus their inversion, hence a total of 2,400
  • there are 144 symmetric sequence
  • there are 420 sequences containing all possible intervals

It is remarkable that such numbers are made only of the first primes, 2,3,5 and 7, suggesting that there is some deep numerological, or geometric explanation for the method.

As the type of key to work upon - e.g., the musical mode is an input to the algorithm, the search has been made for other modes, with any number of notes, and has yielded the following interesting result:

Glass sequences exist only for the seven Greek modes, Ionian, Dorian, Phrygian, Lydian, Mixolydian, Eolian, and Locrian (in other words, for 7-note sequences obtained by playing only the white keys of piano, from any key) - of which Dorian is the "king" mode closed under the operation of inversion and retrogression; and the sequencs are the same no matter which mode one chooses! Obviously this is in agreement with the rules of Glass sequences that, within a key, any order for the notes can be chosen.

Hence there is no convergence (no Glass sequence) for any altered scale, such as the harmonic minor mode, the ascending melodic minor, nor for any mode with a number of notes different from 7. It seems that Glass sequences, used to construct dodecaphonic sequences, curiously underline an important property of diatonic scales - e.g, only scales with one sequence of two tones and one with three tones, separated by one semitone, can be used to generate Glass sequences.

Glass sequences in composition

The existence of Glass sequences, it is claimed, has important consequences for a composer, because they allow definition and concatenation of dodecaphonic series that smoothly blend from one to the next. Namely, as Schenker's theory for tonality, it provides a method for composing and consistently plan the overall structure of an atonal piece.

Prior to the discovery of all possible series by computer, Paul Glass had already used the series of above as the scheme in various compositions - the series is now labeled as no. 866 in the Baggi-Glass catalog.

Here follows a list, by no means complete, of examples of compositions by Paul Glass based on this method, together with the associated series:

  • Fuchs Variations for Cello Solo, 1983, Series no. 866
  • Extemporaneous Concerto for Piano and Orchestra, 1983, Series no. 1398
  • Five Piano Pieces, 1984, Series no.1398
  • The late Nancy Erwin, movie music, 1984, Series no. 866
  • Pianto della Madonna, from Jacopone da Todi, 1986, Series no. 866
  • De' Spiriti Miei, Quando mi vedette, from Guido Cavalcanti, 1987, Series no. 683
  • String Quartet I, 1988, Series no. 783
  • Lamento dell'acqua, 1990, Series no. 763
  • Sinfonia N.4, 1992. Series no. 994
  • Quan Shi Qu, 1994, Series no.683

This example, unlike the preceding and the following ones, is not just a theoretical study in musicology, but a project which has actually helped a real composer to define his methodology and apply the results in his pieces.

NeurSwing, an automatic Jazz rhythm section

This is another example of synthetic musicology, namely the investigation of a musicological problem thanks to a machine constructed for that purpose.

Unlike classical music, the musical merit of a piece of Afro-american Jazz is not really contained in its harmony - relatively simple - , in its melody - often improvised -, in its rhythmic variety - jazz pieces come often only in 4/4 and segments of four measures, sometimes in 6/8, rarely in 5/4 or 7/4 - or in the "purity" of instrumental and vocal sounds - more often than not, these are distorted and personalized on purpose -, but in how well all these elements contribute to the generation of swing. Swing is the blood of jazz, its real medium, through which the jazzman, rather than "making music", "tells a story": as such, it expresses both intellectual elements - the artistic quality of the story - and physical ones, as the push on rhythm - that which makes people want to dance, move or tap their feet, for instance. This is why the score, from which in theory a piece of classical music can be reconstructed (or at least the intention of the composer can be understood at some level) has little meaning in jazz, as swing cannot be annotated: only recordings, from the first acoustical records of 1917 to modern digital recording, can capture swing and allow the establishment of a jazz musicology.

While swing is close to impossible to define, as it evolves and changes throughout jazz history - sometimes taking the form of heavy rhythm accents, sometimes the exact opposite - there are technicalities that can be isolated within a given stylistic period: for instance, the constant opposition of a reference pulsation, often irregular, generated by the rhythm section consisting of, say, piano, bass and drums, and the free floating, improvised line by a soloist, say a saxophonist. It is possible, therefore, to isolate accents and rhythmic embellishments at the melodic level, to use rhythmic and harmonic patterns in putting down piano chords, appoggiaturas by the bass and drum rolls, accents and emphasis on the afterbeat by the drums, to generate swing.

For all practical purposes, a piece of jazz can be annotated as a harmonic grid, a table of squares, each of which represents a measure and contains one or more chords, annotated as Dm7, B7(5) and the like. They specify the harmonic nature of what ought to be played on that beat, though not the actual notes, which can be chosen at will, and not only by the soloist, but even by the members of the rhythm section. Moreover, the players can even substitute chord patterns with other, equivalent ones: for instance, C | C can become Em7 / Em7 | Dm7 / D7.

NeurSwing [reference Baggi 1992] is a system consisting of a computer and a synthesizer that, from a given harmonic grid, simulates a jazz rhythm section and plays in real time piano, bass and drums. Notes, harmony substitutions, rhythmic patterns are selected by the system at random under control of three stylistic knobs: hot/cool - namely, the emphasis of hot expressionism of swing versus cool - dissonant/consonant - borrowed from classical music: dissonances are considered 7ths, 9ths, flatted 5ths, while 3-5 chords and dominant 7ths are more consonant - and as-is / free - namely, stricter adherence to the given grid versus distant, or freer, substitution.

The musicological study can be performed at various levels of machine interaction: first, by changing the setting of the stylistic knobs, which allow possibilities in the order of {infinite to the third power}; second, by altering the stylistic net, namely the dependencies, or membership constants of the stylistic elements that belong to corresponding fuzzy sets; third, by changing the substitution tables for harmony and the tables of the rhythmic patterns. This way, a deeper understanding of the technical elements of swing is possible, or at least, as in the case of the Illiac Suite, an analysis of what effect does a rule cause.

At the practical level, the machine can be used as a training tool by a would-be jazz performer. Learning alone is not efficient, as one tends to speed up at easier passages and slow down at difficult ones, while swing requires perfect mastery of tempo - more so than in classical music. Learning with a metronome does not allow the interaction provided by a dynamically changing rhythm section, and learning from records of rhythm sections is repetitive and inflexible, as no change of tempo and of key, nor any repetition of isolated and difficult passages, are possible. NeurSwing has been used to learn improvisation of jazz pieces, first with full as-is parameter and slow tempos, then with a higher free value and further with higher hot values, when pianist and bassist push forward with denser structures and the drummer accelerates. It can be used therefore as a teaching machine, as it can tirelessly simulate the operation of a rhythm section, with adequate performance to make a serious study possible.


The examples of above represent by no means an exhaustive list of contributions of computer technology to music and musicology. They have been chosen because they illustrate various aspects of interdisciplinary endeavors: from the early studies of Mozart and Haydn to the "ancient" - in regard to history of computing - Illiac Suite, to three projects by this writer of symbol manipulation dealing, respectively, with tonal harmony, dodecaphony and elements of jazz style. No doubt other examples could have been chosen, as a quick examination of the specialized literature would easily show, including studies promoted by the IEEE CS Technical Committee on Computer Generated Music [reference IEEE CS]. What however seems to be common to all these efforts is that computer technology, far from being the "villain" and "soulless", "cold" monster of popular mythology, has already assisted music and musicology and will continue to do so for a long time to come.

This project at the Scala is one of the best illustrations of what has been referred to as Computer Generated Music, an interdisciplinary approach that contains the vast area spanning from artistic music obtained with the help of computers to audio signal processing, with the extremes included. The projects deals with material science for the restoration and preservation of tapes, such as cleaning, baking, strengthening the support; with signal processing for the audio treatment, removal of noise and hiss and recreation of the spectral balance; with esthetic criteria, to decide how to restore false starts, missing musical material, removal of pops, clicks and hum; with musicological criteria, to classify and order all the material; with computing techniques, for the construction of the data base and of its access keys, including musical keys obtained from whistling and humming a few notes to find a matching music piece; and also with legal matters, such as who is the rightful owner of a recorded piece, the performer, the theater, or who recorded it, who restored it, and the like; and certainly other disciplines not mentioned above.

In conclusive words, these are projects that require as much love for technology as is needed for music. They clearly illustrate the common bond between technology and music and will help spread the message of how the deep relationship between science and art is beneficial to both, as it was in the Renaissance and as it is more and more so today.


Baggi, Denis L., Realization of the Unfigured Bass by Digital Computer, Ph.D. Thesis, University of California, Berkeley. Published by Xerox University Microfilms, Ann Arbor, Michigan, 1974.

Baggi, Denis L., Neurswing: An Intelligent Workbench for the Investigation of Swing in Jazz, in Readings in Computer Generated Music, D.Baggi, Editor, IEEE CS Press, Los Alamitos, 1992.

Baggi, Denis L. and M. Negrotti, Musica, linguaggio matematico e intelligenza artificiale, invited paper in Convegno Musica: le ragioni delle emozioni, Conference of the Associazione Nuova Civiltà delle Macchine, May 28, 1997, reprinted in Display, Dipartimento dell'istruzione e dalla cultura, divisione della scuola, centro didattico cantonale, Bellinzona, Switzerland, Feb. 10, 1998.


Cope, David, Recombinant Music: Using Computer to Explore Musical Style, IEEE Computer, July 1991, pp. 22-28.

Haydn, Joseph, Giuoco Filarmonico, ossia maniera facile comporre minuetti, G.Ricordi & C., Milan, Italy, N.108202.

Hiller, Lejaren A., and Leonard M. Isaacson, Experimental Music: Composition With an Electronic Computer, McGraw-Hill, New York, 1959.


IEEE CS, Technical Committee on Computer Generated Music of the Computer Society of the Institute of Electric and Electronic Engineers (IEEE CS TC on CGM), see and in particular the CD-ROM Standards in Computer Generated Music.

Mozart, Wolfgang A., Musikalisches Würfelspiel, K.Anh. 294d, Mainz, Schott's Söhne, Edition 4474.

Pedron, Carlo, Nuova serie di esercizi per lo Studio Progressivo del Basso senza numeri, Carish, Milan, Italy, N.14847.

Rothgeb, John E., Harmonizing the Unfigured Bass: A Computational Study, Ph.D. Thesis, Yale University. Published by Xerox University Microfilms, Ann Arbor, Michigan, 1968.

Slagle, James R., A Heuristic Program That Solves Symbolic Integration Problems in Freshman Calculus, in Feigenbaum, Edward A., and Julian Feldmann, Computers and Thought, McGraw-Hill, New York, 1963, pp.191-203. Also in Slagle, J.R., Artificial Intelligence: The Heuristic Programming Approach, McGraw-Hill, New York, 1971, pp.53-57.


Denis L. Baggi
Chairman and Founder, IEEE CS Technical Committee on Computer Generated Music
Second Vice-chair for European Technical Activities, IEEE CS Technical Activity Board
Speaker, Distinguished Visitor Program, IEEE CS

December 9, 1998