Overview -------- Audiality 2 is an interactive, structured realtime audio engine that generates sound and music using a tree of voices, driven by user defined programs running on a virtual machine. Each voice is controlled by a program that can be given initial arguments, and receive messages for realtime control. A program can (recursively) spawn other programs on subvoices, and control these by sending messages. Timing is subsample accurate, and durations can be specified in milliseconds, or in terms of user defined musical ticks. Modular Voice Structure ----------------------- Audiality 2 voices are modular. When a program is started, the audio DSP part of the voice it will run on is constructed from audio DSP units as specified by the program. By default, a voice has no units, and cannot directly produce sound, but it can still spawn and control subvoices. Output is sent to the nearest 'subvoices' unit up the graph. If no explicit wiring is specified [not yet implemented!], units are autowired according to these rules: 1) Units form chains, the outputs of one unit being wired to the inputs of the next unit. 2) A unit with inputs cannot start a chain. 3) A unit explicitly wired to the voice output (> for output channel count) terminates the current chain. 4) A unit with only outputs... a) ...when placed in a chain, mixes into the audio from the previous units. b) ...when there is no chain (first unit, or chain terminated due to rule 3), and the next unit has no inputs, sends to the voice output. 5) The last unit specified in a struct declaration always sends to the voice output. Voice Structures - Strange Cases -------------------------------- Consider this structure definition: struct { wtosc wtosc panmix } Intuition would have it that we get two oscillators side by side (summing), then processed by the panmix stage. But, no! Since the first oscillator looks ahead in the chain and sees the second one, which has no inputs, it decides it is the end of its chain, and thus decides to send to the voice output instead. As a result, we get two separate chains; one with an oscillator sending directly to the first channel of the voice output, and one with an oscillator and a panmix stage. Subvoices --------- Voices spanwed by programs run as subvoices of the voice the program is running on, recursively forming a tree structure, where audio flows towards the root node. There are two ways of running subvoices; decoupled and inline. Decoupled subvoices send their output to the same destination as their parent voice, and as such, they are essentially independent of any audio processing that the parent voice may be doing. Inline subvoices are wired into the modular voice structure of their parent voice by means of the 'inline' unit, allowing their output to be processed by the units of their parent voice. ///////////////////////////////////////////////////////////////////// // // // WARNING // // // // The rest of this file is slightly out of date! // // // ///////////////////////////////////////////////////////////////////// The Audiality 2 language ---------------------- The Audiality 2 language is a small, domain specific language. It has rudimentary control flow instructions, expressions with no operator precedence rules (evaluation is invariably done left to right), and integrated cooperative threading with message passing. The language itself has only a single type: real number. (In the current implementation, 16:16 fixed point.) However, Audiality 2 also has static objects, such as waves and programs, that are referenced by handle. Keywords as well as symbol names are case sensitive. Comments: // Single line comment /* Multiline comment */ A single line comment can start anywhere, begins with // and ends with a "newline". That is, both full line and "end-of-line" comments are allowed, and use the same syntax. A multiline comment starts with /*, ends with */, and may contain newlines. Code may follow a multiline comment on the same line, after the terminating */. Immediate values: [-][integer][.[fractional]][conversion] That is, no leading zero is required; a fractional number can start with '.'. 'conversion' specifies a conversion to be performed on the value after parsing it, allowing for example pitch values to be expressed in Hz or MIDI style note numbers instead of the native 1.0/octave linear pitch units. Note that conversions have no effect on code generation or run-time operation; they are merely a convenience feature of the parser. Available conversions: f "Frequency." The value is interpreted as a frequency in Hz, and is converted to linear pitch. n "Note." The value is interpreted as a twelve- tone note number, and is converted to linear pitch. That is, the value is divided by 12. Labels: .label Labels are declared with a preceding '.'. Label names must start with a letter ('a'..'z'), followed by any number of letters or decimal figures. It is allowed to continue with another statement directly on the same line, after a label declaration. Variables: !var 2; p var // Declare 'var', init to 2 and write to p rand !random 42 // Declare !random and init using rand The Audiality 2 language provides access to the VM registers as named variables. Control registers have predeclared names, but the remaining general purpose registers can be allocated and named by picking a suitable name and prepending it with '!' (exclamation mark). Variable naming rules are the same as for labels. A variable can only be declared in a way that will guarantee that it is initialized before reading. Due to the simplicity of the parser/assembler, this currently means that a variable has to be initialized by the statement in which it occurs, ie it has to be the target of an assignment or a write-only target term of a statement. Directives: def Define to , so that any occurences of evaluate to . Instructions: Depending on the instruction, there can be zero or more arguments. An argument can be a register name, a label name, a constant name, or an immediate value. (end-of-statement) can be ';' or ',' (for multiple statements on a single line), or "newline". Instructions: sleep return : < run kill force jump loop jz jnz jg jl if ifz ifg ifl else while wz wg wl tick d td phase set Expressions: ( [ [ ...]]) In most places where an argument is expected, it is possible to use an expression, enclosed in parentheses. Audiality 2 expressions are evaluated from left to right, without exception, that is, there is no operator precedence. Operators: + - * / % quant rand p2d In-place operations: Most operators can be used for in-place operations on variables and control registers. This can be thought of as the operator being shorthand for an instruction that takes the target as the first argument. Assignment shorthand: The internal MOVE and LOAD* instructions are not available in the Audiality 2 language. Instead, a variable name followed by another variable name, or an immediate value, is used as a shorthand assignment syntax. The appropriate VM instruction is inferred by the compiler. Control registers: w Oscillator waveform p Oscillator Pitch (linear pitch) sp Oscillator Secondary Pitch (linear pitch) a Oscillator Amplitude tick Musical tick duration (milliseconds) Some control registers ramp internally, rather than instantly accepting new values. The duration of these ramps is controlled indirectly by the timing instructions, so that the control arrives at the new value as the VM continues to run after a delay. When desired, such control registers can be instantly set to the current target using the 'set' instruction. Predefined constants: off (Mute and stop oscillator) square (Square/pulse waveform) saw (Sawtooth waveform) triangle (Triangular waveform) noise (Sample & hold noise generator) Programs and message handlers: progname(argname= argname ...) { 1(argname argname= ...) { ... } 2(argname argname ...) ... } .localprog(...) { ... } Programs are normally exported, but can be kept local (ie only visible to code in the same file) by prepending the name in the declaration with a '.' (period). The CS_EXPORTALL flag to cs_Open() overrides this, essentially ignoring the '.'. Naming rules are the same as for variables and labels. Message handlers are declared inside programs, using their integer entry point indices for names. Message handlers cannot contain timing instructions, such as d, td or sleep. Arguments can be considered equivalent to local variables, initialized with the incoming argument values. Arguments that are not specified by the caller are by default initialized to zero, but different default values can be specified in the declaration argument list by appending '=' followed by the desired value after the argument name. Program and message handler arguments: The number and meaning of arguments is defined entirely by the program code, but it is recommended that all programs use a similar convention, for easier reuse and interoperability. Here are my suggested conventions, which should cover most musical applications as well as 3D sound effects: SomeSound(Pitch Velocity=1 Modulation X Y Z) { 1(Velocity=0 ReleaseVel=1) {} 2(Pitch) {} 3(Modulation) {} 4(X Y Z) {} } Pitch: Linear pitch. Velocity: Note gate and power/velocity. ReleaseVel Release velocity for Velocity 0. Modulation: Vibrato, filter cutoff, drive etc. X, Y, Z: 3D pan position. The logic is that the program arguments intialize internal "controls", that can then be changed in real time using the corresponding messages. The Velocity control should be implemented so that setting it to a non-zero value triggers a new "event", ie a musical note or a new sound event, or just fading to a different power level, whereas setting it to zero switches to a decaying state. In MIDI terms, these operations would correspond to NoteOn and NoteOff, respectively. The Velocity control should be interpreted as note velocity or power, akin to MIDI NoteOn velocity. Changing the Velocity control from zero to a non-zero value is logically similar to M MIDI NoteOn, whereas setting it to zero would correspond to MIDI NoteOff. A series of non-zero values would be similar to MIDI "pressure". To implement MIDI style NoteOff velocity, an additional argument is used for message 1, with Velocity 0. Preferably, a program should implement repeated Velocity on/off transitions in a way that allows clean, seamless monophonic control. If Velocity is set to a non-zero value after the main program has finished, the program is expected to restart in a appropriate way. Linear Pitch: The oscillator frequency is controlled using "linear pitch"; a unit that is very handy in musical applications, as it maps trivially to octaves and notes. Adding 1 doubles the frequency (ie one octave up) whereas subtracting 1 halves the frequency (one octave down) - and the neat part is that this works at ANY pitch! For musical notes, the twelve-tone equal tempered scale is the most common scale, and in terms of "1.0 per octave" linear pitch, that is expressed as 1/12 per semitone. The assembler has a handy conversion 'n' for this, so you can simply type things like "p 7n" or "+p 3n". Audiality 2 defines pitch 0.0 as "C0", which resolves to 261.626 Hz. Uploaded waveforms can have a period length of any integer number of samples, and pitch control is based on those periods, rather than sample rate. Pitch inheritance: As a program starts on a new voice, the secondary pitch register 'sp' is initialized with the sum of 'p' and 'sp' of the parent voice or group. This allows transparent transposition, greatly simplifying musical algorithms and "song data" encoding. If absolute pitch control is desired, in percussion or hardsync or granular synthesis, for example, it is a simple matter of starting the program with 'sp 0' to reset the inherited transposition. Some programs can be simplified and optimized by using both pitch registers for modulation. Pitch inheritance can still be implemented correctly by reading the initial 'sp' value, and feeding that value inte the algorithm in a suitable fashion. Instruction set: end End function/program/handler and return to caller. If there is no caller, wait for the voice to be detached, then detach any subvoices and wait for them to finish, and finally kill the voice. NOTE: Message handlers are still working until the voice is actually killed! Thus, a program may end with subvoices running "indefinitely", using messages to control those voices. jump