Interactions in MIDI and Csound

    

To understand this part in the MIDI environment, you need to read Real-time MIDI.

Several BP3s performing and communicating

From the previous description of interactions via MIDI events — limited for the time being to waiting for a particular note — you may have guessed that a great feature of the Bol Processor BP3 environment is the possibility of running several BP3s, on different machines, or even on a single machine and the same browser… in cooperation with real humans playing MIDI instruments!

Each instance of BP3 can be thought of as a 'musician' with their own compositional skills embedded in a grammar or data (a set of pre-composed musical fragments). We are working on interactions that will allow each musician to modify the behaviour of another musician's grammar, for example by changing rule weights — which may result in some rules being suppressed while others are activated — or changing metronome settings if they need to perform faster/slower, etc. All these features were part of earlier versions (BP2) several decades ago!

Let us start with an extremely simple example using the "wait for note…" script.

Create two projects that contain only data, for example "-da.Beatrix" and "-da.Alan":

-se.Beatrix
{2, C#4 Eb4 A4 G#4 C4 Bb4 F#4 G4 B4 D4 E4}

-se.Alan
_script(wait for E4 channel 1) {2, - F3 C#3 Eb3 A3 G#3 C3 Bb3 F#3 G3 B3 D3 E3}

Note that these melodies do not contain the same number of notes, but they will have the same duration (2 beats) because of their polymetric structures.

We want Alan's performance to start precisely after the last note of Beatrix's performance. As we don't want E4 to overlap with F3, we have put a silence '-' before F3. In the following, we'll have a solution to overcome this limitation.

To manage the interaction in MacOS, we need an additional IAC port which is (automatically) named "Bus2". To do this, open Audi MIDI Setup and click on the IAC driver. Then add a port (see picture). You can create as many ports as you wish.

Set both Beatrix's MIDI output and Alan's MIDI input to "Bus2".

Now we want to hear both performances. Alan's MIDI output is sent to "Bus1" and will therefore be audible on the Pianoteq synthesiser.

Windows and Linux users can connect the two performers more easily: send both Beatrix's and Alan's messages to the external MIDI device, and connect Alan's input to the same MIDI device. But… the input filter should receive events and not forward them to the output, which is the same device, otherwise the loop will produce a disastrous bouncing effect!

Back to MacOS, there are two ways to send Beatrix's performance to the Pianoteq synthesiser:

  • Pianoteq settings make it possible to listen to both "Bus1" and "Bus2" virtual ports.
  • You can set up the MIDI event filter on Alan's project to route input NoteOn/NoteOff events to the current MIDI output. See above for filters.

To play the performance, click PLAY on Alan's project so that it is ready to perform. Then click PLAY on Beatrix's project. This is the result:

Two phrases played in sequence by two instances of the BP3 (Beatrix & Alan).

No doubt this sounds rather unmusical! In fact, we publish tasteless technical examples to encourage musicians to compose interesting pieces! 😀

Using out-time inaudible notes as signals

The idea of beginning Alan's performance with a silence that is filled by Beatrix's final note E4 is unelegant. Below is a better solution:

-se.Beatrix
{2, C#4 Eb4 A4 G#4 C4 Bb4 F#4 G4 B4 D4 E4} _vel(0) <<C0>>

-se.Alan
_script(wait for C0 channel 1) {2, - F3 C#3 Eb3 A3 G#3 C3 Bb3 F#3 G3 B3 D3 E3}

The secret is the expression "_vel(0) <<C0>>" which is an out-time expression of note C0 with velocity zero. The velocity ensures that the note won't be heard, and the out-time property gives it a null duration. Any note can be used here provided that it is mentionend in the "_script(wait…)" instruction.

If "_vel(0) <<C0>>" is followed by more notes, it is necessary to reset the velocity to its default value. The solution is to write it between curly brackets, so that _vel(0) only applies to the content of the expression: "{_vel(0) <<C0>>}"

Checking the time accuracy

Let us check that the real-time synchronisation is not affected by delays. We'll now ask Alan and Beatrix to play the same piece of music (one octave apart) in the same time.

This time, Alan will start:

_transpose(12) _vel(60) E2 • D2 E2 • _vel(65) B2 D2 E2 • G2 B2 D2 E2 • _vel(70) F#2 G2 B2 D2 E2 • Bb2 F#2 G2 B2 D2 E2 • _vel(75) C2 Bb2 F#2 G2 B2 D2 E2 • _vel(77) G#2 C2 Bb2 F#2 G2 B2 D2 E2 • _vel(80) A2 G#2 C2 Bb2 F#2 G2 B2 D2 E2 • _vel(85) Eb2 A2 G#2 C2 Bb2 F#2 G2 B2 D2 E2 • _vel(87) C#2 Eb2 A2 G#2 C2 Bb2 F#2 G2 B2 D2 E2 • _vel(90) F2 C#2 Eb2 A2 G#2 C2 Bb2 F#2 G2 B2 D2 E2

In MacOS, Alan will send MIDI events to "Bus2". Beatrix will listen to "Bus2" and send MIDI events to "Bus1" (Pianoteq). Beatrix will set her input filter to the pass option, routing the incoming events to the output. The Pianoteq synthesiser will be set to listen to "Bus1" only.

This is the score of Beatrix:

_script(wait for E3 channel 1) _transpose(24) _vel(60) E2 • D2 E2 • _vel(65) B2 D2 E2 • G2 B2 D2 E2 • _vel(70) F#2 G2 B2 D2 E2 • Bb2 F#2 G2 B2 D2 E2 • _vel(75) C2 Bb2 F#2 G2 B2 D2 E2 • _vel(77) G#2 C2 Bb2 F#2 G2 B2 D2 E2 • _vel(80) A2 G#2 C2 Bb2 F#2 G2 B2 D2 E2 • _vel(85) Eb2 A2 G#2 C2 Bb2 F#2 G2 B2 D2 E2 • _vel(87) C#2 Eb2 A2 G#2 C2 Bb2 F#2 G2 B2 D2 E2 • _vel(90) F2 C#2 Eb2 A2 G#2 C2 Bb2 F#2 G2 B2 D2 E2

To start the performance, first click on the PLAY button of Beatrix's project, then on the PLAY button of Alan's project.

Have you noticed that Beatrix is waiting for E3, which does not appear in Alan's score? Oh yes, it does! There is a _transpose(12) command that changes E2 (the first note) to E3. So, it works. This is the performance:

Two superimposed performances (one octave apart), played by two instances of the BP3 (Beatrix & Alan).

Not too bad? Despite the lack of musical interest, we must admit that the superimposition is technically acceptable, even if it is not perfect: there is a delay of about 60 milliseconds on the first note, the time it takes Beatrix's machine to detect that it has received a NoteOn for C3. The subsequent notes are programmed to compensate for this delay, but there are still discrepancies (which can be quantified on the Pianoteq MIDI input). They seem to be caused by delays outside BP3.

You can adjust the delay in Beatrix's project settings "-se.Beatrix". There is a parameter called "Sync delay", which is the number of milliseconds Beatrix's output events should be postponed after the synchronisation. We currently find that 380 ms is a good value.

In fact, the superimposition would sound even better if both performances were triggered by the same event, such as the conductor pressing a key on the external keyboard. This exercise was only intended to show that synchronisation between "virtual musicians" works well.

Working with multiple MIDI inputs

In the previous example, we could decide that Alan's performance will start when he receives a particular note from the Pocket Key 25 keyboard. In this case, we need to click on both START buttons, putting both 'musicians' in wait mode, and the performance will not start until the correct key is pressed on the keyboard.

This case is manageable with single inputs on each instance of BP3. More complicated cases, however, require external 'actors', such as a Pocket Key 25 keyboard sending to all the 'musicians' synchronisation messages, or messages modifying parameters in grammars, changing the metronome value, etc.

To achieve this, the Bol Processor is a able to manage multiple MIDI inputs.

The new game is as follows: both Beatrix and Alan will take turns playing variations of Mozart's musical dice game (see '-gr.Mozart'), one octave apart. They will use the Improvise mode to continue throwing the dice and creating unheard variations. But they will wait for a signal from the other to start playing a new variation.

In short, both musicians will use the same grammar, with only a small change for mutual synchronisation. Their settings must be carefully adjusted:

  • Select Italian/Spanish/French as a note convention
  • Check Non-stop improvise
  • Adjust Pclock = 3 and Qclock = 11 to get the same metronome speed of 220 bpm
  • Set Sync delay to 380 ms
  • We don't want both musicians to repeat the same variations. So, set the Seed for randomization to different values, for instance '1' and '2'. Or set it to zero to instruct the machine to seed the random sequence with an arbitrary number of its choice.

In the current version of BP3, the easiest way to send a signal is to send a note with a velocity of zero, which will therefore go unheard. So we need to change the grammar to add these particular notes.

In fact, the same notes should never be part of the score, so that the signal is really sent at the end. This is easy with Mozart's game, for example we can use C# (do#) for the synchronisation. Below are the tops of the grammars used by Beatrix and Alan.

Beatrix '-gr.Beatrix':

-se.Beatrix
ORD
gram#1[1] S --> _script(wait for do#3 channel 1) _vel(80) A B _vel(0) do#2
gram#1[2] A --> A1 A2 A3 A4 A5 A6 A7 A8 A1 A2 A3 A4 A5 A6 A7 A'8
gram#1[3] B --> B1 B2 B3 B4 B5 B6 B7 B8 B1 B2 B3 B4 B5 B6 B7 B8
-------------------
LIN [Select rules randomly and apply from left to right]
etc.

Alan's '-gr.Alan':

-se.Alan
ORD
gram#1[1] S --> _script(wait for do#2 channel 1) _vel(80) _transpose(-12) A B _vel(0) do#4
gram#1[2] A --> A1 A2 A3 A4 A5 A6 A7 A8 A1 A2 A3 A4 A5 A6 A7 A'8
gram#1[3] B --> B1 B2 B3 B4 B5 B6 B7 B8 B1 B2 B3 B4 B5 B6 B7 B8
-------------------
LIN [Select rules randomly and apply from left to right]
etc.

Again, we put do#4 at the end of Alan's performance because it is played as do#3 (one octave lower) due to the _transpose(-12) instruction.

Now we need to set up the MIDI inputs and outputs. Beatrix will send events to "Bus 1" which is the Pianoteq synthesizer. She will receive events from "Bus 2", use them for synchronisation, and forward them to the output.

Alan will send events to "Bus 2" and listen to "Bus 1" for the synchronisation.

This is all perfect on paper, but who is going to start? We have created a chicken and egg situation, so we need a superpower to start the process! Actually, a real human pressing the do#2 key on a Pocket Key 25 keyboard will do.

The interface has a Add an input button. We click it on Alan's project and paste the name Pocket Key 25. We also use the comment fields to remember the use of each port:

MIDI inputs and outputs are identified by their names. The numbers in the leftmost boxes are automatically set by the console during performance.

To start the concert, we'll click START on both projects. The order is irrelevant. Then we'll press a key on the Pocket Key 25. Which key?

If we press the do#2 key, we will certainly trigger Alan's improvisation and the cycle will start. But if we press the do#3 key, nothing will happen because, by default, the filter of the Pocket Key 25's input in "-gr.Alan" does not transmit NoteOns to the output. So Beatrix won't hear it… By setting NoteOn to status '2' (treat and pass) on this filter, it is possible to decide who will start the performance: do#2 for Alan and do#3 for Beatrix.

Here we go (starting with Alan):

Alan and Beatrix (two instances of BP3) playing variations of Mozart's musical dice game

You can interrupt this dual performance at any time by clicking the Pause button (of either project) and resume it by clicking the Continue button. You can also insert musical fragments by running a project such as "-da.tryStopContinue" (see below), if Stop and Continue MIDI events are sent to both "Bus 1" and "Bus 2" ports. In both cases the synchronisation is maintained.

👉  This simple show should convince musicians to create "virtual bands" of BP3s playing different grammars and sending specific synchronisation signals according to which variation has just been produced. Along with human performers who join in the fun!

The "virtual musicians" can be on the same computer or remotely connected via network (BlueTooth) or MIDI cables and USB interfaces. If they are on the same computer, they can be run on different browsers and/or the same browser. In the latter case, BP3 will not allow the same project (grammar or data) to be run in the same session. In general, just like human musicians in an orchestra have individual scores, it makes sense that virtual musicians don't share the same project…

The number of MIDI inputs and outputs in a project is currently limited to 32. It is unlikely that a (human) musician will need more!

Synchronise to a sequence (or list) of notes

The following expression

gram#1[1] S --> _script(wait for C3 channel 1) - _script(wait for E3 channel 1) etc.

synchronises the production to the sequence of notes C3 E3 (whatever the duration and velocity). This creates interesting situations where a "virtual musician" is expected to start playing after receiving a signal (C3) from one partner, then a signal (E3) from another partner.

Note that there is a silence '-' between the two script instructions. If there is no silence, then BP3 will resume playing if either C3 or E3 has been received.

Remember that because of the MIDI channel specification (range 1 to 16), the detection of signals can be selective. They are also inaudible when transmitted by NoteOns with velocity zero.

Crashing the band!

In the example of Alan & Beatrix playing Mozart, the connection seems to create a loop: Beatrix sends events to Pianoteq and Alan ("Bus 1"), who in turn sends events to Beatrix ("Bus 2"). Isn't that dangerous?

The reason it doesn't crash is that Alan's input "fromBeatrix" ("Bus 1") is filtered: NoteOns are received and processed (for synchronisation), but not passed to the output ("Bus 2"), i.e. to Beatrix. You can try to change the filter of input "Bus 1" on Alan's project, setting NoteOns to status '2' (treat + pass): you will get a superb crash after a flood of notes!
➡  This probably will not happen because BP3's MIDI drivers have been equipped with an anti-bouncing mechanism.

Working with multiple MIDI outputs

The Bol Processor currently accepts up to 32 MIDI inputs and outputs.

Example of a project using two inputs and two outputs:

A setup with two MIDI outputs and two MIDI inputs

The procedure for adding outputs is the same as the one for adding inputs: click on the Add an output button, then enter the name of the MIDI device if you know it exactly, otherwise leave it blank and let the machine connect it by default to the next available output, while suggesting other options:

🎹 Your real-time MIDI settings:
MIDI output = 0: “Bus 1” - Pianoteq
MIDI output = 3: “USB MIDI Interface” - Yamaha piano
MIDI input = 1: “Bus 2” - from Alex
MIDI input = 2: “Pocket Key 25” - a small keyboard

🎹 Setting up MacOS MIDI system
MIDI output = 0: “Bus 1” 👉 the name of your choice
MIDI output = 3: “USB MIDI Interface” 👉 the name of your choice
MIDI input = 1: “Bus 2” 👉 the name of your choice
MIDI input 1 makes BP3 interactive
MIDI input = 2: “Pocket Key 25” 👉 the name of your choice
MIDI input 2 makes BP3 interactive

🎶 More MIDI output options were available:
MIDI output = 1: “Bus 2”
MIDI output = 2: “Pocket Key 25”

🎶 More MIDI input options were available:
MIDI input = 0: “Bus 1”
MIDI input = 3: “USB MIDI Interface”

The fact that a MIDI input or output is "available" does not guarantee that it will do what we want it to do. For example, sending MIDI messages to the Pocket Key 25 keyboard will actually do nothing.

Filtering MIDI outputs

In the example above, MIDI output 3 (the Yamaha piano connected to the USB MIDI Interface) has the filter shown on this picture.

The channel filter specifies that the Yamaha piano will get all MIDI channels except those emitted on MIDI channel 2. Filtering MIDI channels makes it possible to send events exclusively to different instruments.

MIDI events can also be filtered by type. The idea is the same as for MIDI input filters, see above.

Filtering on parts has been explained above.

👉  If you do not hear any sound in real-time MIDI, you may consider checking the output MIDI filters before you kick the piano or screw up its cables!  😀

Using standard MIDI control

MIDI has standard control messages, namely Start, Continue and Stop, which can be used to coordinate multiple "virtual musicians" (instances of BP3). The advantage is the clarity of the data and the grammars programmed for interactions. The disadvantage is that these messages are not assigned to specific MIDI channels. This can be a problem with a large number of "virtual musicians". They also introduce a delay of about 250 milliseconds due to the time it takes for the MIDI device to process them.

These MIDI controls are the only ones that you can associate with a real-time Csound performance, since notes sent to the Csound server cannot be received by another instance of the Bol Processor.

Control messages can be used in two ways. The first, which we will now examine, is to "listen" to these messages via a script command. The second (see next section) is to respond to them as interruptions in the performance.

Let us look at a trivial example (of no musical interest), again with Beatrix and Alan playing together. This time, they take turns playing their items (simple sequences of notes).

It is important that the "Respond to Stop/Continue" option is unchecked in both the Alan and Beatrix project settings.

Alan will start first. Below is the data:

E3 D3 C3 _script(MIDI send Start) _script(wait for Continue) A2 B2 C3

In short, he will play three notes (E3 D3 C3), then send a START to Beatrix and wait for a CONTINUE from Beatrix, then on receipt play the final three notes A2 B2 C3.

This is Beatrix's data:

_script(wait for Start) E4 D4 C4 _script(MIDI send Continue)

Start and Continue should be "heard" by Beatrix. Automatically, Stop will have the same status.

Beatrix's project sends its output to "Bus 1", the Pianoteq synthesizer. Its input is connected to the virtual port "Bus 2".

Alan's project sends its output to the virtual port "Bus 2", and its input is connected to "Bus 1".

We'll start the performance with Beatrix. Her machine will stay waiting for START. Then we'll start Alan's part, which will play three notes, then send a START message to Beatrix, who will play her part, and return to Alan, via a CONTINUE message, for the final part…

This all sounds logical, but it doesn't work! We do hear Alan's E3 D3 C3, but then nothing… The first reason is that Beatrix should be able to hear Alan's START command, which is no longer a NoteOn as in the previous examples. This means that the filter of her input "Bus 2", from which she receives Alan's MIDI messages, must be set correctly. The Start event should be received, see picture on the side. Also remember that NoteOn and NoteOff should be received and transmitted to the output (the Pianoteq synth).

Well, now we hear Alan's E3 D3 C3 followed with Beatrix's E4 D4 C4, but then… nothing! 😢

Careful analysis is needed to solve the problem. However, this is simple logic. Remember that Alan is playing on "Bus 2", which is not connected to any MIDI device. If we hear Alan's production, it is because it is received by Beatrix on "Bus 2" and then forwarded to "Bus 1" (the Pianoteq synth). The problem is that the final part A2 B2 C3 is sent to Beatrix, but she has already stopped listening because her own data is finished!

You can imagine a band in which one musician plays an improvisation and then gives a signal to another musician to start their own improvisation, but the careless musician has already vacated the place believing that the programme was finished. The solution is to tell the musicians not to go away until they receive a STOP signal. Maybe a signal from a conductor (here using the Pocket Key 25 keyboard), maybe a signal from the musician who is in charge of ending the performance. So we'll tell Alan to send a STOP signal at the end of his performance, and Beatrix to wait for Alan's STOP signal. Below are the revised scores.

// Alan
E3 D3 C3 _script(MIDI send Start) _script(wait for Continue) A2 B2 C3 _script(MIDI send Stop)

// Beatrix
_script(wait for Start) E4 D4 C4 _script(MIDI send Continue) _script(wait for Stop)

The MIDI messages Start, Continue, Stop have been used here to facilitate the reading of scores (or grammars), but these can be replaced by NoteOns with durations and velocities zero on different MIDI channels when working with a larger number of actors.

By the way, using MIDI messages Start, Continue and Stop can be problematic with physical or virtual MIDI devices. My Yamaha piano, for example, does not transmit these messages. So, when connected to an input, it will only send 3-byte messages such as NoteOn/NoteOffs. In the Windows environment, the Microsoft GS Wavetable Synth also does not transmit any message at all. The best way to exchange messages is via virtual MIDI ports created by "loopMIDI" (see below). In Linux, virtual ports such as 'VirMIDI 0-0' (see below) do not seem to transmit these Start, Continue and Stop messages.

For geeks: In the Bol processor, scripts are appended to the next following sound object. For example, _script(wait for Start) is appended to note E4 in Beatrix's score. But what about scripts at the end of a score? The secret is that BP3 creates an invisible MIDI event (ActiveSensing) at the end of each element to which it can append the final scripts.

Using control messages to interrupt a performance

This is the second way of using control messages. Let's try to stop a performance with a standard Stop MIDI message, play a few notes and then continue the performance with a standard Continue MIDI message.

We create a project "-da.tryStopContinue" as follows:

-se.tryStopContinue
_script(MIDI send Stop) _tempo(2) A2 G#2 C2 Bb2 F#2 G2 B2 _script(MIDI send Continue)

Running this project will interrupt a performance, for example "-da.Watch_What_Happens_by_Oscar_Peterson". For this to work, the "Respond to Stop/Continue" option must be checked in the Oscar Peterson data project.

Let us decide that the performance will be played on the Pianoteq synthesiser connected to "Bus 1" on a Mac.

The "-da.tryStopContinue" will send its data to a virtual port, for instance "Bus 2". To this effect, the "-da.Watch_What_Happens_by_Oscar_Peterson" project will have an input port listening to "Bus 2", accepting all messages and transmitting at least NoteOn/Noteoff.

Another solution is to set up another output port on the "-da.tryStopContinue" project, that sends all data to "Bus 1". The output to "Bus 2" will only send the Stop and Continue messages. The input port listening to "Bus 2" on the Oscar Peterson project will only accept Stop and Continue, and not transmit anything.

👉  It is a good practice to use the filters for restriction of message management to the necessary tasks.

Now, play the Oscar Peterson performance, and at any point start "-da.tryStopContinue". The performance will pause and you will hear the external musical phrase. The performance will then resume. You can, of course, do this several times and insert any variety of musical fragments. For example:

Beginning of Oscar Peterson's "Watch What Happens" with two interruptions

List of scripts for dealing with real-time MIDI

The list below will be kept up to date as there are many scripts on the agenda. These instructions are not case-sensitive.

Scripts tagged [Csound] also work in real-time Csound.

Input scripts

When a note is specified, be sure to use the same note convention as in the project, e.g. C3 or do2 or sa3, etc.

Beware that interactions are limited in Csound because MIDI notes are not transmitted. Therefore, the machine will only react to instructions Start, Continue, Stop, Wait forever, Hold for, and Wait for note if the note has been sent by another MIDI device.

Wait for note channel cWait for a NoteOn of the specified note on channel c (1…16) [Csound]
Wait for StartWait for a Start MIDI message (250) [Csound]
Wait for ContinueWait for a Continue MIDI message (251) [Csound]
Wait for StopWait for a Stop MIDI message (252) [Csound]
Wait foreverWait until STOP or PANIC button is clicked [Csound]
Velocity param Kx = note channel cSet parameter Kx (0 < x < 128) to the velocity (range 0…127) of the specific note on channel c (1…16)
Control param Kx = #y channel cSet parameter Kx (0 < x < 128) to the value (range 0…127) of MIDI controller #y (0 < y < 128) on channel c (1…16)

Output scripts

Hold for x millisecondsDelay all subsequent events by the specified duration x (integer). [Csound]
Send StartSend Start MIDI message (250) [Csound]
Send ContinueSend Continue MIDI message (251) [Csound]
Send StopSend Stop MIDI message (252) [Csound]

Scripts on top of a grammar

(To be continued)

Capture incoming events

The _capture() command allows incoming MIDI events to be recorded to a 'capture' text file. See the Capture MIDI input page for explanations.

Alternatives to IAC

Here are the equivalents of Apple's IAC (Inter-Application Communication) for each system:

Windows environment

On Windows, you can use software like loopMIDI or virtualMIDISynth to create virtual MIDI ports. These tools work similarly to the IAC Driver on macOS:

These tools integrate with software applications that support MIDI, providing a seamless way to connect various MIDI applications without needing external MIDI hardware.

Linux environment

On Linux, ALSA (Advanced Linux Sound Architecture) provides capabilities to create virtual MIDI devices through its sequencing API.

  • snd-virmidi: This ALSA MIDI driver provides virtual MIDI ports for sending and receiving MIDI between applications running on the same system. It's part of the standard ALSA module set and can be configured to provide multiple ports.

To set up virtual MIDI ports on Linux using ALSA, you typically need to load the snd-virmidi module. You can do this by running:

sudo modprobe snd-virmidi midi_devs=2

This command loads the snd-virmidi module and sets it up to provide two virtual MIDI devices (you can increase the number of devices by changing the midi_devs parameter). The virtual ports, namely 'VirMIDI 0-0' and 'VirMIDI 0-1' , can then be accessed by MIDI applications on the Linux system. Please note that they do not appear to transmit the Start, Stop and Continue messages.

👉 This is done automatically by the "install_bp3.sh" shell script installing BP3 on Linux/Ubuntu (download here).

Live coding

Live coding is a performing art form centred on writing code and using interactive programming in an improvisational way. From March 2025, this practice is possible on all Bol Processor implementations: MacOS, Windows, Linux and the standalone application on Mac.

Practically speaking, live coding on the Bol Processor is possible when improvising continuously in real time MIDI. The project is a grammar, and the following real-time changes are currently possible:

  • Edit and save the grammar. The updated version will be immediately reloaded and restarted.
  • Edit and save the settings. The updated version will be immediately reloaded and applied to the next productions.
  • Save another grammar in the same session. It will be loaded immediately to replace the current one and production will restart. The settings (and alphabet, if applicable) will be included.

How to do it

  • Set the grammar in Non-stop improvize mode.
  • Check the Follow grammar(s) and (optional) Follow settings options at the bottom of the settings form. The Track changes option will keep a record of grammar and settings changes.
  • Do the same for all grammars used in the improvisation.
  • Click PRODUCE ITEM(s) on the first grammar.
  • During the performance, clicking SAVE on any of the grammars will save its changes, load it and run it in place of the current one. Its settings and alphabet will also be automatically included.

All the features of a real-time MIDI performance remain available: microtonal adjustments, parts, MIDI input capture, wait instructions, etc.

Example

The following is a trivial example with no musical relevance. It is intended to test the Bol Processor's technical ability to quickly handle changes in a conversational setup.

Two grammars, namely "-gr.livecode1" and "-gr.livecode2", contain a single inference from S to a polymetric expression:

-se.livecode1
S --> {_vel(64) _chan(1){4,{D6 A4 A4{1/2,G5 Bb5}{1/2,D6 C6 Bb5}, 1{Cb4,Eb4,F4,Ab4}{Bb3,D4,Gb4}-}},_vel(64) _chan(2){4,{D4 F3 E3{1/2,G3 Bb3}{1/2,D4 C4 Bb3}, 1{Db2,Ab2}{C2,G2}-}}}

-se.livecode2
S --> {_tempo(41/30) _vel(64) _chan(3){1319/240,{re4{17/120,fa1 fa2}17/1920{119/1920,la2}{17/80,do3 re3 fa3}103/40, 57/40{17/240,la3}601/240, 359/240 4, 359/240{17/120,do4 re4}17/1920{119/1920,fa4}{17/80,la4 do5 re5}499/240, 461/240{17/240,fa5}reb5{1,la3 sol#3}1/120, 479/120 1/120,{re3,sol3,la3}--{1,- fa3}}},_vel(64) _chan(4){4,{la2 -{1,- la0}reb3,{sib1,fa2}2{sol2,la2}}}}

In "-se.livecode1" and "-se.livecode2", the Non-stop improvize and Follow grammar(s) options have been checked.

First run "-gr.livecode1" which is a plain repetition of a musical motif borrowed from Oscar Peterson. Click SAVE to check that it restarts the performance. Then modify the grammar, adding or deleting notes, and click SAVE to check that the modified version will be played immediately.

If the Follow settings option is checked, you will notice that changing a setting, such as tempo, and clicking SAVE to “-se.livecode1” will be reflected in the performance. However, the effect is not immediate, as the machine must first play the items stored in the output buffer. We're working on a way to make changes immediate while maintaining the continuity of the performance.

Listen to a possible result (!):

Grammars "-gr.livecode1" and "-gr.livecode2" in a 'conversational" mode

It is worth noting that the two grammars use different note conventions, namely English in "-gr.livecode1" and Italian/French in "-gr.livecode2".

A more complex example

Let us now mix two complex grammars: "-gr.Mozart" and "-gr.trial.mohanam", which are in the "ctests" folder. Check the Non-stop improvize and Follow grammar(s) options in their settings and start improvising on one of them. Then click SAVE to change the grammar or reload the current one.

In fact, you can use more than two grammars for this exercise. You can also include a grammar that is not in the Non-stop improvize mode, so it will end the performance.

The "-gr.Mozart" grammar uses the French note convention whereas the "-gr.trial.mohanam" grammar uses the Indian convention. The latter also uses an alphabet file "-al.trial.mohanam". The merge is technically satisfactory as you can hear:

Work in progress

We're working on three limitations that seem evident in the context of live coding:

  1. React immediately to changes in the settings.
  2. Create an option to react to changes (or updates) of a grammar only at the moment when an entire element has been sent to MIDI.
  3. When loading a grammar, if the grammar is not valid, the improvisation should continue with the previous grammar.

More suggestions are sure to come as feedback from artists working with live coding. These will give way to challenging developments!

Capture MIDI input

 

Work in progress

Capturing MIDI input opens the way to "learning" from the performance of a musician or another MIDI device. The first step is to use the captured incoming NoteOn/Noteoff events, and optionally ControlChange and PitchBend events, to build a polymetric structure that reproduces the stream.

The difficulty of this task lies in the design of the most significant polymetric structure — for which AI tools may prove helpful in the future. Proper time quantization is also needed to avoid overly complicated results.

A simple form of MIDI capture involves writing down the notes sent by a connected piano keyboard — see the instructions. However, this is a monophonic input with no timing parameters.

We've made it possible to capture MIDI events while other events are playing. For example, the output stream of events can provide a framework for the timing of the composite performance. Consider, for instance, the tempo set by a bass player in a jazz improvisation.

The _capture() command

A single command is used to enable/disable a capture: _capture(x), where x (in the range 1…127) is an identifier of the 'source'. This parameter will be used later to handle different parts of the stream in different ways.

_capture(0) is the default setting: input events are not recorded.

The captured events and the events performed on them are stored in a 'capture' file in the temp_bolprocessor folder. This file is processed by the interface.

(Examples are found in the project "-da.tryCapture".)

The first step to use _capture() is to set up the MIDI input and more specifically its filter. It should at least treat NoteOn and NoteOff events. ControlChange and PitchBend messages can also be captured.

If the pass option is set (see picture), incoming events will also be heard on the output MIDI device. This is useful if the input device is a silent device.

It is possible to create several inputs connected to several sources of MIDI events, each one with its own filter settings. Read the Real-time MIDI page for more explanations.

Another important detail is the quantization setting. If we want to construct polymetric structures, it may be important to set the data to the nearest multiple of a fixed duration, typically 100 milliseconds. This can be set in the settings file "-se.tryCapture".

Simple example

Let us take a look at a very simple example of a capture on top of a performance.

C4 _D4 _capture(104) E4 F4 G4 _capture(0) A4 B4

The machine will play the sequence of notes C4 D4 E4 F4 G4 A4 B4. It will listen to the input while playing E4 F4 G4. It will record both the sequence E4 F4 G4 and the notes received from a source tagged "104".

Suppose that the sequence G3 F3 D3 was played on top of E4 F4 G4. The capture file might look like this:

It should be noted that all the dates are approximated to multiples of 100 milliseconds, i.e. the quantization. For example, the NoteOff of input note G3 falls exactly on the date 3000 ms, which is the NoteOff of the played note E4.

The recording of input and played notes starts at note E4 and ends at note G4, as specified by _capture(104) and _capture(0).

An acceptable approximation of this sequence would be the polymetric expression:

C4 D4 {E4 F4 G4, - - G3 - F3 - D3 - -} A4 B4

Approximations will be automatically created from the capture files at a later stage.

Pause and capture

MIDI input will continue to be captured and timed correctly after the Pause button is clicked. This makes it possible to play the beginning of a piece of music and stop exactly where an input is expected.

Combining 'wait' instructions

Try:

_script(wait for C3 channel 1) C4 D4 _capture(104) E4 F4 G4 _script(wait for D3 channel 1) _capture(0) A4 B4

The recording takes place during the execution of E4 F4 G4 and during the unlimited waiting time for note D3. This allows events to be recorded even when no events are being played.

The C3 and D3 notes have been used for ease of access on a simple keyboard. The dates in the capture file are not incremented by the wait times.

The following is a setup for recording an unlimited sequence of events while no event is being played. Note C0 will not be heard as it has a velocity of zero. Recording ends when the STOP or PANIC button is clicked.

_capture(65) _vel(0) C0 _script(wait forever) C0

Interpreting the recorded input as a polymetric structure will be made more complex by the fact that no rhythmic reference has been provided.

Microtonal corrections

In the following example, both input and output receive microtonal corrections of the just intonation scale.

_scale(just intonation,0) C4 D4 _capture(104) E4 F4 G4 A4 _capture(0) B4

Below is a capture file obtained by entering G3 F3 D3 over the sequence E4 F4 G4 A4. Note the pitchbend corrections in the last column, which indicate microtonal adjustments. The relevant values are those that precede NoteOns on the same channel.

The output events (source 0) are played on MIDI channel 2, and the input events (source 104) on MIDI channel 1. More channels will be used if output notes have an overlap — see the page MIDI microtonality. In this way, pitchbend commands and the notes they address are distributed across different channels.

Added pitchbend

In the following example, a pitchbend correction of +100 cents is applied to the entire piece. It does modify output events, but it has no effect on input events.

_pitchrange(200) _pitchbend(+100) _scale(just intonation,0) C4 D4 _capture(104) E4 F4 G4 A4 _capture(0) B4

Again, after playing G3 F3 D3 over the sequence E4 F4 G4 A4:

Pitchbend corrections applied to the input (source 104) are only those induced by the microtonal scale. Pitchbend corrections applied to the output (source 0) are the combination of microtonal adjustments (see previous example) and the +100 cents of the pitchbend command.

Capturing and recording more events

The _capture() command allows you to capture most types of MIDI events: all 3-byte types, and the 2-byte type Channel pressure (also called Aftertouch).

Below is a (completely unmusical) example of capturing different messages.

The capture will take place in a project called "-da.tryReceive":

_script(wait for C0 channel 1) _capture(111) D4 _pitchrange(200) _pitchbend(+50) _press(35) _mod(42) D4 _script(wait forever)

The recording will have a "111" marker to indicate which events have been received. Only two notes D4 are played during the recording, the second one is raised by 50 cents and has a channel pressure of 35 and a modulation of 42.

After playing back the two D4s, the machine will wait until the STOP button is clicked. This gives the other machine time to send its own data and have it recorded.

The second machine is another instance of BP3 — actually another tag on the interface's browser with the "-da.trySend" project:

{_vel(0) <<C0>>} G3 _press(69) _mod(5430) D3 _pitchrange(200) _pitchbend(+100) E3

This project will start by sending "{_vel(0) <<C0>>} " which is the note C0 with velocity 0 and duration null (an out-time object). This will trigger "-da.tryReceive" which waited for C0. The curly brackets {} restrict velocity 0 to the note C0. Outside of this expression, the velocities are set to their default value (64). In this data, channel pressure, modulation and a pitchbend correction of +100 cents are applied to the final note E3.

The resulting sound is terrible, you've been warned:

Performed and captured events. Forget the way it sounds and look at the file below!

However, the 'capture' file shows that all events have been correctly recorded:

Captured events (from "-da.trySend") are coloured red. They have been automatically assigned to MIDI channel 2, so that corrections will not be mixed between performance and reception.

The pitchbend corrections are shown in the "cents correction" column, each applied to its own channel.

The channel pressure corrections (coloured blue) display the expected values. The modulation corrections (in the range 0 to 16383) are divided into two 3-byte messages, the first carrying the MSB and the second the LSB.

There is a time mismatch of approximately 150 milliseconds between the expected and actual dates, but the durations of the notes are accurate. The mismatch is caused by the delay in the transmission of events over the virtual port. The data looks better if the quantization in the "-da.tryReceive" project is set to 100 ms instead of 10 ms. However, this is of minor importance, as a "normalisation" will take place during the (forthcoming) analysis of the "capture" file.

For geeks: The last column indicates where events have been recorded in the procedure sendMIDIEvent(), file MIDIdriver.c.

Capture events without the need to perform

The setup of the "-da.tryReceive" project should be for example:

_capture(99) _vel(0) C4 _script(wait forever)

Note C4 is not heard due to its velocity 0. It is followed with all MIDI events received by the input until the STOP button is clicked. This makes it possible to record an entire performance. The procedure can also be checked with items produced and performed by the Bol Processor.

Note that the inclusion of pitchbend messages makes it possible to record music played on microtonal scales and (hopefully) identify the closest tunings suitable for reproduction of the piece of music.

For example, try to capture the following phrase from Oscar Peterson's Watch What Happens:

_tempo(2) {4, {{2, F5 Bb4 C5 C4} {3/2, D4} {1/2, Eb4 Db4 D4}, {F4, C5} {1/2, F4, G4} {1/2, F3, G3} {2, Gb3}}}, {4, {C4 {1, Bb3 Bb2} {2, A2}, {D3, A3} {1, Eb3 Eb2} {2, D2}}}

This phrase was imported from a MusicXML score (read page)

A dialogue box appears to analyse or download the captured data:

The result is complex, but it's going to lend itself to being analysed automatically.

The raw table is available to download (as an image) here. The current analysis of this example is available here.

Once the origin of dates has been set to the first NoteOn or NoteOff received, the result is — only important columns are displayed: time, note, event, key, velocity:

This pianoroll represents the input data produced by playing Oscar Peterson's phrase on the Bol Processor (file -da.trySend). The captured data is slightly different in the last part.

0 F5 NoteOn 77 64
0 F4 NoteOn 65 64
0 C5 NoteOn 72 64
0 C4 NoteOn 60 64
0 D3 NoteOn 50 64
0 A3 NoteOn 57 64
230 F5 NoteOff 77 0
230 Bb4 NoteOn 70 64
470 Bb4 NoteOff 70 0
470 C5 NoteOff 72 0
470 C5 NoteOn 72 64
470 F4 NoteOff 65 0
470 C5 NoteOff 72 0
470 F4 NoteOn 65 64
470 G4 NoteOn 67 64
470 C4 NoteOff 60 0
470 Bb3 NoteOn 58 64
470 D3 NoteOff 50 0
470 A3 NoteOff 57 0
470 Eb3 NoteOn 51 64
710 C5 NoteOff 72 0
710 C4 NoteOn 60 64
710 F4 NoteOff 65 0
710 G4 NoteOff 67 0
710 F3 NoteOn 53 64
710 G3 NoteOn 55 64
710 Bb3 NoteOff 58 0
710 Bb2 NoteOn 46 64
710 Eb3 NoteOff 51 0
710 Eb2 NoteOn 39 64
960 C4 NoteOff 60 0
960 D4 NoteOn 62 64
960 F3 NoteOff 53 0
960 G3 NoteOff 55 0
960 F#3 NoteOn 54 64
960 Bb2 NoteOff 46 0
960 A2 NoteOn 45 64
960 Eb2 NoteOff 39 0
960 D2 NoteOn 38 64
1730 D4 NoteOff 62 0
1730 Eb4 NoteOn 63 64
1830 Eb4 NoteOff 63 0
1830 C#4 NoteOn 61 64
1880 C#4 NoteOff 61 0
1880 D4 NoteOn 62 64
1960 D4 NoteOff 62 0
1960 F#3 NoteOff 54 0
1960 A2 NoteOff 45 0
1960 D2 NoteOff 38 0

The next task on our agenda will be to analyse the 'captured' file and reconstruct the original polymetric expression (shown above) or an equivalent version. Follow current AI-recognition work. Then we can consider moving on to grammars, similar to what we've done with imported MusicXML scores (read page).

Constructing a polymetric expression from an arbitrary stream of notes will hopefully be achieved with the help of graph neural networks. The idea is to train the language model with a set of sound files, MIDI files, and their 'translations' as polymetric expressions. Read the AI recognition of polymetric notation page to follow this approach.

« Translate