Author: Bernard Bel

 

Notes can be entered from a connected MIDI device to the Grammar or Data projects of BolProcessor BP3. This is a monophonic input with no timing parameters. For time-tagged MIDI capture, refer to this page.

If you are having problems connecting to MIDI devices, check out the instructions on the Real-time MIDI page. We will remind you of the minimum procedure here. The same applies to Grammar and Data projects.

Let us assume that you connected a MIDI piano keyboard to a USB port of the machine running BP3. My keyboard's name is "Pocket Key 25". The only step required is to declare the keyboard as an input device.

Select Real-time MIDI and click SAVE format. Then click Add an input and SAVE MIDI ports. You will then see the following:

This means that BP3 will attempt to connect to the next available MIDI input and output. Every system has its own built-in MIDI devices, including those that you are currently running on the same machine. Therefore, it is unlikely that your external keyboard will be selected immediately. Never mind; click on the location in your data or grammar where you want to write notes, then click the MIDI enter notes button, which should now be visible since you declared a MIDI input. A process window will appear:

If the notes you play on the MIDI keyboard are immediately displayed on your Data or Grammar page, it means that BP3 has found your external keyboard. This is generally not the case. Clicking STOP will take you here:

On this example you can see that the selected input was "Bus 2", which is not the external keyboard. Click Show process to read details.

This the MacOS version; the others are very similar:

Bol Processor console app
Version 3.4.5 (May 23 2026 - 16:16:40)
Reading MIDI port settings: ../temp_bolprocessor/trace_f51acb6321_-da.tryEnterNotes_midiport
🎹 Your real-time MIDI settings:
MIDI output = 0: “new output” -
MIDI input = -1: “new input” -
🎹 Setting up MacOS MIDI system
Trying to assign ports to 1 output(s) without names but possibly with numbers
MIDI output = 0: “Bus 1” 👉 the number of your choice
Trying to assign ports to 1 input(s) without names but possibly with numbers
MIDI input = 1: “Bus 2” 👉 choice by default
MIDI input 1 makes BP3 interactive
🎶 More MIDI output options are available:
MIDI output = 1: “Bus 2”
MIDI output = 2: “Pocket Key 25”
🎶 More MIDI input options are available:
MIDI input = 0: “Bus 1”
MIDI input = 2: “Pocket Key 25”
MIDI settings saved to ../temp_bolprocessor/trace_f51acb6321_-da.tryEnterNotes_midiport
🎹 Name(s) of MIDI input or/and output changed and will be updated when saving the page of your project

BP3 selected "Bus 2" as the input, but it also says that "Pocket Key 25” is another option. This is the one we need. Let us copy and paste the name, then SAVE MIDI ports:

Now, clicking again the MIDI enter notes button will effectively type the names of notes played (in sequence) on the keyboard.

👉 For now, chords will be interpreted as note sequences rather than polymetric constructions. This could be addressed later.

If it still does not work, check the input filter:

NoteOn should be at least in column 1 (treat). If NoteOn and NoteOff are in column 2 (transmit), the notes will also be heard on the MIDI output device.

You can change the note convention from English to Italian/French or to Indian. Open your project settings and select the desired convention.

Note that you can use the computer keyboard mapping and MIDI keyboard capture of notes at the same time.

For geeks: As of 26 July 2026 (v3.4.7), we switched from SSE design to a polling design, which makes it compatible with Apache, FastCGI, CGI, MAMP, XAMPP, and PHP Desktop.

 

The first version of the Bol Processor (BP1) was an advanced word processor designed to store text representations of musical variations created by Indian drum players. This work required a mapping system that could associate a word with a single key on the computer keyboard.

Although the keyboard mapping was relatively easy to program in the Apple II's 6502 assembly language, its implementation in BP2 and then BP3 was delayed until version 3.4.4 (May 2026). Thus, after many years, the entire technical environment for the study of drum improvisation has been revived, alongside the restored procedures of item parsing and learning rule weights from examples.

Checking keyboard mapping on BP3

Open "-gr.dhadhatite" (in the latest distribution of the "ctests" folder). On top of the grammar you can read:

-se.dhadhatite
-al.dhadhatite
-wg.dhadhatite
-da.dhadhatite
-kb.dhadhatite

The "-kb.dhadhatite" declaration points to a file containing the keyboard mapping. If this file is present in the "ctests" folder, a button appears below the grammar to open it. If no file with this name is found, you will be offered the option of creating a new one.

The display is self-explanatory:

The mapping involves the association of alphabetic keys with any word (or sequence of words). Here we use only the terminal alphabet of the grammar — 'bols' in the language of drum players. But we could map other keys to frequently used variables or expressions in this work environment.

When working with the tabla, Jim Kippen managed to position the words at keyboard locations that facilitated typing at the same speed at which he would play them on the tabla. For this reason, "dha" is located at the far left.

To activate the mapping, click on the desired position in the grammar and then press the 'Escape' key. Now, if you type "qcqqcrd" — or "QcQQCrd", etc. — you will get:

dhatitedhadhatitedheena

This works in MacOS, Windows and Linux environments.

Alphabetic keys that are not mapped, such as 'A' and 'N', will be inactive. Non-alphabetic keys, such as digits and typographic symbols, will function as normal.

If the "Add space after each word" option is checked, and the mapping is saved, typing "qcqqcrd" will produce:

dha tite dha dha tite dhee na

The Bol Processor can segment strings without spaces if they are made of words from the terminal alphabet, here defined in "-al.dhadhatite". However, adding spaces makes it easier to read for untrained humans.

The same keyboard mapping feature can be used on a Data page such as "-da.dhadhatite".

At the bottom of the keyboard form, there is a button labelled "COPY data from this file". This allows you to select a different keyboard file to use for the settings.

 

The "learning rule weights" feature was implemented in the first version of the Bol Processor (BP1) a long time ago. It was designed in response to a problem that arose during fieldwork sessions while Jim Kippen was developing grammars capable of identifying the "language" conveyed by qa‘ida compositions of tabla. Citing Jim (Kippen & Bel, 1992):

[…] sometimes a grammar would reach a point of stagnation where computer-generated variations were judged to be neither very good nor incorrect. Consequently there was no simple way of refining or improving the model. We felt that a solution lay in attributing to each production rule a coefficient of likelihood (or weight) where the probability that certain generative paths would be chosen in preference to others could be examined.

The probabilistic model that has been implemented on the BP is derived from probabilistic grammars/automata as defined by Booth & Thompson (1973), the difference being that a weight rather than a probability is attached to every rule. The rule probability is computed as follows: if the weight is zero then the probability is zero; if the weight is positive then the inference engine calculates the sum of weights of all candidate rules, and the rule probability is the ratio of its own weight to the sum.

Rule weights can be inferred from a set of sentences (Maryanski & Booth 1977:525). The algorithm implemented in the BP is more powerful than the one devised by Maryanski and Booth, since the latter required the choice of a sample set in which all rules had been used. Given a grammar and a subset of the language that this grammar generates (for instance a sample sequence taken from a performance of an expert musician), rule weights may be inferred as follows: first, reset all weights to zero. Then, analyse every sentence and increment the weights of all rules used in the derivation by one unit.

We will demonstrate the inference of rule weights using grammars and data distributed in the ‘ctests’ folder. These grammars are real examples from field work.

Infer weights in the "-gr.dhadhatite" grammar

The ‘ctests’ folder contains both "-gr.dhahatite" and "-dha.dhahatite". The latter is a set of 15 examples designed for the demonstration. They are not of great interest because they were all produced by the grammar and will therefore be parsed successfully. However, you can try modifying details to observe failed parsing.

In addition, the first eight ones are identical, except for their notation. Items can be typed without spaces, given that the terminal alphabet "-al.dhahatite" will create a correct segmentation. They can be set out on several lines separated by single line feeds. Beats can be marked with periods (the period notation) which the polymetric algorithm interprets as identical symbolic durations (see this page).

Let us have a look at the "-gr.dhahatite" grammar which we already used for parsing variations. First create its templates as explained earlier.

Open the settings and check the LEARN option for parsing data files (see image). The option "Start from latest weights file instead of grammar" is selected by default and will be explained further.

Then add "-dha.dhahatite" and "-wg.dhahatite" in declarations on top of the grammar, and save the page.

This modifies the display of the top of the grammar:

Clicking the LEARN weights button will launch the inference of weights from the sample set in "-dha.dhahatite":

The production window shows that all 15 items were successfully parsed although
"13 item(s) failed in the parsing after matching a template"… We knew that this grammar would accept all items, so we need an explanation for the failed parsing.

Click the Show process button to get more details:

🔎 Analyzing new selection…
dhadhatitedhadhadheenadheenatitedheenateenadhadhatitedhadhadheenadhadhatitedhadhateenatatatitetatateenateenatiteteenateenadhadhatitedhadhadheenadhadhatitedhadhadheena
▶︎ Analyzing item [Single line]
👉 Item [Single line] matched template [1]
• Subgrammar 6/6
• Subgrammar 5/6
• Subgrammar 4/6
• Subgrammar 3/6
• Subgrammar 2/6
• Subgrammar 1/6
Item [Single line] matching template [1] rejected by grammar… ❌
Result of failed analysis:
S1F +V8(= S1F) +S2F *(= S1F ++A2 V6)(: S1F) S1F
👉 Item [Single line] matched template [2]
• Subgrammar 6/6
• Subgrammar 5/6
• Subgrammar 4/6
• Subgrammar 3/6
• Subgrammar 2/6
• Subgrammar 1/6
Item [Single line] matching template [2] rejected by grammar… ❌
Result of failed analysis:
(= ++S2F) +V8 S1F +S2F *(: ++S2F) *(= ++A2 V6) S1F S1F
👉 Item [Single line] matched template [3]
• Subgrammar 6/6
• Subgrammar 5/6
• Subgrammar 4/6
• Subgrammar 3/6
• Subgrammar 2/6
• Subgrammar 1/6
Item [Single line] matching template [3] accepted by grammar… ✅
Etc.

This report shows that the first item failed to match two templates but was later correctly parsed after matching template [3]. Other items displayed similar behaviour. At the top of the page, there is a link to a trace of the parsing because the "Trace production or parsing" option is selected in the settings. There is also a "Detailed trace" option available for automated analysis, which is not discussed here.

Now, move down to the bottom of the grammar and click the "-wg.dhadhatite" button. This file was created by the BP3 console upon completion of its analysis. It contains the new weights of the grammar, i.e. for each rule, the original weight (usually 100) plus the number of times the rule has been used in the parsing process.

For example, the weight of rule [1] of gram#1 would be raised from 100 to 115, since it was used for all 15 items. However, the weight of rule [5] of gram#1 would stay at 100, since it was not used in this analysis. Changes are easy to check comparing the content of "-wg.dhadhatite" with weights in the grammar.

The "-wg.dhadhatite" display includes several buttons that are easy to understand:

• The COPY current weights button will copy all the weights in the grammar to the file "-wg.dhadhatite";
• The SET rule weights and RESET rule weights buttons will set or reset all weights in the file "-wg.dhadhatite", yet not in the grammar.
• The COPY BACK rule weights button will copy weights from "-wg.dhadhatite" to the "-gr.dhadhatite" grammar. Then you are offered the option to save the grammar if these weights are correct, or to reload the grammar otherwise:

• The "-wg.dhadhatite" page also has buttons that allow you to save a copy of its weights to a new '-wg' file, or copy weights from another file.

In real life…

This "-wg.dhadhatite" layout shows how to proceed in real-life situations. You have a working Bol Processor grammar, as well as large sets of examples produced by the grammar and validated (or provided) by the expert you are working with. Proceed as follows:

1. Open the "-wg" file and RESET all weights to 0 (in that file);
2. Click the LEARN weights button;
3. Keep an eye on items that are rejected. These may indicate an incomplete grammar. They may also be incorrect. If so, make changes and go back to step 1;
4. If the option "Start from latest weights file instead of grammar" is selected, you can repeat this process with more sets of examples, as rule weights will add up.

Although rule weights can be very large integers, we find it more practical to keep them within a small range, such as 0 to 127. The 127 limit is just a convention that can be changed in the PHP interface. Currently, if the weight of a rule is not displayed, its value is 127, and vice versa.

References

Kippen, Jim, & Bernard Bel (1992) Modelling music with grammars: formal language representation in the Bol Processor. In A. Marsden & A. Pople (eds.): Computer Representations and Models in Music, London, Academic Press, 1992, p. 207-238.

Booth, T.L. and R.A. Thompson (1973). Applying Probability Measures to Abstract Languages, IEEE Transactions on Computers, Vol. C-22, n°5, p. 442-450.

Maryanski, F.J., and T.L. Booth (1977). Inference of Finite-State Probabilistic Grammars, IEEE Transactions on Computers, Vol. C-26, n°6, p. 521-536.
[Some anomalies of this paper are corrected in B.R. Gaine's paper Maryanski's Grammatical Inferencer, IEEE Transactions on Computers, Vol. C-27, n°1, 1979: 62-64]

The Bol Processor can analyse musical variations (strings of terminal symbols) using a grammar. A successful parse (membership test) indicates that the variation was produced by, or could be produced by, the grammar. This process was used extensively for modelling improvisation and composition in north Indian tabla drumming — read Jim Kippen's interview.

On this page we will introduce the real data sets "-da.dhahatite" versus "-gr.dhahatite", and "-da.dhin--" versus "-gr.dhin--". However, let us first deal with a very simple example.

A simple true Bol Processor grammar

Grammar "-gr.tryAllItems0" in the "ctests" folder:

RND
gram#1[1] S <-> (= X) X (: X)
gram#1[2] S <-> X X
-----
RND
gram#2[1] X <-> a
gram#2[2] X <-> b

This grammar is a "true Bol Processor grammar":

• It does not contain erase rules such as X --> lambda;
• Rules can be used for producing and parsing items, as indicated by the "<->" derivation sign;
• Rules do not contain any /flag/;
• Rules do not have decreasing weights such as "<K3-20>", "<100-50>", etc.;
• Rules do not contain any procedure such as "_goto", "_repeat", "_destru", etc.;
• Note, however, that rules can contain left/right contexts, including remote ones.

This can be summarised by the following, less technical yet highly relevant statement: a true Bol Processor grammar will deterministically parse every item it can produce. The "deterministic" property means that the algorithm will not backtrack if there are no available candidate rules. This is a fundamental feature of recognising formal languages that represent sets of musical variations.

When a grammar is read, the interface checks whether it is "true BP". If so, a "Create templates" button is displayed.

(For geeks)The interface uses the is_true_bp() function to check the grammar.

This grammar uses the terminal alphabet "-al.abc" which is linked to a set of sound-objects, but we won't use this feature here. Set the output mode to "BP data file".

Let us first produce all items, i.e. the language of this grammar. In the settings, check "Produce all items" in the PRODUCTION section. You can set "Max items produced" to a large number, for instance 500, as we assume that the actual size of the language is much smaller.

Click "PRODUCE". The result is:

a a a
a b a
b a b
b b b
a a
a b
b a
b b

The first four items have been created by gram#1[1], the first rule of the first subgrammar. The rule contains a pattern (=X)…(:X) meaning that the first and last parts must be identical.

The gram#1[2] rule produces two instances of variable X that are rewritten as 'a' or 'b' in gram#2. All possibilities are displayed in the last four items.

These 8 items have been copied to "-da.tryAnalyse", with additional empty lines to separate them.

Parse "-da.tryAnalyse"

In order to parse items in "-da.tryAnalyse", we need to write the name of the grammar on top of the Data page, along with links to the alphabet and settings:

-se.tryAnalyse
-al.abc
-gr.tryAllItems0

Now, "Analyse" buttons are shown for each item. However, if you click on any of these, the parsing will fail for the first four. The reason is that these should be recognised by gram#1[1], which is a pattern rule. In other words, we need to tell the engine that the pattern has been recognised.

👉  This is why we need to create templates in true BP grammars that contain pattern rules.

Create templates for "-gr.tryAllItems0"

Click the "Create templates" button, then click the "output file" link. We get:

----------
TEMPLATES:
[1] (@0 _)_(@1 )
[2] _ _
---------

In this simple grammar, the two templates represent the pattern types of each rule in gram#1. Although no theoretical knowledge of templates is required to use them, let us have a little explanation:

Template [2] contains two occurrences of '_', which means it will match any item containing two terminal symbols.

Template [1] contains the structural markers "@0" indicating a master parenthesis, and "@1" its slave copy. Note that "@0" is followed by a single "_", indicating a single terminal symbol, but "@1" is alone in its parenthesis, as it is meant to be the exact copy of its master.

Copy the templates, paste them at the bottom of grammar "-gr.tryAllItems0", then save the grammar. Note that the template button now appears as "Update templates".

Parse "-da.tryAnalyse" using the templates in "-gr.tryAllItems0"

Return to the "-da.tryAnalyse" and click the "Analyse" buttons. Now, all items are parsed successfully. The trace of the analysis of the first item "a a a" make it clear:

Analysing this item
Compiling grammar…
Compiling subgrammar #1…
Compiling subgrammar #2…
Compiling subgrammar #3…
Parsing completed
Errors: 0
Template(s) found, position 333
👉 We will try all templates, as per your settings
Analyzing selection…
Item matched template [1]
• Subgrammar 3/3
• Subgrammar 2/3
• Subgrammar 1/3
👉 Item matching template [1] accepted by grammar… ✅

Evidently, the four first items will match the "template [1]", and the four last items will match the "template [2]". The trace is self-explanatory:

Selected: gram#2[1] X <-> a
(= a) X(: a)
Selected: gram#2[1] X <-> a
(= X) X(: X)
Selected: gram#1[1] S <-> (= X) X(: X)
S

A few words about the creation of templates. Firstly, this procedure is restricted to finite languages. A grammar containing recursive rules would produce items of unrestricted length, resulting in an infinite number of templates. The Bol Processor has grammar procedures, such as "_repeat()", that tell the number of times a rule can be applied, thereby limiting the length or duration of any production. However, these procedures are currently not applicable to the parsing of items.

Secondly, at first glance, creating templates for the simple grammar amounts to producing the entire language, replacing terminals with '_', and eliminating duplicate instances. This would be impractical for most real-world grammars because, although the language is finite, it can be very large. To avoid this, the machine first determines up to which subgrammar structural rules are found. Then it explores all derivations of these subgrammars, and for each derivation it produces only one item which is converted to its template.

Structural rules are the ones that contain syntactic structures (master-slave parentheses) or/and structural markers if they are not in the contexts of the rule.

Structural markers are the glyphs '+', ':', ';', '=' and '\'. We'll see their usage in grammars for tabla compositions.

Analysis of "-da.acceleration" (without templates)

The "-gr.acceleration" grammar has no template, but you can check that it is able to parse "-da.acceleration" which it had created. Note that all rules have '<->' derivation signs instead of '-->' as in the old version.

A particular feature of this grammar is that a rule contains a period (beat marker) :

gram#1[2] A <-> E2 •

The item produced by this grammar contains periods that create the accelerating tempo:

_transpose(12) _vel(60) E2 • D2 E2 • _vel(65) B2 D2 E2 • G2 B2 D2 E2 • _vel(70) F#2 G2 B2 D2 E2 • Bb2 F#2 G2 B2 D2 E2 • _vel(75) C2 Bb2 F#2 G2 B2 D2 E2 • _vel(77) G#2 C2 Bb2 F#2 G2 B2 D2 E2 • _vel(80) A2 G#2 C2 Bb2 F#2 G2 B2 D2 E2 • _vel(85) Eb2 A2 G#2 C2 Bb2 F#2 G2 B2 D2 E2 • _vel(87) C#2 Eb2 A2 G#2 C2 Bb2 F#2 G2 B2 D2 E2 • _vel(90) F2 C#2 Eb2 A2 G#2 C2 Bb2 F#2 G2 B2 D2 E2 •

These periods are taken into account for the analysis because they have been produced by the grammar. For example, if you erase the last one, the parsing will fail.

Performance controls such as "_transpose()" and "_vel()" are ignored. The trace of the (successful) parsing ends as follows:

…
Selected: gram#1[3] B <-> D2 A
E2 . B C D E F G H I J K L
Selected: gram#1[2] A <-> E2 .
A B C D E F G H I J K L
Selected: gram#1[1] S <-> A B C D E F G H I J K L
S

Remember that "." and "•" are identical glyphs on BP3.

Rule selection criteria

To ensure the deterministic analysis is successful, several rules and selection criteria must be observed.

When multiple rules are candidates for rewriting a string W_i​, the system selects based on:

• (D1) Position: Preference for the rightmost possible derivation.
• (D2) Context Length: Preference for the rule that maximizes the fulfillment of context conditions.
• (D3) Pattern Length: "Priority to the largest aggregates of symbols." Longest patterns are recognized first.
• (D4) Order: The reverse order of appearance in the grammar.

To prevent ambiguity and "checkmate" scenarios in analysis, the following rules are applied:

• The Chunk Rule: The right-hand side of a rule f_i​ cannot be a substring of the right-hand side of a rule f_j​ where j < i.
• The Context Rule: In a LIN subgrammar, a right-hand context can only contain symbols from the subgrammar's external alphabet. Every symbol in the external alphabet of a subgrammar G_i is a terminal of a subgrammar G_j with j < i.

Explanations and proofs are found in chapter 4 of Bel's (1990) thesis: French and (bad English) versions.

Analysis of tabla compositions

Grammars "-gr.dhahatite" and "-gr.dhin--" are authentic examples of qa‘ida, the basic composition/improvisation form in the Lucknow school of tabla, as taught to Jim Kippen by Ustad Afaq Husain Khan in the early 1980s. (Read our joint paper in Anthropological Quarterly, 1989.)

The methods described here were first implemented in 1981 on an Apple IIc computer with just 64 kilobytes of memory… The Bol Processor BP1 was programmed in 6502 assembly language. Still, it was efficient enough to be used as an expert system for field research with leading exponents of the tabla.

The Bol Processor grammar concept emerged from formulating hypotheses about the structures of a qa‘idas within a teaching context. The aim was to generate as many variations as possible that would be deemed correct by an expert. However, proper identification of the language implied that the machine would be able to recognise good and bad variations submitted by experts and students alike. For this reason, each grammar should work "in reverse": given a musical variation (a string of terminals typed on the keyboard), the rules are applied from bottom to top until no rule remains applicable. If the final symbol is "S", the starting symbol, then the musical variation is assessed as "correct".

The "-gr.dhadhatite" grammar

The "dhadhatite" qa‘ida is taught to tabla beginners. This is because it is technically easy while also following complex syntactic structures, which traditional musicians refer to as "qavaid" — a term meaning "grammar" in Urdu (and Arabic). Therefore, the idea of using formal grammars to describe this compositional type was very promising.

The Bol Processor enables users to program the computer's keyboard to map words to keys instead of characters. For example, typing "q" on an English keyboard would type "dha" into the text.

This is possible in "-gr.dhahatite" and "-da.dhadhatite" because "-kb.dhadhatite" is declared at the top of the grammar or data. The mapping is effective after pressing the "escape" key — read details.

When you open the "-gr.dhahatite" grammar, you will see that it is recognised as a "true BP grammar". Click the "Create templates" button, click the "output file" link, and copy the templates at the bottom of the grammar:

TEMPLATES:
[1] ________+________(@0 ________)+________ * (@0 ________++________)(@2 )________
[2] (@0 ++________)+________________+________ * (@1 ) * (@0 ++________)________________
[3] ________(@0 ______+__)________+________ * (@0 ________) * (@2 )________________
[4] (@0 ++____________+____)________+________ * (@1 )________________
[5] (@0 ++______________+__)________+________ * (@1 )________________
[6] (@0 ++________________)________+________ * (@1 )________________

These six templates contain markers of master-slave parentheses created by subgramar #2, including markers of a homomorphism, notated "*", that modifies the content of the following parenthesis, applying a mapping defined in the "-al.dhadhatite" alphabet:

*
dha --> ta
ti
te
na
dhee --> tee
tr

The terminal alphabet of this grammar is made of sound-objects named "dha", "ta", etc. These are quasi-onomatopoeic mnemonics, in an oral notation system, that represent drum-strokes. The "*" homomorphism reflects the musical concept of replacing "open" (resonating) strokes, such as "dha" and "dhee", with their "closed" counterparts, here "ta" and "tee". So, for instance "*(dhadhatite)" should be played "tatatite". You can use any word you like instead of "*" to label the mapping.

The templates also contain structural markers "+" also produced by subgrammar #2, for example:

gram#2[1] S1F S2F S1V S2F E32 <-> S1F +S2F (= V8) +S2F * (= S1F ++ S2F) (: V8) S1F

These markers are used as (proximate) contexts in subgrammar #5, for example:

gram#5[5] ++ A2 <-> ++dheena
gram#5[6] #+ S1F <-> #+ dhadhatitedhadhadheena

In rule [5], the string "++" is a left context, but in rule [6] the string "#+" is a negative context, meaning anything but '+'.

The last rule of subrammar #5 deserves our attention:

gram#5[8] ++ S2F <-- ++ dhadhatitedhadhadheena

It is using the derivation sign "<--" instead of "<->". This means that this rule can only be used in the analysis. It is easy to guess that when the sequence of strokes "dhadhatitedhadhadheena" is found preceded by a "++" (picked up from the template), it is immediately identified as "S2F". Sequences S1F and S2F appear in typical rhythmic contexts shown in subgrammar #1, for example:

gram#1[3] S64 <-> S1V S2F S1F S2F E32

In subgrammar #2, some of these fixed units are replaced with variations:

gram#2[2] S1V S2F S1F S2F E32 <-> (=++ A1 V7 ) +S2F S1F +S2F * (:++ A1 V7 ) * (= ++ S2F ) S1F S1F

To make things easier, the analyst labelled the variables with numbers indicating their durations: 1 for A1, for example, and 7 for V7. This is not compulsory, however.

In subgrammar #3, variations are broken down into smaller units. For example:

gram#3[18] V7 <-> T1 V6
gram#3[19] V7 <-> T2 V5

Then in the following subgrammars, small units are rewritten as sequences of strokes, for instance:

gram#4[5] T2 <-> dheena
…
gram#5[2] + B4 <-> +dhadhateena

Experts familiar with this musical genre will be convinced by an in-depth analysis of this grammar and experiments of production that its construction is based on musical concepts that are embodied perfectly in the formal grammars of the Bol Processor.

There is little to say about the derivation modes of these subgrammars. Most of them could be set to "RND", but "LIN" produces equivalent output in less computation time. This was critical in the Apple II era… Subgrammar #5 is "ORD", the fastest option, but "LIN" or "RND" would also be acceptable.

Now, let us produce a few variations of the "-gr.dhahatite" grammar. In the settings, check the "Non-stop improvize" option and set "Maxitems produced" to a small number, e.g. "4"., which is safe in terms of computation time and disk usage. Set the output file to "BP data file", since no sounds are expected.

If the "Seed for randomization" is set to 0 in the settings, you will get a different sequence each time you click the "PRODUCE ITEM(s)" button, for example:

4+4+4+4/4 dha dha ti te • dha dha dhee na • dha dha dhee na • ti te tee na •
dha dha ti te • dha dha dhee na • dha dha ti te • dha dha tee na •
ta ta ti te • ta ta tee na • ta ta tee na • ti te tee na •
dha dha ti te • dha dha dhee na • dha dha ti te • dha dha dhee na

4+4+4+4/4 dha dha tee na • dha dha dhee na • dha dha ti te • dha dha tee na •
dha dha ti te • dha dha dhee na • dha dha ti te • dha dha tee na •
ta ta tee na • ta ta tee na • ta ta ti te • ta ta tee na •
dha dha ti te • dha dha dhee na • dha dha ti te • dha dha dhee na

4+4+4+4/4 dha dha ti te • dha dha dhee na • dhee na ti te • dhee na tee na •
dha dha ti te • dha dha dhee na • dha dha ti te • dha dha tee na •
ta ta ti te • ta ta tee na • tee na ti te • tee na tee na •
dha dha ti te • dha dha dhee na • dha dha ti te • dha dha dhee na

4+4+4+4/4 dha dha ti te • dha dha dhee na • dha dha ti te • dha dha tee na •
tee na dhee na • dha dha ti te • dha dha ti te • dha dha tee na •
ta ta ti te • ta ta tee na • ta ta ti te • ta ta tee na •
tee na dhee na • dha dha ti te • dha dha ti te • dha dha dhee na

The "4+4+4+4/4" sections marking at the start of each variation indicates a layout most suitable for musicians: the beats are separated by periods ("." or "•"), with each beat containing four strokes — hence the "/4". These periods will be ignored in the parsing because they are not created by any rule in the grammar.

The expression "4+4+4+4" indicates that there are four lines, each containing four beats. As expected, the total duration is 16 beats.

👉 An old, still-valid format for "4+4+4+4/4" is "4+4+4+4*1/4".

Note that if "Split terminal symbols" is unchecked, you get a more compact representation, for instance:

4+4+4+4/4 dhadhatite . dhadhadheena . dhadha-- . titeteena .
dhadhatite . dhadhadheena . dhadhatite . dhadhateena .
tatatite . tatateena . tata-- . titeteena .
dhadhatite . dhadhadheena . dhadhatite . dhadhadheena

There is no guarantee that all variations will be different. Another method that ensures this (at the cost of computation time) is to select "Produce all items" instead of "Non-stop improvize" in the settings. Now, the machine only keeps productions that have not been found in the list before.

Items produced by this grammar can be copied to a Data project, here "-da.dhadhatite". The file in "ctests" contains eight variations. Beware that items should be separated by empty lines, which is automatically the case if you checked "Add lines between items" in the settings of "-gr.dhadhatite". In the settings of "-da.dhadhatite", check "Trace production or parsing", then save the settings and data and click "Analyze" near the first item. The "trace file" displays the detailed process which we won't comment. The "Show process" button displays a summary:

4+4+4+4/4 dhadhatite • dhadhadheena • dheenatite • dheenateena •
dhadhatite • dhadhadheena • dhadhatite • dhadhateena •
tatatite • tatateena • teenatite • teenateena •
dhadhatite • dhadhadheena • dhadhatite • dhadhadheena

Template(s) found
👉 We will try all templates, as per your settings
Analyzing selection…
Interpreting structure…
Expanding polymetric expression…
Using quantization = 10 ms with compression rate = 1
Phase diagram contains 2 lines
👉 Item matched template [1]
Subgrammar 6/6
Subgrammar 5/6
Subgrammar 4/6
Subgrammar 3/6
Subgrammar 2/6
Subgrammar 1/6
Item matching template [1] rejected by grammar… ❌
Result of failed analysis:
S1F +V8(= S1F) +S2F *(= S1F ++V8)(: S1F) S1F
👉 Item matched template [2]
Subgrammar 6/6
Subgrammar 5/6
Subgrammar 4/6
Subgrammar 3/6
Subgrammar 2/6
Subgrammar 1/6
Item matching template [2] rejected by grammar… ❌
Result of failed analysis:
(= ++S2F) +V8 S1F +S2F *(: ++S2F) *(= ++V8) S1F S1F
👉 Item matched template [3]
Subgrammar 6/6
Subgrammar 5/6
Subgrammar 4/6
Subgrammar 3/6
Subgrammar 2/6
Subgrammar 1/6
Item matching template [3] accepted by grammar… ✅
👉 Item matched template [4]
Subgrammar 6/6
Subgrammar 5/6
Subgrammar 4/6
Subgrammar 3/6
Subgrammar 2/6
Subgrammar 1/6
Item matching template [4] rejected by grammar… ❌
Result of failed analysis:
(= ++S2F V4 +B4) S1F +S2F *(: ++S2F V4 +B4) S1F S1F
👉 Item matched template [5]
Subgrammar 6/6
Subgrammar 5/6
Subgrammar 4/6
Subgrammar 3/6
Subgrammar 2/6
Subgrammar 1/6
Item matching template [5] rejected by grammar… ❌
Result of failed analysis:
(= ++S2F V6 +B2) S1F +S2F *(: ++S2F V6 +B2) S1F S1F
👉 Item matched template [6]
Subgrammar 6/6
Subgrammar 5/6
Subgrammar 4/6
Subgrammar 3/6
Subgrammar 2/6
Subgrammar 1/6
Item matching template [6] rejected by grammar… ❌
Result of failed analysis:
(= ++S2F V8) S1F +S2F *(: ++S2F V8) S1F S1F

This summary shows that this variation matches all six templates. However, only one leads to successful parsing, ending with the start string  "S". This helps determine the syntactic structure of the musical example, in addition to telling that it is "correct".

The initial beat and tempo expression "4+4+4+4/4" can be ignored. The following versions will also be parsed successfully:

/4 dhadhatite • dhadhadheena • dheenatite • dheenateena •
dhadhatite • dhadhadheena • dhadhatite • dhadhateena •
tatatite • tatateena • teenatite • teenateena •
dhadhatite • dhadhadheena • dhadhatite • dhadhadheena

dhadhatite • dhadhadheena • dheenatite • dheenateena •
dhadhatite • dhadhadheena • dhadhatite • dhadhateena •
tatatite • tatateena • teenatite • teenateena •
dhadhatite • dhadhadheena • dhadhatite • dhadhadheena

dhadhatitedhadhadheenadheenatitedheenateena
dhadhatitedhadhadheenadhadhatitedhadhateena
tatatitetatateenateenatiteteenateena
dhadhatitedhadhadheenadhadhatitedhadhadheena

dhadhatitedhadhadheenadheenatitedheenateenadhadhatitedhadhadheenadhadhatitedhadhateenatatatitetatateenateenatiteteenateenadhadhatitedhadhadheenadhadhatitedhadhadheena

The "-gr.dhin--" grammar

This grammar describes two sets of variations associated with the qa'ida. The first set covers 16 beats, i.e. 96 strokes at a speed of 6 per beat. The second set covers 32 beats, i.e. 192 strokes.

The 16-beat version was taught to Jim Kippen by Ustad Afaq Husain Khan in the 1980s. Within weeks, Jim had learned to identify the "language" of this qa‘ida, meaning that the machine successfully parsed all examples provided by the expert musician. However, at the end of the training, when Afaq Husain Khan played the same qa‘ida in a concert, he deliberately expanded variations to 32 beats. None of these were recognised by the grammar. Therefore, the grammar was adapted to produce 32-beat variations, and its validity was verified by analysing the variations performed at the concert.

Each set is represented by a variable named "S96" or "S192", created by subgrammar #1:

RND
gram#1 [1] <0> S <-> 4+4/6 S96
gram#1 [2] <5> S <-> 4+4+4+4/6 S192

The weights of rules 0 and 5 indicate that the weights of the rules have been inferred from a set of examples, five of which belonged to the 'S192' variant and none of which belonged to the 'S96' variant. Weight inference is explained on this page.

The weights of rules 0 and 5 indicate that the weights of the rules have been inferred from a set of examples, five of which belonged to the 'S192' variant and none of which belonged to the 'S96' variant. We will explain weight inference later.

'S192' and 'S96' are broken down further into fixed or variable blocks in subgrammar #2, which also introduces master-slave parentheses and the open-closed homomorphism notated "*":

RND
gram#2 [1] <5> S192 <-> (= F48 ) (= V24 ) F'24 *(: F48 ) (: V24 ) F24
gram#2 [2] <0> S96 <-> (= V24 ) F'24 *(: V24 ) F24
gram#2 [3] <0> V24 <-> (= V12 ) (: V12 )
gram#2 [4] <1> V24 <-> (= V12 ) *(: V12 )
gram#2 [5] <0> V24 <-> Q24
gram#2 [6] <0> V24 <-> V12 V12
gram#2 [7] <4> V24 <-> B24
gram#2 [8] <0> V12 <-> (= B6 ) *(: B6 )
gram#2 [9] <1> V12 <-> B12

Once again, we see that several weight 0 rules have not been used to analyse the example set. No explanation is required for other subgrammars.

Let's ask the machine to produce randomly 5 variations using this grammar and rule weights to determine their probabilities:

4+4+4+4/6 dhin--dhagena . dha--dhagena . dhatigegenaka . dheenedheenagena .
tagetirakita . dhin--dhagena . dhatigegenaka . teeneteenakena .
dheenedheenagena . dhagenadhin-- . dheenedheenagena . dheenedha-dheene .
tagetirakita . dhin--dhagena . dhatigegenaka . teeneteenakena .
tin--takena . ta--takena . tatikekenaka . teeneteenakena .
taketirakita . tin--takena . tatikekenaka . teeneteenakena .
dheenedheenagena . dhagenadhin-- . dheenedheenagena . dheenedha-dheene .
tagetirakita . dhin--dhagena . dhatigegenaka . dheenedheenagena

4+4+4+4/6 dhin--dhagena . dha--dhagena . dhatigegenaka . dheenedheenagena .
tagetirakita . dhin--dhagena . dhatigegenaka . teeneteenakena .
dhin--dhagena . dha--dhin-- . dheenedheenagena . dheenedheenagena .
tagetirakita . dhin--dhagena . dhatigegenaka . teeneteenakena .
tin--takena . ta--takena . tatikekenaka . teeneteenakena .
taketirakita . tin--takena . tatikekenaka . teeneteenakena .
dhin--dhagena . dha--dhin-- . dheenedheenagena . dheenedheenagena .
tagetirakita . dhin--dhagena . dhatigegenaka . dheenedheenagena

4+4+4+4/6 dhin--dhagena . dha--dhagena . dhatigegenaka . dheenedheenagena .
tagetirakita . dhin--dhagena . dhatigegenaka . teeneteenakena .
dhagenadha-- . dheenedheenagena . dheenedheenagena . dheenedheenagena .
tagetirakita . dhin--dhagena . dhatigegenaka . teeneteenakena .
tin--takena . ta--takena . tatikekenaka . teeneteenakena .
taketirakita . tin--takena . tatikekenaka . teeneteenakena .
dhagenadha-- . dheenedheenagena . dheenedheenagena . dheenedheenagena .
tagetirakita . dhin--dhagena . dhatigegenaka . dheenedheenagena

4+4+4+4/6 dhin--dhagena . dha--dhagena . dhatigegenaka . dheenedheenagena .
tagetirakita . dhin--dhagena . dhatigegenaka . teeneteenakena .
dhagenadha-- . dheenedheenagena . dheenedha-dheene . dhagenadhin-- .
tagetirakita . dhin--dhagena . dhatigegenaka . teeneteenakena .
tin--takena . ta--takena . tatikekenaka . teeneteenakena .
taketirakita . tin--takena . tatikekenaka . teeneteenakena .
dhagenadha-- . dheenedheenagena . dheenedha-dheene . dhagenadhin-- .
tagetirakita . dhin--dhagena . dhatigegenaka . dheenedheenagena

4+4+4+4/6 dhin--dhagena . dha--dhagena . dhatigegenaka . dheenedheenagena .
tagetirakita . dhin--dhagena . dhatigegenaka . teeneteenakena .
dhagenadha-- . dheenedheenagena . dhatigegenaka . teeneteenakena .
tagetirakita . dhin--dhagena . dhatigegenaka . teeneteenakena .
tin--takena . ta--takena . tatikekenaka . teeneteenakena .
taketirakita . tin--takena . tatikekenaka . teeneteenakena .
dhagenadha-- . dheenedheenagena . dhatigegenaka . teeneteenakena .
tagetirakita . dhin--dhagena . dhatigegenaka . dheenedheenagena

The variations differ slightly. These subtle differences are intended to create a "poetic" effect for listeners familiar with the "language" of the tabla.

Create 16 templates in the "-gr.dhin--" grammar. You will notice that the first eight cover 16 beats and the next eight cover 32 beats. This indicates that the creation of templates disregards rule weights and takes all rules in order. The same is true when creating variations with the "Produce all items" option.

A set of three 32-beat variations is provided in the "-da.dhin--" Data project. You can check that all these variations are successfully parsed. The parsing of the third item deserves our attention:

dhin--dhagena • dha--dhagena • dhatigegenaka • dheenedheenagena •
tagetirakita • dhin--dhagena • dhatigegenaka • teeneteenakena •
dheenedheenagena • dheenedheenagena • teeneteenakena • teeneteenakena •
tagetirakita • dhin--dhagena • dhatigegenaka • teeneteenakena •
tin--takena • ta--takena • tatikekenaka • teeneteenakena •
taketirakita • tin--takena • tatikekenaka • teeneteenakena •
dheenedheenagena • dheenedheenagena • teeneteenakena • teeneteenakena •
tagetirakita • dhin--dhagena • dhatigegenaka • dheenedheenagena

👉 Item matched template [12]
Subgrammar 6/6
Subgrammar 5/6
Subgrammar 4/6
Subgrammar 3/6
Subgrammar 2/6
Subgrammar 1/6
Item matching template [12] accepted by grammar… ✅
👉 Item matched template [13]
Subgrammar 6/6
Subgrammar 5/6
Subgrammar 4/6
Subgrammar 3/6
Subgrammar 2/6
Subgrammar 1/6
Item matching template [13] accepted by grammar… ✅
👉 Item matched template [16]
Subgrammar 6/6
Subgrammar 5/6
Subgrammar 4/6
Subgrammar 3/6
Subgrammar 2/6
Subgrammar 1/6
Item matching template [16] rejected by grammar… ❌

In the settings of "-da.dhin--", the option "When parsing, check all templates" is selected. This means that even after a successful analysis, the machine will try all the other templates. In this example, templates [12] and [13] match the composition and both lead to successful parsing. Therefore, there is a syntactic ambiguity in this piece, which we may see as part of its "poetic" dimension.

These demos highlight an important feature of the Bol Processor which is summarised by the following:

1. Every language produced by a true Bol Processor grammar is finite;
2. The grammar produces a finite (hopefully small) number of templates;
3. The parsing of an item matched against a template is done in a deterministic way;
4. Consequently, a true Bol Processor grammar is an identification of its finite language.

These qa‘idas are discussed in detail in our paper Modelling music with grammars (Kippen & Bel, 1992). The paper demonstrates the ability of the model to handle complex structures by taking real examples from the repertoire. It also questions the relevance of attempting to model irregularities encountered in actual performance.

(For geeks) An appendix of our Kippen & Bel (1992) paper explains the process of context-sensitive canonic rightmost derivation  which is used for the parsing.
More details are found in Bel (1987) and in chapter 4 of Bel's (1990) thesis: French and (bad English) versions.

👉 We recommend continuing your reading with the "Learning rule weights" page.

---

References

Kippen, Jim, & Bernard Bel (1992) Modelling music with grammars: formal language representation in the Bol Processor. In A. Marsden & A. Pople (eds.): Computer Representations and Models in Music, London, Academic Press, 1992, p. 207-238.

Bel, Bernard (1987) Grammaires de génération et de reconnaissance de phrases rythmiques Grammars for the production and recognition of rhythmic sentences).
Actes du 6^e congrès AFCET/INRIA, Antibes, 1987b: 353-366.

 

Let us compare productions by the same grammar
“-gr.tryDerivationModes” in all derivation modes.

The grammar comprises two subgrammars. The second one (gram#2) uses the ORD derivation mode in all examples. We show the trace of a derivation and the resulting production.

'A' and 'B' are variables. 'a', 'b', 'e' and 'f' are terminal symbols that are used to label sound objects.

[Derivation mode]
gram#1[1] S --> A B B A B B A
gram#1[2] A B --> a B
gram#1[3] B A --> B b
gram#1[4] B B A --> e B A
gram#1[5] B B --> f B

ORD
gram#2[1] f B --> f f
gram#2[2] e B --> e e
gram#2[3] A e --> c e

---

ORD

[Step #1] Selected: [1] LEFT S --> A B B A B B A
A B B A B B A
[Step #2] Selected: [2] LEFT A B --> a B
a B B A B B A
[Step #3] Selected: [2] LEFT A B --> a B
a B B a B B A
[Step #4] Selected: [3] LEFT B A --> B b
a B B a B B b
[Step #5] Selected: [5] LEFT B B --> f B
a f B a B B b
[Step #6] Selected: [5] LEFT B B --> f B
a f B a f B b
[Step #7] Selected: gram#2[1] LEFT f B --> f f
a f f a f B b
[Step #8] Selected: gram#2[1] LEFT f B --> f f
a f f a f f b

Result:  a f f a f f b

In this mode, the rules are applied in order and the rewriting positions are searched from left to right. This process is the fastest way to find a unique solution.

---

RND  (default derivation mode)

[Step #1] Selected: [1] RND S --> A B B A B B A
A B B A B B A
[Step #2] Selected: [4] RND B B A --> e B A
A e B A B B A
[Step #3] Selected: [3] RND B A --> B b
A e B A B B b
[Step #4] Selected: [2] RND A B --> a B
A e B a B B b
[Step #5] Selected: [5] RND B B --> f B
A e B a f B b
[Step #6] Selected: gram#2[1] LEFT f B --> f f
A e B a f f b
[Step #7] Selected: gram#2[2] LEFT e B --> e e
A e e a f f b
[Step #8] Selected: gram#2[3] LEFT A e --> c e
c e e a f f b

Result:  c e e a f f b
More results:
a f f a f f b
a f f a e e b
a f f b f f b
a f f b e e b
a e e a f f b
a e e b f f b
a e e a e e b
a e e b e e b
a e e c e e b
a f f c e e b
c e e b f f b
c e e a e e b
c e e b e e b
c e e c e e b

Rules are selected at random, with probabilities depending on their weights. The rewriting position is also selected at random unless it is specified as "LEFT" or "RIGHT."

---

LIN

[Step #1] Selected: [1] LEFT S --> A B B A B B A
A B B A B B A
[Step #2] Selected: [2] LEFT A B --> a B
a B B A B B A
[Step #3] Selected: [4] LEFT B B A --> e B A
a e B A B B A
[Step #4] Selected: [3] LEFT B A --> B b
a e B b B B A
[Step #5] Selected: [4] LEFT B B A --> e B A
a e B b e B A
[Step #6] Selected: [3] LEFT B A --> B b
a e B b e B b
[Step #7] Selected: gram#2[2] LEFT e B --> e e
a e e b e B b
[Step #8] Selected: gram#2[2] LEFT e B --> e e
a e e b e e b

Result:  a e e b e e b
More results:
a e e a f f b
a e e b e e b
a e e b f f b
a f f a e e b
a f f a f f b
a f f b e e b
a f f b f f b

Here, rules are selected at random but the rewriting position is searched from left to right, unless it is specified as "RIGHT" or "RND".

---

SUB

[Step #1] Selected: [1] RND S --> A B B A B B A
A B B A B B A
[Step #2] Selected: [2] RND A B --> a B
[Step #3] Selected: [4] RND B B A --> e B A
[Step #4] Selected: [2] RND A B --> a B
[Step #5] Selected: [4] RND B B A --> e B A
[Step #6] Selected: [3] RND B A --> B b
[Step #7] Selected: gram#2[2] LEFT e B --> e e
[Step #8] Selected: gram#2[2] LEFT e B --> e e

Result:  a e e a e e b

Here, all eligible rules are applied at once. The process is repeated until no rule is eligible.

---

SUB1

[Step #1] Selected: [1] LEFT S --> A B B A B B A
[Step #2] Selected: [2] LEFT A B --> a B
[Step #3] Selected: [2] LEFT A B --> a B
[Step #4] Selected: [3] LEFT B A --> B b
[Step #5] Selected: [5] LEFT B B --> f B
[Step #6] Selected: [5] LEFT B B --> f B
[Step #7] Selected: gram#2[1] LEFT f B --> f f
[Step #8] Selected: gram#2[1] LEFT f B --> f f

Result:  a f f a f f b

This is similar to "SUB," except that rewriting positions are searched from left to right, and the set of eligible rules is applied only once.

---

POSLONG

[Step #1] Selected: [1] LEFT S --> A B B A B B A
A B B A B B A
[Step #2] Selected: [2] LEFT A B --> a B
a B B A B B A
[Step #3] Selected: [4] LEFT B B A --> e B A
a e B A B B A
[Step #4] Selected: [2] LEFT A B --> a B
a e B a B B A
[Step #5] Selected: [4] LEFT B B A --> e B A
a e B a e B A
[Step #6] Selected: [3] LEFT B A --> B b
a e B a e B b
[Step #7] Selected: gram#2[2] LEFT e B --> e e
a e e a e B b
[Step #8] Selected: gram#2[2] LEFT e B --> e e

Result:  a e e a e e b

This is similar to SUB1, except that rewritings only occur in the positions of the longest substrings matched by a rule.