B8) Two Directions, Three Modes

A composer writes music. A musicologist analyzes it. A pedagogue wants to verify the conformity of a performance. BP3 does all of this — with the same grammar.

Where does this article fit in?

In L13, we discovered that any formal grammar has two sides: it can produce sentences (generation) or analyze them (recognition). We saw that these two directions are asymmetrical — it’s easier to produce than to analyze.

This article shows how BP3 (Bol Processor 3, cf. I2) puts this duality into practice. Unlike most formal musical systems that only work in one direction, BP3 was designed from the outset to operate in three modes: producing, analyzing, and exhaustively exploring the grammar’s possible structures.

Why is this important?

Most formal musical tools are unidirectional (some examples):

GTTM (M6): analytical only — it assigns structures to existing pieces but composes nothing
EMI (David Cope): generative only — it composes from extracted patterns but offers no formal analysis mode
SuperCollider (I3): generative only — it produces sound, it does not analyze it structurally

According to our comparative studies, BP3 is a priori the only grammatical system where the same grammar serves both directions — producing and analyzing — within the same formalism:

System	Generates?	Formal Analysis?	Infers?	Same Formalism?
GTTM (Lerdahl & Jackendoff)	No	Yes (preference rules)	No	—
Steedman CCG (jazz)	Yes (1984)	Yes, parsing (2014)	No	Yes (2/3)
EMI (David Cope)	Yes	No (pattern matching)	Yes (extraction)	No (separate processes)
GF (Ranta)	Yes	Yes (parsing)	No	Yes (2/3)
Continuator (Pachet)	Yes	No (no membership test)	Yes (Markov)	No
BP3 (Bel & Kippen)	Yes (PROD)	Yes (ANAL)	QAVAID†	Yes (2/3) + QAVAID†

† QAVAID = separate Prolog II program (Bel & Kippen, 1988-1990), never integrated into BP3, abandoned. The Bel-Kippen ecosystem covered all 3 directions, but BP3 itself covers 2.

It is this dual capability — producing and analyzing with the same grammar — that allowed Bernard Bel and James Kippen to conduct their ethnomusicological project on Lucknow tabla. With QAVAID, a separate program written in Prolog II, they also aimed for grammatical inference — but this third direction remained experimental [BelKippen1992a].

The idea in one sentence

BP3 uses the same grammar in three modes: producing music (PROD), verifying that a sequence is grammatical (ANAL), and exhaustively cataloging possible structural forms (TEMP).

Let’s explain step by step

1. PROD Mode — the composer

The PROD (production) mode is the one we already know: the grammar generates musical sequences by derivation. This is the subject of articles B3 to B6.

In logic, this mode corresponds to modus ponens (mode of affirmation) — direct reasoning:

Premise :   Grammar G has the rule  S → A B
Premise :   We want to derive S
Conclusion : We obtain A B

Modus ponens moves forward: “if the grammar says that S can become A B, then I produce A B.” This is the reasoning of the composer: I know the rules, I apply them to create.

In BP3, PROD mode uses the five derivation modes (B3) — ORD, RND, LIN, SUB, SUB1 — with probabilities (B1), flags (B4), variables, and homomorphisms (B6).

Example:

gram#1[1] S --> |x| - |x| - |x|
gram#1[2] |x| --> dha tirakita dhin dhin
gram#1[3] |x| --> tin ta dha ge na

In PROD mode, this grammar produces a tihāī (a cadence where a motif is repeated three times to land on sam, the first beat of the tāla cycle — cf. B6).

The translation of these constructions into SuperCollider — flags, decremental weights, Prout and embedInStream — is detailed in B7.

2. ANAL Mode — the musicologist

The ANAL (analysis) mode reverses the direction: instead of producing, the grammar tests whether a given sequence belongs to the language. This is the membership test — the question “is this sequence grammatical?”

In logic, this mode corresponds to modus tollens (mode of negation) — inverse reasoning:

Premise :   Grammar G generates language L(G)
Observation : Sequence s is NOT in L(G)
Conclusion : s is not a valid production of G

Modus tollens moves backward: “if the sequence does not correspond to any possible derivation of the grammar, then it is not grammatical.” This is the reasoning of the musicologist: I receive a performance, I check if it respects the rules.

Sidebar: Modus ponens and modus tollens — the composer and the critic

Imagine a tablist and a guru (master):

The tablist (modus ponens / PROD): “The rules say I can play dha tirakita dhin dhin. I play it.”

The guru (modus tollens / ANAL): “My student played dha tirakita tin na. Does this correspond to a valid derivation of the kayda grammar?”

The guru does not compose — he evaluates. BP3 allows formalizing both roles with the same grammar [KippenBel1989b].

How ANAL mode works

The analysis process relies on template matching. The source code reveals that ANAL has two recognition strategies:

Direct strategy (always available): BP3 receives a sequence to analyze and attempts to match it to the grammar rules by searching the derivation space.
Template strategy (optional): if templates have been pre-calculated by TEMP mode (cf. §3), BP3 offers the user the option to use them. It then first checks if the form of the sequence corresponds to a known skeleton, before attempting full parsing.

In both cases, if a match is found, the sequence is accepted as belonging to the language.

In Shannon’s terms, ANAL mode is the decoder of the musical communication chain (cf. L13). It operates in two steps: first, separating signal from noise — what is grammatical from what is not —, then extracting meaning from information — assigning a structure, an interpretation to the accepted sequence. And like any decoding, this operation is not unambiguous: the same sequence can admit several valid analyses, several derivation trees. Ambiguity is not an accident — it is a fundamental property of recognition.

Note: membership test ≠ language identification

ANAL mode tests the membership of a sequence to a single grammar — the one that is loaded. It does not allow testing a sequence against a family of grammars to determine which one it belongs to. This is the difference between the membership test (“s ∈ L(G) ?”) and language identification (“to which L(Gᵢ) does s belong?”). This second problem — “does this performance fall under kayda X or kayda Y?” — is a classification problem, more difficult and not covered by BP3 in its current state.

In ANAL mode, wildcards (?1, ?2 — cf. B6) take on their full meaning; they are the pattern matching tools that allow the grammar to “search” for motifs in a received sequence.

Wildcards in analytical mode

BP3 wildcards are analogous to capture groups in regular expressions (regex), but operate within the framework of formal grammars:

Syntax	Role in PROD	Role in ANAL
`?`	Anonymous placeholder	Accepts any terminal
`?1`	Named capture	Captures a motif and verifies its consistency
`?1 ... ?1`	Replication of the captured motif	Verifies that the same motif appears at both positions

Example in ANAL mode:

gram#1[1] S --> ?1 - ?1 - ?1

This rule, in ANAL mode, means: “the sequence to be analyzed must be of the form X – X – X, where X is the same motif in all three positions.” This is exactly the tihāī test — the grammar verifies that the cadence respects the triple repetition structure.

In PROD mode, the same rule generates a motif for ?1 and replicates it in all three positions — it is a tihāī generator. In ANAL mode, it captures the motif at the first occurrence and verifies that it is identical to the other two — it is a tihāī verifier. Both directions work, with complementary roles: the homomorphism from B6 ensures consistency in both directions.

Weight learning

A remarkable aspect of ANAL mode is weight learning. When a sequence is successfully analyzed, the rules that were used during recognition have their weights incremented (B4). In other words:

Rules frequently confirmed by examples become more probable
Rules rarely confirmed lose relative probability
The grammar gradually adapts to the analyzed corpus

This is an embryonic learning mechanism: the grammar does not change its rules, but adjusts their relative frequency based on the data. We move from an “a priori” PCFG (Probabilistic Context-Free Grammar — B1) to a PCFG calibrated by experience.

3. TEMP Mode — the structure cartographer

The TEMP (producing templates) mode answers a simple question: what structural forms can this grammar produce?

What TEMP does

Imagine a grammar that can produce hundreds of variations of a kayda. Each variation has a different form — a structural arrangement of motifs — but the motifs themselves vary. TEMP exhaustively explores all these possible forms.

To do this, TEMP derives the grammar exhaustively — it explores all possible structures, whereas PROD randomly draws one according to weights. It is not interested in terminals (the concrete bols), only in skeletons: where are the positions, how many motifs, what nesting.

The result is a catalog of all structural forms that the grammar allows — its templates — displayed directly in the grammar window under a TEMPLATES: section.

A design tool in itself

TEMP is accessible as an independent command in BP3 — no need to run an analysis to use it. It is primarily a tool for the grammar designer:

Explore: “what forms can my grammar produce?” — see the complete structural space at a glance, whereas PROD only shows a random sample
Verify: “does my grammar cover all the structures I intended?” — detect missing forms or unexpected derivations
Debug: when a grammar produces surprising results, TEMP helps understand why by showing all possible structures

It’s the equivalent of a “print preview” mode for grammars: you see the form without the content.

TEMP and ANAL: an optional relationship

TEMP is not a prerequisite for ANAL. The BP3 source code confirms this: ANAL has two independent recognition paths:

Without templates: ANAL performs recognition directly from the grammar rules — classic parsing by searching the derivation space.
With templates: if templates have been pre-calculated by TEMP, BP3 offers the user the option to use them (“Use templates?”). ANAL can then first check if the form of the sequence corresponds to a known skeleton, before attempting full parsing.

The second strategy is faster — comparing a form to a catalog of skeletons is less costly than full parsing — but it is optional. ANAL works perfectly without TEMP ever having been launched.

TEMP is therefore better described as a structural exploration tool that can also speed up analysis, rather than as a mere auxiliary to ANAL.

Two key differences from PROD

TEMP is exhaustive, PROD is stochastic. PROD respects the weights of the rules (B1) and produces one probabilistically chosen sequence. TEMP ignores weights and enumerates all possible structural derivations.

TEMP stops at the structure, PROD goes all the way to the terminals. PROD derives all rules until concrete bols are obtained (dha tirakita dhin dhin). TEMP stops at abstract skeletons — the form without the content.

It is here that a unique feature of BP3 takes on its full meaning: directional arrows. Each rule carries a direction annotation — --> (production only), <-- (analysis only), or <-> (bidirectional). This distinction, absent from classical grammatical formalisms, allows the same grammar to finely control which rules are active in each mode.

Sidebar: QAVAID — the aborted third direction

Beyond BP3’s three modes, Bel and Kippen developed QAVAID (Question Answer Validated Analytical Inference Device, قواعد qavā’id = “grammar” in Arabic/Urdu) — a separate Prolog II program for interactive grammatical inference (1988-1990). The musician served as an oracle: the machine proposed generalizations from tabla examples, and the guru accepted or rejected them. The algorithm built an almost minimal finite automaton by state merging [Bel1990-JFA].

QAVAID was never integrated into BP3. The project was abandoned: machines were too slow, corpora insufficient, and James Kippen moved to the University of Toronto. As Kippen summarized in 2021: “it only proved that it would be feasible” [Kippen2021].

The code is archived at bolprocessor.org/misc/QAVAID/. The ambition remains: contemporary ML approaches could take up the torch of interactive musical grammatical inference (cf. M10).

4. The complete cycle

The power of BP3 lies in the loop between its modes:

         PROD                    ANAL
Grammar ──→ Sequences    Sequences ──→ Validation
    ↑                                        │
    │              Adjusted Weights          │
    └────────────────────────────────────────┘

         TEMP
Grammar ──→ Catalog of structural skeletons
              (optionally used by ANAL)

The actual cycle that BP3 implements:

PROD mode generates musical sequences by stochastic derivation.
ANAL mode verifies if new examples belong to the language (optionally assisted by TEMP templates).
The weights of the rules confirmed by analysis are incremented — the grammar adapts.
Return to step 2: new productions reflect the adjusted weights.

This is an analysis-by-synthesis process applied to music — the same paradigm that Halle and Stevens (1962) proposed for speech perception, but here embodied in an interactive human-machine system [KippenBel1989].

QAVAID aimed to add a third direction — inference — to completely close the cycle: automatically building the grammar from examples, rather than writing it manually. This separate Prolog II program did not succeed (cf. sidebar above), but the idea remains relevant.

The ultimate test — the round-trip test — is: “are the sequences produced by the grammar accepted as valid by the expert musician?” If so, the grammar has captured something of the tacit rules of improvisation.

Three theoretical frameworks: Chomsky, Shannon, Morris

BP3’s three modes are not an isolated invention. They fit into three complementary theoretical frameworks — a triptych that illuminates a different dimension of bidirectionality.

Chomsky: structural complexity

The three modes operate on grammars classified in Chomsky’s hierarchy (L1). BP3 is a Type 2+ formalism — context-free augmented by flags, weights, and homomorphisms that push it towards mildly context-sensitive languages (L9).

This position in the hierarchy constrains what each mode can do. The computational asymmetry described in L13 applies directly:

In PROD mode, derivation is linear in the length of the output — each step is a local choice.
In ANAL mode, the membership test is at least O(n³) for CFGs — and potentially more costly for BP3’s extensions.

Producing is intrinsically easier than analyzing. This is not an implementation flaw — it is a mathematical property of the formalism.

Shannon: encoder, channel, decoder

Claude Shannon’s communication model (1948) directly maps BP3’s modes:

Musical ──→ PROD ──→ Sequence ──→ ANAL ──→ Recognized
Intention   (encoder)  of bols    (decoder)  Structure

PROD is the encoder: it transforms a musical intention — the meaning captured by the grammar — into a sequence of symbols. Each derivation is an act of encoding: the choice of a form from among the possibilities.

ANAL is the decoder: it receives the sequence and must extract its meaning. As we saw above, this decoding operates in two steps — separating signal from noise (the membership test: is it grammatical?), then extracting meaning from information (assigning a structure: what derivation tree?). And this reconstruction of intention is intrinsically ambiguous: the same sequence of bols can correspond to several valid derivations, just as the same sentence can have several syntactic analyses.

TEMP builds the decoder’s dictionary: by exhaustively enumerating possible structural templates, it provides ANAL with an optional reference table that can speed up decoding — like a codebook dictionary that allows the receiver to recognize valid messages faster. But ANAL can also decode without a dictionary, directly from the rules.

The channel noise, in music, takes multiple forms: variations in interpretation, transcription errors, ambiguity between resonant and dry bols, the musician’s stylistic freedom. Shannon’s fundamental asymmetry — it is easier to encode than to decode a noisy message — is exactly the PROD/ANAL asymmetry.

Morris: cycle correction

In 1973, James Morris formalized compiler correctness with a commutative diagram (L11): compiling and then executing the target program must yield the same result as directly interpreting the source.

The cycle that BP3 actually implements — seen in the previous section — is a Morris commutativity test applied to music:

Grammar G ──PROD──→ Sequences S
     ↑                       │
     │     Adjusted Weights ANAL
     │                       │
     └───────────────────────┘

The diagram commutes if: the sequences produced by grammar G are accepted as grammatical by ANAL, and the weights adjusted by ANAL lead PROD to produce sequences that the expert musician judges valid. This is the musical version of a compiler correctness proof.

QAVAID aimed to enrich this cycle by adding inference — building G automatically rather than manually. If this third direction had succeeded, the diagram would have had an additional path: examples → QAVAID → inferred grammar G₂, with the test L(G₂) ≈ L(G₁). But in BP3 as it exists, the cycle closes by weight adjustment, not by inference.

Polymetry as concurrency

When modes operate on polymetric structures ({v1, v2} — cf. B5), they enter the domain of concurrency formalized in L12:

PROD on polymetry = generating a parallel composition in the sense of CCS ( $v_1 \mid v_2$ ) or CSP ( $v_1 \parallel_{\text{downbeat}} v_2$ ). The producer creates two independent temporal flows that synchronize at the downbeats.
ANAL on polymetry = parsing concurrent flows — a significantly more difficult problem than sequential parsing. It is necessary to verify that each voice is grammatical and that their temporal relationships are consistent.

Polymetry transforms parsing from a sequential problem (a string of symbols) into a concurrent system verification problem — a leap in complexity that process algebras and Petri nets allow to formalize.

Position in Series B

Article	Direction	Covered Constructions
B1 – B6	→ PROD (production)	Rules, modes, weights, flags, polymetry, variables, homomorphisms
B7	→ PROD → SC	Translation to SuperCollider
B8 (this article)	← ANAL + TEMP (structural exploration)	Membership test, analytical wildcards, structural templates

Articles B1 to B7 cover the production direction of BP3. This article completes the picture by exploring the inverse direction — analysis — and the TEMP mode of structural exploration, showing why BP3 is a fundamentally bidirectional system, not just a generator.

Key takeaways

BP3 has three modes for using a grammar: PROD (produce), ANAL (analyze), TEMP (exhaustively explore possible structures). These three modes cover two directions: production and analysis.
PROD mode (modus ponens) is the composer — it produces sequences by stochastic derivation.
ANAL mode (modus tollens) is the musicologist — it separates signal and noise, then extracts meaning with its ambiguities of interpretation.
TEMP mode is the structure cartographer — it exhaustively enumerates all possible structural skeletons (ignoring weights). ANAL can optionally use this catalog to speed up recognition, but also works without it.
Wildcards (?1, ?2) work in both directions: in PROD mode, they generate and replicate (tihāī generator); in ANAL mode, they capture and verify (tihāī checker).
The weights of the rules automatically adjust in ANAL mode — the grammar learns from what it analyzes.
QAVAID (qavā’id = grammar in Arabic/Urdu) was a separate Prolog II program (1988-1990), never integrated into BP3, aiming for interactive grammatical inference. The project was abandoned.
The actual BP3 cycle — PROD → ANAL → adjusted weights → re-PROD — makes BP3 a rare case of a bidirectional musical grammatical system.
The Chomsky-Shannon-Morris triptych illuminates the dimensions: Chomsky constrains complexity (PROD linear vs ANAL cubic), Shannon maps directions (encoder/decoder/noise), Morris formalizes cycle correction (commutativity round-trip test).
Polymetry transforms each mode into a concurrency problem (L12): producing {v1, v2} means generating a parallel composition, analyzing it means parsing concurrent flows.

Glossary

ANAL (mode): BP3’s analytical mode — the grammar tests if a sequence belongs to the language. Uses template matching and wildcards. Corresponds to modus tollens.
Analysis-by-synthesis: A paradigm where recognition uses generation as an internal subprocess. The receiver generates hypotheses and compares them to the received signal (cf. L13).
Almost minimal finite automaton: A compact representation of a regular language built by QAVAID (separate Prolog II program) from examples. “Almost” minimal because exact minimization can over-generalize.
Kayda: A tabla composition with a fixed theme and systematic variations. The typical framework for grammatical inference with QAVAID.
Membership test: The decision problem: “does this sequence belong to the language generated by this grammar?”. In BP3, solved by template matching in ANAL mode.
Modus ponens: Direct logical reasoning: “if A implies B, and A is true, then B is true.” In BP3: if the grammar has the rule S → A B, we can produce A B. Direction: PROD.
Modus tollens: Inverse logical reasoning: “if A implies B, and B is false, then A is false.” In BP3: if a sequence does not correspond to any derivation of the grammar, it is not grammatical. Direction: ANAL.
Oracle: In the context of grammatical inference, the expert (the musician) who validates or rejects generalizations proposed by the system.
PROD (mode): BP3’s production mode — the grammar generates musical sequences by derivation. Corresponds to modus ponens.
QAVAID (Question Answer Validated Analytical Inference Device): A separate Prolog II program (1988-1990) for interactive grammatical inference, developed by Bel & Kippen. Never integrated into BP3, abandoned. The name also means “grammar” (قواعد) in Arabic/Urdu.
Round-trip test: A validation test where sequences produced by an inferred grammar are submitted to the musician who provided the original examples. If accepted, the grammar has captured the tacit rules. Musical equivalent of Morris’s commutativity test.
TEMP (mode): BP3’s structural exploration mode — exhaustively enumerates all structural skeletons that the grammar can produce (ignoring weights). The resulting catalog can optionally be used by ANAL as an alternative recognition strategy. TEMP is not a prerequisite for ANAL. In Shannon’s terms: builds the decoder’s dictionary.
Template matching: The central mechanism of ANAL mode: superimposing a sequence to be analyzed onto the patterns defined by the grammar rules.
Chomsky-Shannon-Morris Triptych: Three complementary theoretical frameworks for understanding BP3’s modes. Chomsky = structural complexity (hierarchy), Shannon = encoding/decoding (direction), Morris = cycle correction (commutativity).
Morris Diagram: A commutative diagram (Morris, 1973) that formalizes compiler correctness: the result must be the same whether one follows the “compile then execute” path or “interpret directly.” Applied to the PROD→ANAL→adjusted weights→re-PROD cycle.

Prerequisites: B3, B6, L13
Reading time: 15 min
Tags: #BP3 #production #analysis #TEMP #wildcards #tabla