S10) Under the Hood

Compiler, Engine, and Dispatcher

The Complete Pipeline — From .bp File to Sound

A .bp file goes in. Sound comes out. In between: a JavaScript compiler, a C engine compiled to WebAssembly, a resolver that translates names into frequencies, and a dispatcher that sends everything to Web Audio.

This article describes the current and target architecture. What is implemented today runs in the browser — inline JS + Web Audio. The target architecture (external REPLs, OSC, MIDI) is designed but not yet operational. The sections specify this.

Where Does This Article Fit In?

After the five declarative layers (S9), we open the hood. This article is technical — it describes the software components that transform a .bp file into concrete events.

The Pipeline in a Diagram

 

flowchart TD
    subgraph BPscript["BPscript"]
        ED["Editor (.bp)"]
        COMP["JS Compiler"]
        ENG["BP3 WASM Engine"]
    end

    ALPH[("alphabet.json")]
    CTRL[("controls.json")]
    FILT[("filter.json")]
    ROUT[("routing.json")]
    TUN[("tuning.json")]

    SEQ["Timed tokens"]

    subgraph Dispatch["Dispatcher"]
        RES["Resolver"]
        CV["CV engine"]
        DIS["Dispatch"]
    end

    subgraph Transports["Transports"]
        WAUDIO["Web Audio"]
        OSC["OSC"]
        MIDI["MIDI"]
    end

    subgraph Evals["Evaluators"]
        JSEVAL["JS inline"]
        SCLANG["sclang"]
        PYTHON["Python"]
    end

    ED -->|".bp"| COMP
    ALPH -.-> COMP
    CTRL -.-> COMP
    FILT -.-> COMP
    COMP -->|"Interface 1"| ENG
    ENG -->|"Interface 2"| SEQ
    SEQ --> RES
    TUN -.-> RES
    ROUT -.-> RES
    RES --> CV
    CV --> DIS

    DIS -->|"terminals"| WAUDIO
    DIS -->|"backticks"| JSEVAL
    JSEVAL -->|"values"| DIS

    DIS -.-> OSC
    DIS -.-> MIDI
    DIS -.-> SCLANG
    DIS -.-> PYTHON

    style ED fill:#4a7ab5,stroke:#3a6aa5,color:#fff
    style COMP fill:#4a9a8a,stroke:#3a8a7a,color:#fff
    style ENG fill:#c8842a,stroke:#b8742a,color:#fff
    style RES fill:#7a4a9a,stroke:#6a3a8a,color:#fff
    style CV fill:#7a4a9a,stroke:#6a3a8a,color:#fff
    style DIS fill:#8a5ab5,stroke:#7a4aa5,color:#fff
    style WAUDIO fill:#2a6a4a,stroke:#1a5a3a,color:#fff
    style JSEVAL fill:#4a5ab5,stroke:#3a4aa5,color:#fff
    style OSC fill:#555,stroke:#777,color:#999
    style MIDI fill:#555,stroke:#777,color:#999
    style SCLANG fill:#555,stroke:#777,color:#999
    style PYTHON fill:#555,stroke:#777,color:#999
    style ALPH fill:#333,stroke:#666,color:#ccc
    style CTRL fill:#333,stroke:#666,color:#ccc
    style FILT fill:#333,stroke:#666,color:#ccc
    style TUN fill:#333,stroke:#666,color:#ccc
    style ROUT fill:#333,stroke:#666,color:#ccc
    style SEQ fill:#444,stroke:#666,color:#ddd

 

Figure 1 — BPscript Pipeline. Solid line = implemented: the compiler produces a grammar with a flat alphabet, the WASM engine derives timed tokens, the resolver applies temperament and routing, the dispatcher sends terminals to Web Audio and backticks to JS inline (in-browser evaluation via new Function()). Inline JS returns values to the dispatcher (bidirectional arrow). Dotted line = planned: OSC/MIDI transports and external REPLs (sclang, Python).

The BPscript Compiler

The compiler is written in JavaScript and runs client-side (browser or Node.js). It transforms the .bp source into a BP3 grammar in three steps:

Step 1 — Tokenizer

The .bp source is broken down into a stream of tokens (lexical units): the 3 reserved words (gate, trigger, cv), the 24 symbols, the 7 operators, identifiers, literals, and backticks.

 

@alphabet.raga:sc             →  [@, alphabet.raga, :, sc]
Sa(vel:120)                   →  [Sa, (, vel, :, 120, )]
[phase==1] S -> A             →  [[, phase, ==, 1, ], S, ->, A]

 

Backticks are tokenized as opaque blocks — their content is not analyzed.

Step 2 — Parser

Tokens become an AST (Abstract Syntax Tree — see L4*). The BPscript AST has about ten node types: Scene, Directive, Declaration, Macro, Subgrammar, Rule, Expression, etc.

The parser checks syntax and registers declarations (gate Sa:sc). There is no separate type checking pass — a dedicated type-checker that would automatically validate the double contract (S3) is planned but not yet implemented. For now, type errors are detected at runtime.

Details of the AST are in S13.

Step 3 — Encoder

The AST is translated into BP3 grammar text: gram#N blocks, mode directives (@mode:random), rules with BP3 syntax (-->, <-->, flags /flag=N/), and a flat alphabet of terminals.

The flat alphabet — a major change

The historical OCT (Object-Controlled-Tempo) format of BP3 has been abandoned. BP3 no longer knows what a note is — it sees opaque names (C4, env1, Sa). Controls (velocity, tempo, etc.) are no longer encoded in terminals but passed via _script(CTn) with a separate control table:

 

// BPscript : [vel:120]Sa
// BP3 Grammar : _script(CT3) Sa
// CT3 in the control table : { type: "vel", value: 120 }

 

This separation is fundamental: BP3 handles structure (when, how long, in what order), the downstream JS resolver handles content (what frequency, what channel, what sound).

The BP3 WASM Engine

The C code of BP3, compiled to WebAssembly (WASM) via Emscripten. This is the same code that Bernard Bel has maintained since 1989, ported to run in a browser — with extensions.

The engine receives the encoded grammar and produces timed tokens — an array of timestamped events:

 

[
  { terminal: "Sa",  start: 0,    end: 1000 },
  { terminal: "dha", start: 0,    end: 0 },
  { terminal: "Re",  start: 1000, end: 2000 },
  ...
]

 

Terminals are simple names — BP3 does not know what Sa means or what frequency it corresponds to. The downstream resolver performs this mapping.

The engine manages:

  • Grammatical derivation (7 derivation types, 3 scan directions)
  • Polymetry resolution (PolyMake algorithm — see B13)
  • Calculation of indeterminate rests (...)
  • Flags, captures, templates, homomorphisms
  • Weights and their decrement
  • Controls via _script(CTn) (lookup in the control table)

WASM Extensions

The engine is not an untouched black box — two extensions have been added:

  • bp3_set_object_duration: API to give a concrete duration to custom terminals (sampled sounds, envelopes). Historical BP3 only managed symbolic durations; this API allows JS to communicate a duration in milliseconds to the engine.
  • _script(CTn): Table-based control mechanism — BPscript qualifiers ([vel:120], [tempo:160]) are compiled into indexed _script calls, which the engine inserts into the stream as zero-duration instructions.

The Dispatcher — Resolver, CV, Dispatch

The dispatcher is written in JavaScript. It is the richest component of the pipeline — it does much more than just routing.

The Resolver

Before dispatch, the resolver transforms opaque names into concrete events:

  • Temperament: abstract degree (Sa, degree 1) → frequency (261.63 Hz) via tuning.json
  • Routing: symbol → transport or REPL via routing.json
  • Control state: _script(CTn) lookup → concrete parameters (vel:120, tempo:160)

It is the resolver, not BP3, that knows what a note is.

The CV Engine

CV (Control Voltage — ADSR envelopes, LFOs, ramps — see S3*) objects are a layer in themselves. Declared in the BPscript header and placed in the grammar as ordinary symbols, they are resolved by the dispatcher:

  • ADSR: amplitude envelope (attack, decay, sustain, release)
  • LFO: low-frequency oscillator (periodic modulation)
  • Ramp: linear interpolation between two bounds

The CV engine translates these objects into audio buses (Web Audio), sequences of CC messages (MIDI), or interpolated parameters (OSC) depending on the destination transport.

The Dispatch — Two Flows

The dispatcher separates two distinct flows:

  1. Typed terminals (gates, triggers, CV) → transports according to routing — this is fire-and-forget (send without waiting for a response)
  2. Backticks → code evaluation according to the tag — inline JS today, external REPLs eventually

Today: backticks are evaluated in JS in the browser via new Function(). No external REPL, no persistent session — the JS code executes directly. The main transport is Web Audio.

Target architecture: tagged backticks (sc:, py:) will be routed to external REPLs (sclang, Python) via stdin/pipe. OSC and MIDI transports will complement Web Audio.

Advanced Management

  • Simultaneity: an ! point dispatches to multiple outputs at the same timestamp
  • Hot-swap: hot recompilation for live coding (quantized by default — see S11 (coming soon))
  • Loop mode: automatic re-derivation — the grammar is re-derived at each cycle, potentially producing different variations with each loop

Code Evaluation — Inline JS and REPLs

Current state: backticks are evaluated in inline JavaScript in the browser (new Function()). No external session.

Target architecture: external REPLs (sclang, Python, GHCi+Tidal) will take over for tagged backticks.

Code evaluation is not a simple output adapter at the end of the chain. It is an enrichment component at the heart of the pipeline, with a bidirectional flow:

Two Roles

Role Direction What happens
Enrichment REPL → dispatcher The REPL evaluates code and returns a value or an event that BPscript reinjects into the scene
Execution Dispatcher → REPL The REPL receives code to execute at time T (playback)

Enrichment is the most original role: it allows the power of SuperCollider or Python to be integrated into the grammar without inventing a programming language. The REPL evaluates rrand(40,127) and returns 87 — BPscript injects this value into the vel parameter of the Sa symbol.

The Three Times

The three times of backticks (S4) are managed by the REPLs:

  1. Init: initialization backticks — executed before derivation (define SynthDefs, import modules)
  2. Playback: in-flow backticks — executed at time T calculated by BP3 (modify runtime state)
  3. Resolution: parameter backticks — the REPL evaluates the expression and returns the value to the dispatcher

The resolution case is a round trip: dispatcher → REPL → value → dispatcher. This is the only time the dispatcher waits for a response.

The Incoming Trigger <!

The return flow also allows for external synchronization (S7): a MIDI signal, an OSC message, or a Python event can trigger a <! that unblocks derivation. The REPL receives the external signal and transmits it to the dispatcher, which restarts the pending derivation.

Universal Interface

The interface is the same for all REPLs (~100 lines per language):

 

connect()              // open the REPL session
eval(code, time)       // send code at time T (playback)
getValue(expr)         // evaluate and return a value (resolution)
close()                // close the session

 

Evaluator Language Output Status
JS inline JavaScript Web Audio, DOM Implemented
sclang SuperCollider Audio (via scsynth) Planned
Python Python DMX, GPIO, computation Planned
GHCi + Tidal Haskell Audio (via SuperDirt) Planned

Transports — Universal Data

Transports send structured data at a precise moment, without evaluating code.

Current state: only Web Audio is implemented — it is the main transport, everything runs in the browser.

Target architecture: OSC and MIDI will complement Web Audio to control external hardware and software.

Transport Protocol Usage Status
Web Audio Browser API Sound in the browser Implemented
OSC UDP SuperCollider (scsynth), Processing, TouchDesigner Planned
MIDI Web MIDI API Hardware synthesizers, DAWs, controllers Planned

No session, no state. The dispatcher translates the event into a Web Audio call (today) or an OSC/MIDI message (eventually) and sends it at time T. This is fire-and-forget.

A .bp file without backticks only needs transports — the dispatcher is a pure sequencer.

The Three Interfaces

Interface Between Format Status
Interface 1 Compiler → BP3 Engine BP3 grammar text + control table Exists
Interface 2 BP3 Engine → Dispatcher JS array of timed tokens Exists
Interface 3 Dispatcher → Transports + REPLs Timestamped messages + code Exists

All three interfaces are operational. Interface 3 handles both transport messages (OSC bundles, MIDI events, Web Audio calls) and REPL commands (eval, getValue).

Key Takeaways

  1. 3 compilation steps: tokenizer → parser → encoder (type-checker planned, not yet implemented)
  2. Flat alphabet: BP3 sees opaque names — the OCT format is abandoned, the JS resolver does the mapping
  3. The BP3 WASM engine has extensions: bp3_set_object_duration and _script(CTn)
  4. The dispatcher is the richest component: resolver (tuning + routing), CV engine (ADSR, LFO, ramp), control state, loop mode, hot-swap
  5. Code evaluation is bidirectional: enrichment (returned values, <! triggers) + execution (playback at time T)
  6. Today: inline JS (new Function()) + Web Audio — all in the browser
  7. Target architecture: external REPLs (sclang, Python) + OSC/MIDI transports as complements

Glossary

  • Compiler: JS program that transforms .bp source into BP3 grammar + control table
  • Tokenizer: First step — breaks source into tokens
  • Parser: Second step — builds the AST, checks syntax
  • Encoder: Third step — translates the AST into BP3 grammar with a flat alphabet
  • Flat alphabet: Convention where terminals are simple names (no OCT encoding) — BP3 does not know what they mean
  • Timed token: Timestamped event produced by BP3: {terminal, start, end}
  • Resolver: Dispatcher component that translates names → frequencies (tuning) and names → routes (routing)
  • CV engine: Dispatcher component that resolves CV objects (ADSR, LFO, ramp) into audio buses or CC messages
  • Transport: Output protocol for timestamped data (OSC, MIDI, Web Audio) — universal, stateless
  • REPL: Persistent code session (sclang, Python) — bidirectional: executes code AND returns values
  • _script(CTn): Table-based control mechanism — BPscript qualifiers are compiled into indices in a control table
  • bp3_set_object_duration: WASM API to communicate a concrete duration (ms) to a custom terminal

Links in the Series

  • S9 — The five layers — alphabets, tuning, and routing consumed by the pipeline
  • S11 (coming soon) — Live — how hot-swap and loop mode interact with the pipeline
  • S12 — The EBNF — the formal grammar implemented by the parser
  • S13 — The AST — the node types produced by the parser
  • S3 — The types — gate, trigger, cv and the double contract
  • S4 — The backticks — the three times (init, playback, resolution)
  • B13 — PolyMake — the polymetric expansion algorithm in the engine

Prerequisite: S9
Reading time: 14 min
Tags: #BPscript #architecture #compiler #WebAssembly #dispatcher

Next article: S11 — Live: modifying while playing

← Back to index