Cholidean Harmony Structure

Quick start examples |MIDIport ▶ | FluidSynth ▶ | Windows ▶

Cholidean Harmony Structure is a projection of 12-tone Equal Temperament (12ET) music systems into 3D space. The twelve tones are placed on a 3D parametric closed curve. The fact that each tone is related to two and only two other tones, creates strongly the perception of a two-dimensional surface strip that curves in three-dimensional space to fit the surface of an umbilic torus.

Project's integration with FluidSynth, as a MIDI backend player, demonstrates a powerful method for visualizing and exploring harmony theories.

*At its heart, this project offers full 3D harmony exploration — a feature that works on all platforms when run with the null backend (no audio), so anyone can experience the concept without setup hurdles. *

📖 Two ways into this world:

The Watch and the Twelve Realms — read the origin fable (first watch story at → https://jimishol.github.io/post/circle_of_fifths)
Tonality Structure in Music — read the tonal structure article

Installation Prerequisites and Steps 🚀

This project embeds 3DreamEngine — an awesome 3D engine for LÖVE — directly in its codebase. Users only need to install LÖVE to run the project; no separate installation of 3DreamEngine is required.

🐧 Linux

If you’re on Linux and don’t have LÖVE installed:

sudo zypper install love

midiport backend needs alsa-devel:

sudo zypper install alsa-devel

Running, from project's ROOT directory, is simple:

love .

or

./run.sh

this way enables the restart capability.

To enable MIDI playback with the fluidsynth backend, install fluidsynth via your package manager.

sudo zypper install fluidsynth

Then download some nice SoundFont like FluidR3_GM.sf2:

If your repository includes them, install

sudo zypper install fluid-soundfont-gm

Alternatively, download the raw file from FluidR3_GM.sf2 and place it in the project’s root directory.
For a cleaner setup, put it in the soundfonts/ folder and set:
M.soundfonts = "soundfonts/FluidR3_GM.sf2"
If no valid SoundFont is found, the Fluidsynth backend will fall back to its configured default (if available); otherwise, no sound will be produced.

🍎 macOS (Untested)

LÖVE and FluidSynth are available via Homebrew:

brew install love  
brew install fluidsynth
brew install coreutils

You can try running the project with:

love .

Notes:

On macOS, only the null and fluidsynth backends are supported. The midiport backend depends on ALSA and is Linux‑only.
The Fluidsynth backend requires gstdbuf (from GNU coreutils) to ensure real‑time note events are flushed immediately. Without it, you may experience delayed or batched note updates. coreutils provides gstdbuf for line‑buffered output.
FluidSynth requires a SoundFont to produce sound.

On some Linux distributions, a default GM SoundFont is installed automatically with FluidSynth, so you don’t need to provide one.

On macOS (and some Linux setups), no default SoundFont is bundled. If no system‑wide default SoundFont is available, place at least one .sf2 file inside the project (root or soundfonts/ is recommended). For files outside the root, add their relative path in src/constants.lua so the fluidsynth backend can find them.
CoreAudio is used automatically by FluidSynth for audio output, so no extra drivers are needed.

macOS support has not been verified. Contributions or feedback from macOS users—especially developers—are warmly welcomed to help improve compatibility and ease of use.

🪟 Windows

✅ On Windows, the recommended backend is udpMidi, which can provide both the full 3D harmonic visualisation and audio/MIDI I/O — provided you have:

A MIDI player
loopMIDI (or equivalent virtual MIDI cable)
A synth engine installed

loopMIDI setup: Create a virtual MIDI port named midiBridgePort (exact spelling) and route your MIDI player’s output to it. The udpMidi bridge will automatically connect to this port.

To use udpMidi:

Set M.backend = "udpMidi" in src/constants.lua.

Launch the bridge binary before starting the visualiser:

src/backends/udpMidi/binaries/udp-midi-bridge-windows.exe

If you prefer a no‑audio mode, you can still set M.backend = "null".

Prepare the Project Directory
Clone the repo (including asset_pipeline/ and docs/ldoc/), or download and unzip the release ZIP.
Install LÖVE
Install LÖVE for Windows from the official site: https://love2d.org/
Use the installer so LÖVE is added to your system PATH.
Launch the Visualiser
From the command line:
```
love .
```

Known Issues:

Line‑buffered output isn’t working:
FluidSynth’s note‑on/off events arrive in batches under Windows; see issue #4.
Spaces in filenames:
May break playback through the winpty layer — use underscores instead.
Restart‑on‑exit disabled:
The batch wrapper doesn’t propagate non‑zero exit codes, so automatic restart on exit code 42 is unavailable.

Prerequisites

If you haven’t already installed the project via Releases, this project uses Git Large File Storage (LFS) to manage assets (mainly normal maps). Before you clone, build, or contribute, make sure Git LFS is installed and initialized:

# Install Git LFS (once per machine):

# macOS (Homebrew)
brew install git-lfs

# Windows (Chocolatey)
choco install git-lfs

# Debian/Ubuntu
sudo apt-get install git-lfs

# Initialize Git LFS
git lfs install

🚀 Basic Usage & Developer Integration

🎧 For Regular Users

This project works like a minimalist music player — but with a twist. Instead of just playing sound, it visually projects musical harmony into a 3D space, offering a unique and immersive way to experience music. The active joints (notes) become self-illuminating and grow in size. Of these, the bass notes strive valiantly to stand out with distinctive dots. The outgoing edges emit light discreetly to activate their destination. Spectral surfaces, when unambiguously indicated by active joints, materialize and emit light to attract the attention of minor or major scales that could incorporate them. The compositions are visualized in an anticipated dance of surprising steps.

Supported Format: Currently supports MIDI files via FluidSynth or via the Linux ALSA midiport backend—both emit real-time note ON/OFF events for 3D visualization. On Windows, use the null backend for full visualisation without audio.
Interactive Controls: Users can pause playback or slow down tempo, making it ideal for music students or harmony learners.
No Technical Setup Required: Just launch the app, load a MIDI file, and enjoy the visual harmony.

📝 Note: Future versions may support additional formats, depending on backend contributions.

🛠️ For Backend Developers

The project is designed to be extensible. Developers can integrate alternative backends as long as they can emit note ON/OFF events in real time. The backend manager is src/backends/init.lua.

🔄 How It Works

The core visual engine listens on the active_notes Love2D thread channel for updated note lists.
Backend threads (FluidSynth, midiport, udpMidi) push active‑note tables directly into that channel in real time.
The null backend falls back to disk I/O, reading from active_notes.lua when no channel updates occur.

🧪 Backend Options

udpMidi (Cross‑platform)
- Streams MIDI input events over UDP from a small Node.js bridge binary.
- On Windows, works with loopMIDI or another virtual MIDI cable.
- Windows setup: Create a virtual MIDI port named midiBridgePort (exact spelling) and route your MIDI player’s output to it. Launch src/backends/udpMidi/binaries/udp-midi-bridge-windows.exe before starting the visualiser.
- On Linux/macOS, can connect to any ALSA/CoreMIDI port.
- See src/backends/udpMidi/README.md for full setup instructions.
FluidSynth
- Launched as a thread by the project.
- Outputs note events via terminal stdout.
- Accepts playback commands via TCP.
- A watcher thread connects via TCP and sends commands (e.g., play, pause, tempo).
- Parses FluidSynth’s noteon/noteoff output and publishes the current active‑notes list on a Love2D thread channel (active_notes), eliminating any file I/O for note state.
Null Backend (Manual Mode)
- Teachers or developers can manually edit active_notes.lua to simulate note activity.
- Useful for demonstrations, teaching, or testing without a live music source.
- Recommended for exploring harmony in 3D space without audio.
midiport (Linux only)
- Sniffs an ALSA MIDI port directly via FFI (default Midi Through 14:0).
- Merges and publishes active notes at ~50 Hz to active_notes.lua.
- Sends control commands (gain, player_start, etc.) over a persistent non‑blocking TCP socket, bypassing stdout buffering.
- Does not support autoplayback of MIDI files (unlike the FluidSynth backend), but offers ~80 ms faster note tracking on average.
- Configure in src/constants.lua:
```
M.backend            = "midiport"
M.DEFAULT_MIDI_PORT  = "14:0"
M.shellHost          = "localhost"
M.shellPort          = 9800
```

💡 Tip — Clearing lingering notes (midiport / udpMidi)
With the midiportor udpMidi backends, if you suddenly change songs, any notes still active from the previous song will keep their status until cleared. Restart the current song (Backtab) or move to the next song (Enter) to reset the list.

💡 Tip — Using midiport.sh to wire up your MIDI chain (midiport only)
The midiport.sh script isn’t part of this project — it’s a standalone helper that launches whichever synth engine you specify (via SYNTH_CMD) and wires your MIDI player → ALSA MIDI port → synth engine.

For a deep dive into the asset pipeline, see FOR_DEVELOPERS.

📦 Example Usage Scenarios

🎼 1. MIDI Playback Mode

Launch the visualizer with FluidSynth to render harmony in 3D space using real-time MIDI input.

Linux:

./run.sh

Windows:

love .

FluidSynth playback will follow whatever exists in midi_files/ folder.

Camera Position Setup:

Most likely, when examining the structure, you will find some position more suitable than others in terms of understanding it. Press d and copy the camera position to the M.initialCameraPosition field in src/constants.lua, so that you always start from that position. If you have preferred lightning, copy 24h Day time to the M.day_night field of the same file.

Tonic Repositioning:

Quite often, you will feel that the scale of a piece is such that you would like its tonic to be in a different position than it is. With Shift + ← or Shift + →, you can move the tonic to the position you desire.

🎹 2. Live MIDI Input with Fluidsynth Backend

Even when the playlist is empty or playback has ended, the Fluidsynth backend remains active and can receive live MIDI input from a connected device.

🧩 Why Use This Mode?

Ideal for teachers demonstrating chords, scales, or harmonic concepts live.
Enables interactive performances without relying on preloaded MIDI files.
Keeps the system responsive and visual even after automated playback ends.

🔌 Connect Your MIDI Device (Linux)

Use aconnect to route your USB MIDI device to the Fluidsynth backend.

List available MIDI ports:

aconnect -l

Example output:

client 24: 'USB Midi' [type=kernel]
    0 'USB Midi MIDI 1 '
client 128: 'FLUID Synth' [type=user]
    0 'FLUID Synth MIDI Input'

Connect your device to Fluidsynth:

aconnect 24:0 128:0

Replace 24:0 and 128:0 with the actual port numbers from your system.

🧠 What Happens Next?

Notes played on the MIDI device are routed directly to Fluidsynth.
The backend thread listens for noteon and noteoff events.
active_notes.lua is updated in real time, allowing the main thread to visualize the notes.

💡 For lower latency (~80ms faster), the midiport backend offers direct ALSA access, though it does not support autoplayback.

🔄 Tip: Use This as a Fallback

If the playlist is empty or has finished playing, this setup allows users to continue interacting with the system using a physical MIDI device — no need to restart or reconfigure the backend.

🎷 3. Live MIDI Sniffing (`midiport` / `udpMidi`)

Both the midiport (Linux) and udpMidi (cross‑platform) backends operate independently of any playback or synth engine, continuously monitoring and visualising connected MIDI devices in real time.

📌 Setup — Linux (ALSA `midiport`)

List your ALSA ports
```
aconnect -l
```

Configure the midiport backend in src/constants.lua

-- src/constants.lua
M.backend    = "midiport"     -- select the ALSA‐sniffing backend
M.midiport   = "14:0"         -- ALSA client:port to sniff
M.shellHost  = "localhost"    -- TCP host for control commands
M.shellPort  = 9800           -- TCP port for control commands

Route your MIDI device (if needed)
```
aconnect 24:0 14:0
```
Launch the visualiser
```
love .
```

📌 Setup — Windows / macOS / Linux (`udpMidi`)

Prepare a MIDI input port
- On Windows, create a virtual MIDI port named midiBridgePort in loopMIDI and route your MIDI player’s output to it.
- On macOS/Linux, use any available CoreMIDI/ALSA port.

Configure the udpMidi backend in src/constants.lua

-- src/constants.lua
M.backend = "udpMidi"

Launch the bridge binary before starting the visualiser:
- Windows:
```
src/backends/udpMidi/binaries/udp-midi-bridge-windows.exe
```
- macOS/Linux: run the Node.js bridge script or platform‑specific binary.
Launch the visualiser
```
love .
```

🎶 What Happens

Notes from your MIDI device (USB keyboard, virtual port, or player output) are captured in real time.
The backend merges them into the active_notes Love2D thread channel (no file I/O delays).
The visualiser immediately reflects live playing.
Control commands (gain, player_start, etc.) flow over TCP exactly like in FluidSynth mode — no stdout buffering issues.

🎮 Keybindings & Controls

These keybindings fall into two categories:

Playback and command-menu commands sent over TCP to the active backend (Fluidsynth by default).
Local controls handled entirely by the visualizer core.

Only the h, d, and Ctrl + Q keys are consumed by the visualizer itself. All other bindings below are relayed as TCP messages when using a backend that accepts them (Fluidsynth by default). Future backends can repurpose or extend these.

Key	Function
p	Toggle play / pause
tab	Play current song from start
Enter	Move to next song
h	Toggle “instant” vs “offset” note-off mode
d	Toggle debug overlay (FPS, camera info, note OFF mode)
Ctrl + Q	Quit

Command-Menu Controls

Key	Function
:	Open the command menu
a	Set tempo in BPM
b	Set relative speed (e.g. `0.5` = half speed)
c	Play the file once, then loop it `<count>` more times (`0` = cancel loop; `-1` = infinite)
d	Seek to an absolute tick in the current MIDI file (1 quarter-note @ 100 BPM = 600 ticks)
e	Raw mode: send any command string directly to the backend’s TCP server
h	Help: show a scrollable list of all supported commands provided by the backend

📖 Documentation

🔄 Fluidsynth Backend Integration

The Fluidsynth backend is launched as a separate thread and communicates with the main visualizer via shared Lua channels. It handles MIDI playback and tracks active notes in real time.

🧠 How It Works

A thread spawns the Fluidsynth process using platform-specific commands.
Fluidsynth outputs noteon and noteoff events to its terminal (stdout).
The thread reads these events line-by-line and maintains a table of currently active notes.

📁 Output Format

The active_notes.lua file now serves two purposes:

Null backend
You hand-edit this file to define which notes are “on.”
midiport backend
On startup it’s cleared, then on each sniff cycle your live ALSA notes are merged with whatever you’ve hand-defined here before being pushed over the channel.

The FluidSynth backend does not read or write this file — it pushes note lists entirely in memory via Love2D channels.

Example `active_notes.lua`

-- Active MIDI notes (manual/merged)
return {
    60, 64, 67,  -- C major triad
}

⚙️ Configuration Channels

Channel Name	Purpose
backend	Backend identifier (`"fluidsynth"`, `"midiport"`, or `"null"`)
soundfonts	Path or comma-separated list of SoundFont files
songs	Comma-separated list of MIDI files to play
shellHost	TCP host for backend control commands
shellPort	TCP port for backend control commands
platform	OS identifier (`"linux"`, `"windows"`, etc.)
track_control	Signal the FluidSynth tracker to clear its active notes
quit	Signal the midiport sniffing thread to exit
midiPort	ALSA client:port string for the midiport backend (e.g. `"14:0"`)
active_notes	Table of currently active MIDI note keys

🧪 Notes on Stability

This system depends on:

FluidSynth emitting clean, parseable output
Channels being correctly populated before launch
The subprocess staying alive and responsive

If any part fails (e.g. malformed output, missing soundfont, broken pipe), the tracker may silently stop updating. For this reason, a fallback mode (null backend) is available for manual control.

💬 Community & Discussions

Your feedback drives this project! Join one of our GitHub Discussions below:

💭 Project's Usage & Feedback – share your experiences and questions about the project itself.
🎼 Interpretation of structure – Share ideas on different interpretations of the structure’s elements.

License 📝

This project is licensed under the GNU General Public License v3.0.
You can find the full license text in the LICENSE file.

Third-Party Licenses 📦

This project includes third-party assets that are distributed under their respective license terms.
Please refer to the individual files in the THIRD_PARTY_LICENSES/ directory for full details:

Asset Type	License File
3D Engine components	3dreamengine.md
Material textures & HDRIs	materials.md
MIDI files	midis.md
Node.js runtime	Node.js.md
Third‑Party Licenses for udpMidi Binaries	udpMidi_binaries_THIRD_PARTY_LICENSES.md

Acknowledgments 🙏

This project took shape thanks to the insight and encouragement of Edgar Delgado Vega.

Although the idea had been explored by 20th-century music–math theorists, it was only when E.D.V. encountered the concept that he immediately recognized its potential for new approaches in 12ET harmony. He urged me to share it more widely and encouraged me to bring it into academic and creative circles.

That encouragement transformed a dormant idea into a living project. From OpenSCAD to MeshLab, to Blender, to 3DreamEngine, to MIDI events, each stage brought new challenges and discoveries. Without Edgar Delgado’s vision and determination, this journey might never have begun.

The revolutionary idea of such a clear projection of 12‑ET into three‑dimensional space is entirely my own. Yet the code and documentation were realised with the assistance of AI — specifically Microsoft Copilot — guided through exhaustive patience and persistence on my part. Without the help of this tool, it would have been impossible for me to bring a project of this scale to life.

Name		Name	Last commit message	Last commit date
Latest commit History 898 Commits
.github/workflows		.github/workflows
3DreamEngine		3DreamEngine
THIRD_PARTY_LICENSES		THIRD_PARTY_LICENSES
asset_pipeline		asset_pipeline
assets		assets
docs		docs
extensions		extensions
midi_files		midi_files
shaders		shaders
soundfonts		soundfonts
src		src
.gitattributes		.gitattributes
.gitignore		.gitignore
LICENSE		LICENSE
README.md		README.md
active_notes.lua		active_notes.lua
conf.lua		conf.lua
main.lua		main.lua
midiport.sh		midiport.sh
run.sh		run.sh

License

jimishol/cholidean-harmony-structure

Folders and files

Latest commit

History

Repository files navigation