WikiMusic
the free music knowledge encyclopaedia

• GNU Guix, whether as a standalone distro (Guix System) or on top of any other GNU/Linux distribution. This is the one thing to install yourself, and it is also the whole of the setup: manifest.scm in the repository names every package and tool the project uses – Guile, Artanis, SQLite, GNU Make, gettext, watchexec, Emacs, and the Guile libraries wikimusic is built on – and channels.scm pins the Guile and Guix revision behind them.
• Not on Guix? You can still run wikimusic: read manifest.scm as the list of exactly what you need, and install those packages with your own package manager. The manifest is the honest answer to "what does this need", and it is what a non-Guix setup should be measured against.
• A database, created for you: SQLite by default, and the project is compatible with PostgreSQL and MySQL too (db.dbd in ~conf/artanis.conf= picks one).
• Optional: direnv, which reads the .envrc in the repository and enters the same environment for you as you cd into it.

A first run downloads the pinned toolchain, which takes a while; after that the environment is in the store and starting the site is seconds.

Every task the project defines lives in maak.scm, and each runs inside the pinned environment described above. maak --list names them all; the ones you will reach for most:

You will need a database to be present in the root folder of the project (alongside app, lib, etc.).

Wikimusic uses SQLite by default but is also compatible with PostgreSQL and MySQL.

You can automate the creation of the database, the enabling of WAL mode for concurrent writes, and the applying of the schema with maak init-db:

maak init-db

The schema is the migrations kept in the tree, under resources/migrations/sqlite, and they are applied by mutastructura, the migration tool pinned in manifest.scm. It keeps a ledger of what it has run in a migrations_history table of the database, applies only the migrations that are missing from it – oldest first – and records each file with a SHA256 so a change to one is noticed.

To apply migrations to a database that already exists – which is safe to run at any time, and the way to bring a deployment up to what the source in front of it expects – run the same tool through its own task:

maak migrate

Neither the server nor any other task applies migrations, and nothing reads the SQL files and hands them to sqlite3 itself: maak init-db and maak migrate are the two ways in. A copy of the SQLite file, or a JSON dump loaded into a fresh database (see Export, archival, and exchanging between instances), are the other ways to arrive at the tables.

Upon starting a blank new wikimusic deployment you will have no users in the database, and there is no sign-up route: the first account is made from the shell, and it should hold two permissions – wikimusic:modify-permissions, the super grant, and wikimusic:user-create, which is what the routes that make an account ask for. The second is not covered by the first: a route asks for the permissions it names, and what the super grant overrides is ownership, not that (see Accounts, permissions and ownership).

A password is stored as what hash-password derives – Argon2id where this build's libgcrypt offers it, and PBKDF2-SHA256 otherwise – with the algorithm, its cost and its salt written into the stored string itself. Any characters may be in a password: the string reaches hash-password as a string and is never evaluated, and the salt is drawn fresh inside that call, which is why the users table's own salt column is left empty here.

guix shell -m manifest.scm -- guile -L ./lib -L . -c \
  '(load "env.scm") (use-modules (wikimusic auth)) (display (hash-password "correct horse battery staple"))'

Take the string that prints, and insert the account with it. The permissions column is read as a Scheme list and is never evaluated, so the permissions named there are the ones the holder holds:

guix shell -m manifest.scm -- sqlite3 wikimusic.db <<'SQL'
insert into users (identifier, email, password, salt, permissions, username)
  values (lower(hex(randomblob(16))), 'you@example.com',
          '<the string hash-password printed>', '',
          '(wikimusic:modify-permissions wikimusic:user-create)', 'you');
SQL

The heredoc is quoted, so nothing in it is expanded by the shell: the hash contains $ and may as well contain anything else, and it is stored as the bytes hash-password printed rather than as whatever a shell made of them.

The rest of the accounts are then made from the running site, by a holder of wikimusic:user-create (see Accounts, permissions and ownership). Sign in with the email and the password you hashed, and the instance is yours to fill.

If running locally, it's recommended to use localhost instead of 127.0.0.1 as this will work better with the authentication/authorization and other cookies.

If you are hacking away at the project, I would suggest using maak dev since this will effectively run the project with hot-reloading capabilities and it will use the default artanis.conf where we configure hostname, db path, etc.

Running the test suite is maak test, and maak --list names every other task the project defines. maak build-manual renders this document to HTML, and maak deploy copies that rendering to the instance that serves it.

The unit tests live in test/veritas/unit, one file per area, and each is run by the framework with the deployment's own env.scm loaded. What a spec needs beyond that – being a deployment for the length of an assertion, when what it is proving is a switch a deployment turns off or a host it names – is a helper module rather than a test of its own, and it lives outside the unit directory for that reason: test/support/deployment.scm.

If you are hosting this for production, I would suggest running Wikimusic as a Shepherd service/daemon.

You can alternatively just feel free to do this any other way, I recommend using the manifest.scm . Check maak.scm for every task the project defines.

Here follows an example of a Guix (Shepherd service):

(define-module (wolk-jjba wikimusic)
  #:use-module (gnu)
  #:use-module (guix))

(use-service-modules shepherd)

(define-public (wolk-jjba-wikimusic-service config)
  (list (shepherd-service (documentation "Run wikimusic as a daemon")
                          (provision '(wikimusic))
                          (requirement '())
                          (start #~(make-forkexec-constructor '("make"
                                                                "production-server")
                                    #:directory
                                    "/etc/wikimusic/job-clone/wikimusic/trunk"
                                    #:environment-variables '("GUILE_LOAD_PATH=/run/current-system/profile/share/guile/site/3.0"
                                                              "GUILE_DBD_PATH=/run/current-system/profile/lib"
                                                              "C_INCLUDE_PATH=/run/current-system/profile/include"
                                                              "GUILE_LOAD_COMPILED_PATH=/run/current-system/profile/lib/guile/3.0/site-ccache:/run/current-system/profile/share/guile/site/3.0"
                                                              "LIBRARY_PATH=/run/current-system/profile/lib"
                                                              "GIT_SSL_NO_VERIFY=1"
                                                              "PATH=/run/privileged/bin:/run/current-system/profile/bin:/run/current-system/profile/sbin")))
                          (stop #~(make-kill-destructor))
                          (auto-start? #t)
                          (respawn? #t))))

(define-public wolk-jjba-wikimusic-service-type
  (service-type (name 'wikimusic)
                (description "Run wikimusic as a daemon")
                (extensions (list (service-extension
                                   shepherd-root-service-type
                                   wolk-jjba-wikimusic-service)))
                (default-value '())))

A session is a sealed token that names the signed-in user and the permissions they hold. A route states the permission it needs, and a request that lacks it is refused – that is the permission half of the rule. Wikimusic also has an ownership half: a contributor may change what they created.

The rule, in one sentence: a request may change a row when the session either holds the permission the route asks for and is the row's creator, or holds wikimusic:modify-permissions, the grant that overrides ownership for every row. It is enforced on every mutation – an entity's own edit route, and the rows that hang off it: an artwork, an external source, a song version, a link between two entities, and a comment's author. There is no route that rewrites or deletes a row without asking who made it.

• wikimusic:modify-permissions is the super grant. Its holder edits and deletes anything, on any entity, regardless of who created it. This is the permission to hand to the instance's maintainers.
• For everyone else, a mutation is allowed only when the row's created_by column names the session's user id. A song's edit route, for instance, looks the song up and refuses when created_by is someone else.
• The check has two shapes for the two ways a route meets a row. Where the page's own entity is at hand, guard-owner in (wikimusic controller) takes the row's created_by and answers #t when the change may go ahead. Where the row has to be read first – an artwork addressed by its own identifier, a relation addressed by the pair it joins – require-row-owner reads that row's created_by and defers to guard-owner with it.
• A refusal raises, it does not emit: the handler's return value is what Artanis renders, so a route that emitted a 403 and then carried on would have it thrown away and answer 500 instead. Both helpers raise the 403 that unwinds the route (require-row-owner a 404 first, when no such row exists). A route therefore only has to wrap its mutation in (when (guard-owner ...) ...) and cannot forget to refuse.
• Under both, may-mutate? in (wikimusic controller) answers #t for the super grant, and otherwise compares created_by with the session's owner-id. A refusal is a 403, rendered by the same error pages as every other status.

Reading is not restricted: anyone may read any public row. Only writing to a row you did not create needs the super grant.

The .gp5 is picked on the version form exactly as a PDF is: the browser turns the chosen file into a data URL and posts that, and the row keeps it. The file field accepts .gp5 files.

When a version carries a tab, its section lists what the tab's own header declares – the title, artist and album its author typed, who tabbed it, its tempo, its key, and how many measures and tracks the score holds – and offers the file for download under a name that says which song and which version it is. The facts come from the file and not from the row, so a reader can see what the download holds before spending it. A field the author left blank is left out, and the numbers a readable header always carries are always listed.

The reader stops at the counts of measures and tracks. Everything a player would play – measure headers, tracks, beats, notes – lies behind them and is left where it is; those parts are long, version-dependent and structured, and nothing the page shows needs them. The reader is strict about its place in the file: every string's declared length must match the bytes it occupies, and the file must still hold as many bytes as the next field needs, so a file that does not add up answers "the header of this tab could not be read" rather than a guess.

A file the site cannot parse is still its author's file and is still offered for download – it simply carries no facts on the page. A file of another Guitar Pro generation is refused by the same header check, since its fields are laid out differently. The reader is written against the Guitar Pro 5 format, and its module – (wikimusic guitarpro) – is the one place that knows the file's byte layout.

The tab is not only offered for download. Beside it is an offer to draw the tablature in the page itself, notation and tablature staff together, and to play it back: pressing the offer loads a renderer – alphaTab – and draws the score, and the play control that appears with it plays the tab through the browser's own audio.

Neither the renderer nor the sound it plays with is fetched until a reader asks for a tab. They are about two megabytes together, and a reader who came to read about a song should not pay for a tablature they never open; a page whose tabs are never opened costs a reader only the small script that offers them. When a reader does ask, the words describing what is happening – loading, drawn, or could not be drawn – are the page's own: the view writes every sentence the player can show and the script only un-hides one of them, so a translated page stays translated while a tablature loads.

What the renderer and the sound are, where they came from, and the licences they travel under are written down beside them in pub/vendor/alphatab/README.md. They are fetched by maak alphatab and committed, so an instance serves its own files and reaches no CDN at run time; raising the release in maak.scm and running that task again is how they are updated.

A tab the site's own reader cannot follow is still offered to the renderer, and a tab the renderer cannot draw says so on the page while leaving the file downloadable: the two readers are independent, and a file that defeats one of them is not therefore a file the site will not hold.

The list of file extensions artanis serves statically – in ENTRY – carries sf2 for the SoundFont, so a deployment that narrows that list should keep it.

A version that carries an ASCII tab – the plain-text transcription so many players have written and passed around – is a page to practise from rather than only to read.

• Auto-scrolling moves the tab down the screen at a speed the reader chooses, a few lines at a time, so a hand is free to play.
• The chords the tab names are collected and shown under it as a small guitar chord guide, each chord with the fingering the transposition implies (Am x02210, F 133211), and a control highlights those chord names in the tab itself.
• The metronome under the song is a click track whose sound is made in the browser – twenty milliseconds of a shaped tone per beat – rather than fetched, so a reader who never opens it pays nothing for it, and its tempo, its meter and its beat lamps are all the page's own Scheme.

None of the three is needed to read the song, and none of them is a video or a third-party widget: they are three small scripts in pub/js/, offered by the page, hidden until the script that drives them is there.

GET /export/json

curl -O https://wikimusic.example/export/json

The answer is application/json holding every content table row by row in a shape this same instance can read back:

{
  "format": "wikimusic-export",
  "version": 1,
  "exported-at": 1791907095,
  "tables": [
    { "table": "artists", "rows": [ { "identifier": "...", ... } ] },
    { "table": "songs",   "rows": [ ... ] }
  ]
}

tables is a list in an order that respects the foreign keys among the tables, so an entity is always written before anything that points at it. Each row is the raw column values – the dump is not rendered – and the values are what a script or another instance reads back.

The tables carried are the content itself: artists, genres, songs, their artworks, external sources and relations, a song's contents, and every entity's opinions and comments. users and password_reset are deliberately absent: a share is a link to somebody's account, not a copy of it, and each row's created_by is resolved by the receiving instance on its own.

GET /export/html

curl -O https://wikimusic.example/export/html

The same rows, as one long self-contained page: every artist, genre and song with its name linked to its page and its description already rendered from Org. It is what an archivist wants – a copy that reads in a browser with no server behind it – and it links to the JSON dump above for the machine's copy.

POST /import/json

curl -X POST -H 'Content-Type: application/json' \
     --data @wikimusic-export.json https://other.example/import/json

The body is the JSON the export above produced. Import is the one write here and the change is to the whole database, so it is gated: the session must hold wikimusic:modify-permissions, the super grant, and the route is never reachable anonymously. Rows are inserted one at a time, and a row that fails – a foreign key pointing at a user this instance does not have, or an identifier already taken – is counted and skipped rather than aborting the share, so a partial import is still a partial success. The answer says what happened:

{ "format": "wikimusic-import", "inserted": 812, "skipped": 47 }

A table is only ever written if it is in the controller's fixed list of content tables, so a crafted body cannot reach a table of its own choosing.

For a whole-instance backup the SQLite file may simply be copied – see Running WikiMusic – but the two export routes and the import are the supported, format-stable way to move content between instances.

Every sentence a reader sees is written in the source as a literal argument of G_, which is what puts it in the template and in front of a translator:

(G_ "browse the songs")

The sentences are extracted from the source into a template (sys/i18n/locale/translate.pot), each language has its own catalog beside it (sys/i18n/locale/<lang>/LC_MESSAGES/translate.po and the compiled .mo the server reads), and the server is told where they are in ENTRY – the text domain is bound to ./sys/i18n/locale and conf/artanis.conf sets session.i18n = locale.

# after changing a string in the source: refresh the template, then the catalogs
maak i18n-update-template
maak i18n-merge

# a new catalog for a language that has none yet (en, nl, es and ja ship with one)
maak i18n-init-nl

# compile every catalog into the .mo the server reads
maak i18n-compile

A language that is not in maak.scm's i18n-locales is not compiled, and one that is not in wikimusic-locales in lib/wikimusic/html.scm is not offered in the language control; adding both is what makes a new language a first-class one. Translators are very welcome – a catalog is ordinary gettext, and no Scheme is needed to fill one in.

A string that is reworded is a string no catalog holds any more, so it silently falls back to English in the three translated locales. The sentences the project says about itself are therefore written once, in lib/wikimusic/prose.scm, where that rule is stated; everywhere else, the words a page shows are written where they are read.

Nothing here is a big ask, and every one of these is a real contribution:

• Tell me what is missing or awkward. A ticket on the board, a message to <jjbigorra@gmail.com>, or a note under the entry you were reading – a bug report from somebody actually using the thing is worth more than a feature list.
• Send a PR/patch. Branches and patches on the forge are how the code arrives; a small, explained change is easier to review than a large one, and a change that comes with the story of why is easier still.
• Translate, or fix a translation. Dutch, Spanish and Japanese are in; more are welcome. See Translated into four languages – a catalog is ordinary gettext and needs no Scheme.
• Write a description. The encyclopaedia is the point, and an artist, a genre or a song with a good description in Org mode is worth several commits of code.
• Run an instance. Fork it, name it, fill it with your own corner of music; every one of these sections is a description of a thing you are free to change.

maak test      # the whole suite, one spec at a time
maak fmt       # format every Scheme file the way the Guix style guide does

The suite lives in test/veritas/unit, one file per area of the project, and each spec is run with the deployment's own env.scm loaded – the same file the server loads, so a test proves the thing a deployment runs. A spec that needs to be a deployment for one assertion (a host it names, a switch it turns off) uses the helper in test/support/deployment.scm instead of a test of its own.

A push to the trunk branch is picked up by Woodpecker (.woodpecker/byggsteg.yaml), which submits a job to the jointhefreeworld Byggsteg service. The job that runs on the deployment is the project's own tasks, in order: maak-test, pull-and-restart, maak-deploy – so what is merged is what is tested, and then what is served.

This project adheres to the jointhefreeworld AI (Artificial Intelligence) policy.

Our core principle is simple: AI should assist human creativity and problem-solving, never replace human reasoning.

While tools like Large Language Models (LLMs) and interactive chatbots can be beneficial for reviewing, refactoring small functions, or acting as a sounding board, they should be used with moderation.

We require a human in the loop for all contributions. The use of autonomous AI agents to automatically generate and submit pull requests to this project is strictly prohibited.

This project adheres to the jointhefreeworld code of conduct. Find it here:

https://jointhefreeworld.org/blog/articles/personal/jointhefreeworld-code-of-conduct/index.html

In summary, we foster an inclusive, respectful, and cooperative environment for all contributors and users of this free software project. Inspired by the ideals of the GNU Project, we strive to uphold freedom, equality, and community as guiding principles. We believe that collaboration in a community of mutual respect is essential to creating excellent free software.

wikimusic serves its own assets, and some of them are somebody else's work, vendored into the tree rather than linked to, so that an instance reaches no CDN at run time and works with no network of its own:

• alphaTab (MPL-2.0) draws a stored Guitar Pro tab as notation and tablature and plays it back;
• the SONiVOX SoundFont (Apache-2.0, © Sonic Network Inc.) is what makes a played-back tab audible, and fits beside it;
• Bravura (SIL OFL 1.1) is the music font a tablature is drawn with;
• Intel One Mono and Plus Jakarta Sans (both SIL OFL 1.1) are the two web families the interface is set in beside Lucida, and their licence texts travel with them;
• Phosphor (MIT) is the icon set every control on the site wears;
• the three Lucida families the interface is set in, and the picture the project shows of itself, which are the author's own files rather than a third party's – what the project takes the position on the Lucida files to be is set out below, under On Lucida fonts.

Two tasks fetch them, and each is the only way its own files should change: maak vendor-assets brings the typefaces, the glyphs and the project's picture into pub/ – writing google.css and phosphor.css with every address made relative on the way – and maak alphatab brings the renderer, the music font and the SoundFont into pub/vendor/alphatab/. Their licences, their provenance and the task that fetches them are written down beside the files, in pub/vendor/alphatab/README.md, pub/vendor/fonts/README.md and pub/vendor/phosphor/README.md. They are unmodified, and their source stays where it came from.

The rewriting a refresh does is the part of it that no fetch can be trusted to show – a copy that went on naming the host it came from would be a tree full of files nobody reads – so it is pinned by the test suite against a written stylesheet rather than a live one, with no network: test/veritas/unit/assets-spec.scm.

Lucida fonts are proprietary typefaces originally developed by Bigelow & Holmes. They remain a long-time favorite for many people and are very legible in many situations and hardware/software. While they are bundled pre-installed on many operating systems and platforms, they remain proprietary fonts.

You may use these font files under principles of fair use or for local development and non-commercial purposes, provided it is not for profit and used without securing a proper commercial license where required by law. Users and contributors are solely responsible for ensuring their usage complies with applicable software licensing terms and intellectual property rights.