Jamaican Music Historical Catalogue

Status: Seed v1.0 (locked reference, 2026-09-22) — not a complete census. Public hub: /catalogue · Documents on-site: /catalogue/docs/ · Stats API: GET /api/catalogue/stats · Seed manifest: src/data/catalogue/manifest.json

v1.0 is the version of the reference documents and the seed data they lock (SHA-256 checksums in the manifest). The catalogue’s coverage is still a seed plus one MusicBrainz relationship hop — see the coverage table on the hub and seed §30.

A research / historical-census layer for JamaicansMusic.com. Its job is to answer “how many songs, albums, riddims and artists has Jamaica produced?” in a way that can defend every number: what was counted, what was excluded, which duplicates were merged, and which source supports each fact.

The two source-of-truth documents are checked in alongside this file:

1. Purpose and non-goals

Goal. Build the closest possible historical census of Jamaican genres, works, recordings, release groups, releases, riddims, artists, labels, studios and their relationships — supporting both historical research (evidence, citations, uncertainty, alternate names) and computational counting (deduplicated totals by genre, decade, origin, riddim, label, …).

Non-goals for Seed v1.0.

• It is not a claim to contain every Jamaican song, artist, album or riddim.
• It does not publish a single “total”. Large figures are estimates, labelled as such.
• It does not assert any artist’s birthplace, nationality or heritage (see §4).

2. Relationship to the Listen catalog

The site already has a Listen music library (music_artists, canonical_songs, recording_videos, riddims, labels, album_projects, …) built around what can be played from YouTube today. The Historical Catalogue is a different thing: what history records, whether or not a playable video exists.

They are deliberately not joined in v1:

How they may be bridged later (not in v1): a nullable hist_recording_id on canonical_songs and/or a hist_recording_links table (hist_recording_id, canonical_song_id, link_status, source_id). Any bridge is a link, never a merge — a Listen song may map to several historical recordings (versions), and most historical recordings will have no playable video. The riddims table in Listen (“Top Riddims”) and hist_riddims are likewise separate until a reviewed alias graph exists.

3. Non-negotiable counting rules (summary of seed §2, §20)

1. A song is not a recording. Work (composition) → recording (performance/mix) → track (recording on a release) → release group (album concept) → release (pressing). The headline album count is count(distinct hist_release_groups.id where primary_type = 'album'); reissues/editions live in hist_releases and never inflate it.
2. Riddims are first-class. One riddim, many recordings. known_recording_count on hist_riddims is a snapshot with a source and access date, never a lifetime total.
3. Artists are deduplicated by evidence, not by name: same MBID, documented legal identity + alias relationship, or same group identity.
4. Genre membership is many-to-many. Totals are deduplicated by recording id (or use a single primary genre); genre columns are never summed.
5. Explicit date precision — YYYY, YYYY-MM or YYYY-MM-DD as text, never a guessed day.
6. Preserve printed spellings (title_as_printed) separately from normalised titles.
7. Never overwrite a disputed value silently — hist_change_log and hist_source_assertions keep the prior claim.

4. Evidence vocabulary and origin policy

Every historically important field carries one of (hist_evidence_status enum):

Field-level confidence is high | medium | low | unknown (hist_confidence).

Artist origin (hist_artists.origin_bucket, enum hist_origin_bucket) is one of jamaican_born, jamaican_descent_born_abroad, non_jamaican_born_jamaican_genre_artist, mixed_or_multinational_group, unknown. The default — and the value for every seeded artist, including the 84 imported from MusicBrainz — is unknown. It may only change when a hist_source_assertions row documents birthplace / parentage. tests/catalogue.test.js fails if the seed asserts anything else.

5. Schema (migration 0011_historical_catalogue.sql)

24 tables, all hist_-prefixed. Mirrors seed §16 as a focused v1.

Deferred to v0.2: hist_release_tracks, hist_recording_credits (controlled roles from seed §15), hist_work_writers, hist_artist_origins history table.

The migration is idempotent and runs on Netlify DB / Neon / local Postgres / PGlite. Apply manually with npm run db:migrate:sql -- 0011_historical_catalogue if not on Netlify.

6. Seed data

Checked-in JSON under src/data/catalogue/ (typed by src/lib/catalogue/seed.ts). Two kinds of file: hand-curated registries (edited by historians, mirrored in the seed document) and generated files (written by scripts, never hand-edited, re-runnable). Counts are Seed v1.0 (2026-09-22).

Totals after seeding: 137 artists (53 + 84), 240 aliases, 104 memberships, 13 riddims, 34 recordings, 2,883 release groups (2,127 albums), 10 labels, 37 genres, 17 sources, ≈ 6,000 source assertions.

npm run catalogue:seed upserts everything into hist_* (keyed by seed id, safe to re-run, chunked), writes a hist_source_assertions row for each seeded fact (title / MBID / first-release date per release group, one per accepted musicbrainz_artist_id citing SRC008, …), rebuilds the seed-script memberships, and files an origin_evidence research note for every artist plus riddim_lineage notes for the JIS candidates.

MusicBrainz guardrails (Phase A + B): bulk MusicBrainz data lives only in the scratch database named by CATALOGUE_SCRATCH_DATABASE_URL; the Phase B neighbourhood import uses the public WS/2 API at ≤ 1 request / 1.5 s with an identifying User-Agent and requests no tags (CC BY-NC-SA); origin_bucket is never set from a MusicBrainz area; MusicBrainz tags are never imported as hist_genres; no MBID is invented (ambiguous seed matches go to review); generated seed ids (A100001…, AL100001…) are stable across re-runs.

validateCatalogueSeed() enforces: unique ids/slugs/MBIDs, controlled vocabularies, resolvable references, date precision, origin_bucket = unknown, and that every generated row is DATABASE_DERIVED with an MBID and a registered snapshot source.

Seed manifest (v1.0 lock). npm run catalogue:manifest writes manifest.json and the <!-- seed-manifest --> block in MASTER-CATALOGUE-SEED.md; npm run catalogue:manifest -- --check and npm test fail when a seed file changed without regenerating both. Editing seed data therefore always means: edit JSON → npm test → npm run catalogue:manifest → commit all three.

7. Public surface

• /catalogue (src/pages/catalogue.astro, static) — states it is a seed, carries a “Data as of” stamp (the manifest’s generatedAt, i.e. when the seed data last changed, plus the seed version), shows the four estimate cards (visibly tagged ESTIMATE), a Listen library box (songs / albums / riddims from src/data/music/meta.json and artist profiles from the content collection — the same sources /listen and /artists use, shown as a separate layer), the seeded-so-far strip (hand-curated vs MusicBrainz neighbourhood split), the coverage statistics table, counting rules, evidence legend, estimates-by-genre table, genre taxonomy, seed riddims (+ lineage candidates), the hand-curated artist registry (names link to /artists/<slug> where a profile exists — resolved through src/lib/music/artistSlug.ts so no ._ slug can leak — with MusicBrainz as a secondary link), the MusicBrainz neighbourhood summary (memberships, release groups by type), sources, the phased plan with live status and links to these documents.
• /catalogue/docs/<slug>/ (src/pages/catalogue/docs/[slug].astro, static) — every markdown file in this folder rendered as a site page at build time (content collection catalogueDocs, registry in src/lib/catalogue/docs.js), with canonical URLs, meta descriptions, a contents list and sitemap entries; /catalogue/docs/ is the index. The repository is private, so GitHub links are never shown to visitors; in-document links between these files are rewritten to the site pages (src/lib/catalogue/rehypeCatalogueDocLinks.js). The locked Kartel v1 document is served noindex as superseded.
• Planned sub-pages that were never built (/catalogue/artists, /albums, /riddims, /labels, /genres, /eras) 301 to the closest existing page (netlify.toml).
• GET /api/catalogue/stats (netlify/functions/catalogue-stats.mts → src/lib/catalogue/handleCatalogueStatsRequest.ts) — { seedVersion, seeded: { source, counts }, coverage[], musicbrainzNeighbourhood, estimates, docs }. seeded.source is database when the hist_* tables answer, seed_files otherwise; both report the same keys (artists, artistsHandCurated, artistsFromMusicbrainz, artistAliases, artistMemberships, releaseGroups, releaseGroupsAlbums, …). docs holds the on-site document URLs.

7.1 Automation — monthly seed refresh

.github/workflows/catalogue-seed-refresh.yml runs on the 2nd of every month (and on workflow_dispatch, with a dry_run input). It re-runs the Phase B neighbourhood import against the public MusicBrainz WS/2 API and opens a pull request only when the data changed:

1. npm run catalogue:mb-phase-b -- --neighbourhood — rate-limited to one request per 1.5 s with the identifying User-Agent from src/lib/catalogue/musicbrainz/dump.ts, 503/429 back-off.
2. npm run catalogue:neighbourhood-diff -- --restore --sync-source — compares the new file with the committed one ignoring the run stamps (accessedDate, resolvedAt, userAgent). No material change → the file is restored and the job ends. Otherwise SRC016’s accessedDate in sources.json is moved to the new snapshot date and a before/after summary is written.
3. npm run catalogue:manifest → npm run catalogue:validate → npm run catalogue:manifest -- --check.
4. peter-evans/create-pull-request opens (or updates) a PR from automation/catalogue-seed-refresh with the summary and a review checklist. Only the default GITHUB_TOKEN is used; nothing is pushed to main; the site database, Listen catalog and AdSense/GA configuration are untouched.

The exact-count locks in tests/catalogue.test.js and the figures quoted in this README are meant to move with such a PR — the reviewer updates them (and decides on a document version bump) before merging. The hub’s “Data as of” stamp follows the manifest date automatically.

8. Phased plan

Seed §30 separates the document/seed version (v1.0, locked) from catalogue coverage milestones: next is coverage 0.2 (score the Phase A extracts, label MBIDs, recording-level reconciliation for the §11 seed and SRC017, origin evidence for the first 50 hand-curated artists, riddim alias graph, duplicate reports); the catalogue 1.0 target (historian dashboard, alias workflow, CSV export) is unchanged.

9. What historians can do next

Work queues are already modelled (hist_research_notes.queue):

1. Origin evidence — 137 open notes, one per seeded artist (53 hand-curated first). Attach a source assertion for birth_place / birth_country / jamaican_parentage_note, then set origin_bucket.
2. Riddim lineage — verify Answer ↔ “Never Let Go”, Real Rock’s Studio One origin, and the My Conversation / Queen Majesty / Lonely Street candidates; set source_recording_id, first_known_year and hist_riddim_relations (same_as, remake_of, …).
3. Identity — Burning Spear person/group (MusicBrainz resolves the seed MBID to a Person; the §7 row stays unknown until reviewed); Charlie Chaplin (A000050, deejay vs actor — MBID is the disambiguated reggae entry); disambiguate General [1] and Volcano [2]; pick or reject the four review MBIDs (Lynn Taitt, Lord Fly, Harold Richardson, Sugar Belly Walker); review the 84 neighbourhood artists (all one documented hop from a seed anchor, none hand-verified).
4. Chronology — the nine §13 albums now have MBIDs and first-release dates; review the 195 undated and the 704 compilation release groups in the neighbourhood; decide which MusicBrainz “Album” rows are really reissue compilations.
5. Works — link the 34 seed recordings to hist_works; decide which [Dub …] / Version entries share a work; resolve the six Riddim Guide Dub … credits on Stalag to artists.

Add rows by editing the JSON in src/data/catalogue/ (validated by npm test), re-running npm run catalogue:manifest and npm run catalogue:seed; larger imports go through the Phase A/B/C scripts.