Files
strapi/examples/complex/scripts/db-mariadb.js
T
DMehaffy 19fef31e08 chore(examples): migration performance benchmark harness + mariadb/sqlite + anti-pattern schemas (#26036)
* chore(examples): add mariadb + sqlite + podman support to complex

Extends the complex example's DB tooling to cover all Strapi-supported
database dialects and both container runtimes, as groundwork for a
migration performance benchmark harness:

- New compose.js runtime shim auto-detects podman compose / podman-compose
  / docker compose / docker-compose and the matching container CLI; all
  existing db-* scripts now go through it so podman-only environments
  work without installing docker
- New db-mariadb.js mirrors db-mysql.js using mariadb-dump / mariadb CLIs
  and adds a mariadb:11 service on port 3307 to docker-compose.dev.yml
- New db-sqlite.js handles file-based snapshot/restore/wipe/check via
  fs.copy / better-sqlite3
- db-utils.js falls back to `<runtime> ps --filter name=` for container
  lookup since podman-compose doesn't support `ps -q`
- develop-with-db.js and the v4 templates (develop-with-db.js,
  seed-with-db.js) handle mariadb + sqlite (sqlite skips compose)
- setup-v4-project.js includes better-sqlite3 in v4 deps, database.js
  template covers all 4 clients, and compose.js is copied into the
  v4 scaffold scripts dir (dep of db-utils.js)

All four DBs smoke-tested locally against podman: start/check/snapshot/
restore/wipe cycle works for mariadb; cp-based snapshot cycle works
for sqlite.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore(examples): add migration perf benchmark harness

Three new scripts enable per-migration timing and baseline-vs-candidate
comparison reports for v4→v5 migrations in the complex example:

- bench-hook.js: Node --require preload that intercepts require('umzug')
  and subscribes to Umzug's native `migrating`/`migrated` events for
  sub-ms timing. Captures every migration that runs (including dynamically
  registered ones like discard-drafts and EE-only release migrations)
  without hardcoding names. Dumps to a JSON file on process exit; self-
  disables when STRAPI_BENCH_HOOK_OUTPUT is unset.
- bench.js: orchestrator with `run`, `seed`, and `suite` subcommands.
  `run` restores a snapshot, spawns Strapi in migrate-then-exit mode
  with the hook preload, collects row counts, and writes a result JSON
  with baseline/candidate attribution, env capture (node, CPU, memory,
  DB version, host type), and config (multiplier, seed/hook modes).
  `seed` wipes the DB, runs the v4 seed via seed-with-db.js, then
  snapshots. First iteration supports --strapi-source=local only;
  experimental/pinned are stubbed with a clear error.
- bench-compare.js: takes N labels and emits both a clipboard-friendly
  markdown report (stdout + results/compare-*.md) and a self-contained
  HTML report (results/compare-*.html) with inline SVG bar charts,
  per-DB grid, sortable tables, collapsible raw JSON, and a light/dark
  adaptive theme via prefers-color-scheme. No CDN deps.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore(examples): bench harness smoke-test fixes

Fixes discovered during the end-to-end smoke test on the existing 6
content types at multiplier=1:

- bench-hook.js: switch from subclass-based wrapping to in-place
  Umzug.prototype.up patching. The subclass approach replaced the module
  export at require time, but Node's module cache hands out the original
  class on subsequent requires, so listeners weren't attached on all
  instances. In-place prototype patching works for every instance
  regardless of how Umzug was imported.
- bench-hook.js: flush incrementally after each recorded migration.
  Strapi's shutdown path can bypass process.on('exit') handlers under
  some conditions (signal or explicit exit from deep inside), causing
  fully-collected timing data to be lost. Writing after each recording
  makes the benchmark resilient to any exit path.
- bench.js: compile TypeScript configs via @strapi/typescript-utils
  before createStrapi().load(). The examples/complex project has .ts
  config files; the Strapi CLI compiles them to dist/ before boot but
  our direct node -e loader skipped this, producing
  "db.config.connection undefined" failures.
- bench.js: propagate STRAPI_BENCH_HOOK_DEBUG to the Strapi child so
  debug output is visible when tracing hook behavior.
- bench-compare.js: rework the SVG chart. Dynamic label column sized
  to the longest migration name (up to 420px), 80px reserved on the
  right for value labels so they never clip, inlined monospace font
  (SVG text doesn't reliably inherit CSS variables from the surrounding
  stylesheet), and `dominant-baseline="middle"` for proper vertical
  centering.

Verified: full pipeline (setup:v4 → seed → snapshot → bench:run →
bench:compare) works against postgres at multiplier=1. Ran a baseline
vs cherry-picked PR #25988 comparison — captured all 7 v4→v5 migrations,
produced both markdown and HTML reports with correct test-setup
attribution and delta coloring.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(examples): run ANALYZE before db:check to get fresh row counts

pg_stat_user_tables.n_live_tup and information_schema.tables.table_rows
are approximate and can lag behind reality by minutes or hours depending
on autovacuum / ANALYZE cadence. For a benchmark harness that publishes
row-count numbers in its reports, stale counts are misleading.

Trigger a refresh via ANALYZE (postgres) / ANALYZE TABLE per-table
(mysql/mariadb) before each db:check invocation. Best-effort on the
mysql/mariadb side — fall through to stale stats if ANALYZE fails rather
than error the whole command.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(examples): add hc-m2m-source/target anti-pattern schemas

First anti-pattern schema pair for migration benchmark stress-testing.
A high-cardinality many-to-many relation that forces the v4→v5
discard-drafts migration's copyRelationTableRows code path to span
multiple chunks (>1000 rows) — the same scenario PR #25988's caching
fixes target.

- src/api/hc-m2m-source: collection type with DP and a manyToMany
  relation to hc-m2m-target (owning side)
- src/api/hc-m2m-target: collection type with DP and the inverse
  manyToMany back to source
- setup-v4-project.js: include both in the v4 scaffold CONTENT_TYPES
- seed-v4.js: seedHcM2m() method that creates sources + targets and
  fans out 10 targets-per-source via the M2M relation. BASE counts at
  m=1 are tiny (15 pub + 5 draft per side) but at m=100 produce ~2K
  sources × ~2K targets × 10 = 20K join rows, crossing the 1000-row
  chunk boundary multiple times

Intentionally NOT a realistic content-type design — this is a
stress-test fixture. See the description in schema.json.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(examples): render multiplier x db matrix in bench-compare

Rework bench-compare to index results by (label, multiplier, dbEngine)
triples pulled from each result JSON's own fields, rather than parsing
labels out of filenames. Lets the same canonical baseline/candidate
label span any number of (multiplier, db) combinations and produces:

- A speedup matrix at the top: rows = multipliers, cols = databases,
  cells = "baseline -> candidate (delta%)". Missing cells render as
  "-" so partial data still produces a useful report.
- A data-availability matrix listing what ran vs what's still missing.
- Per-(db, multiplier) detail sections as collapsible details in
  HTML, all expanded in markdown.

Also:
- New flag syntax: --baseline <label> / --candidate <label>, with
  positional args kept for backward compat.
- Legacy labels that embedded the multiplier (e.g. "baseline-m100")
  are normalized to their base form ("baseline"), letting older
  result files keep working.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(examples): force TCP for mysql/mariadb CLI in containers

The mysql/mariadb CLI tools default to connecting via unix socket at
/var/run/mysqld/mysqld.sock, which isn't populated in the official
mysql:8 / mariadb:11 container images. Every invocation (check,
snapshot, restore, wipe, readiness probe, version probe) needs an
explicit -h 127.0.0.1 to force TCP via the container's loopback.

Without this fix, bench:seed and bench:run error out with
"Can't connect to local MySQL server through socket" on anything
requiring the CLI inside the container (pg_stat-style row-count
queries, snapshot restore, etc.).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* enhancement(examples): parallelize entity creation in seed-v4

Replace sequential for-loops of `await entityService.create(...)` with
a `concurrentMap(count, concurrency, taskFn)` helper that runs N tasks
in flight at once. At SEED_CONCURRENCY=5 (default), a seed that was
strictly serial now fans out into 5 parallel creates.

Concurrency chosen conservatively: Strapi v4's default knex pool is
`{min: 2, max: 10}`, and entity-heavy creates (components + DZs +
localizations) can use multiple connections per call. 5 keeps us well
under the pool ceiling. Tune via `SEED_CONCURRENCY=<n>` env var if
you've also raised the pool max.

Applied to: seedBasic, seedBasicDp, updateComponentRelations,
seedBasicDpI18n, seedRelation, seedRelationDp, seedRelationDpI18n,
seedHcM2m (all entity-creation loops plus their follow-up
self-reference update loops).

Not yet done: incremental seeding (restore previous snapshot + seed
delta) — a separate optimization tracked as a follow-up.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs(examples): update complex README for new bench tooling + DBs

README was documenting just the original 6-type, postgres+mysql
workflow. Updated to cover everything this branch adds:

- 8 content types (added hc-m2m-source/target anti-patterns)
- 4 supported databases (added mariadb + sqlite)
- Container runtime auto-detection (podman compose / podman-compose /
  docker compose / docker-compose) with STRAPI_BENCH_RUNTIME override
- Benchmark harness workflow (bench:seed / bench:run / bench:compare /
  bench:suite) for reviewing migration-performance PRs
- SEED_CONCURRENCY, STRAPI_BENCH_HOOK_OUTPUT, STRAPI_BENCH_HOOK_DEBUG,
  and the existing port-override env vars
- MariaDB port default 3307 to avoid colliding with MySQL on 3306

Also collapsed the redundant per-DB command sections (postgres and
mysql both had identical copy-pasted blocks) into a single
'yarn db:<op>:<db>' table since the commands are symmetric across
all four dialects.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(examples): align better-sqlite3 version with monorepo convention

I picked `11.3.0` arbitrarily. Every other example and tests/app-template
use `12.8.0`, and the root yarn.lock already resolves that version.
Without alignment CI's `yarn install --immutable` fails with 'lockfile
would have been modified', cascading every subsequent job (build, pretty,
commitlint, aggregate_test_result) to red.

Bumping to `12.8.0` to match, regenerating yarn.lock.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: throw instead of return to fail fast

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Ben Irvin <ben@innerdvations.com>
2026-04-27 18:28:08 +02:00

220 lines
7.4 KiB
JavaScript

#!/usr/bin/env node
const { execSync } = require('child_process');
const fs = require('fs');
const path = require('path');
const {
getContainerName: getComposeContainerName,
getComposeEnv,
COMPOSE_PROJECT_NAME,
} = require('./db-utils');
const { getComposeCommand, getContainerCommand } = require('./compose');
const SCRIPT_DIR = __dirname;
const COMPLEX_DIR = path.resolve(SCRIPT_DIR, '..');
const DOCKER_COMPOSE_FILE = path.join(COMPLEX_DIR, 'docker-compose.dev.yml');
const SNAPSHOTS_DIR = path.join(COMPLEX_DIR, 'snapshots');
const DB_NAME = process.env.DATABASE_NAME || 'strapi';
const DB_USER = process.env.DATABASE_USERNAME || 'strapi';
const DB_PASSWORD = process.env.DATABASE_PASSWORD || 'strapi';
function composeCmd() {
const { exe, prefixArgs } = getComposeCommand();
return [exe, ...prefixArgs].join(' ');
}
function containerCmd() {
return getContainerCommand();
}
function getContainerName() {
const name = getComposeContainerName(DOCKER_COMPOSE_FILE, COMPLEX_DIR, 'mariadb');
if (!name) {
throw new Error(
`MariaDB container not found. Start it with "yarn db:start:mariadb" (COMPOSE_PROJECT_NAME=${COMPOSE_PROJECT_NAME}).`
);
}
return name;
}
const command = process.argv[2];
const snapshotName = process.argv[3];
function ensureSnapshotsDir() {
if (!fs.existsSync(SNAPSHOTS_DIR)) {
fs.mkdirSync(SNAPSHOTS_DIR, { recursive: true });
}
}
function execShell(cmd) {
try {
execSync(cmd, { stdio: 'inherit', cwd: COMPLEX_DIR, env: getComposeEnv(), shell: '/bin/bash' });
} catch (error) {
console.error(`Error executing: ${cmd}`);
process.exit(1);
}
}
switch (command) {
case 'start':
console.log('Starting mariadb container...');
execShell(`${composeCmd()} -f ${DOCKER_COMPOSE_FILE} up -d mariadb`);
console.log('✅ MariaDB started');
break;
case 'stop':
console.log('Stopping mariadb container...');
execShell(`${composeCmd()} -f ${DOCKER_COMPOSE_FILE} stop mariadb`);
console.log('✅ MariaDB stopped');
break;
case 'snapshot':
if (!snapshotName) {
console.error('Error: Snapshot name is required');
console.error('Usage: node db-mariadb.js snapshot <name>');
process.exit(1);
}
ensureSnapshotsDir();
const snapshotPath = path.join(SNAPSHOTS_DIR, `mariadb-${snapshotName}.sql`);
console.log(`Creating snapshot: ${snapshotName}...`);
{
const containerName = getContainerName();
try {
// `mariadb-dump` is the modern binary; older images fall back to `mysqldump` alias.
execSync(
`${containerCmd()} exec ${containerName} mariadb-dump -h 127.0.0.1 -u${DB_USER} -p${DB_PASSWORD} ${DB_NAME} > ${snapshotPath}`,
{ stdio: 'inherit', cwd: COMPLEX_DIR, shell: '/bin/bash' }
);
console.log(`✅ Snapshot created: ${snapshotPath}`);
} catch (error) {
console.error(`Error creating snapshot: ${error.message}`);
process.exit(1);
}
}
break;
case 'restore':
if (!snapshotName) {
console.error('Error: Snapshot name is required');
console.error('Usage: node db-mariadb.js restore <name>');
process.exit(1);
}
const restorePath = path.join(SNAPSHOTS_DIR, `mariadb-${snapshotName}.sql`);
if (!fs.existsSync(restorePath)) {
console.error(`Error: Snapshot not found: ${restorePath}`);
process.exit(1);
}
console.log(`Restoring snapshot: ${snapshotName}...`);
{
const containerName = getContainerName();
try {
// Drop and recreate database
execSync(
`${containerCmd()} exec ${containerName} mariadb -h 127.0.0.1 -u${DB_USER} -p${DB_PASSWORD} -e "DROP DATABASE IF EXISTS ${DB_NAME}; CREATE DATABASE ${DB_NAME};"`,
{ stdio: 'inherit', cwd: COMPLEX_DIR, shell: '/bin/bash' }
);
// Restore from snapshot
execSync(
`${containerCmd()} exec -i ${containerName} mariadb -h 127.0.0.1 -u${DB_USER} -p${DB_PASSWORD} ${DB_NAME} < ${restorePath}`,
{ stdio: 'inherit', cwd: COMPLEX_DIR, shell: '/bin/bash' }
);
console.log(`✅ Snapshot restored: ${snapshotName}`);
} catch (error) {
console.error(`Error restoring snapshot: ${error.message}`);
process.exit(1);
}
}
break;
case 'wipe':
console.log('Wiping mariadb database...');
{
const containerName = getContainerName();
try {
execSync(
`${containerCmd()} exec ${containerName} mariadb -h 127.0.0.1 -u${DB_USER} -p${DB_PASSWORD} -e "DROP DATABASE IF EXISTS ${DB_NAME}; CREATE DATABASE ${DB_NAME};"`,
{ stdio: 'inherit', cwd: COMPLEX_DIR, shell: '/bin/bash' }
);
console.log('✅ Database wiped');
} catch (error) {
console.error(`Error wiping database: ${error.message}`);
process.exit(1);
}
}
break;
case 'check':
{
const containerName = getContainerName();
try {
// Refresh information_schema.tables.table_rows via ANALYZE TABLE;
// without it, stats can lag reality by hours.
const analyzeQuery = `
SELECT CONCAT('ANALYZE TABLE ', table_name, ';') AS cmd
FROM information_schema.tables
WHERE table_schema = '${DB_NAME}';
`;
try {
const analyzeList = execSync(
`${containerCmd()} exec ${containerName} mariadb -h 127.0.0.1 -u${DB_USER} -p${DB_PASSWORD} -D ${DB_NAME} -e "${analyzeQuery.trim()}" -s -N`,
{ encoding: 'utf8', cwd: COMPLEX_DIR, shell: '/bin/bash' }
);
const cmds = analyzeList.trim().split('\n').filter(Boolean).join(' ');
if (cmds) {
execSync(
`${containerCmd()} exec ${containerName} mariadb -h 127.0.0.1 -u${DB_USER} -p${DB_PASSWORD} -D ${DB_NAME} -e "${cmds}"`,
{ stdio: 'ignore', cwd: COMPLEX_DIR, shell: '/bin/bash' }
);
}
} catch {
// Best-effort; fall through to stale stats rather than fail.
}
const query = `
SELECT
table_name,
table_rows
FROM information_schema.tables
WHERE table_schema = '${DB_NAME}'
ORDER BY table_name;
`;
const output = execSync(
`${containerCmd()} exec ${containerName} mariadb -h 127.0.0.1 -u${DB_USER} -p${DB_PASSWORD} -D ${DB_NAME} -e "${query.trim()}" -s -N`,
{ encoding: 'utf8', cwd: COMPLEX_DIR, shell: '/bin/bash' }
);
const lines = output
.trim()
.split('\n')
.filter((l) => l.trim());
if (lines.length === 0) {
console.log('📊 No tables found (database is empty or wiped)');
} else {
console.log('📊 Database Tables (approximate row counts):\n');
console.log('Table Name | Row Count');
console.log('------------------------------------|----------');
for (const line of lines) {
const [table, count] = line.trim().split('\t');
const paddedName = (table || '').padEnd(35);
const rowCount = count || '0';
console.log(`${paddedName} | ${rowCount}`);
}
}
} catch (error) {
console.error(`Error checking database: ${error.message}`);
process.exit(1);
}
}
break;
default:
console.error('Error: Unknown command');
console.error('Usage: node db-mariadb.js <start|stop|snapshot|restore|wipe|check> [name]');
process.exit(1);
}