mirror of
https://github.com/strapi/strapi.git
synced 2026-05-03 16:22:30 +00:00
19fef31e08
* chore(examples): add mariadb + sqlite + podman support to complex Extends the complex example's DB tooling to cover all Strapi-supported database dialects and both container runtimes, as groundwork for a migration performance benchmark harness: - New compose.js runtime shim auto-detects podman compose / podman-compose / docker compose / docker-compose and the matching container CLI; all existing db-* scripts now go through it so podman-only environments work without installing docker - New db-mariadb.js mirrors db-mysql.js using mariadb-dump / mariadb CLIs and adds a mariadb:11 service on port 3307 to docker-compose.dev.yml - New db-sqlite.js handles file-based snapshot/restore/wipe/check via fs.copy / better-sqlite3 - db-utils.js falls back to `<runtime> ps --filter name=` for container lookup since podman-compose doesn't support `ps -q` - develop-with-db.js and the v4 templates (develop-with-db.js, seed-with-db.js) handle mariadb + sqlite (sqlite skips compose) - setup-v4-project.js includes better-sqlite3 in v4 deps, database.js template covers all 4 clients, and compose.js is copied into the v4 scaffold scripts dir (dep of db-utils.js) All four DBs smoke-tested locally against podman: start/check/snapshot/ restore/wipe cycle works for mariadb; cp-based snapshot cycle works for sqlite. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore(examples): add migration perf benchmark harness Three new scripts enable per-migration timing and baseline-vs-candidate comparison reports for v4→v5 migrations in the complex example: - bench-hook.js: Node --require preload that intercepts require('umzug') and subscribes to Umzug's native `migrating`/`migrated` events for sub-ms timing. Captures every migration that runs (including dynamically registered ones like discard-drafts and EE-only release migrations) without hardcoding names. Dumps to a JSON file on process exit; self- disables when STRAPI_BENCH_HOOK_OUTPUT is unset. - bench.js: orchestrator with `run`, `seed`, and `suite` subcommands. `run` restores a snapshot, spawns Strapi in migrate-then-exit mode with the hook preload, collects row counts, and writes a result JSON with baseline/candidate attribution, env capture (node, CPU, memory, DB version, host type), and config (multiplier, seed/hook modes). `seed` wipes the DB, runs the v4 seed via seed-with-db.js, then snapshots. First iteration supports --strapi-source=local only; experimental/pinned are stubbed with a clear error. - bench-compare.js: takes N labels and emits both a clipboard-friendly markdown report (stdout + results/compare-*.md) and a self-contained HTML report (results/compare-*.html) with inline SVG bar charts, per-DB grid, sortable tables, collapsible raw JSON, and a light/dark adaptive theme via prefers-color-scheme. No CDN deps. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore(examples): bench harness smoke-test fixes Fixes discovered during the end-to-end smoke test on the existing 6 content types at multiplier=1: - bench-hook.js: switch from subclass-based wrapping to in-place Umzug.prototype.up patching. The subclass approach replaced the module export at require time, but Node's module cache hands out the original class on subsequent requires, so listeners weren't attached on all instances. In-place prototype patching works for every instance regardless of how Umzug was imported. - bench-hook.js: flush incrementally after each recorded migration. Strapi's shutdown path can bypass process.on('exit') handlers under some conditions (signal or explicit exit from deep inside), causing fully-collected timing data to be lost. Writing after each recording makes the benchmark resilient to any exit path. - bench.js: compile TypeScript configs via @strapi/typescript-utils before createStrapi().load(). The examples/complex project has .ts config files; the Strapi CLI compiles them to dist/ before boot but our direct node -e loader skipped this, producing "db.config.connection undefined" failures. - bench.js: propagate STRAPI_BENCH_HOOK_DEBUG to the Strapi child so debug output is visible when tracing hook behavior. - bench-compare.js: rework the SVG chart. Dynamic label column sized to the longest migration name (up to 420px), 80px reserved on the right for value labels so they never clip, inlined monospace font (SVG text doesn't reliably inherit CSS variables from the surrounding stylesheet), and `dominant-baseline="middle"` for proper vertical centering. Verified: full pipeline (setup:v4 → seed → snapshot → bench:run → bench:compare) works against postgres at multiplier=1. Ran a baseline vs cherry-picked PR #25988 comparison — captured all 7 v4→v5 migrations, produced both markdown and HTML reports with correct test-setup attribution and delta coloring. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(examples): run ANALYZE before db:check to get fresh row counts pg_stat_user_tables.n_live_tup and information_schema.tables.table_rows are approximate and can lag behind reality by minutes or hours depending on autovacuum / ANALYZE cadence. For a benchmark harness that publishes row-count numbers in its reports, stale counts are misleading. Trigger a refresh via ANALYZE (postgres) / ANALYZE TABLE per-table (mysql/mariadb) before each db:check invocation. Best-effort on the mysql/mariadb side — fall through to stale stats if ANALYZE fails rather than error the whole command. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(examples): add hc-m2m-source/target anti-pattern schemas First anti-pattern schema pair for migration benchmark stress-testing. A high-cardinality many-to-many relation that forces the v4→v5 discard-drafts migration's copyRelationTableRows code path to span multiple chunks (>1000 rows) — the same scenario PR #25988's caching fixes target. - src/api/hc-m2m-source: collection type with DP and a manyToMany relation to hc-m2m-target (owning side) - src/api/hc-m2m-target: collection type with DP and the inverse manyToMany back to source - setup-v4-project.js: include both in the v4 scaffold CONTENT_TYPES - seed-v4.js: seedHcM2m() method that creates sources + targets and fans out 10 targets-per-source via the M2M relation. BASE counts at m=1 are tiny (15 pub + 5 draft per side) but at m=100 produce ~2K sources × ~2K targets × 10 = 20K join rows, crossing the 1000-row chunk boundary multiple times Intentionally NOT a realistic content-type design — this is a stress-test fixture. See the description in schema.json. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(examples): render multiplier x db matrix in bench-compare Rework bench-compare to index results by (label, multiplier, dbEngine) triples pulled from each result JSON's own fields, rather than parsing labels out of filenames. Lets the same canonical baseline/candidate label span any number of (multiplier, db) combinations and produces: - A speedup matrix at the top: rows = multipliers, cols = databases, cells = "baseline -> candidate (delta%)". Missing cells render as "-" so partial data still produces a useful report. - A data-availability matrix listing what ran vs what's still missing. - Per-(db, multiplier) detail sections as collapsible details in HTML, all expanded in markdown. Also: - New flag syntax: --baseline <label> / --candidate <label>, with positional args kept for backward compat. - Legacy labels that embedded the multiplier (e.g. "baseline-m100") are normalized to their base form ("baseline"), letting older result files keep working. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(examples): force TCP for mysql/mariadb CLI in containers The mysql/mariadb CLI tools default to connecting via unix socket at /var/run/mysqld/mysqld.sock, which isn't populated in the official mysql:8 / mariadb:11 container images. Every invocation (check, snapshot, restore, wipe, readiness probe, version probe) needs an explicit -h 127.0.0.1 to force TCP via the container's loopback. Without this fix, bench:seed and bench:run error out with "Can't connect to local MySQL server through socket" on anything requiring the CLI inside the container (pg_stat-style row-count queries, snapshot restore, etc.). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * enhancement(examples): parallelize entity creation in seed-v4 Replace sequential for-loops of `await entityService.create(...)` with a `concurrentMap(count, concurrency, taskFn)` helper that runs N tasks in flight at once. At SEED_CONCURRENCY=5 (default), a seed that was strictly serial now fans out into 5 parallel creates. Concurrency chosen conservatively: Strapi v4's default knex pool is `{min: 2, max: 10}`, and entity-heavy creates (components + DZs + localizations) can use multiple connections per call. 5 keeps us well under the pool ceiling. Tune via `SEED_CONCURRENCY=<n>` env var if you've also raised the pool max. Applied to: seedBasic, seedBasicDp, updateComponentRelations, seedBasicDpI18n, seedRelation, seedRelationDp, seedRelationDpI18n, seedHcM2m (all entity-creation loops plus their follow-up self-reference update loops). Not yet done: incremental seeding (restore previous snapshot + seed delta) — a separate optimization tracked as a follow-up. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(examples): update complex README for new bench tooling + DBs README was documenting just the original 6-type, postgres+mysql workflow. Updated to cover everything this branch adds: - 8 content types (added hc-m2m-source/target anti-patterns) - 4 supported databases (added mariadb + sqlite) - Container runtime auto-detection (podman compose / podman-compose / docker compose / docker-compose) with STRAPI_BENCH_RUNTIME override - Benchmark harness workflow (bench:seed / bench:run / bench:compare / bench:suite) for reviewing migration-performance PRs - SEED_CONCURRENCY, STRAPI_BENCH_HOOK_OUTPUT, STRAPI_BENCH_HOOK_DEBUG, and the existing port-override env vars - MariaDB port default 3307 to avoid colliding with MySQL on 3306 Also collapsed the redundant per-DB command sections (postgres and mysql both had identical copy-pasted blocks) into a single 'yarn db:<op>:<db>' table since the commands are symmetric across all four dialects. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(examples): align better-sqlite3 version with monorepo convention I picked `11.3.0` arbitrarily. Every other example and tests/app-template use `12.8.0`, and the root yarn.lock already resolves that version. Without alignment CI's `yarn install --immutable` fails with 'lockfile would have been modified', cascading every subsequent job (build, pretty, commitlint, aggregate_test_result) to red. Bumping to `12.8.0` to match, regenerating yarn.lock. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: throw instead of return to fail fast --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Ben Irvin <ben@innerdvations.com>
220 lines
7.4 KiB
JavaScript
220 lines
7.4 KiB
JavaScript
#!/usr/bin/env node
|
|
|
|
const { execSync } = require('child_process');
|
|
const fs = require('fs');
|
|
const path = require('path');
|
|
|
|
const {
|
|
getContainerName: getComposeContainerName,
|
|
getComposeEnv,
|
|
COMPOSE_PROJECT_NAME,
|
|
} = require('./db-utils');
|
|
const { getComposeCommand, getContainerCommand } = require('./compose');
|
|
|
|
const SCRIPT_DIR = __dirname;
|
|
const COMPLEX_DIR = path.resolve(SCRIPT_DIR, '..');
|
|
const DOCKER_COMPOSE_FILE = path.join(COMPLEX_DIR, 'docker-compose.dev.yml');
|
|
const SNAPSHOTS_DIR = path.join(COMPLEX_DIR, 'snapshots');
|
|
|
|
const DB_NAME = process.env.DATABASE_NAME || 'strapi';
|
|
const DB_USER = process.env.DATABASE_USERNAME || 'strapi';
|
|
const DB_PASSWORD = process.env.DATABASE_PASSWORD || 'strapi';
|
|
|
|
function composeCmd() {
|
|
const { exe, prefixArgs } = getComposeCommand();
|
|
return [exe, ...prefixArgs].join(' ');
|
|
}
|
|
|
|
function containerCmd() {
|
|
return getContainerCommand();
|
|
}
|
|
|
|
function getContainerName() {
|
|
const name = getComposeContainerName(DOCKER_COMPOSE_FILE, COMPLEX_DIR, 'mariadb');
|
|
if (!name) {
|
|
throw new Error(
|
|
`MariaDB container not found. Start it with "yarn db:start:mariadb" (COMPOSE_PROJECT_NAME=${COMPOSE_PROJECT_NAME}).`
|
|
);
|
|
}
|
|
return name;
|
|
}
|
|
|
|
const command = process.argv[2];
|
|
const snapshotName = process.argv[3];
|
|
|
|
function ensureSnapshotsDir() {
|
|
if (!fs.existsSync(SNAPSHOTS_DIR)) {
|
|
fs.mkdirSync(SNAPSHOTS_DIR, { recursive: true });
|
|
}
|
|
}
|
|
|
|
function execShell(cmd) {
|
|
try {
|
|
execSync(cmd, { stdio: 'inherit', cwd: COMPLEX_DIR, env: getComposeEnv(), shell: '/bin/bash' });
|
|
} catch (error) {
|
|
console.error(`Error executing: ${cmd}`);
|
|
process.exit(1);
|
|
}
|
|
}
|
|
|
|
switch (command) {
|
|
case 'start':
|
|
console.log('Starting mariadb container...');
|
|
execShell(`${composeCmd()} -f ${DOCKER_COMPOSE_FILE} up -d mariadb`);
|
|
console.log('✅ MariaDB started');
|
|
break;
|
|
|
|
case 'stop':
|
|
console.log('Stopping mariadb container...');
|
|
execShell(`${composeCmd()} -f ${DOCKER_COMPOSE_FILE} stop mariadb`);
|
|
console.log('✅ MariaDB stopped');
|
|
break;
|
|
|
|
case 'snapshot':
|
|
if (!snapshotName) {
|
|
console.error('Error: Snapshot name is required');
|
|
console.error('Usage: node db-mariadb.js snapshot <name>');
|
|
process.exit(1);
|
|
}
|
|
ensureSnapshotsDir();
|
|
const snapshotPath = path.join(SNAPSHOTS_DIR, `mariadb-${snapshotName}.sql`);
|
|
console.log(`Creating snapshot: ${snapshotName}...`);
|
|
{
|
|
const containerName = getContainerName();
|
|
try {
|
|
// `mariadb-dump` is the modern binary; older images fall back to `mysqldump` alias.
|
|
execSync(
|
|
`${containerCmd()} exec ${containerName} mariadb-dump -h 127.0.0.1 -u${DB_USER} -p${DB_PASSWORD} ${DB_NAME} > ${snapshotPath}`,
|
|
{ stdio: 'inherit', cwd: COMPLEX_DIR, shell: '/bin/bash' }
|
|
);
|
|
console.log(`✅ Snapshot created: ${snapshotPath}`);
|
|
} catch (error) {
|
|
console.error(`Error creating snapshot: ${error.message}`);
|
|
process.exit(1);
|
|
}
|
|
}
|
|
break;
|
|
|
|
case 'restore':
|
|
if (!snapshotName) {
|
|
console.error('Error: Snapshot name is required');
|
|
console.error('Usage: node db-mariadb.js restore <name>');
|
|
process.exit(1);
|
|
}
|
|
const restorePath = path.join(SNAPSHOTS_DIR, `mariadb-${snapshotName}.sql`);
|
|
if (!fs.existsSync(restorePath)) {
|
|
console.error(`Error: Snapshot not found: ${restorePath}`);
|
|
process.exit(1);
|
|
}
|
|
console.log(`Restoring snapshot: ${snapshotName}...`);
|
|
{
|
|
const containerName = getContainerName();
|
|
try {
|
|
// Drop and recreate database
|
|
execSync(
|
|
`${containerCmd()} exec ${containerName} mariadb -h 127.0.0.1 -u${DB_USER} -p${DB_PASSWORD} -e "DROP DATABASE IF EXISTS ${DB_NAME}; CREATE DATABASE ${DB_NAME};"`,
|
|
{ stdio: 'inherit', cwd: COMPLEX_DIR, shell: '/bin/bash' }
|
|
);
|
|
// Restore from snapshot
|
|
execSync(
|
|
`${containerCmd()} exec -i ${containerName} mariadb -h 127.0.0.1 -u${DB_USER} -p${DB_PASSWORD} ${DB_NAME} < ${restorePath}`,
|
|
{ stdio: 'inherit', cwd: COMPLEX_DIR, shell: '/bin/bash' }
|
|
);
|
|
console.log(`✅ Snapshot restored: ${snapshotName}`);
|
|
} catch (error) {
|
|
console.error(`Error restoring snapshot: ${error.message}`);
|
|
process.exit(1);
|
|
}
|
|
}
|
|
break;
|
|
|
|
case 'wipe':
|
|
console.log('Wiping mariadb database...');
|
|
{
|
|
const containerName = getContainerName();
|
|
try {
|
|
execSync(
|
|
`${containerCmd()} exec ${containerName} mariadb -h 127.0.0.1 -u${DB_USER} -p${DB_PASSWORD} -e "DROP DATABASE IF EXISTS ${DB_NAME}; CREATE DATABASE ${DB_NAME};"`,
|
|
{ stdio: 'inherit', cwd: COMPLEX_DIR, shell: '/bin/bash' }
|
|
);
|
|
console.log('✅ Database wiped');
|
|
} catch (error) {
|
|
console.error(`Error wiping database: ${error.message}`);
|
|
process.exit(1);
|
|
}
|
|
}
|
|
break;
|
|
|
|
case 'check':
|
|
{
|
|
const containerName = getContainerName();
|
|
try {
|
|
// Refresh information_schema.tables.table_rows via ANALYZE TABLE;
|
|
// without it, stats can lag reality by hours.
|
|
const analyzeQuery = `
|
|
SELECT CONCAT('ANALYZE TABLE ', table_name, ';') AS cmd
|
|
FROM information_schema.tables
|
|
WHERE table_schema = '${DB_NAME}';
|
|
`;
|
|
try {
|
|
const analyzeList = execSync(
|
|
`${containerCmd()} exec ${containerName} mariadb -h 127.0.0.1 -u${DB_USER} -p${DB_PASSWORD} -D ${DB_NAME} -e "${analyzeQuery.trim()}" -s -N`,
|
|
{ encoding: 'utf8', cwd: COMPLEX_DIR, shell: '/bin/bash' }
|
|
);
|
|
const cmds = analyzeList.trim().split('\n').filter(Boolean).join(' ');
|
|
if (cmds) {
|
|
execSync(
|
|
`${containerCmd()} exec ${containerName} mariadb -h 127.0.0.1 -u${DB_USER} -p${DB_PASSWORD} -D ${DB_NAME} -e "${cmds}"`,
|
|
{ stdio: 'ignore', cwd: COMPLEX_DIR, shell: '/bin/bash' }
|
|
);
|
|
}
|
|
} catch {
|
|
// Best-effort; fall through to stale stats rather than fail.
|
|
}
|
|
|
|
const query = `
|
|
SELECT
|
|
table_name,
|
|
table_rows
|
|
FROM information_schema.tables
|
|
WHERE table_schema = '${DB_NAME}'
|
|
ORDER BY table_name;
|
|
`;
|
|
|
|
const output = execSync(
|
|
`${containerCmd()} exec ${containerName} mariadb -h 127.0.0.1 -u${DB_USER} -p${DB_PASSWORD} -D ${DB_NAME} -e "${query.trim()}" -s -N`,
|
|
{ encoding: 'utf8', cwd: COMPLEX_DIR, shell: '/bin/bash' }
|
|
);
|
|
|
|
const lines = output
|
|
.trim()
|
|
.split('\n')
|
|
.filter((l) => l.trim());
|
|
|
|
if (lines.length === 0) {
|
|
console.log('📊 No tables found (database is empty or wiped)');
|
|
} else {
|
|
console.log('📊 Database Tables (approximate row counts):\n');
|
|
console.log('Table Name | Row Count');
|
|
console.log('------------------------------------|----------');
|
|
|
|
for (const line of lines) {
|
|
const [table, count] = line.trim().split('\t');
|
|
const paddedName = (table || '').padEnd(35);
|
|
const rowCount = count || '0';
|
|
console.log(`${paddedName} | ${rowCount}`);
|
|
}
|
|
}
|
|
} catch (error) {
|
|
console.error(`Error checking database: ${error.message}`);
|
|
process.exit(1);
|
|
}
|
|
}
|
|
break;
|
|
|
|
default:
|
|
console.error('Error: Unknown command');
|
|
console.error('Usage: node db-mariadb.js <start|stop|snapshot|restore|wipe|check> [name]');
|
|
process.exit(1);
|
|
}
|