mirror of
https://github.com/strapi/strapi.git
synced 2026-05-03 16:22:30 +00:00
19fef31e08
* chore(examples): add mariadb + sqlite + podman support to complex Extends the complex example's DB tooling to cover all Strapi-supported database dialects and both container runtimes, as groundwork for a migration performance benchmark harness: - New compose.js runtime shim auto-detects podman compose / podman-compose / docker compose / docker-compose and the matching container CLI; all existing db-* scripts now go through it so podman-only environments work without installing docker - New db-mariadb.js mirrors db-mysql.js using mariadb-dump / mariadb CLIs and adds a mariadb:11 service on port 3307 to docker-compose.dev.yml - New db-sqlite.js handles file-based snapshot/restore/wipe/check via fs.copy / better-sqlite3 - db-utils.js falls back to `<runtime> ps --filter name=` for container lookup since podman-compose doesn't support `ps -q` - develop-with-db.js and the v4 templates (develop-with-db.js, seed-with-db.js) handle mariadb + sqlite (sqlite skips compose) - setup-v4-project.js includes better-sqlite3 in v4 deps, database.js template covers all 4 clients, and compose.js is copied into the v4 scaffold scripts dir (dep of db-utils.js) All four DBs smoke-tested locally against podman: start/check/snapshot/ restore/wipe cycle works for mariadb; cp-based snapshot cycle works for sqlite. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore(examples): add migration perf benchmark harness Three new scripts enable per-migration timing and baseline-vs-candidate comparison reports for v4→v5 migrations in the complex example: - bench-hook.js: Node --require preload that intercepts require('umzug') and subscribes to Umzug's native `migrating`/`migrated` events for sub-ms timing. Captures every migration that runs (including dynamically registered ones like discard-drafts and EE-only release migrations) without hardcoding names. Dumps to a JSON file on process exit; self- disables when STRAPI_BENCH_HOOK_OUTPUT is unset. - bench.js: orchestrator with `run`, `seed`, and `suite` subcommands. `run` restores a snapshot, spawns Strapi in migrate-then-exit mode with the hook preload, collects row counts, and writes a result JSON with baseline/candidate attribution, env capture (node, CPU, memory, DB version, host type), and config (multiplier, seed/hook modes). `seed` wipes the DB, runs the v4 seed via seed-with-db.js, then snapshots. First iteration supports --strapi-source=local only; experimental/pinned are stubbed with a clear error. - bench-compare.js: takes N labels and emits both a clipboard-friendly markdown report (stdout + results/compare-*.md) and a self-contained HTML report (results/compare-*.html) with inline SVG bar charts, per-DB grid, sortable tables, collapsible raw JSON, and a light/dark adaptive theme via prefers-color-scheme. No CDN deps. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore(examples): bench harness smoke-test fixes Fixes discovered during the end-to-end smoke test on the existing 6 content types at multiplier=1: - bench-hook.js: switch from subclass-based wrapping to in-place Umzug.prototype.up patching. The subclass approach replaced the module export at require time, but Node's module cache hands out the original class on subsequent requires, so listeners weren't attached on all instances. In-place prototype patching works for every instance regardless of how Umzug was imported. - bench-hook.js: flush incrementally after each recorded migration. Strapi's shutdown path can bypass process.on('exit') handlers under some conditions (signal or explicit exit from deep inside), causing fully-collected timing data to be lost. Writing after each recording makes the benchmark resilient to any exit path. - bench.js: compile TypeScript configs via @strapi/typescript-utils before createStrapi().load(). The examples/complex project has .ts config files; the Strapi CLI compiles them to dist/ before boot but our direct node -e loader skipped this, producing "db.config.connection undefined" failures. - bench.js: propagate STRAPI_BENCH_HOOK_DEBUG to the Strapi child so debug output is visible when tracing hook behavior. - bench-compare.js: rework the SVG chart. Dynamic label column sized to the longest migration name (up to 420px), 80px reserved on the right for value labels so they never clip, inlined monospace font (SVG text doesn't reliably inherit CSS variables from the surrounding stylesheet), and `dominant-baseline="middle"` for proper vertical centering. Verified: full pipeline (setup:v4 → seed → snapshot → bench:run → bench:compare) works against postgres at multiplier=1. Ran a baseline vs cherry-picked PR #25988 comparison — captured all 7 v4→v5 migrations, produced both markdown and HTML reports with correct test-setup attribution and delta coloring. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(examples): run ANALYZE before db:check to get fresh row counts pg_stat_user_tables.n_live_tup and information_schema.tables.table_rows are approximate and can lag behind reality by minutes or hours depending on autovacuum / ANALYZE cadence. For a benchmark harness that publishes row-count numbers in its reports, stale counts are misleading. Trigger a refresh via ANALYZE (postgres) / ANALYZE TABLE per-table (mysql/mariadb) before each db:check invocation. Best-effort on the mysql/mariadb side — fall through to stale stats if ANALYZE fails rather than error the whole command. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(examples): add hc-m2m-source/target anti-pattern schemas First anti-pattern schema pair for migration benchmark stress-testing. A high-cardinality many-to-many relation that forces the v4→v5 discard-drafts migration's copyRelationTableRows code path to span multiple chunks (>1000 rows) — the same scenario PR #25988's caching fixes target. - src/api/hc-m2m-source: collection type with DP and a manyToMany relation to hc-m2m-target (owning side) - src/api/hc-m2m-target: collection type with DP and the inverse manyToMany back to source - setup-v4-project.js: include both in the v4 scaffold CONTENT_TYPES - seed-v4.js: seedHcM2m() method that creates sources + targets and fans out 10 targets-per-source via the M2M relation. BASE counts at m=1 are tiny (15 pub + 5 draft per side) but at m=100 produce ~2K sources × ~2K targets × 10 = 20K join rows, crossing the 1000-row chunk boundary multiple times Intentionally NOT a realistic content-type design — this is a stress-test fixture. See the description in schema.json. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(examples): render multiplier x db matrix in bench-compare Rework bench-compare to index results by (label, multiplier, dbEngine) triples pulled from each result JSON's own fields, rather than parsing labels out of filenames. Lets the same canonical baseline/candidate label span any number of (multiplier, db) combinations and produces: - A speedup matrix at the top: rows = multipliers, cols = databases, cells = "baseline -> candidate (delta%)". Missing cells render as "-" so partial data still produces a useful report. - A data-availability matrix listing what ran vs what's still missing. - Per-(db, multiplier) detail sections as collapsible details in HTML, all expanded in markdown. Also: - New flag syntax: --baseline <label> / --candidate <label>, with positional args kept for backward compat. - Legacy labels that embedded the multiplier (e.g. "baseline-m100") are normalized to their base form ("baseline"), letting older result files keep working. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(examples): force TCP for mysql/mariadb CLI in containers The mysql/mariadb CLI tools default to connecting via unix socket at /var/run/mysqld/mysqld.sock, which isn't populated in the official mysql:8 / mariadb:11 container images. Every invocation (check, snapshot, restore, wipe, readiness probe, version probe) needs an explicit -h 127.0.0.1 to force TCP via the container's loopback. Without this fix, bench:seed and bench:run error out with "Can't connect to local MySQL server through socket" on anything requiring the CLI inside the container (pg_stat-style row-count queries, snapshot restore, etc.). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * enhancement(examples): parallelize entity creation in seed-v4 Replace sequential for-loops of `await entityService.create(...)` with a `concurrentMap(count, concurrency, taskFn)` helper that runs N tasks in flight at once. At SEED_CONCURRENCY=5 (default), a seed that was strictly serial now fans out into 5 parallel creates. Concurrency chosen conservatively: Strapi v4's default knex pool is `{min: 2, max: 10}`, and entity-heavy creates (components + DZs + localizations) can use multiple connections per call. 5 keeps us well under the pool ceiling. Tune via `SEED_CONCURRENCY=<n>` env var if you've also raised the pool max. Applied to: seedBasic, seedBasicDp, updateComponentRelations, seedBasicDpI18n, seedRelation, seedRelationDp, seedRelationDpI18n, seedHcM2m (all entity-creation loops plus their follow-up self-reference update loops). Not yet done: incremental seeding (restore previous snapshot + seed delta) — a separate optimization tracked as a follow-up. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(examples): update complex README for new bench tooling + DBs README was documenting just the original 6-type, postgres+mysql workflow. Updated to cover everything this branch adds: - 8 content types (added hc-m2m-source/target anti-patterns) - 4 supported databases (added mariadb + sqlite) - Container runtime auto-detection (podman compose / podman-compose / docker compose / docker-compose) with STRAPI_BENCH_RUNTIME override - Benchmark harness workflow (bench:seed / bench:run / bench:compare / bench:suite) for reviewing migration-performance PRs - SEED_CONCURRENCY, STRAPI_BENCH_HOOK_OUTPUT, STRAPI_BENCH_HOOK_DEBUG, and the existing port-override env vars - MariaDB port default 3307 to avoid colliding with MySQL on 3306 Also collapsed the redundant per-DB command sections (postgres and mysql both had identical copy-pasted blocks) into a single 'yarn db:<op>:<db>' table since the commands are symmetric across all four dialects. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(examples): align better-sqlite3 version with monorepo convention I picked `11.3.0` arbitrarily. Every other example and tests/app-template use `12.8.0`, and the root yarn.lock already resolves that version. Without alignment CI's `yarn install --immutable` fails with 'lockfile would have been modified', cascading every subsequent job (build, pretty, commitlint, aggregate_test_result) to red. Bumping to `12.8.0` to match, regenerating yarn.lock. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: throw instead of return to fail fast --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Ben Irvin <ben@innerdvations.com>
173 lines
5.4 KiB
JavaScript
173 lines
5.4 KiB
JavaScript
#!/usr/bin/env node
|
|
/* eslint-disable no-console, global-require */
|
|
|
|
/**
|
|
* Migration benchmark timing hook (Node `--require` preload).
|
|
*
|
|
* Captures per-migration start/end times by subscribing to Umzug's native
|
|
* `migrating` / `migrated` events. Umzug is a stable public dep of Strapi
|
|
* (currently 3.8.1); its event API is documented and unlikely to change,
|
|
* so this is more durable than patching anything inside @strapi/database.
|
|
*
|
|
* Strategy:
|
|
* 1. Patch Module._load so each `require('umzug')` returns a module whose
|
|
* `Umzug` export is a subclass that auto-attaches our listeners.
|
|
* 2. Accumulate timings in a module-level array.
|
|
* 3. On process exit, flush the array to STRAPI_BENCH_HOOK_OUTPUT (JSON).
|
|
*
|
|
* Activation:
|
|
* STRAPI_BENCH_HOOK_OUTPUT=/tmp/bench-$$.json \
|
|
* node --require /abs/path/to/bench-hook.js <strapi launcher>
|
|
*
|
|
* If STRAPI_BENCH_HOOK_OUTPUT is unset, the hook self-disables — cheap to
|
|
* leave `--require` in dev configs without affecting normal runs.
|
|
*/
|
|
|
|
const fs = require('fs');
|
|
const Module = require('module');
|
|
|
|
const OUTPUT_PATH = process.env.STRAPI_BENCH_HOOK_OUTPUT;
|
|
const HOOK_DEBUG = process.env.STRAPI_BENCH_HOOK_DEBUG === '1';
|
|
|
|
function debug(...args) {
|
|
if (HOOK_DEBUG) {
|
|
console.error('[bench-hook]', ...args);
|
|
}
|
|
}
|
|
|
|
if (!OUTPUT_PATH) {
|
|
debug('STRAPI_BENCH_HOOK_OUTPUT not set — hook disabled');
|
|
return;
|
|
}
|
|
|
|
/** @type {Array<{name: string, startedAt: number, durationMs: number}>} */
|
|
const migrations = [];
|
|
|
|
/** @type {Map<string, number>} */
|
|
const inflight = new Map();
|
|
let umzugInstanceCount = 0;
|
|
|
|
function recordStart(name) {
|
|
inflight.set(name, performance.now());
|
|
}
|
|
|
|
function recordEnd(name) {
|
|
const start = inflight.get(name);
|
|
if (start == null) {
|
|
debug(`migrated event without matching migrating event: ${name}`);
|
|
return;
|
|
}
|
|
inflight.delete(name);
|
|
const durationMs = performance.now() - start;
|
|
migrations.push({
|
|
name,
|
|
startedAt: Date.now() - durationMs, // wall-clock approximation
|
|
durationMs,
|
|
});
|
|
debug(`recorded ${name}: ${durationMs.toFixed(2)}ms`);
|
|
// Flush incrementally — some Strapi shutdown paths bypass exit handlers
|
|
// (process.exit via signal or uncaught error), and we don't want to lose
|
|
// timing data we already collected.
|
|
flush();
|
|
}
|
|
|
|
/**
|
|
* Extract a migration name from an Umzug event payload.
|
|
* Umzug v3 emits `{name, path, context, ...}` — sometimes just the name in older shapes.
|
|
*/
|
|
function getMigrationName(event) {
|
|
if (!event) return '<unknown>';
|
|
if (typeof event === 'string') return event;
|
|
if (typeof event.name === 'string') return event.name;
|
|
return '<unknown>';
|
|
}
|
|
|
|
/**
|
|
* Patch the Umzug prototype in place so every instance auto-attaches our
|
|
* listeners the first time its `up()` runs. This avoids cache-aliasing issues
|
|
* the subclass-then-replace-export approach would hit — Node caches the
|
|
* original module object, so subsequent `require('umzug')` calls return the
|
|
* original class from cache, not our subclass wrapper.
|
|
*/
|
|
function instrumentUmzugClass(Umzug) {
|
|
if (!Umzug || typeof Umzug !== 'function' || Umzug.__strapiBenchHookPatched) {
|
|
return false;
|
|
}
|
|
|
|
const originalUp = Umzug.prototype.up;
|
|
if (typeof originalUp !== 'function') {
|
|
debug('Umzug.prototype.up is not a function — cannot patch');
|
|
return false;
|
|
}
|
|
|
|
Umzug.prototype.up = async function patchedUp(...args) {
|
|
if (!this.__strapiBenchHookListeners) {
|
|
this.__strapiBenchHookListeners = true;
|
|
umzugInstanceCount += 1;
|
|
try {
|
|
this.on('migrating', (evt) => recordStart(getMigrationName(evt)));
|
|
this.on('migrated', (evt) => recordEnd(getMigrationName(evt)));
|
|
debug(`attached listeners to Umzug instance #${umzugInstanceCount}`);
|
|
} catch (err) {
|
|
debug('failed to attach listeners:', err.message);
|
|
}
|
|
}
|
|
return originalUp.apply(this, args);
|
|
};
|
|
|
|
Object.defineProperty(Umzug, '__strapiBenchHookPatched', {
|
|
value: true,
|
|
enumerable: false,
|
|
});
|
|
|
|
return true;
|
|
}
|
|
|
|
/**
|
|
* Intercept require('umzug') to patch the class the moment it's first loaded —
|
|
* before Strapi calls `new Umzug(...)`. Since we mutate the class prototype
|
|
* in place, subsequent loads that hit the cache still see the patched class.
|
|
*/
|
|
const originalLoad = Module._load;
|
|
|
|
Module._load = function patchedLoad(request, parent, ...rest) {
|
|
const mod = originalLoad.call(this, request, parent, ...rest);
|
|
if (request === 'umzug' && mod && mod.Umzug && !mod.Umzug.__strapiBenchHookPatched) {
|
|
const ok = instrumentUmzugClass(mod.Umzug);
|
|
if (ok) debug('patched umzug class prototype');
|
|
}
|
|
return mod;
|
|
};
|
|
|
|
/**
|
|
* Flush collected timings to disk. Sync write so we don't lose data if the
|
|
* parent process exits immediately after Strapi finishes migrating.
|
|
*/
|
|
function flush() {
|
|
const payload = {
|
|
hookVersion: 1,
|
|
instanceCount: umzugInstanceCount,
|
|
migrations,
|
|
capturedAt: new Date().toISOString(),
|
|
};
|
|
try {
|
|
fs.writeFileSync(OUTPUT_PATH, JSON.stringify(payload, null, 2), 'utf8');
|
|
debug(`flushed ${migrations.length} migration entries to ${OUTPUT_PATH}`);
|
|
} catch (err) {
|
|
console.error('[bench-hook] failed to write output:', err.message);
|
|
}
|
|
}
|
|
|
|
process.on('exit', flush);
|
|
// Safety net — some Strapi boot failures call process.exit via uncaught errors.
|
|
process.on('SIGINT', () => {
|
|
flush();
|
|
process.exit(130);
|
|
});
|
|
process.on('SIGTERM', () => {
|
|
flush();
|
|
process.exit(143);
|
|
});
|
|
|
|
debug(`initialized — output: ${OUTPUT_PATH}`);
|