* chore(examples): add mariadb + sqlite + podman support to complex
Extends the complex example's DB tooling to cover all Strapi-supported
database dialects and both container runtimes, as groundwork for a
migration performance benchmark harness:
- New compose.js runtime shim auto-detects podman compose / podman-compose
/ docker compose / docker-compose and the matching container CLI; all
existing db-* scripts now go through it so podman-only environments
work without installing docker
- New db-mariadb.js mirrors db-mysql.js using mariadb-dump / mariadb CLIs
and adds a mariadb:11 service on port 3307 to docker-compose.dev.yml
- New db-sqlite.js handles file-based snapshot/restore/wipe/check via
fs.copy / better-sqlite3
- db-utils.js falls back to `<runtime> ps --filter name=` for container
lookup since podman-compose doesn't support `ps -q`
- develop-with-db.js and the v4 templates (develop-with-db.js,
seed-with-db.js) handle mariadb + sqlite (sqlite skips compose)
- setup-v4-project.js includes better-sqlite3 in v4 deps, database.js
template covers all 4 clients, and compose.js is copied into the
v4 scaffold scripts dir (dep of db-utils.js)
All four DBs smoke-tested locally against podman: start/check/snapshot/
restore/wipe cycle works for mariadb; cp-based snapshot cycle works
for sqlite.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore(examples): add migration perf benchmark harness
Three new scripts enable per-migration timing and baseline-vs-candidate
comparison reports for v4→v5 migrations in the complex example:
- bench-hook.js: Node --require preload that intercepts require('umzug')
and subscribes to Umzug's native `migrating`/`migrated` events for
sub-ms timing. Captures every migration that runs (including dynamically
registered ones like discard-drafts and EE-only release migrations)
without hardcoding names. Dumps to a JSON file on process exit; self-
disables when STRAPI_BENCH_HOOK_OUTPUT is unset.
- bench.js: orchestrator with `run`, `seed`, and `suite` subcommands.
`run` restores a snapshot, spawns Strapi in migrate-then-exit mode
with the hook preload, collects row counts, and writes a result JSON
with baseline/candidate attribution, env capture (node, CPU, memory,
DB version, host type), and config (multiplier, seed/hook modes).
`seed` wipes the DB, runs the v4 seed via seed-with-db.js, then
snapshots. First iteration supports --strapi-source=local only;
experimental/pinned are stubbed with a clear error.
- bench-compare.js: takes N labels and emits both a clipboard-friendly
markdown report (stdout + results/compare-*.md) and a self-contained
HTML report (results/compare-*.html) with inline SVG bar charts,
per-DB grid, sortable tables, collapsible raw JSON, and a light/dark
adaptive theme via prefers-color-scheme. No CDN deps.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore(examples): bench harness smoke-test fixes
Fixes discovered during the end-to-end smoke test on the existing 6
content types at multiplier=1:
- bench-hook.js: switch from subclass-based wrapping to in-place
Umzug.prototype.up patching. The subclass approach replaced the module
export at require time, but Node's module cache hands out the original
class on subsequent requires, so listeners weren't attached on all
instances. In-place prototype patching works for every instance
regardless of how Umzug was imported.
- bench-hook.js: flush incrementally after each recorded migration.
Strapi's shutdown path can bypass process.on('exit') handlers under
some conditions (signal or explicit exit from deep inside), causing
fully-collected timing data to be lost. Writing after each recording
makes the benchmark resilient to any exit path.
- bench.js: compile TypeScript configs via @strapi/typescript-utils
before createStrapi().load(). The examples/complex project has .ts
config files; the Strapi CLI compiles them to dist/ before boot but
our direct node -e loader skipped this, producing
"db.config.connection undefined" failures.
- bench.js: propagate STRAPI_BENCH_HOOK_DEBUG to the Strapi child so
debug output is visible when tracing hook behavior.
- bench-compare.js: rework the SVG chart. Dynamic label column sized
to the longest migration name (up to 420px), 80px reserved on the
right for value labels so they never clip, inlined monospace font
(SVG text doesn't reliably inherit CSS variables from the surrounding
stylesheet), and `dominant-baseline="middle"` for proper vertical
centering.
Verified: full pipeline (setup:v4 → seed → snapshot → bench:run →
bench:compare) works against postgres at multiplier=1. Ran a baseline
vs cherry-picked PR #25988 comparison — captured all 7 v4→v5 migrations,
produced both markdown and HTML reports with correct test-setup
attribution and delta coloring.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(examples): run ANALYZE before db:check to get fresh row counts
pg_stat_user_tables.n_live_tup and information_schema.tables.table_rows
are approximate and can lag behind reality by minutes or hours depending
on autovacuum / ANALYZE cadence. For a benchmark harness that publishes
row-count numbers in its reports, stale counts are misleading.
Trigger a refresh via ANALYZE (postgres) / ANALYZE TABLE per-table
(mysql/mariadb) before each db:check invocation. Best-effort on the
mysql/mariadb side — fall through to stale stats if ANALYZE fails rather
than error the whole command.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(examples): add hc-m2m-source/target anti-pattern schemas
First anti-pattern schema pair for migration benchmark stress-testing.
A high-cardinality many-to-many relation that forces the v4→v5
discard-drafts migration's copyRelationTableRows code path to span
multiple chunks (>1000 rows) — the same scenario PR #25988's caching
fixes target.
- src/api/hc-m2m-source: collection type with DP and a manyToMany
relation to hc-m2m-target (owning side)
- src/api/hc-m2m-target: collection type with DP and the inverse
manyToMany back to source
- setup-v4-project.js: include both in the v4 scaffold CONTENT_TYPES
- seed-v4.js: seedHcM2m() method that creates sources + targets and
fans out 10 targets-per-source via the M2M relation. BASE counts at
m=1 are tiny (15 pub + 5 draft per side) but at m=100 produce ~2K
sources × ~2K targets × 10 = 20K join rows, crossing the 1000-row
chunk boundary multiple times
Intentionally NOT a realistic content-type design — this is a
stress-test fixture. See the description in schema.json.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(examples): render multiplier x db matrix in bench-compare
Rework bench-compare to index results by (label, multiplier, dbEngine)
triples pulled from each result JSON's own fields, rather than parsing
labels out of filenames. Lets the same canonical baseline/candidate
label span any number of (multiplier, db) combinations and produces:
- A speedup matrix at the top: rows = multipliers, cols = databases,
cells = "baseline -> candidate (delta%)". Missing cells render as
"-" so partial data still produces a useful report.
- A data-availability matrix listing what ran vs what's still missing.
- Per-(db, multiplier) detail sections as collapsible details in
HTML, all expanded in markdown.
Also:
- New flag syntax: --baseline <label> / --candidate <label>, with
positional args kept for backward compat.
- Legacy labels that embedded the multiplier (e.g. "baseline-m100")
are normalized to their base form ("baseline"), letting older
result files keep working.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(examples): force TCP for mysql/mariadb CLI in containers
The mysql/mariadb CLI tools default to connecting via unix socket at
/var/run/mysqld/mysqld.sock, which isn't populated in the official
mysql:8 / mariadb:11 container images. Every invocation (check,
snapshot, restore, wipe, readiness probe, version probe) needs an
explicit -h 127.0.0.1 to force TCP via the container's loopback.
Without this fix, bench:seed and bench:run error out with
"Can't connect to local MySQL server through socket" on anything
requiring the CLI inside the container (pg_stat-style row-count
queries, snapshot restore, etc.).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* enhancement(examples): parallelize entity creation in seed-v4
Replace sequential for-loops of `await entityService.create(...)` with
a `concurrentMap(count, concurrency, taskFn)` helper that runs N tasks
in flight at once. At SEED_CONCURRENCY=5 (default), a seed that was
strictly serial now fans out into 5 parallel creates.
Concurrency chosen conservatively: Strapi v4's default knex pool is
`{min: 2, max: 10}`, and entity-heavy creates (components + DZs +
localizations) can use multiple connections per call. 5 keeps us well
under the pool ceiling. Tune via `SEED_CONCURRENCY=<n>` env var if
you've also raised the pool max.
Applied to: seedBasic, seedBasicDp, updateComponentRelations,
seedBasicDpI18n, seedRelation, seedRelationDp, seedRelationDpI18n,
seedHcM2m (all entity-creation loops plus their follow-up
self-reference update loops).
Not yet done: incremental seeding (restore previous snapshot + seed
delta) — a separate optimization tracked as a follow-up.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs(examples): update complex README for new bench tooling + DBs
README was documenting just the original 6-type, postgres+mysql
workflow. Updated to cover everything this branch adds:
- 8 content types (added hc-m2m-source/target anti-patterns)
- 4 supported databases (added mariadb + sqlite)
- Container runtime auto-detection (podman compose / podman-compose /
docker compose / docker-compose) with STRAPI_BENCH_RUNTIME override
- Benchmark harness workflow (bench:seed / bench:run / bench:compare /
bench:suite) for reviewing migration-performance PRs
- SEED_CONCURRENCY, STRAPI_BENCH_HOOK_OUTPUT, STRAPI_BENCH_HOOK_DEBUG,
and the existing port-override env vars
- MariaDB port default 3307 to avoid colliding with MySQL on 3306
Also collapsed the redundant per-DB command sections (postgres and
mysql both had identical copy-pasted blocks) into a single
'yarn db:<op>:<db>' table since the commands are symmetric across
all four dialects.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(examples): align better-sqlite3 version with monorepo convention
I picked `11.3.0` arbitrarily. Every other example and tests/app-template
use `12.8.0`, and the root yarn.lock already resolves that version.
Without alignment CI's `yarn install --immutable` fails with 'lockfile
would have been modified', cascading every subsequent job (build, pretty,
commitlint, aggregate_test_result) to red.
Bumping to `12.8.0` to match, regenerating yarn.lock.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: throw instead of return to fail fast
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Ben Irvin <ben@innerdvations.com>
* enhancement: update progress per chunk
* fix: check stageprogress exists to make ts happy
* chore: split progress tracker into two methods
* test: fix lint
* enhancement: display readable times
* fix: speed indication for assets
* fix: restore speed indicator and fix double counting
* chore: clean up code
* fix: data transfer memory leak
* fix: yarn.lock
* chore: fix memory logging
* ci: complex project remote transfer
* enhancement: optimized transfer mode
* test(cli): data transfer and env passthrough
* chore: only send one message warning of legacy mode
* enhancement: show transfer totals and estimated time
* test(cli): fix deterministic transfer test files
* fix: push and pull shared utils backwards compatibility
* fix(data-transfer): extract legacy asset chunk parsing and tighten transfer logging/test coverage
* enhancement: checksum negotiation
* enhancement: show skipped file warnings on client
* fix: transfer diagnostics
* test: fix open handle
* fix: clear stall timeout for assets
* chore: fix misleading comments and variables
* test: fix misleading test
* test: fix typo
* test: make checks deterministic, less flaky
* enhancement(data-transfer): speed up asset totals; widen assets start reply window on remote pull
* fix(data-transfer): harden WebSocket JSON serialization for transfer frames
* fix(data-transfer): more transfer hardening
* test: fix test imports
* fix: await async write
* fix(data-transfer): resolve push transfer deadlock and harden async writes
- Extract createAssetsDestinationWritable so Writable callbacks run before
uploadStream completes (same WS batch as PassThrough chunks).
- Add writable-async-write (write callback + drain/finished race; avoid hang
on destroy).
- Wire push/pull, remote-source, file & directory sources to shared write().
- Fire-and-forget pull flush: Promise.resolve(flush).catch(onError); guard
missing stream inside try.
- Add regression tests (assets writable, writable-async-write, handler checks).
* fix(data-transfer): write push stream batches sequentially
Use a for-loop with await write() instead of Promise.all over msg.data
so non-asset stages respect one in-flight write per objectMode Writable
and backpressure from writable-async-write.
- Validate minChunksForBackpressure in assertReadStreamBackpressure
- Add engine test for non-Buffer asset chunk byte progress (counts as 1)
- Assert push.ts keeps sequential msg.data handling in static handler test
* fix(data-transfer): align push streamAsset with remote-source and harden tests
- Push handler: combine stream/end under one branch, error when start is missing
or action is invalid; shorten stage write comments.
- Engine version-matching tests: use a fresh createDestination() per engine so
parallel transfers do not share destination writables (MaxListeners warnings).
- File destination tests: mock createWriteStream with a new Writable per call.
- CLI transfer tests: mock progress.stream so transfer::finish runs after transfer
and clears the progress setInterval (fixes Jest worker hang).
- Misc test cleanup: assets-destination timeout clearTimeout, collect listeners,
writable-async-write teardown; tighten push/static test descriptions.
* test: remove parity test
* fix(data-transfer): harden collect() and stabilize transfer tests
- collect(): settle once, remove listeners on resolve/reject, avoid double completion
- engine tests: add expectHeapGrowthWithinNoise for heap smoke checks
- CLI transfer tests: console spies in beforeAll; jest.restoreAllMocks in afterAll
- stream test: remove removeAllListeners workaround
* fix: add params actually works
* fix: rename addInputParams
* test: fix register timing
* chore: update getstarted
* test(api): remove ai comment
* revert: do not merge in type unneccessarily
* revert: do not add type to route
* chore: restore complex attributes
(cherry picked from commit b278fe35b6)
* chore: add dynamic zone to about schema
(cherry picked from commit 09f2d1f056)
* feat: add content source maps service
* chore: refactor to fp and async.pipe
* chore: use header instead of query param
* fix: ignore polymorphic relations
* chore: add error handling
* fix: arrays of relations and medias
* enhancement(getstarted): show nested values in preview
* fix: update highlights when the dom changes
* fix: highlight dimensions after text update
* fix: strip away invisible characters from dom
* fix: don't encode slugs
* fix: handle fields in single components
* enhancement: add rawWithIndices to traverseEntity patj
* fix: handle components and repeatable components
* fix: strapi utils build
* fix: dynamic zones
* chore: use dots for array indices not brackets
* fix: content source map base url fallback
* chore: use URLSearchParams instead of URL
* chore: refinements
* test: fieldUtils testing
* fix: traverse-entity doc
* chore: add traverse-entity tests
* feat: scroll element into view on focus
* chore: make email clickable in dummy preview
* fix: single and double click behavior
* chore: use css hover instead of mouseenter
* enhancement: double click to edit notification
* fix: display translated errors
* fix: unit test
* fix: null blocks value
* enhancement: filter attributes in nested fields
* fix: prevent losing iframe pointer events after closing popover
* feat: add content source maps service
* chore: refactor to fp and async.pipe
* chore: use header instead of query param
* fix: ignore polymorphic relations
* chore: add error handling
* fix: arrays of relations and medias
* enhancement(getstarted): show nested values in preview
* fix: update highlights when the dom changes
* fix: highlight dimensions after text update
* fix: strip away invisible characters from dom
* fix: don't encode slugs
* chore: mark feedback
* chore: rearrange observers
* feat: add content source maps service
* chore: refactor to fp and async.pipe
* chore: use header instead of query param
* fix: ignore polymorphic relations
* chore: add error handling
* fix: arrays of relations and medias
* chore: marc feedback
* chore: use traverseEntity util
* fix: make backend unit test async
* chore: refactor types
Nico André
DMehaffy
Filip Ónodi
Ben Irvin
dependabot[bot]
Bassel Kanso
mathildeleg
Ben Irvin
Andrei L
markkaylor
Leonid Vinogradov
Adrien L
HichamELBSI
Rémi de Juvigny
Araksya Gevorgyan
Rémi de Juvigny
Lucas Boilly
Jan Fässler