mirror of https://github.com/strapi/strapi.git synced 2026-05-03 16:22:30 +00:00

Files

T

History

Ben Irvin bac520e9f8 fix(data-transfer): fix large transfer crashes; show transfer progress (#23479 )

* enhancement: update progress per chunk

* fix: check stageprogress exists to make ts happy

* chore: split progress tracker into two methods

* test: fix lint

* enhancement: display readable times

* fix: speed indication for assets

* fix: restore speed indicator and fix double counting

* chore: clean up code

* fix: data transfer memory leak

* fix: yarn.lock

* chore: fix memory logging

* ci: complex project remote transfer

* enhancement: optimized transfer mode

* test(cli): data transfer and env passthrough

* chore: only send one message warning of legacy mode

* enhancement: show transfer totals and estimated time

* test(cli): fix deterministic transfer test files

* fix: push and pull shared utils backwards compatibility

* fix(data-transfer): extract legacy asset chunk parsing and tighten transfer logging/test coverage

* enhancement: checksum negotiation

* enhancement: show skipped file warnings on client

* fix: transfer diagnostics

* test: fix open handle

* fix: clear stall timeout for assets

* chore: fix misleading comments and variables

* test: fix misleading test

* test: fix typo

* test: make checks deterministic, less flaky

* enhancement(data-transfer): speed up asset totals; widen assets start reply window on remote pull

* fix(data-transfer): harden WebSocket JSON serialization for transfer frames

* fix(data-transfer): more transfer hardening

* test: fix test imports

* fix: await async write

* fix(data-transfer): resolve push transfer deadlock and harden async writes

- Extract createAssetsDestinationWritable so Writable callbacks run before
  uploadStream completes (same WS batch as PassThrough chunks).
- Add writable-async-write (write callback + drain/finished race; avoid hang
  on destroy).
- Wire push/pull, remote-source, file & directory sources to shared write().
- Fire-and-forget pull flush: Promise.resolve(flush).catch(onError); guard
  missing stream inside try.
- Add regression tests (assets writable, writable-async-write, handler checks).

* fix(data-transfer): write push stream batches sequentially

Use a for-loop with await write() instead of Promise.all over msg.data
so non-asset stages respect one in-flight write per objectMode Writable
and backpressure from writable-async-write.

- Validate minChunksForBackpressure in assertReadStreamBackpressure
- Add engine test for non-Buffer asset chunk byte progress (counts as 1)
- Assert push.ts keeps sequential msg.data handling in static handler test

* fix(data-transfer): align push streamAsset with remote-source and harden tests

- Push handler: combine stream/end under one branch, error when start is missing
  or action is invalid; shorten stage write comments.
- Engine version-matching tests: use a fresh createDestination() per engine so
  parallel transfers do not share destination writables (MaxListeners warnings).
- File destination tests: mock createWriteStream with a new Writable per call.
- CLI transfer tests: mock progress.stream so transfer::finish runs after transfer
  and clears the progress setInterval (fixes Jest worker hang).
- Misc test cleanup: assets-destination timeout clearTimeout, collect listeners,
  writable-async-write teardown; tighten push/static test descriptions.

* test: remove parity test

* fix(data-transfer): harden collect() and stabilize transfer tests

- collect(): settle once, remove listeners on resolve/reject, avoid double completion
- engine tests: add expectHeapGrowthWithinNoise for heap smoke checks
- CLI transfer tests: console spies in beforeAll; jest.restoreAllMocks in afterAll
- stream test: remove removeAllListeners workaround

2026-04-20 16:08:05 +02:00

config

fix(data-transfer): fix large transfer crashes; show transfer progress (#23479 )

2026-04-20 16:08:05 +02:00

public

test: migration testing project (#25151 )

2026-01-27 14:50:08 +00:00

scripts

fix(homepage): render widgets progressively and batch permission checks (#25846 )

2026-03-30 16:43:43 +03:00

snapshots

test: migration testing project (#25151 )

2026-01-27 14:50:08 +00:00

src

fix: universal polymorphic

2026-02-24 15:45:36 +01:00

.env.example

test: migration testing project (#25151 )

2026-01-27 14:50:08 +00:00

.gitignore

test: migration testing project (#25151 )

2026-01-27 14:50:08 +00:00

docker-compose.dev.yml

test: migration testing project (#25151 )

2026-01-27 14:50:08 +00:00

favicon.png

test: migration testing project (#25151 )

2026-01-27 14:50:08 +00:00

package.json

fix(data-transfer): fix large transfer crashes; show transfer progress (#23479 )

2026-04-20 16:08:05 +02:00

README.md

fix(homepage): render widgets progressively and batch permission checks (#25846 )

2026-03-30 16:43:43 +03:00

tsconfig.json

test: migration testing project (#25151 )

2026-01-27 14:50:08 +00:00

README.md

Complex Example Project

This project contains complex Strapi schemas for testing migrations between Strapi v4 and v5.

Content Types

The project includes 6 content types with different combinations of features:

basic - Basic content type (no draft/publish, no i18n)
basic-dp - Basic content type with draft/publish
basic-dp-i18n - Basic content type with draft/publish and i18n
relation - Relation content type (no draft/publish, no i18n)
relation-dp - Relation content type with draft/publish
relation-dp-i18n - Relation content type with draft/publish and i18n

Migration Testing Workflow

This project includes tools for testing migrations between Strapi v4 and v5 by creating an isolated v4 project and managing database snapshots. The complex example ships its own docker-compose.dev.yml so the database containers are independent of the monorepo root.

Setup

Create/Update the external v4 project:
```
yarn setup:v4
```
This creates a Strapi v4 project outside the monorepo (default: a sibling directory named complex-v4). You can override the location via V4_OUTSIDE_DIR.
Navigate to the v4 project (use the path printed by setup):
```
cd <path-printed-by-setup>
```
Configure the v4 project (only if you need custom DB creds):
```
cp .env.example .env
# Edit .env as needed
```
Start the v4 project:
```
yarn develop:postgres
```

Database Management

PostgreSQL

Start PostgreSQL container:

yarn db:start:postgres

Stop PostgreSQL container:

yarn db:stop:postgres

Create a snapshot:

yarn db:snapshot:postgres <name>

Example: yarn db:snapshot:postgres mybackup

Restore from snapshot:

yarn db:restore:postgres <name>

Example: yarn db:restore:postgres mybackup

Wipe database (drop and recreate):

yarn db:wipe:postgres

Check database (show table row counts):

yarn db:check:postgres

This displays a table showing how many records are in each table, useful for quickly seeing if the database is empty, has data, etc.

MySQL

Start MySQL container:

yarn db:start:mysql

Stop MySQL container:

yarn db:stop:mysql

Create a snapshot:

yarn db:snapshot:mysql <name>

Example: yarn db:snapshot:mysql mybackup

Restore from snapshot:

yarn db:restore:mysql <name>

Example: yarn db:restore:mysql mybackup

Wipe database (drop and recreate):

yarn db:wipe:mysql

Check database (show table row counts):

yarn db:check:mysql

This displays a table showing how many records are in each table, useful for quickly seeing if the database is empty, has data, etc.

Typical Migration Testing Workflow

Setup v4 project (if not already done):
```
yarn setup:v4
```
Wipe the database (ensures v4 format, no v5 schema):
```
yarn db:wipe:postgres
```
Start v4 project (in separate terminal, use the path printed by setup):
```
cd <path-printed-by-setup>
yarn develop:postgres
```
(v4 will automatically start its database if needed)
Seed test data in the v4 project:
```
yarn seed
```

Create snapshot:

cd examples/complex
yarn db:snapshot:postgres mybackup

Stop v4 server (Ctrl+C in v4 terminal)
Start v5 server with the same database:
```
yarn develop:postgres
```
Migrations will run automatically on startup.
Validate migration (no HTTP server needed):
```
yarn test:migration
```
Test and fix bugs as needed
Restore snapshot to reset database:

yarn db:restore:postgres mybackup

Repeat from step 7 to test fixes

Note: The database container stays running even after stopping Strapi, so you can inspect the database or run multiple tests without restarting the container. The complex example uses its own Compose project name (strapi_complex) so it does not collide with other containers.

Snapshots

Database snapshots are stored in the snapshots/ directory:

PostgreSQL: snapshots/postgres-<name>.sql
MySQL: snapshots/mysql-<name>.sql

Snapshots are gitignored and should not be committed to the repository.

Development Commands

Simplified Database Commands

The easiest way to start Strapi with a specific database:

Start with PostgreSQL:

yarn develop:postgres

Start with MySQL:

yarn develop:mysql

These commands will:

✅ Automatically start the database container if it's not already running
✅ Configure Strapi to use the specified database (no manual config needed)
✅ Start the Strapi development server
✅ Keep the database container running when you press Ctrl+C (only Strapi stops)

Note: The database containers use the standard ports by default and can be overridden:

PostgreSQL: port 5432 (override with POSTGRES_PORT)
MySQL: port 3306 (override with MYSQL_PORT)

Standard Strapi Commands

yarn develop - Start development server (defaults to PostgreSQL; requires a running DB)
yarn build - Build for production
yarn start - Start production server
yarn strapi - Run Strapi CLI commands

V5 Seeding (Large Dataset)

Use the v5 seeder in this project to generate a large dataset for homepage perf testing:

yarn seed:v5

You can scale the volume with a multiplier:

yarn seed:v5 -- --multiplier 20

Or:

SEED_MULTIPLIER=20 yarn seed:v5