PHPStan narrowed $dsnScheme to literal 'redis' after the other arms
were ruled out, making the default arm dead code.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Stage-5 re-review C-Re2 outcome: the C8 decision was that Redis stays
on logs/console only, NOT on documentsdb/vectorsdb/database. However,
.github/workflows/ci.yml currently runs a Redis matrix entry with
_APP_DB_ADAPTER=redis that flows through the database pool.
Removing 'redis' from the database pool schemes here would break that
CI matrix entry. Per the fixup spec, do not remove it until the matrix
is reframed (point Redis at logs/console only) or dropped (rely on the
utopia-php/database adapter trait suite for coverage).
This commit replaces the soft NOTE with a loud WARNING that:
- states unambiguously that Redis is logs/console only;
- explains why 'redis' is still listed on the database pool today;
- documents the two paths to remove it (CI reframe or matrix drop);
- reaffirms that documentsdb/vectorsdb must NEVER allow 'redis'.
documentsdb and vectorsdb schemes were already redis-free; verified.
CONFLICT FLAGGED: this is the deviation called out in the fixup spec.
The C-Re2 fix cannot be completed in this pass without breaking CI.
Follow-up work: decide between reframing the CI matrix or dropping the
Redis matrix entry, then drop 'redis' from the 'database' and 'console'
pool schemes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
C8: Add a comment block above the connection pool definitions warning
that the `redis` scheme on the `database` and `logs` pools is for the
test matrix only — operators who set `_APP_DB_ADAPTER=redis` in
production will lose data on cache eviction or process restart.
Redis stays OFF the `documentsdb` and `vectorsdb` pools, which have
stricter durability and indexing requirements that the in-memory
Utopia Redis adapter does not provide.
M12: Restore the matrix-generator logic that picks the per-PR matrix
based on whether `composer.lock` changed `utopia-php/database`. PRs
that bump the database package run the full Mariadb+Postgres+Mongo+
Redis matrix in both dedicated and shared modes; everything else
runs the SQLite+Redis short matrix in shared mode. The previous
"TEMP — revert before merge" early `return` short-circuited that
logic and forced every PR through the short matrix.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Wires the new Redis database adapter through the Appwrite stack:
- registers.php: add 'redis' to schemes for console/database/logs pools,
add an explicit redis resource arm honouring the optional db segment,
and dispatch to Utopia\Database\Adapter\Redis in the database match
- composer.json: switch utopia-php/database constraint to
dev-feat-redis-adapter to pick up the new adapter (lockfile bump
follows after the database PR is pushed to origin)
- docker-compose.yml: add a dedicated redis-mirror service with
noeviction policy and AOF persistence so the database backend stays
isolated from the cache instance
- ci.yml: add Redis to default and full database matrices and export
the matching env vars (_APP_DB_HOST=redis-mirror, port 6379)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
SQLite has no statement-level timeout primitive, so the test's expectation
that a slow query returns 408 is not implementable on this adapter. Skip
explicitly rather than letting it surface as a failure on the SQLite matrix.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Previously every connection pool (console, database, documentsdb,
vectorsdb, logs) opened the same SQLite file, so they all serialised
through one writer lock — even though they're conceptually independent
databases. Treat _APP_DB_SQLITE_PATH as a directory + stem template and
replace the stem with the pool key, so each gets its own file and
unrelated writes no longer queue against each other.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Tune SQLite for high-concurrency test workloads:
- synchronous=OFF skips fsync (safe for ephemeral test data only).
- 256 MB page cache + 2 GB mmap window cuts read I/O dramatically.
- temp_store=MEMORY keeps temporary tables off disk.
- busy_timeout to 60s and wal_autocheckpoint to every 10k pages so
long write bursts don't fight the checkpointer.
- 64 MB WAL size cap.
Mount the SQLite docker volume as tmpfs so writes never reach disk.
The test DB is wiped between CI runs so durability is irrelevant.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
SQLite serialises writes through a single file lock. Running services
like Databases and TablesDB with paratest_processes=3 produces SQLITE_BUSY
storms — 97 lock errors in Databases and 283 in TablesDB on the prior run
even with a 30s busy_timeout. Drop concurrency to 1 for the SQLite matrix
specifically; the legacy adapter matrix keeps its existing parallelism.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Short-circuit the matrix script so this PR runs SQLite shared-tables only
while we drive the SQLite test suite to green. Revert this commit before
merging so the upstream branch-aware matrix logic kicks back in.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Default per-PR runs target SQLite/shared only. PRs that bump
utopia-php/database swap to the legacy MariaDB/PostgreSQL/MongoDB matrix
across both modes — SQLite is excluded there since the SQLite layer is
already covered by every other PR.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Default per-PR runs target only SQLite shared-tables, since SQLite is the
test backend and most PRs don't touch the database adapter layer. When
utopia-php/database changes, run the full MariaDB/PostgreSQL/MongoDB
matrix across both modes to catch adapter regressions.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This branch is exclusively about wiring SQLite as the test backend, so
running the rest of the matrix is wasted CI. Strip databases to SQLite
and modes to shared.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
E2E test backend was hitting SQLITE_BUSY ("database is locked") under
Swoole's concurrent worker fanout, since SQLite serialises writes through
a single file lock. The previous 5s timeout was too tight for parallel
test load. NORMAL synchronous halves write fsync cost on WAL — fine for
ephemeral test databases.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pulls latest utopia-php/database (e44768f) with the emulateMySQL flag and
opts SQLite into MariaDB-shape behaviour so the existing test suite, which
assumes MariaDB semantics, continues to pass against SQLite.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pulls in the FTS5 fulltext implementation (utopia-php/database#870) plus
the relationship/batch-attrs/schema-introspection capability flips. Drops
the need for the Audit::setup() workarounds and the bootstrap fulltext
filter on the SQLite path — those will come out in a follow-up once CI
confirms the upstream branch is stable on its own.
Same upstream-broken Utopia\Exception in audit's Adapter\Database::setup().
This call site fires every time a project is created, so any test that
creates projects (which is almost all of them) crashes when running
against SQLite. Mirror the http.php fix: build the audit collection
directly through the adapter's schema documents.
The original fulltext filter only covered the createDatabase() helper, but
http.php has three more inline createCollection paths — two for default
buckets and one for project metadata collections — each of which builds
indexes via array_map without consulting adapter capability flags. SQLite
doesn't support fulltext, so the project-collection path tripped during
EVENT_START and crashed every worker.
Extract buildSupportedIndexes() and use it from all four sites so the
filter is applied consistently. The 1-line ?? defaults on lengths/orders
match the dbForProject site's tolerance for raw config without those keys.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
utopia-php/audit's Adapter\Database::setup() calls $db->exists($db->getDatabase())
and throws Utopia\Exception on false. Two issues:
1. SQLite adapter's exists($db, null) always returns false (it has no concept
of named databases — once the file is open, the DB exists), so the throw
path always fires.
2. The audit lib's `use Utopia\Exception` references a class that doesn't
exist anywhere in the dependency tree, so the throw fatals with
`Class "Utopia\Exception" not found`.
Both are upstream bugs, but for this PR I just sidestep the broken setup()
and create the audit collection directly through the regular Database API
using the adapter's schema documents. Same end result, no exists() probe.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two fixes for the SQLite test wiring:
1. .env had emptied _APP_DB_USER/_APP_DB_PASS/_APP_DB_ROOT_PASS, but the
mongodb/mariadb/postgresql containers still consume those during init
regardless of which adapter Appwrite uses. Empty values left the
side-running mongodb (needed for documentsdb) refusing to start, which
blocked docker compose up --wait for every CI matrix row, including
MongoDB. Restoring the original test credentials.
2. Bootstrap was unconditionally creating fulltext indexes from collection
configs. SQLite adapter explicitly returns false for
getSupportForFulltextIndex(), so it threw on the first such index. Now
the bootstrap consults the adapter and skips fulltext indexes when not
supported.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Jake BarnbyandClaude Opus 4.7
Matej BačoandGitHub
Matej Bačo
Shmuel FogelandGitHub
Luke B. SilverandGitHub
frubio
Chirag AggarwalandGitHub
Torsten DittmannandGitHub