Commit Graph

16966 Commits

Author SHA1 Message Date
Jake Barnby 1b77a78121 (fix): store externalId on collection metadata, eliminate all mapping queries
- Database library: createCollection() accepts metadata parameter
  for arbitrary key-value pairs on the _metadata document
- Appwrite: passes externalId (user-facing collection ID) when
  creating collections via createCollection(metadata: ['externalId' => $collectionId])
- Metadata decorator: reads $collection->getAttribute('externalId')
  directly from the collection metadata — zero queries, zero caches,
  zero overhead
- getDatabasesDB: removed dbForProject dependency and all mapping logic

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 21:59:25 +13:00
Jake Barnby 1b665fdb6f (fix): bulk-load related collection mappings instead of findOne by $sequence
The findOne by $sequence query fails validation because $sequence is an
internal id-type attribute. Instead, load ALL collections for the
database once on first relationship encounter and cache statically per
database per Swoole worker. No per-attribute queries needed.

Also fixes UUID sequence extraction — the previous explode('_') split
UUIDs incorrectly.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 18:39:31 +13:00
Jake Barnby f66150aa8b (fix): defer event copy in UserEvents to fire-time for correct user context
The UserEvents hook copied event state at resource-resolution time
(before the init hook set the user), causing webhooks to receive null
user IDs. Now stores the source queueForEvents as a reference and
copies at fire-time when the user Document has been populated.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 13:56:49 +13:00
Jake Barnby 48a01919bc (fix): add static cache for related collection lookups
The findOne query per relationship attribute ran on every getDatabasesDB
call. Static cache keyed by database+internal name caches results across
requests within the same Swoole worker process.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 04:17:23 +13:00
Jake Barnby 4d85ab0dcd (fix): fix related collection lookup by sequence, increase schema timeouts
- Related collection lookup used internal name as document ID, but
  Appwrite collections have user-facing IDs. Extract sequence from
  internal name and use findOne by $sequence instead.
- Increase all schema polling timeouts from 240s to 360s for CI
  dedicated mode parallel load.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 03:40:01 +13:00
Jake Barnby 61a623fcc0 (fix): inject dbForProject into getDatabasesDB for related collection lookups
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 03:14:15 +13:00
Jake Barnby 95552031ef (fix): register related collection ID mappings and update migration lib
- getDatabasesDB now registers mappings for related collections from
  relationship attributes, not just the primary collection. This fixes
  testOneToOneRelationship where nested documents showed internal names.
- Update utopia-php/migration to handle Index objects in createCollection.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 03:01:58 +13:00
Jake Barnby 5740cecbe5 (fix): pass collection to getDatabasesDB for zero-query ID mapping
Endpoints pass the collection document they already have to
getDatabasesDB. The factory registers the single collection mapping
on the Metadata decorator — no bulk find queries, no static cache,
no dbForProject dependency.

The decorator is now purely stateless with zero database overhead.
Collection ID resolution uses only the pre-registered mapping from
the endpoint's request parameters.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 02:08:19 +13:00
Jake Barnby 3f7261167b Merge remote-tracking branch 'origin/1.9.x' into feat-query-lib 2026-04-02 00:56:28 +13:00
Jake Barnby d7b61e6afc (fix): move collection mapping query from decorator to getDatabasesDB with static cache
The decorator no longer queries dbForProject directly. Instead,
getDatabasesDB pre-populates the mapping via setCollectionId() and
uses a static cache (Metadata::getCachedMap) so the query runs at most
ONCE per database per Swoole worker process. The decorator is now
stateless — no database dependencies, just a pre-set map.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 23:24:46 +13:00
Jake Barnby dc77abf161 (fix): restore Metadata decorator with lazy collection mapping, remove processDocument
Remove processDocument() and all its calls — the decorator approach is
the intended design. The Metadata decorator lazily loads the collection
ID mapping from dbForProject on first use, wrapped in silent() to
prevent lifecycle hooks. Maps both relative (collection_N) and full
(database_M_collection_N) keys to user-facing IDs.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 21:42:00 +13:00
Jake Barnby 088f1c2953 (fix): ensure float range min/max are cast to float with fallback
The formatOptions min/max may be integers after JSON decode or may be
on the attribute directly (from range filter). Use floatval() and check
both formatOptions and direct attribute keys.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 21:27:50 +13:00
Jake Barnby 1eb9c386f6 (fix): replace Metadata decorator with processDocument() in endpoint actions
The Metadata decorator required a collection ID mapping query that added
overhead and caused Redis OOM in CI. Instead, stamp $databaseId and
$collectionId/$tableId directly in endpoint actions where user-facing
IDs are available from request parameters — matching the 1.9.x approach.

- Remove Metadata decorator from getDatabasesDB hooks
- Restore processDocument() on Documents/Action base class
- Add processDocument() calls in Get, Create, XList, Update, Upsert
- Simplify Metadata.php to stateless map-only decorator (no queries)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 19:58:07 +13:00
Jake Barnby 33b971a2c1 (fix): pre-compute collection mapping in getDatabasesDB, remove all queries from Metadata
Move the collection ID mapping query from the Metadata decorator to
getDatabasesDB where it runs once during resource init. The mapping
includes both relative keys (collection_N) and full keys
(database_M_collection_N) for dedicated mode compatibility.

The Metadata decorator is now purely stateless — no database queries,
no dbForProject dependency. It only uses the pre-set map from
setCollectionId().

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 18:56:14 +13:00
Jake Barnby 124002f419 (fix): lazy-load collection mapping with silent/skipValidation to reduce Redis pressure
Move collection ID mapping query from getDatabasesDB init to lazy
loading inside the Metadata decorator. The query is wrapped in
silent() and skipValidation() to prevent triggering lifecycle hooks
(Usage, Events) that were contributing to Redis OOM in CI.

The mapping is loaded once on first need, not during resource init,
reducing overhead for requests that don't access document endpoints.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 13:19:52 +13:00
Jake Barnby b05489bbc8 (fix): inject dbForProject into getDatabasesDB resource for collection mapping
The collection ID mapping query needs dbForProject which was missing
from the getDatabasesDB resource's dependency injection.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 12:22:20 +13:00
Jake Barnby dca59528df (fix): load collection ID mapping in getDatabasesDB instead of decorator
Move the collection mapping query from inside the Metadata decorator to
the getDatabasesDB resource factory. This avoids decorator-level
database queries that can fail in various contexts and cause cascading
issues.

The mapping is loaded once when getDatabasesDB creates the database
instance, and the results are passed to the Metadata decorator via
setCollectionId(). The decorator itself is now stateless with respect
to database queries.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 06:27:34 +13:00
Jake Barnby 4cb70799ab (fix): register floatRange format against ColumnType::Double
Float attributes created via API use type 'double' (ColumnType::Double)
since the Float/Create endpoint was updated. The format registration
must match to pass format validation during attribute creation.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 02:54:13 +13:00
Prem Palanisamy 4dfdfb5e59 Merge remote-tracking branch 'origin/1.9.x' into feat/import-export-json 2026-03-31 12:37:17 +01:00
Prem Palanisamy 5d1009b324 fix: correct resourceType routing, schemaless validation, and E2E tests for migrations
- Add getDatabaseResourceType() helper to map database types to resource constants
- Use database-specific resourceType for CSV/JSON import/export instead of hardcoded TYPE_DATABASE
- Skip attribute validation for schemaless databases (DocumentsDB/VectorsDB) in exports
- Parse JSON export queries in migration worker
- Restore MigrationsBase from 1.9.x and append VectorsDB/DocumentsDB E2E tests
2026-03-31 12:35:18 +01:00
Jake Barnby f9aee4de5d (fix): clear stale install data before starting new installation 2026-03-31 23:08:31 +13:00
Jake Barnby 2cf04e4b0f (fix): skip Metadata decorator for internal documents, add Tenancy hook, fix Mongo tenant null
- Metadata decorator now skips _metadata collection documents to prevent
  injected $databaseId/$collectionId attributes from leaking into SQL
  UPDATE statements via getAttributes()
- Add Tenancy hook to dbForProject in shared mode (was missing, causing
  401 on session verification)
- Update Collection/Table response models to use $databaseId with $
  prefix matching the decorator
- Update utopia-php/database with MongoDB tenant null filter fix

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 21:43:50 +13:00
Jake Barnby b47ac00ca8 (refactor): rename migrate param and add --migrate flag to upgrade task 2026-03-31 21:08:29 +13:00
Jake Barnby 2f53d09c5b (feat): add database migration step to upgrade installer 2026-03-31 20:58:33 +13:00
Jake Barnby 1fa1aa8621 (fix): guard against missing Host header in dispatch 2026-03-31 20:58:22 +13:00
Jake Barnby 6c5f028d24 (fix): use property access on GroupedQueries return type
Query::groupByType() now returns a GroupedQueries readonly class instead
of an associative array. All bracket access ($grouped['filters'], etc.)
must use arrow syntax ($grouped->filters).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 18:53:11 +13:00
Jake Barnby 26f70357c8 Merge remote-tracking branch 'origin/1.9.x' into feat-query-lib
# Conflicts:
#	app/config/collections/projects.php
#	src/Appwrite/Platform/Modules/Databases/Http/DocumentsDB/Collections/Indexes/Create.php
#	src/Appwrite/Platform/Modules/Storage/Http/Buckets/Files/Update.php
2026-03-31 17:09:24 +13:00
Jake Barnby 5c84efdd7c (fix): upgrade UI — remove min-height, hide secret key row, fix copy 2026-03-31 15:58:45 +13:00
Prem Palanisamy a80ecd0cb6 fix: alphabetize imports, update phpstan baseline count for migrations tests 2026-03-30 17:24:58 +01:00
premtsd-code 3bb6a8bcc8 Merge branch '1.9.x' into feat/import-export-json 2026-03-30 16:15:16 +01:00
Prem Palanisamy 2611bf4af1 fix: add setPlatform to JSON import trigger for consistency 2026-03-30 15:43:21 +01:00
Prem Palanisamy de219de31d fix: restore original databasetype block in CSV export 2026-03-30 15:26:09 +01:00
Prem Palanisamy d8bbd82556 fix: remove duplicate database fetch, add null-safe queries fallback, add schemaless comment 2026-03-30 14:55:05 +01:00
Prem Palanisamy aaebeec61e fix: remove spatial from documentsdb indexes, parse JSON export queries, skip schema validation for schemaless exports 2026-03-30 09:09:50 +01:00
ArnabChatterjee20k 995e9fdf39 Merge branch '1.9.x' into update-db-size 2026-03-30 13:07:30 +05:30
Claude 32005c0a49 fix: remove redundant new User(getArrayCopy()) wrapping
Since setDocumentType('users', User::class) is registered on all
database instances, getDocument('users', ...) already returns User
instances. The new User($doc->getArrayCopy()) pattern was redundant
and could lose internal state managed by the database layer.

https://claude.ai/code/session_01JLPDurUgyj7qViA8JqQFTH
2026-03-29 03:04:43 +00:00
Damodar Lohani 98c63a87d1 Merge branch '1.9.x' into claude/apply-cloud-user-pattern-az7qg 2026-03-29 08:30:11 +05:45
Chirag Aggarwal f901b6e0ac feat: move static SDKs off platform specs 2026-03-28 17:24:20 +05:30
premtsd-code 53d0e14f97 Merge branch '1.9.x' into feat/import-export-json 2026-03-28 04:23:07 +00:00
ArnabChatterjee20k 325b1d67b4 Merge branch '1.9.x' into update-db-size 2026-03-27 19:47:35 +05:30
ArnabChatterjee20k 3ff7cadcab updated project size 2026-03-27 18:39:26 +05:30
Jake Barnby 4b3de66e19 (fix): register Permissions hook on all DB instances in resources.php, worker.php, cli.php 2026-03-27 14:19:57 +13:00
Jake Barnby 8280bdaf9e (fix): remove manual Permissions/Tenancy from non-user-data DB instances 2026-03-27 11:31:47 +13:00
Jake Barnby 0ff9426d08 (fix): replace remaining array access on config Attribute objects 2026-03-27 04:42:05 +13:00
Jake Barnby 84d16aaa73 (fix): add missing Method import and fix double QueryQueryMethod replacement 2026-03-27 02:48:29 +13:00
Jake Barnby efa7ccc8fc (fix): alias Query\Method as QueryMethod to avoid SDK\Method collision 2026-03-27 02:37:41 +13:00
Jake Barnby 3e2504e75a (fix): replace all remaining Query::TYPE_* with Method enum across entire codebase 2026-03-27 02:26:51 +13:00
Jake Barnby 91727a4d11 (fix): replace remaining Document::SET_TYPE_* and Query::TYPE_* with enums 2026-03-27 02:15:24 +13:00
Jake Barnby 03e8e79424 (fix): remove dbForProject dependency from getDatabasesDB to prevent pool exhaustion 2026-03-27 02:02:19 +13:00
Jake Barnby 9d199ec0e5 (fix): pass ColumnType enum to Structure::addFormat and hasFormat 2026-03-27 01:51:07 +13:00