Commit Graph

347 Commits

Author SHA1 Message Date
Jake Barnby 1b77a78121 (fix): store externalId on collection metadata, eliminate all mapping queries
- Database library: createCollection() accepts metadata parameter
  for arbitrary key-value pairs on the _metadata document
- Appwrite: passes externalId (user-facing collection ID) when
  creating collections via createCollection(metadata: ['externalId' => $collectionId])
- Metadata decorator: reads $collection->getAttribute('externalId')
  directly from the collection metadata — zero queries, zero caches,
  zero overhead
- getDatabasesDB: removed dbForProject dependency and all mapping logic

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 21:59:25 +13:00
Jake Barnby 1b665fdb6f (fix): bulk-load related collection mappings instead of findOne by $sequence
The findOne by $sequence query fails validation because $sequence is an
internal id-type attribute. Instead, load ALL collections for the
database once on first relationship encounter and cache statically per
database per Swoole worker. No per-attribute queries needed.

Also fixes UUID sequence extraction — the previous explode('_') split
UUIDs incorrectly.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 18:39:31 +13:00
Jake Barnby f66150aa8b (fix): defer event copy in UserEvents to fire-time for correct user context
The UserEvents hook copied event state at resource-resolution time
(before the init hook set the user), causing webhooks to receive null
user IDs. Now stores the source queueForEvents as a reference and
copies at fire-time when the user Document has been populated.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 13:56:49 +13:00
Jake Barnby 48a01919bc (fix): add static cache for related collection lookups
The findOne query per relationship attribute ran on every getDatabasesDB
call. Static cache keyed by database+internal name caches results across
requests within the same Swoole worker process.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 04:17:23 +13:00
Jake Barnby 4d85ab0dcd (fix): fix related collection lookup by sequence, increase schema timeouts
- Related collection lookup used internal name as document ID, but
  Appwrite collections have user-facing IDs. Extract sequence from
  internal name and use findOne by $sequence instead.
- Increase all schema polling timeouts from 240s to 360s for CI
  dedicated mode parallel load.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 03:40:01 +13:00
Jake Barnby 61a623fcc0 (fix): inject dbForProject into getDatabasesDB for related collection lookups
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 03:14:15 +13:00
Jake Barnby 95552031ef (fix): register related collection ID mappings and update migration lib
- getDatabasesDB now registers mappings for related collections from
  relationship attributes, not just the primary collection. This fixes
  testOneToOneRelationship where nested documents showed internal names.
- Update utopia-php/migration to handle Index objects in createCollection.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 03:01:58 +13:00
Jake Barnby 5740cecbe5 (fix): pass collection to getDatabasesDB for zero-query ID mapping
Endpoints pass the collection document they already have to
getDatabasesDB. The factory registers the single collection mapping
on the Metadata decorator — no bulk find queries, no static cache,
no dbForProject dependency.

The decorator is now purely stateless with zero database overhead.
Collection ID resolution uses only the pre-registered mapping from
the endpoint's request parameters.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 02:08:19 +13:00
Jake Barnby d7b61e6afc (fix): move collection mapping query from decorator to getDatabasesDB with static cache
The decorator no longer queries dbForProject directly. Instead,
getDatabasesDB pre-populates the mapping via setCollectionId() and
uses a static cache (Metadata::getCachedMap) so the query runs at most
ONCE per database per Swoole worker process. The decorator is now
stateless — no database dependencies, just a pre-set map.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 23:24:46 +13:00
Jake Barnby dc77abf161 (fix): restore Metadata decorator with lazy collection mapping, remove processDocument
Remove processDocument() and all its calls — the decorator approach is
the intended design. The Metadata decorator lazily loads the collection
ID mapping from dbForProject on first use, wrapped in silent() to
prevent lifecycle hooks. Maps both relative (collection_N) and full
(database_M_collection_N) keys to user-facing IDs.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 21:42:00 +13:00
Jake Barnby 1eb9c386f6 (fix): replace Metadata decorator with processDocument() in endpoint actions
The Metadata decorator required a collection ID mapping query that added
overhead and caused Redis OOM in CI. Instead, stamp $databaseId and
$collectionId/$tableId directly in endpoint actions where user-facing
IDs are available from request parameters — matching the 1.9.x approach.

- Remove Metadata decorator from getDatabasesDB hooks
- Restore processDocument() on Documents/Action base class
- Add processDocument() calls in Get, Create, XList, Update, Upsert
- Simplify Metadata.php to stateless map-only decorator (no queries)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 19:58:07 +13:00
Jake Barnby 33b971a2c1 (fix): pre-compute collection mapping in getDatabasesDB, remove all queries from Metadata
Move the collection ID mapping query from the Metadata decorator to
getDatabasesDB where it runs once during resource init. The mapping
includes both relative keys (collection_N) and full keys
(database_M_collection_N) for dedicated mode compatibility.

The Metadata decorator is now purely stateless — no database queries,
no dbForProject dependency. It only uses the pre-set map from
setCollectionId().

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 18:56:14 +13:00
Jake Barnby 124002f419 (fix): lazy-load collection mapping with silent/skipValidation to reduce Redis pressure
Move collection ID mapping query from getDatabasesDB init to lazy
loading inside the Metadata decorator. The query is wrapped in
silent() and skipValidation() to prevent triggering lifecycle hooks
(Usage, Events) that were contributing to Redis OOM in CI.

The mapping is loaded once on first need, not during resource init,
reducing overhead for requests that don't access document endpoints.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 13:19:52 +13:00
Jake Barnby b05489bbc8 (fix): inject dbForProject into getDatabasesDB resource for collection mapping
The collection ID mapping query needs dbForProject which was missing
from the getDatabasesDB resource's dependency injection.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 12:22:20 +13:00
Jake Barnby dca59528df (fix): load collection ID mapping in getDatabasesDB instead of decorator
Move the collection mapping query from inside the Metadata decorator to
the getDatabasesDB resource factory. This avoids decorator-level
database queries that can fail in various contexts and cause cascading
issues.

The mapping is loaded once when getDatabasesDB creates the database
instance, and the results are passed to the Metadata decorator via
setCollectionId(). The decorator itself is now stateless with respect
to database queries.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 06:27:34 +13:00
Jake Barnby 2cf04e4b0f (fix): skip Metadata decorator for internal documents, add Tenancy hook, fix Mongo tenant null
- Metadata decorator now skips _metadata collection documents to prevent
  injected $databaseId/$collectionId attributes from leaking into SQL
  UPDATE statements via getAttributes()
- Add Tenancy hook to dbForProject in shared mode (was missing, causing
  401 on session verification)
- Update Collection/Table response models to use $databaseId with $
  prefix matching the decorator
- Update utopia-php/database with MongoDB tenant null filter fix

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 21:43:50 +13:00
Jake Barnby 26f70357c8 Merge remote-tracking branch 'origin/1.9.x' into feat-query-lib
# Conflicts:
#	app/config/collections/projects.php
#	src/Appwrite/Platform/Modules/Databases/Http/DocumentsDB/Collections/Indexes/Create.php
#	src/Appwrite/Platform/Modules/Storage/Http/Buckets/Files/Update.php
2026-03-31 17:09:24 +13:00
Claude 32005c0a49 fix: remove redundant new User(getArrayCopy()) wrapping
Since setDocumentType('users', User::class) is registered on all
database instances, getDocument('users', ...) already returns User
instances. The new User($doc->getArrayCopy()) pattern was redundant
and could lose internal state managed by the database layer.

https://claude.ai/code/session_01JLPDurUgyj7qViA8JqQFTH
2026-03-29 03:04:43 +00:00
Jake Barnby 4b3de66e19 (fix): register Permissions hook on all DB instances in resources.php, worker.php, cli.php 2026-03-27 14:19:57 +13:00
Jake Barnby 8280bdaf9e (fix): remove manual Permissions/Tenancy from non-user-data DB instances 2026-03-27 11:31:47 +13:00
Jake Barnby 3e2504e75a (fix): replace all remaining Query::TYPE_* with Method enum across entire codebase 2026-03-27 02:26:51 +13:00
Jake Barnby 03e8e79424 (fix): remove dbForProject dependency from getDatabasesDB to prevent pool exhaustion 2026-03-27 02:02:19 +13:00
Jake Barnby 1230f8201f (feat): add hook pipeline and move usage/permissions/metadata to hooks 2026-03-27 00:50:59 +13:00
Claude 9aa488c961 fix: wrap getDocument('users') results in User instances
The user resource and realtime handlers return Document objects from
getDocument(), but isPrivileged()/isApp() are now instance methods on
the User class. Wrapping results with new User() ensures the correct
type is returned for all code paths.

https://claude.ai/code/session_01JLPDurUgyj7qViA8JqQFTH
2026-03-26 02:47:57 +00:00
ArnabChatterjee20k 8ae07ac61f Merge remote-tracking branch 'origin/1.9.x' into revert-11585-revert-11402-sync-mongodb 2026-03-23 10:47:23 +05:30
eldadfux b81f3f8267 Merge origin/1.8.x into feat-user-impersonation 2026-03-19 22:12:38 +01:00
ArnabChatterjee20k c7907932e4 Revert "Revert "Documentsdb + vectordb (latest)"" 2026-03-19 20:30:42 +05:30
ArnabChatterjee20k 9917f95dfd Revert "Documentsdb + vectordb (latest)" 2026-03-19 19:18:27 +05:30
Matej Bačo 7e7cac017c Merge pull request #11582 from appwrite/fix-webhooks-duplication
Fix: webhook endpoints duplication
2026-03-19 14:16:30 +01:00
ArnabChatterjee20k 6fa34aef04 linting 2026-03-19 16:39:19 +05:30
Matej Bačo 6f8a54273d AI review fixes 2026-03-19 11:47:02 +01:00
Matej Bačo 37a7c70c2b Fix webhook endpoints duplication 2026-03-19 11:27:13 +01:00
ArnabChatterjee20k 826ac1abe9 dummy commit to check some tests and edge cases 2026-03-19 12:45:16 +05:30
eldadfux 85fcc52b84 Merge origin/1.8.x into feat-user-impersonation 2026-03-18 10:23:03 +01:00
ArnabChatterjee20k 8d58383c2e Merge remote-tracking branch 'origin/1.8.x' into sync-mongodb 2026-03-18 11:38:37 +05:30
Jake Barnby 09c963abda fix: use $sequence for team lookup instead of $internalId
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-14 10:51:18 +13:00
Jake Barnby eb2f0b1cea fix: restore team lookup by $internalId and remove duplicate composer key
The team resource query was changed from $internalId to $sequence,
breaking console session authentication (401 on project creation).
Also removed duplicate utopia-php/servers entry in composer.json
and updated composer.lock hash.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-14 10:40:29 +13:00
Jake Barnby 01a9340eaf Merge branch '1.8.x' into feat-installer 2026-03-14 10:30:29 +13:00
eldadfux f6d38fe1ce Merge remote-tracking branch 'origin/1.8.x' into feat-user-impersonation
Made-with: Cursor

# Conflicts:
#	app/controllers/shared/api.php
2026-03-13 21:48:41 +01:00
Jake Barnby 6fee0cb73e Revert "fix: cast tenant to int for SQL adapters to fix strict type comparison"
This reverts commit 840b1fbef3.
2026-03-14 01:16:14 +13:00
Jake Barnby 840b1fbef3 fix: cast tenant to int for SQL adapters to fix strict type comparison
Document::getTenant() casts numeric tenants to (int), but
adapter->getTenant() held a string from getSequence(). The strict
!== comparison in Database::getCollection() then failed, returning
"Collection not found" for all shared-table MariaDB projects.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-14 01:06:43 +13:00
eldadfux b85cf2fdb6 applied new logic for logs 2026-03-13 09:18:39 +01:00
Jake Barnby 871795efa9 fix: remove conditional tenant int cast, use string tenant directly
The $tenant attribute type in the database library is being changed from
VAR_INTEGER to VAR_ID, which handles both SQL (integer) and MongoDB
(UUID7 string) adapters natively. This removes the now-unnecessary
conditional casting pattern throughout the codebase.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 19:53:42 +13:00
Jake Barnby d91e3543c8 (fix): use adapter ID type to determine tenant value for MongoDB compatibility 2026-03-13 19:13:07 +13:00
eldadfux 8304a8e0e4 Add impersonation feature for user management
- Introduced a new API endpoint to update user impersonator capability.
- Enhanced user model to include impersonator attributes.
- Updated database schema to support impersonation.
- Implemented impersonation logic in the request handling to allow users with impersonator capability to act as other users.
- Added relevant API documentation for impersonation headers.

This feature allows users with the appropriate permissions to impersonate other users, enhancing flexibility in user management.
2026-03-12 19:08:25 +01:00
loks0n a804cba999 Refactor usage metrics to stateless publisher pattern
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 15:36:16 +00:00
ArnabChatterjee20k ffa4645d25 Refactor database adapter initialization for attribute support
- Moved the `setSupportForAttributes` method call from the adapter initialization to the database resource creation in `getDatabasesDB`.
- Updated the logic to ensure attribute support is set correctly based on the database type.
2026-03-10 17:32:38 +05:30
ArnabChatterjee20k e4178d9ae6 Merge remote-tracking branch 'origin/sync-mongodb' into sync-mongodb 2026-03-10 16:34:10 +05:30
ArnabChatterjee20k bf0d49cc76 Refactor embedding timeout and clean database logic
- Updated embedding agent timeout to be configurable via environment variable.
- Removed commented code in XList for clarity.
- Refactored database cleaning logic into separate methods for better readability and maintainability.
2026-03-10 15:41:32 +05:30
ArnabChatterjee20k f6655343c7 pr comments addressed
* Added new exception `MIGRATION_DATABASE_TYPE_UNSUPPORTED` with proper error metadata and HTTP 400 response.
* Replaced generic CSV database type errors with the new migration-specific exception for clearer error handling.
* Added support for `DOCUMENTSDB` in migration transfer resource service mapping.
* Fixed Appwrite report initialization by correctly injecting `getDatabasesDB`.
* Updated database adapter initialization to conditionally disable attribute support for `DOCUMENTSDB`.
* Moved `setSupportForAttributes` logic from pool initialization to database resource creation.
* Removed duplicate `getDatabasesDB` resource definition and redundant database event listener setup.
* Cleaned up unused variables and minor code inconsistencies.
* Fixed docblock formatting in `TransactionState`.
* Adjusted metrics handling in VectorDB embeddings text creation (removed unnecessary trigger/reset flow).
2026-03-10 14:30:57 +05:30