Merge remote-tracking branch 'origin/1.9.x' into presence-api

This commit is contained in:
ArnabChatterjee20k
2026-04-30 16:41:48 +05:30
124 changed files with 3763 additions and 2014 deletions
@@ -0,0 +1,174 @@
# Parallel Chunk Upload Support for utopia-php/storage
## Context
The Appwrite API now supports out-of-order chunked uploads (chunks can arrive in any sequence). The next step is **parallel uploads** — multiple chunks uploaded simultaneously via separate HTTP requests. The SDK guarantees the first chunk is sent before any parallel chunks, so the document creation race is handled at the API layer. However, the storage device layer has a race condition that must be fixed.
## Problem: `Local::joinChunks()` Race
When two requests upload the final missing chunks in parallel, both can observe `countChunks() == $chunks` and call `joinChunks()` simultaneously.
### Current behavior (loser throws)
```php
// Local::joinChunks()
$dest = \fopen($tmpAssemble, 'wb');
// ... stream all parts into $tmpAssemble ...
if (! \rename($tmpAssemble, $path)) {
\unlink($tmpAssemble);
throw new Exception('Failed to finalize assembled file '.$path);
}
```
The winner succeeds with `rename()`. The loser gets `false` from `rename()` (file already exists at `$path`) and throws a 500-error exception. The client that lost the race receives an error even though the file is fully assembled.
### Required behavior
If `$path` already exists, another request already assembled the file. The loser should **silently succeed** — the file is complete, nothing more to do.
## Proposed Changes
### 1. `Local::joinChunks()` — Handle assembly race
Before opening `$tmpAssemble`, check if the final file already exists. If it does, skip assembly entirely.
```php
private function joinChunks(string $path, int $chunks): void
{
// Race winner already assembled the file
if (\file_exists($path)) {
return;
}
$tmp = \dirname($path).DIRECTORY_SEPARATOR.'tmp_'.asename($path);
$tmpAssemble = \dirname($path).DIRECTORY_SEPARATOR.'tmp_assemble_'.asename($path);
// ... rest of assembly logic ...
if (! \rename($tmpAssemble, $path)) {
// Another request may have won the race between fclose and rename
if (\file_exists($path)) {
\unlink($tmpAssemble);
return;
}
\unlink($tmpAssemble);
throw new Exception('Failed to finalize assembled file '.$path);
}
// ... cleanup ...
}
```
### 2. `Local::countChunks()` — Reliability under concurrent writes
`countChunks()` uses `glob()` on the temp directory. Under heavy parallel load, `glob()` might miss files or return inconsistent counts. The current implementation is already fairly robust (it validates `.part.\d+` suffix), but we should document that the return value is a best-effort snapshot.
No code change needed here unless tests reveal issues.
### 3. Tests — Concurrent chunk uploads
Add a test that simulates two parallel requests completing a multi-chunk upload:
```php
public function testParallelChunkUpload(): void
{
$storage = $this->makeJoinTestStorage();
$dest = $storage->getRoot().DIRECTORY_SEPARATOR.'parallel.dat';
// Upload chunk 1 (creates temp directory)
$storage->uploadData('AAAA', $dest, 'application/octet-stream', 1, 2);
// Simulate two parallel requests uploading the last chunk
// In a real test, use pcntl_fork() or pthreads for true concurrency
// For the test suite, sequential calls are sufficient if we verify
// the second call doesn't throw after the first completed assembly
$storage->uploadData('BBBB', $dest, 'application/octet-stream', 2, 2);
// Verify file exists and is correct
$this->assertTrue(\file_exists($dest));
$this->assertSame('AAAABBBB', \file_get_contents($dest));
// Verify second assembly attempt doesn't throw
// (This simulates the race where another request already assembled)
try {
$storage->uploadData('BBBB', $dest, 'application/octet-stream', 2, 2);
} catch (\Exception $e) {
$this->fail('Duplicate assembly should not throw: '.$e->getMessage());
}
$storage->delete($storage->getRoot(), true);
}
```
A more realistic concurrent test using `pcntl_fork()`:
```php
public function testParallelChunkUploadWithFork(): void
{
if (!\function_exists('pcntl_fork')) {
$this->markTestSkipped('pcntl extension required for fork-based concurrency test');
}
$storage = $this->makeJoinTestStorage();
$dest = $storage->getRoot().DIRECTORY_SEPARATOR.'parallel-fork.dat';
// Pre-upload chunk 1
$storage->uploadData('AAAA', $dest, 'application/octet-stream', 1, 2);
$pid = pcntl_fork();
if ($pid === -1) {
$this->fail('Failed to fork');
} elseif ($pid === 0) {
// Child process: upload chunk 2
try {
$storage->uploadData('BBBB', $dest, 'application/octet-stream', 2, 2);
exit(0);
} catch (\Exception $e) {
exit(1);
}
}
// Parent process: also upload chunk 2 (race condition)
$parentSuccess = true;
try {
$storage->uploadData('BBBB', $dest, 'application/octet-stream', 2, 2);
} catch (\Exception $e) {
$parentSuccess = false;
}
pcntl_waitpid($pid, $status);
$childSuccess = pcntl_wexitstatus($status) === 0;
// At least one should succeed
$this->assertTrue($parentSuccess || $childSuccess, 'At least one parallel upload should succeed');
// File should be correctly assembled
$this->assertTrue(\file_exists($dest));
$this->assertSame('AAAABBBB', \file_get_contents($dest));
$storage->delete($storage->getRoot(), true);
}
```
## S3 Device
S3 already handles out-of-order multipart uploads natively. The `completeMultipartUpload` call with `ksort()` sorts parts by number regardless of upload order. However, parallel `completeMultipartUpload` calls for the same `uploadId` would still be problematic.
This is an **API-layer concern** — the Appwrite API should ensure only one request calls `completeMultipartUpload` per upload. The S3 device itself does not need changes.
## Files to Change
| File | Change |
|------|--------|
| `src/Storage/Device/Local.php` | Add `file_exists($path)` guard at start of `joinChunks()` and in `rename()` failure handler |
| `tests/Storage/Device/LocalTest.php` | Add `testParallelChunkUpload` and `testParallelChunkUploadWithFork` |
## Backwards Compatibility
Fully backwards compatible. The change only affects the error path when `rename()` fails due to an existing file. Previously it threw; now it returns silently. No public API signatures change.
## Related PRs
- Appwrite server PR: https://github.com/appwrite/appwrite/pull/12138 (out-of-order upload support)
- This storage PR is a prerequisite for the follow-up Appwrite PR that enables parallel chunk uploads at the API level.
+2
View File
@@ -47,6 +47,8 @@ _APP_DB_SCHEMA=appwrite
_APP_DB_USER=user
_APP_DB_PASS=password
_APP_DB_ROOT_PASS=rootsecretpassword
_APP_DATABASE_SHARED_TABLES=
_APP_DATABASE_SHARED_NAMESPACE=
_APP_DB_ADAPTER_DOCUMENTSDB=mongodb
_APP_DB_HOST_DOCUMENTSDB=mongodb
_APP_DB_PORT_DOCUMENTSDB=27017
+93 -57
View File
@@ -7,6 +7,7 @@ concurrency:
env:
COMPOSE_FILE: docker-compose.yml
IMAGE: appwrite-dev
REGISTRY_IMAGE: ghcr.io/${{ github.repository }}/appwrite-dev
K6_VERSION: '0.53.0'
on:
@@ -19,6 +20,10 @@ on:
type: string
default: ''
permissions:
contents: read
packages: write
jobs:
dependencies:
name: Checks / Dependencies
@@ -258,32 +263,30 @@ jobs:
username: ${{ vars.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Login to GHCR
uses: docker/login-action@v4
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v4
- name: Build Appwrite
- name: Build and push Appwrite
uses: docker/build-push-action@v6
with:
context: .
push: false
tags: ${{ env.IMAGE }}
load: true
push: true
tags: ${{ env.REGISTRY_IMAGE }}:${{ github.sha }}
cache-from: type=gha
cache-to: type=gha,mode=max
outputs: type=docker,dest=/tmp/${{ env.IMAGE }}.tar
target: development
build-args: |
DEBUG=false
TESTING=true
VERSION=dev
- name: Upload Docker Image
uses: actions/upload-artifact@v7
with:
name: ${{ env.IMAGE }}
path: /tmp/${{ env.IMAGE }}.tar
retention-days: 1
unit:
name: Tests / Unit
runs-on: ubuntu-latest
@@ -291,26 +294,32 @@ jobs:
permissions:
contents: read
pull-requests: write
packages: read
steps:
- name: checkout
uses: actions/checkout@v6
- name: Download Docker Image
uses: actions/download-artifact@v7
with:
name: ${{ env.IMAGE }}
path: /tmp
- name: Login to Docker Hub
uses: docker/login-action@v4
with:
username: ${{ vars.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Login to GHCR
uses: docker/login-action@v4
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Pull Docker Image
run: |
docker pull ${{ env.REGISTRY_IMAGE }}:${{ github.sha }}
docker tag ${{ env.REGISTRY_IMAGE }}:${{ github.sha }} ${{ env.IMAGE }}
- name: Load and Start Appwrite
timeout-minutes: 5
run: |
docker load --input /tmp/${{ env.IMAGE }}.tar
docker compose pull --quiet --ignore-buildable
docker compose up -d --quiet-pull --wait
@@ -338,26 +347,32 @@ jobs:
permissions:
contents: read
pull-requests: write
packages: read
steps:
- name: checkout
uses: actions/checkout@v6
- name: Download Docker Image
uses: actions/download-artifact@v7
with:
name: ${{ env.IMAGE }}
path: /tmp
- name: Login to Docker Hub
uses: docker/login-action@v4
with:
username: ${{ vars.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Login to GHCR
uses: docker/login-action@v4
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Pull Docker Image
run: |
docker pull ${{ env.REGISTRY_IMAGE }}:${{ github.sha }}
docker tag ${{ env.REGISTRY_IMAGE }}:${{ github.sha }} ${{ env.IMAGE }}
- name: Load and Start Appwrite
timeout-minutes: 5
run: |
docker load --input /tmp/${{ env.IMAGE }}.tar
docker compose pull --quiet --ignore-buildable
docker compose up -d --quiet-pull --wait
@@ -396,6 +411,7 @@ jobs:
permissions:
contents: read
pull-requests: write
packages: read
strategy:
fail-fast: false
matrix:
@@ -451,16 +467,10 @@ jobs:
- name: Checkout repository
uses: actions/checkout@v6
- name: Download Docker Image
uses: actions/download-artifact@v7
with:
name: ${{ env.IMAGE }}
path: /tmp
- name: Set environment
run: |
echo "_APP_OPTIONS_ROUTER_PROTECTION=enabled" >> $GITHUB_ENV
if [ "${{ matrix.database }}" = "MariaDB" ]; then
echo "COMPOSE_PROFILES=mariadb" >> $GITHUB_ENV
echo "_APP_DB_ADAPTER=mariadb" >> $GITHUB_ENV
@@ -484,6 +494,18 @@ jobs:
username: ${{ vars.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Login to GHCR
uses: docker/login-action@v4
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Pull Docker Image
run: |
docker pull ${{ env.REGISTRY_IMAGE }}:${{ github.sha }}
docker tag ${{ env.REGISTRY_IMAGE }}:${{ github.sha }} ${{ env.IMAGE }}
- name: Load and Start Appwrite
timeout-minutes: 5
env:
@@ -492,7 +514,6 @@ jobs:
_APP_DATABASE_DOCUMENTSDB_SHARED_TABLES: ${{ matrix.mode != 'dedicated' && 'documentsdb_db_main' || '' }}
_APP_DATABASE_VECTORSDB_SHARED_TABLES: ${{ matrix.mode != 'dedicated' && 'vectorsdb_db_main' || '' }}
run: |
docker load --input /tmp/${{ env.IMAGE }}.tar
docker compose pull --quiet --ignore-buildable
docker compose up -d --quiet-pull --wait
@@ -546,6 +567,7 @@ jobs:
permissions:
contents: read
pull-requests: write
packages: read
strategy:
fail-fast: false
matrix:
@@ -556,18 +578,24 @@ jobs:
with:
fetch-depth: 1
- name: Download Docker Image
uses: actions/download-artifact@v7
with:
name: ${{ env.IMAGE }}
path: /tmp
- name: Login to Docker Hub
uses: docker/login-action@v4
with:
username: ${{ vars.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Login to GHCR
uses: docker/login-action@v4
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Pull Docker Image
run: |
docker pull ${{ env.REGISTRY_IMAGE }}:${{ github.sha }}
docker tag ${{ env.REGISTRY_IMAGE }}:${{ github.sha }} ${{ env.IMAGE }}
- name: Load and Start Appwrite
timeout-minutes: 5
env:
@@ -576,7 +604,6 @@ jobs:
_APP_DATABASE_DOCUMENTSDB_SHARED_TABLES: ${{ matrix.mode != 'dedicated' && 'documentsdb_db_main' || '' }}
_APP_DATABASE_VECTORSDB_SHARED_TABLES: ${{ matrix.mode != 'dedicated' && 'vectorsdb_db_main' || '' }}
run: |
docker load --input /tmp/${{ env.IMAGE }}.tar
docker compose pull --quiet --ignore-buildable
docker compose up -d --quiet-pull --wait
@@ -607,6 +634,7 @@ jobs:
permissions:
contents: read
pull-requests: write
packages: read
strategy:
fail-fast: false
matrix:
@@ -615,18 +643,24 @@ jobs:
- name: Checkout repository
uses: actions/checkout@v6
- name: Download Docker Image
uses: actions/download-artifact@v7
with:
name: ${{ env.IMAGE }}
path: /tmp
- name: Login to Docker Hub
uses: docker/login-action@v4
with:
username: ${{ vars.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Login to GHCR
uses: docker/login-action@v4
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Pull Docker Image
run: |
docker pull ${{ env.REGISTRY_IMAGE }}:${{ github.sha }}
docker tag ${{ env.REGISTRY_IMAGE }}:${{ github.sha }} ${{ env.IMAGE }}
- name: Load and Start Appwrite
timeout-minutes: 5
env:
@@ -634,7 +668,6 @@ jobs:
_APP_DATABASE_DOCUMENTSDB_SHARED_TABLES: ${{ matrix.mode != 'dedicated' && 'documentsdb_db_main' || '' }}
_APP_DATABASE_VECTORSDB_SHARED_TABLES: ${{ matrix.mode != 'dedicated' && 'vectorsdb_db_main' || '' }}
run: |
docker load --input /tmp/${{ env.IMAGE }}.tar
docker compose pull --quiet --ignore-buildable
docker compose up -d --quiet-pull --wait
@@ -676,28 +709,31 @@ jobs:
contents: read
issues: write
pull-requests: write
packages: read
steps:
- name: Checkout repository
uses: actions/checkout@v6
with:
fetch-depth: 1
- name: Download Docker Image
uses: actions/download-artifact@v7
with:
name: ${{ env.IMAGE }}
path: /tmp
- name: Login to Docker Hub
uses: docker/login-action@v4
with:
username: ${{ vars.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Load Appwrite image
- name: Login to GHCR
uses: docker/login-action@v4
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Pull Appwrite image
run: |
docker load --input /tmp/${{ env.IMAGE }}.tar
docker tag ${{ env.IMAGE }} ${{ env.IMAGE }}:after
docker pull ${{ env.REGISTRY_IMAGE }}:${{ github.sha }}
docker tag ${{ env.REGISTRY_IMAGE }}:${{ github.sha }} ${{ env.IMAGE }}
docker tag ${{ env.REGISTRY_IMAGE }}:${{ github.sha }} ${{ env.IMAGE }}:after
- name: Setup k6
uses: grafana/setup-k6-action@ffe7d7290dfa715e48c2ccc924d068444c94bde2
+31 -1
View File
@@ -5,6 +5,11 @@ on:
types:
- closed
permissions:
actions: write
contents: read
packages: write
jobs:
cleanup:
runs-on: ubuntu-latest
@@ -36,4 +41,29 @@ jobs:
done
done
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Cleanup GHCR image
continue-on-error: true
run: |
package_path="${GITHUB_REPOSITORY#*/}/appwrite-dev"
encoded_path="$(printf '%s' "$package_path" | jq -Rr @uri)"
gh api --paginate "/repos/${GITHUB_REPOSITORY}/pulls/${{ github.event.pull_request.number }}/commits" --jq '.[].sha' | while read -r sha; do
version_ids=$(gh api --paginate -H "Accept: application/vnd.github+json" \
"/orgs/${GITHUB_REPOSITORY_OWNER}/packages/container/${encoded_path}/versions" \
--jq ".[] | select(.metadata.container.tags | index(\"${sha}\")) | .id")
if [ -z "$version_ids" ]; then
echo "No GHCR version found for SHA ${sha}"
continue
fi
echo "$version_ids" | while read -r version_id; do
gh api --method DELETE -H "Accept: application/vnd.github+json" \
"/orgs/${GITHUB_REPOSITORY_OWNER}/packages/container/${encoded_path}/versions/${version_id}"
echo "Deleted ${package_path}:${sha} (version ${version_id})"
done
done
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+6 -2
View File
@@ -24,9 +24,11 @@ jobs:
ignore-unfixed: 'false'
severity: 'CRITICAL,HIGH'
- name: Upload Docker Image Scan Results
uses: github/codeql-action/upload-sarif@v2
uses: github/codeql-action/upload-sarif@v4
if: always() && hashFiles('trivy-image-results.sarif') != ''
with:
sarif_file: 'trivy-image-results.sarif'
category: 'trivy-image'
scan-code:
name: Scan Code
@@ -42,6 +44,8 @@ jobs:
output: 'trivy-fs-results.sarif'
severity: 'CRITICAL,HIGH'
- name: Upload Code Scan Results
uses: github/codeql-action/upload-sarif@v2
uses: github/codeql-action/upload-sarif@v4
if: always() && hashFiles('trivy-fs-results.sarif') != ''
with:
sarif_file: 'trivy-fs-results.sarif'
category: 'trivy-source'
+7 -2
View File
@@ -12,7 +12,7 @@ RUN composer install --ignore-platform-reqs --optimize-autoloader \
--no-plugins --no-scripts --prefer-dist \
`if [ "$TESTING" != "true" ]; then echo "--no-dev"; fi`
FROM appwrite/base:1.0.1 AS base
FROM appwrite/base:1.2.1 AS base
LABEL maintainer="team@appwrite.io"
@@ -24,6 +24,10 @@ ENV _APP_VERSION=$VERSION \
_APP_HOME=https://appwrite.io
RUN \
if [ "$DEBUG" != "true" ]; then \
rm -f /usr/local/etc/php/conf.d/docker-php-ext-xdebug.ini && \
rm -f /usr/local/lib/php/extensions/no-debug-non-zts-*/xdebug.so; \
fi && \
if [ "$DEBUG" == "true" ]; then \
apk add boost boost-dev; \
fi
@@ -100,7 +104,8 @@ RUN mkdir -p /etc/letsencrypt/live/ && chmod -Rf 755 /etc/letsencrypt/live/
FROM base AS production
RUN rm -rf /usr/src/code/app/config/specs && \
rm -f /usr/local/lib/php/extensions/no-debug-non-zts-20240924/xdebug.so && \
rm -f /usr/local/etc/php/conf.d/docker-php-ext-xdebug.ini /usr/local/etc/php/conf.d/xdebug.ini && \
rm -f /usr/local/lib/php/extensions/no-debug-non-zts-*/xdebug.so && \
find /usr -name '*.a' -delete 2>/dev/null || true && \
find /usr -type d -name '__pycache__' -exec rm -rf {} + 2>/dev/null || true && \
find /usr -name '*.pyc' -delete 2>/dev/null || true
+20
View File
@@ -157,12 +157,19 @@ $container->set('getProjectDB', function (Group $pools, Database $dbForPlatform,
}
if (isset($databases[$dsn->getHost()])) {
/** @var array $collections */
$collections = Config::getParam('collections', []);
$projectCollections = $collections['projects'] ?? [];
$projectsGlobalCollections = array_keys($projectCollections);
$projectsGlobalCollections[] = 'audit';
$database = $databases[$dsn->getHost()];
$sharedTables = \explode(',', System::getEnv('_APP_DATABASE_SHARED_TABLES', ''));
if (\in_array($dsn->getHost(), $sharedTables)) {
$database
->setSharedTables(true)
->setGlobalCollections($projectsGlobalCollections)
->setTenant($project->getSequence())
->setNamespace($dsn->getParam('namespace'));
} else {
@@ -182,9 +189,16 @@ $container->set('getProjectDB', function (Group $pools, Database $dbForPlatform,
$sharedTables = \explode(',', System::getEnv('_APP_DATABASE_SHARED_TABLES', ''));
if (\in_array($dsn->getHost(), $sharedTables)) {
/** @var array $collections */
$collections = Config::getParam('collections', []);
$projectCollections = $collections['projects'] ?? [];
$projectsGlobalCollections = array_keys($projectCollections);
$projectsGlobalCollections[] = 'audit';
$database
->setSharedTables(true)
->setTenant($project->getSequence())
->setGlobalCollections($projectsGlobalCollections)
->setNamespace($dsn->getParam('namespace'));
} else {
$database
@@ -212,6 +226,11 @@ $container->set('getLogsDB', function (Group $pools, Cache $cache, Authorization
return $database;
}
/** @var array $collections */
$collections = Config::getParam('collections', []);
$logsCollections = $collections['logs'] ?? [];
$logsCollections = array_keys($logsCollections);
$adapter = new DatabasePool($pools->get('logs'));
$database = new Database($adapter, $cache);
@@ -220,6 +239,7 @@ $container->set('getLogsDB', function (Group $pools, Cache $cache, Authorization
->setAuthorization($authorization)
->setSharedTables(true)
->setNamespace('logsV1')
->setGlobalCollections($logsCollections)
->setTimeout(APP_DATABASE_TIMEOUT_MILLISECONDS_TASK)
->setMaxQueryValues(APP_DATABASE_QUERY_MAX_VALUES);
+10
View File
@@ -28,6 +28,16 @@
"emails.invitation.thanks": "Gracias.,",
"emails.invitation.buttonText": "Aceptar invitación a {{team}}",
"emails.invitation.signature": "El equipo de {{project}}",
"emails.sessionAlert.subject": "Alerta de seguridad: nueva sesión en tu cuenta de {{project}}",
"emails.sessionAlert.preview": "Nuevo inicio de sesión detectado en {{project}} a las {{time}} UTC.",
"emails.sessionAlert.hello": "Hola {{user}},",
"emails.sessionAlert.body": "Se ha creado una nueva sesión en tu cuenta de {{b}}{{project}}{{/b}}, {{b}}el {{date}} de {{year}} a las {{time}} UTC{{/b}}.\nEstos son los detalles de la nueva sesión:",
"emails.sessionAlert.listDevice": "Dispositivo: {{b}}{{device}}{{/b}}",
"emails.sessionAlert.listIpAddress": "Dirección IP: {{b}}{{ipAddress}}{{/b}}",
"emails.sessionAlert.listCountry": "País: {{b}}{{country}}{{/b}}",
"emails.sessionAlert.footer": "Si has sido tú, no tienes que hacer nada más.\nSi no has iniciado esta sesión o sospechas actividad no autorizada, protege tu cuenta.",
"emails.sessionAlert.thanks": "Gracias,",
"emails.sessionAlert.signature": "El equipo de {{project}}",
"locale.country.unknown": "Desconocido",
"countries.af": "Afganistán",
"countries.ao": "Angola",
+5 -5
View File
@@ -23,8 +23,8 @@ $member = [
'projects.read',
'locale.read',
'avatars.read',
'execution.read',
'execution.write',
'executions.read',
'executions.write',
'targets.read',
'targets.write',
'subscribers.write',
@@ -85,8 +85,8 @@ $admins = [
'sites.write',
'log.read',
'log.write',
'execution.read',
'execution.write',
'executions.read',
'executions.write',
'rules.read',
'rules.write',
'migrations.read',
@@ -127,7 +127,7 @@ return [
'files.write',
'locale.read',
'avatars.read',
'execution.write',
'executions.write',
],
],
User::ROLE_USERS => [
+332 -231
View File
@@ -1,14 +1,341 @@
<?php
return [ // List of publicly visible scopes
'sessions.write' => [
'description' => 'Access to create, update, and delete user sessions',
// List of publicly visible scopes
return [
// Project
"project.read" => [
"description" =>
"Access to read project\'s information",
"category" => "Project",
],
"project.write" => [
"description" =>
"Access to update project\'s information",
"category" => "Project",
],
"keys.read" => [
"description" =>
"Access to read project\'s keys",
"category" => "Project",
],
"keys.write" => [
"description" =>
"Access to create, update, and delete project\'s keys",
"category" => "Project",
],
"platforms.read" => [
"description" =>
"Access to read project\'s platforms",
"category" => "Project",
],
"platforms.write" => [
"description" =>
"Access to create, update, and delete project\'s platforms",
"category" => "Project",
],
"mocks.read" => [
"description" =>
"Access to read project\'s mocks",
"category" => "Project",
],
"mocks.write" => [
"description" =>
"Access to create, update, and delete project\'s mocks",
"category" => "Project",
],
"policies.read" => [
"description" =>
"Access to read project\'s policies",
"category" => "Project",
],
"policies.write" => [
"description" =>
"Access to update project\'s policies",
"category" => "Project",
],
"templates.read" => [
"description" =>
"Access to read project\'s templates",
"category" => "Project",
],
"templates.write" => [
"description" =>
"Access to create, update, and delete project\'s templates",
"category" => "Project",
],
"oauth2.read" => [
"description" =>
"Access to read project\'s OAuth2 configuration",
"category" => "Project",
],
"oauth2.write" => [
"description" =>
"Access to update project\'s OAuth2 configuration",
"category" => "Project",
],
// Auth
'users.read' => [
'description' => 'Access to read your project\'s users',
'description' => 'Access to read users',
'category' => 'Auth',
],
'users.write' => [
'description' => 'Access to create, update, and delete your project\'s users',
'description' => 'Access to create, update, and delete users',
'category' => 'Auth',
],
'sessions.read' => [
'description' => 'Access to read user sessions',
'category' => 'Auth',
],
'sessions.write' => [
'description' => 'Access to create, update, and delete user sessions',
'category' => 'Auth',
],
'teams.read' => [
'description' => 'Access to read teams',
'category' => 'Auth',
],
'teams.write' => [
'description' => 'Access to create, update, and delete teams',
'category' => 'Auth',
],
// Databases
'databases.read' => [
'description' => 'Access to read databases',
'category' => 'Databases',
],
'databases.write' => [
'description' => 'Access to create, update, and delete databases',
'category' => 'Databases',
],
'tables.read' => [
'description' => 'Access to read database tables',
'category' => 'Databases',
],
'tables.write' => [
'description' => 'Access to create, update, and delete database tables',
'category' => 'Databases',
],
'columns.read' => [
'description' => 'Access to read database table columns',
'category' => 'Databases',
],
'columns.write' => [
'description' => 'Access to create, update, and delete database table columns',
'category' => 'Databases',
],
'indexes.read' => [
'description' => 'Access to read database table indexes',
'category' => 'Databases',
],
'indexes.write' => [
'description' => 'Access to create, update, and delete database table indexes',
'category' => 'Databases',
],
'rows.read' => [
'description' => 'Access to read database table rows',
'category' => 'Databases',
],
'rows.write' => [
'description' => 'Access to create, update, and delete database table rows',
'category' => 'Databases',
],
'collections.read' => [
'description' => 'Access to read database collections',
'category' => 'Databases',
'deprecated' => true,
],
'collections.write' => [
'description' => 'Access to create, update, and delete database collections',
'category' => 'Databases',
'deprecated' => true,
],
'attributes.read' => [
'description' => 'Access to read database collection attributes',
'category' => 'Databases',
'deprecated' => true,
],
'attributes.write' => [
'description' => 'Access to create, update, and delete database collection attributes',
'category' => 'Databases',
'deprecated' => true,
],
'documents.read' => [
'description' => 'Access to read database collection documents',
'category' => 'Databases',
'deprecated' => true,
],
'documents.write' => [
'description' => 'Access to create, update, and delete database collection documents',
'category' => 'Databases',
'deprecated' => true,
],
// Storage
'buckets.read' => [
'description' => 'Access to read storage buckets',
'category' => 'Storage',
],
'buckets.write' => [
'description' => 'Access to create, update, and delete storage buckets',
'category' => 'Storage',
],
'files.read' => [
'description' => 'Access to read storage files and preview images',
'category' => 'Storage',
],
'files.write' => [
'description' => 'Access to create, update, and delete storage files',
'category' => 'Storage',
],
'tokens.read' => [
'description' => 'Access to read storage file tokens',
'category' => 'Storage',
],
'tokens.write' => [
'description' => 'Access to create, update, and delete storage file tokens',
'category' => 'Storage',
],
// Functions
'functions.read' => [
'description' => 'Access to read functions and deployments',
'category' => 'Functions',
],
'functions.write' => [
'description' => 'Access to create, update, and delete functions and deployments',
'category' => 'Functions',
],
'executions.read' => [
'description' => 'Access to read function executions',
'category' => 'Functions',
],
'executions.write' => [
'description' => 'Access to create function executions',
'category' => 'Functions',
],
// Sites
'sites.read' => [
'description' => 'Access to read sites and deployments',
'category' => 'Sites',
],
'sites.write' => [
'description' => 'Access to create, update, and delete sites and deployments',
'category' => 'Sites',
],
'log.read' => [
'description' => 'Access to read site logs',
'category' => 'Sites',
],
'log.write' => [
'description' => 'Access to update, and delete site logs',
'category' => 'Sites',
],
// Messaging
'providers.read' => [
'description' => 'Access to read messaging providers',
'category' => 'Messaging',
],
'providers.write' => [
'description' => 'Access to create, update, and delete messaging providers',
'category' => 'Messaging',
],
'topics.read' => [
'description' => 'Access to read messaging topics',
'category' => 'Messaging',
],
'topics.write' => [
'description' => 'Access to create, update, and delete messaging topics',
'category' => 'Messaging',
],
'subscribers.read' => [
'description' => 'Access to read messaging subscribers',
'category' => 'Messaging',
],
'subscribers.write' => [
'description' => 'Access to create, update, and delete messaging subscribers',
'category' => 'Messaging',
],
'targets.read' => [
'description' => 'Access to read messaging targets',
'category' => 'Messaging',
],
'targets.write' => [
'description' => 'Access to create, update, and delete messaging targets',
'category' => 'Messaging',
],
'messages.read' => [
'description' => 'Access to read messaging messages',
'category' => 'Messaging',
],
'messages.write' => [
'description' => 'Access to create, update, and delete messaging messages',
'category' => 'Messaging',
],
// Other
"webhooks.read" => [
"description" =>
"Access to read webhooks",
'category' => 'Other',
],
"webhooks.write" => [
"description" =>
"Access to create, update, and delete webhooks",
'category' => 'Other',
],
'locale.read' => [
'description' => 'Access to use Locale service',
'category' => 'Other',
],
'avatars.read' => [
'description' => 'Access to use Avatars service',
'category' => 'Other',
],
'health.read' => [
'description' => 'Access to use Health service',
'category' => 'Other',
],
'assistant.read' => [
'description' => 'Access to use Assistant service',
'category' => 'Other',
],
'migrations.read' => [
'description' => 'Access to read migrations',
'category' => 'Other',
],
'migrations.write' => [
'description' => 'Access to create, update, and delete migrations.',
'category' => 'Other',
],
// TODO: Figure out where to move those
'schedules.read' => [
'description' => 'Access to read schedules.',
'category' => 'Other',
],
'schedules.write' => [
'description' => 'Access to create, update, and delete schedules.',
'category' => 'Other',
],
'vcs.read' => [
'description' => 'Access to read resources under VCS service.',
'category' => 'Other',
],
'vcs.write' => [
'description' => 'Access to create, update, and delete resources under VCS service.',
'category' => 'Other',
],
'rules.read' => [
'description' => 'Access to read proxy rules.',
'category' => 'Other',
],
'rules.write' => [
'description' => 'Access to create, update, and delete proxy rules.',
'category' => 'Other',
],
'presences.read' => [
'description' => 'Access to read your project\'s presences',
@@ -16,230 +343,4 @@ return [ // List of publicly visible scopes
'presences.read' => [
'description' => 'Access to create, update, and delete your project\'s presences',
],
'teams.read' => [
'description' => 'Access to read your project\'s teams',
],
'teams.write' => [
'description' => 'Access to create, update, and delete your project\'s teams',
],
'databases.read' => [
'description' => 'Access to read your project\'s databases',
],
'databases.write' => [
'description' => 'Access to create, update, and delete your project\'s databases',
],
'collections.read' => [
'description' => 'Access to read your project\'s database collections',
],
'collections.write' => [
'description' => 'Access to create, update, and delete your project\'s database collections',
],
'tables.read' => [
'description' => 'Access to read your project\'s database tables',
],
'tables.write' => [
'description' => 'Access to create, update, and delete your project\'s database tables',
],
'attributes.read' => [
'description' => 'Access to read your project\'s database collection\'s attributes',
],
'attributes.write' => [
'description' => 'Access to create, update, and delete your project\'s database collection\'s attributes',
],
'columns.read' => [
'description' => 'Access to read your project\'s database table\'s columns',
],
'columns.write' => [
'description' => 'Access to create, update, and delete your project\'s database table\'s columns',
],
'indexes.read' => [
'description' => 'Access to read your project\'s database table\'s indexes',
],
'indexes.write' => [
'description' => 'Access to create, update, and delete your project\'s database table\'s indexes',
],
'documents.read' => [
'description' => 'Access to read your project\'s database documents',
],
'documents.write' => [
'description' => 'Access to create, update, and delete your project\'s database documents',
],
'rows.read' => [
'description' => 'Access to read your project\'s database rows',
],
'rows.write' => [
'description' => 'Access to create, update, and delete your project\'s database rows',
],
'files.read' => [
'description' => 'Access to read your project\'s storage files and preview images',
],
'files.write' => [
'description' => 'Access to create, update, and delete your project\'s storage files',
],
'buckets.read' => [
'description' => 'Access to read your project\'s storage buckets',
],
'buckets.write' => [
'description' => 'Access to create, update, and delete your project\'s storage buckets',
],
'functions.read' => [
'description' => 'Access to read your project\'s functions and code deployments',
],
'functions.write' => [
'description' => 'Access to create, update, and delete your project\'s functions and code deployments',
],
'sites.read' => [
'description' => 'Access to read your project\'s sites and deployments',
],
'sites.write' => [
'description' => 'Access to create, update, and delete your project\'s sites and deployments',
],
'log.read' => [
'description' => 'Access to read your site\'s logs',
],
'log.write' => [
'description' => 'Access to update, and delete your site\'s logs',
],
'execution.read' => [
'description' => 'Access to read your project\'s execution logs',
],
'execution.write' => [
'description' => 'Access to execute your project\'s functions',
],
'locale.read' => [
'description' => 'Access to access your project\'s Locale service',
],
'avatars.read' => [
'description' => 'Access to access your project\'s Avatars service',
],
'health.read' => [
'description' => 'Access to read your project\'s health status',
],
'providers.read' => [
'description' => 'Access to read your project\'s providers',
],
'providers.write' => [
'description' => 'Access to create, update, and delete your project\'s providers',
],
'messages.read' => [
'description' => 'Access to read your project\'s messages',
],
'messages.write' => [
'description' => 'Access to create, update, and delete your project\'s messages',
],
'topics.read' => [
'description' => 'Access to read your project\'s topics',
],
'topics.write' => [
'description' => 'Access to create, update, and delete your project\'s topics',
],
'subscribers.read' => [
'description' => 'Access to read your project\'s subscribers',
],
'subscribers.write' => [
'description' => 'Access to create, update, and delete your project\'s subscribers',
],
'targets.read' => [
'description' => 'Access to read your project\'s targets',
],
'targets.write' => [
'description' => 'Access to create, update, and delete your project\'s targets',
],
'rules.read' => [
'description' => 'Access to read your project\'s proxy rules',
],
'rules.write' => [
'description' => 'Access to create, update, and delete your project\'s proxy rules',
],
'schedules.read' => [
'description' => 'Access to read your project\'s schedules',
],
'schedules.write' => [
'description' => 'Access to create, update, and delete your project\'s schedules',
],
'migrations.read' => [
'description' => 'Access to read your project\'s migrations',
],
'migrations.write' => [
'description' => 'Access to create, update, and delete your project\'s migrations.',
],
'vcs.read' => [
'description' => 'Access to read your project\'s VCS repositories',
],
'vcs.write' => [
'description' => 'Access to create, update, and delete your project\'s VCS repositories',
],
'assistant.read' => [
'description' => 'Access to read the Assistant service',
],
'tokens.read' => [
'description' => 'Access to read your project\'s tokens',
],
'tokens.write' => [
'description' => 'Access to create, update, and delete your project\'s tokens',
],
"webhooks.read" => [
"description" =>
"Access to read project\'s webhooks",
],
"webhooks.write" => [
"description" =>
"Access to create, update, and delete project\'s webhooks",
],
"project.read" => [
"description" =>
"Access to read project\'s information",
],
"project.write" => [
"description" =>
"Access to update project\'s information",
],
"keys.read" => [
"description" =>
"Access to read project\'s keys",
],
"keys.write" => [
"description" =>
"Access to create, update, and delete project\'s keys",
],
"platforms.read" => [
"description" =>
"Access to read project\'s platforms",
],
"platforms.write" => [
"description" =>
"Access to create, update, and delete project\'s platforms",
],
"mocks.read" => [
"description" =>
"Access to read project\'s mocks",
],
"mocks.write" => [
"description" =>
"Access to create, update, and delete project\'s mocks",
],
"policies.read" => [
"description" =>
"Access to read project\'s policies",
],
"policies.write" => [
"description" =>
"Access to update project\'s policies",
],
"templates.read" => [
"description" =>
"Access to read project\'s templates",
],
"templates.write" => [
"description" =>
"Access to create, update, and delete project\'s templates",
],
"oauth2.read" => [
"description" =>
"Access to read project\'s OAuth2 configuration",
],
"oauth2.write" => [
"description" =>
"Access to update project\'s OAuth2 configuration",
],
];
+1 -1
View File
@@ -286,7 +286,7 @@ return [
'name' => 'Migrations',
'subtitle' => 'The Migrations service allows you to migrate third-party data to your Appwrite project.',
'description' => '/docs/services/migrations.md',
'controller' => 'api/migrations.php',
'controller' => '', // Uses modules
'sdk' => true,
'docs' => true,
'docsUrl' => 'https://appwrite.io/docs/migrations',
File diff suppressed because it is too large Load Diff
+4 -4
View File
@@ -856,7 +856,7 @@ Http::get('/v1/users/:userId/targets/:targetId')
Http::get('/v1/users/:userId/sessions')
->desc('List user sessions')
->groups(['api', 'users'])
->label('scope', 'users.read')
->label('scope', ['users.read', 'sessions.read'])
->label('sdk', new Method(
namespace: 'users',
group: 'sessions',
@@ -2314,7 +2314,7 @@ Http::post('/v1/users/:userId/sessions')
->desc('Create session')
->groups(['api', 'users'])
->label('event', 'users.[userId].sessions.[sessionId].create')
->label('scope', 'users.write')
->label('scope', ['users.write', 'sessions.write'])
->label('audits.event', 'session.create')
->label('audits.resource', 'user/{request.userId}')
->label('usage.metric', 'sessions.{scope}.requests.create')
@@ -2470,7 +2470,7 @@ Http::delete('/v1/users/:userId/sessions/:sessionId')
->desc('Delete user session')
->groups(['api', 'users'])
->label('event', 'users.[userId].sessions.[sessionId].delete')
->label('scope', 'users.write')
->label('scope', ['users.write', 'sessions.write'])
->label('audits.event', 'session.delete')
->label('audits.resource', 'user/{request.userId}')
->label('sdk', new Method(
@@ -2521,7 +2521,7 @@ Http::delete('/v1/users/:userId/sessions')
->desc('Delete user sessions')
->groups(['api', 'users'])
->label('event', 'users.[userId].sessions.delete')
->label('scope', 'users.write')
->label('scope', ['users.write', 'sessions.write'])
->label('audits.event', 'session.delete')
->label('audits.resource', 'user/{user.$id}')
->label('sdk', new Method(
+1
View File
@@ -244,6 +244,7 @@ const APP_AUTH_TYPE_KEY = 'Key';
const APP_AUTH_TYPE_ADMIN = 'Admin';
// Response related
const MAX_OUTPUT_CHUNK_SIZE = 10 * 1024 * 1024; // 10MB
const APP_LIMIT_UPLOAD_CHUNK_SIZE = 5 * 1024 * 1024; // 5MB
const APP_FUNCTION_LOG_LENGTH_LIMIT = 1000000;
const APP_FUNCTION_ERROR_LENGTH_LIMIT = 1000000;
// Function headers
+6
View File
@@ -159,10 +159,16 @@ $container->set('getLogsDB', function (Group $pools, Cache $cache, Authorization
$adapter = new DatabasePool($pools->get('logs'));
$database = new Database($adapter, $cache);
/** @var array $collections */
$collections = Config::getParam('collections', []);
$logsCollections = $collections['logs'] ?? [];
$logsCollections = array_keys($logsCollections);
$database
->setDatabase(APP_DATABASE)
->setAuthorization($authorization)
->setSharedTables(true)
->setGlobalCollections($logsCollections)
->setNamespace('logsV1')
->setTimeout(APP_DATABASE_TIMEOUT_MILLISECONDS_API)
->setMaxQueryValues(APP_DATABASE_QUERY_MAX_VALUES);
+28
View File
@@ -204,9 +204,16 @@ return function (Container $container): void {
$sharedTables = \explode(',', System::getEnv('_APP_DATABASE_SHARED_TABLES', ''));
if (\in_array($dsn->getHost(), $sharedTables)) {
/** @var array $collections */
$collections = Config::getParam('collections', []);
$projectCollections = $collections['projects'] ?? [];
$projectsGlobalCollections = array_keys($projectCollections);
$projectsGlobalCollections[] = 'audit';
$database
->setSharedTables(true)
->setTenant($project->getSequence())
->setGlobalCollections($projectsGlobalCollections)
->setNamespace($dsn->getParam('namespace'));
} else {
$database
@@ -223,6 +230,11 @@ return function (Container $container): void {
$adapter = null;
return function (?Document $project = null) use ($pools, $cache, $authorization, &$adapter) {
/** @var array $collections */
$collections = Config::getParam('collections', []);
$logsCollections = $collections['logs'] ?? [];
$logsCollections = array_keys($logsCollections);
$adapter ??= new DatabasePool($pools->get('logs'));
$database = new Database($adapter, $cache);
@@ -230,6 +242,7 @@ return function (Container $container): void {
->setDatabase(APP_DATABASE)
->setAuthorization($authorization)
->setSharedTables(true)
->setGlobalCollections($logsCollections)
->setNamespace('logsV1')
->setTimeout(APP_DATABASE_TIMEOUT_MILLISECONDS_API)
->setMaxQueryValues(APP_DATABASE_QUERY_MAX_VALUES);
@@ -690,8 +703,15 @@ return function (Container $container): void {
$sharedTables = \explode(',', System::getEnv('_APP_DATABASE_SHARED_TABLES', ''));
if (\in_array($dsn->getHost(), $sharedTables)) {
/** @var array $collections */
$collections = Config::getParam('collections', []);
$projectCollections = $collections['projects'] ?? [];
$projectsGlobalCollections = array_keys($projectCollections);
$projectsGlobalCollections[] = 'audit';
$database
->setSharedTables(true)
->setGlobalCollections($projectsGlobalCollections)
->setTenant($project->getSequence())
->setNamespace($dsn->getParam('namespace'));
} else {
@@ -1292,6 +1312,12 @@ return function (Container $container): void {
$database = new Database($adapter, $cache);
$sharedTables = \array_filter(\explode(',', System::getEnv('_APP_DATABASE_SHARED_TABLES', '')));
/** @var array $collections */
$collections = Config::getParam('collections', []);
$projectCollections = $collections['projects'] ?? [];
$projectsGlobalCollections = array_keys($projectCollections);
$projectsGlobalCollections[] = 'audit';
$database
->setDatabase(APP_DATABASE)
->setAuthorization($authorization)
@@ -1314,6 +1340,7 @@ return function (Container $container): void {
if (\in_array($databaseHost, $dbTypeSharedTables)) {
$database
->setSharedTables(true)
->setGlobalCollections($projectsGlobalCollections)
->setTenant($project->getSequence())
->setNamespace($databaseDSN->getParam('namespace'));
} else {
@@ -1325,6 +1352,7 @@ return function (Container $container): void {
} elseif (\in_array($dsn->getHost(), $sharedTables)) {
$database
->setSharedTables(true)
->setGlobalCollections($projectsGlobalCollections)
->setTenant($project->getSequence())
->setNamespace($dsn->getParam('namespace'));
} else {
+38
View File
@@ -14,6 +14,7 @@ use Appwrite\Utopia\Database\Documents\User;
use Utopia\Audit\Adapter\Database as AdapterDatabase;
use Utopia\Audit\Audit as UtopiaAudit;
use Utopia\Cache\Cache;
use Utopia\Config\Config;
use Utopia\Console;
use Utopia\Database\Adapter\Pool as DatabasePool;
use Utopia\Database\Database;
@@ -90,8 +91,15 @@ return function (Container $container): void {
$sharedTables = \explode(',', System::getEnv('_APP_DATABASE_SHARED_TABLES', ''));
if (\in_array($dsn->getHost(), $sharedTables)) {
/** @var array $collections */
$collections = Config::getParam('collections', []);
$projectCollections = $collections['projects'] ?? [];
$projectsGlobalCollections = array_keys($projectCollections);
$projectsGlobalCollections[] = 'audit';
$database
->setSharedTables(true)
->setGlobalCollections($projectsGlobalCollections)
->setTenant($project->getSequence())
->setNamespace($dsn->getParam('namespace'));
} else {
@@ -130,8 +138,15 @@ return function (Container $container): void {
$sharedTables = \explode(',', System::getEnv('_APP_DATABASE_SHARED_TABLES', ''));
if (\in_array($dsn->getHost(), $sharedTables)) {
/** @var array $collections */
$collections = Config::getParam('collections', []);
$projectCollections = $collections['projects'] ?? [];
$projectsGlobalCollections = array_keys($projectCollections);
$projectsGlobalCollections[] = 'audit';
$database
->setSharedTables(true)
->setGlobalCollections($projectsGlobalCollections)
->setTenant($project->getSequence())
->setNamespace($dsn->getParam('namespace'));
} else {
@@ -152,8 +167,15 @@ return function (Container $container): void {
$sharedTables = \explode(',', System::getEnv('_APP_DATABASE_SHARED_TABLES', ''));
if (\in_array($dsn->getHost(), $sharedTables)) {
/** @var array $collections */
$collections = Config::getParam('collections', []);
$projectCollections = $collections['projects'] ?? [];
$projectsGlobalCollections = array_keys($projectCollections);
$projectsGlobalCollections[] = 'audit';
$database
->setSharedTables(true)
->setGlobalCollections($projectsGlobalCollections)
->setTenant($project->getSequence())
->setNamespace($dsn->getParam('namespace'));
} else {
@@ -210,6 +232,14 @@ return function (Container $container): void {
$sharedTables = \array_filter(\explode(',', System::getEnv('_APP_DATABASE_SHARED_TABLES', '')));
/** @var array $collections */
$collections = Config::getParam('collections', []);
$projectCollections = $collections['projects'] ?? [];
$projectsGlobalCollections = array_keys($projectCollections);
$projectsGlobalCollections[] = 'audit';
$database->setGlobalCollections($projectsGlobalCollections);
// For separate pools (documentsdb/vectorsdb), check their own shared tables config.
// If not configured, use dedicated mode to avoid cross-engine tenant type mismatches.
if ($databaseHost !== $dsn->getHost()) {
@@ -222,6 +252,7 @@ return function (Container $container): void {
if (\in_array($databaseHost, $dbTypeSharedTables)) {
$database
->setSharedTables(true)
->setGlobalCollections($projectsGlobalCollections)
->setTenant($projectDocument->getSequence())
->setNamespace($databaseDSN->getParam('namespace'));
} else {
@@ -233,6 +264,7 @@ return function (Container $container): void {
} elseif (\in_array($dsn->getHost(), $sharedTables, true)) {
$database
->setSharedTables(true)
->setGlobalCollections($projectsGlobalCollections)
->setTenant($projectDocument->getSequence())
->setNamespace($dsn->getParam('namespace'));
} else {
@@ -257,6 +289,11 @@ return function (Container $container): void {
return $database;
}
/** @var array $collections */
$collections = Config::getParam('collections', []);
$logsCollections = $collections['logs'] ?? [];
$logsCollections = array_keys($logsCollections);
$adapter = new DatabasePool($pools->get('logs'));
$database = new Database($adapter, $cache);
@@ -264,6 +301,7 @@ return function (Container $container): void {
->setDatabase(APP_DATABASE)
->setAuthorization($authorization)
->setSharedTables(true)
->setGlobalCollections($logsCollections)
->setNamespace('logsV1')
->setTimeout(APP_DATABASE_TIMEOUT_MILLISECONDS_WORKER)
->setMaxQueryValues(APP_DATABASE_QUERY_MAX_VALUES_WORKER);
+6
View File
@@ -162,8 +162,14 @@ if (!function_exists('getProjectDB')) {
$sharedTables = \explode(',', System::getEnv('_APP_DATABASE_SHARED_TABLES', ''));
if (\in_array($dsn->getHost(), $sharedTables)) {
$collections = Config::getParam('collections', []);
$projectCollections = $collections['projects'] ?? [];
$projectsGlobalCollections = array_keys($projectCollections);
$projectsGlobalCollections[] = 'audit';
$database
->setSharedTables(true)
->setGlobalCollections($projectsGlobalCollections)
->setTenant($project->getSequence())
->setNamespace($dsn->getParam('namespace'));
} else {
+1 -1
View File
@@ -82,7 +82,7 @@
"utopia-php/queue": "0.17.*",
"utopia-php/servers": "0.3.*",
"utopia-php/registry": "0.5.*",
"utopia-php/storage": "1.0.*",
"utopia-php/storage": "2.*",
"utopia-php/system": "0.10.*",
"utopia-php/telemetry": "0.2.*",
"utopia-php/vcs": "3.*",
Generated
+40 -41
View File
@@ -4,7 +4,7 @@
"Read more about it at https://getcomposer.org/doc/01-basic-usage.md#installing-dependencies",
"This file is @generated automatically"
],
"content-hash": "805802552f7482eaeae4bdaa505ae982",
"content-hash": "4bee36b21a57e754d2b3417e72dc9599",
"packages": [
{
"name": "adhocore/jwt",
@@ -3351,16 +3351,16 @@
},
{
"name": "utopia-php/abuse",
"version": "1.2.2",
"version": "1.2.3",
"source": {
"type": "git",
"url": "https://github.com/utopia-php/abuse.git",
"reference": "20bee84fd14dbe81d50ecabf1ffd81cceca06152"
"reference": "53f4274939353522ba331f55bcff6e6011ffc56c"
},
"dist": {
"type": "zip",
"url": "https://api.github.com/repos/utopia-php/abuse/zipball/20bee84fd14dbe81d50ecabf1ffd81cceca06152",
"reference": "20bee84fd14dbe81d50ecabf1ffd81cceca06152",
"url": "https://api.github.com/repos/utopia-php/abuse/zipball/53f4274939353522ba331f55bcff6e6011ffc56c",
"reference": "53f4274939353522ba331f55bcff6e6011ffc56c",
"shasum": ""
},
"require": {
@@ -3397,9 +3397,9 @@
],
"support": {
"issues": "https://github.com/utopia-php/abuse/issues",
"source": "https://github.com/utopia-php/abuse/tree/1.2.2"
"source": "https://github.com/utopia-php/abuse/tree/1.2.3"
},
"time": "2026-02-02T10:43:10+00:00"
"time": "2026-04-29T11:19:08+00:00"
},
{
"name": "utopia-php/agents",
@@ -4062,16 +4062,16 @@
},
{
"name": "utopia-php/domains",
"version": "1.0.5",
"version": "1.0.6",
"source": {
"type": "git",
"url": "https://github.com/utopia-php/domains.git",
"reference": "0edf6bb2b07f30db849a267027077bf5abb994c6"
"reference": "c87ba0a1da4cbf75d2cff9d3ea0262b78f1d86f6"
},
"dist": {
"type": "zip",
"url": "https://api.github.com/repos/utopia-php/domains/zipball/0edf6bb2b07f30db849a267027077bf5abb994c6",
"reference": "0edf6bb2b07f30db849a267027077bf5abb994c6",
"url": "https://api.github.com/repos/utopia-php/domains/zipball/c87ba0a1da4cbf75d2cff9d3ea0262b78f1d86f6",
"reference": "c87ba0a1da4cbf75d2cff9d3ea0262b78f1d86f6",
"shasum": ""
},
"require": {
@@ -4118,9 +4118,9 @@
],
"support": {
"issues": "https://github.com/utopia-php/domains/issues",
"source": "https://github.com/utopia-php/domains/tree/1.0.5"
"source": "https://github.com/utopia-php/domains/tree/1.0.6"
},
"time": "2026-03-03T09:20:50+00:00"
"time": "2026-04-29T11:08:10+00:00"
},
{
"name": "utopia-php/dsn",
@@ -4530,16 +4530,16 @@
},
{
"name": "utopia-php/migration",
"version": "1.9.3",
"version": "1.9.5",
"source": {
"type": "git",
"url": "https://github.com/utopia-php/migration.git",
"reference": "111f6221d04578a6f721c23ac872002375f176ae"
"reference": "952a4dfe232702f80e45c35129466a8d8cb4c599"
},
"dist": {
"type": "zip",
"url": "https://api.github.com/repos/utopia-php/migration/zipball/111f6221d04578a6f721c23ac872002375f176ae",
"reference": "111f6221d04578a6f721c23ac872002375f176ae",
"url": "https://api.github.com/repos/utopia-php/migration/zipball/952a4dfe232702f80e45c35129466a8d8cb4c599",
"reference": "952a4dfe232702f80e45c35129466a8d8cb4c599",
"shasum": ""
},
"require": {
@@ -4550,7 +4550,7 @@
"php": ">=8.1",
"utopia-php/database": "5.*",
"utopia-php/dsn": "0.2.*",
"utopia-php/storage": "1.0.*"
"utopia-php/storage": "2.*"
},
"require-dev": {
"ext-pdo": "*",
@@ -4579,9 +4579,9 @@
],
"support": {
"issues": "https://github.com/utopia-php/migration/issues",
"source": "https://github.com/utopia-php/migration/tree/1.9.3"
"source": "https://github.com/utopia-php/migration/tree/1.9.5"
},
"time": "2026-04-22T07:13:26+00:00"
"time": "2026-04-29T11:19:13+00:00"
},
{
"name": "utopia-php/mongo",
@@ -5020,16 +5020,16 @@
},
{
"name": "utopia-php/storage",
"version": "1.0.1",
"version": "2.0.1",
"source": {
"type": "git",
"url": "https://github.com/utopia-php/storage.git",
"reference": "f014be445f0baa635d0764e1673196f412511618"
"reference": "8a2e3a86fd01aaed675884146665308c2122264e"
},
"dist": {
"type": "zip",
"url": "https://api.github.com/repos/utopia-php/storage/zipball/f014be445f0baa635d0764e1673196f412511618",
"reference": "f014be445f0baa635d0764e1673196f412511618",
"url": "https://api.github.com/repos/utopia-php/storage/zipball/8a2e3a86fd01aaed675884146665308c2122264e",
"reference": "8a2e3a86fd01aaed675884146665308c2122264e",
"shasum": ""
},
"require": {
@@ -5043,9 +5043,8 @@
"utopia-php/validators": "0.2.*"
},
"require-dev": {
"laravel/pint": "1.2.*",
"phpunit/phpunit": "^9.3",
"vimeo/psalm": "4.0.1"
"laravel/pint": "^1.21",
"phpunit/phpunit": "^9.3"
},
"type": "library",
"autoload": {
@@ -5067,9 +5066,9 @@
],
"support": {
"issues": "https://github.com/utopia-php/storage/issues",
"source": "https://github.com/utopia-php/storage/tree/1.0.1"
"source": "https://github.com/utopia-php/storage/tree/2.0.1"
},
"time": "2026-02-23T05:59:32+00:00"
"time": "2026-04-29T09:05:48+00:00"
},
{
"name": "utopia-php/system",
@@ -5466,16 +5465,16 @@
"packages-dev": [
{
"name": "appwrite/sdk-generator",
"version": "1.24.0",
"version": "1.25.1",
"source": {
"type": "git",
"url": "https://github.com/appwrite/sdk-generator.git",
"reference": "6d4d26659bc7a1c347c1d4d8dae3b77b5562e0cb"
"reference": "f21a556b9acdbf75bbdcdc90a078af641646eade"
},
"dist": {
"type": "zip",
"url": "https://api.github.com/repos/appwrite/sdk-generator/zipball/6d4d26659bc7a1c347c1d4d8dae3b77b5562e0cb",
"reference": "6d4d26659bc7a1c347c1d4d8dae3b77b5562e0cb",
"url": "https://api.github.com/repos/appwrite/sdk-generator/zipball/f21a556b9acdbf75bbdcdc90a078af641646eade",
"reference": "f21a556b9acdbf75bbdcdc90a078af641646eade",
"shasum": ""
},
"require": {
@@ -5511,9 +5510,9 @@
"description": "Appwrite PHP library for generating API SDKs for multiple programming languages and platforms",
"support": {
"issues": "https://github.com/appwrite/sdk-generator/issues",
"source": "https://github.com/appwrite/sdk-generator/tree/1.24.0"
"source": "https://github.com/appwrite/sdk-generator/tree/1.25.1"
},
"time": "2026-04-24T12:50:05+00:00"
"time": "2026-04-28T11:12:22+00:00"
},
{
"name": "brianium/paratest",
@@ -6222,11 +6221,11 @@
},
{
"name": "phpstan/phpstan",
"version": "2.1.51",
"version": "2.1.54",
"dist": {
"type": "zip",
"url": "https://api.github.com/repos/phpstan/phpstan/zipball/dc3b523c45e714c70de2ac5113b958223b55dc59",
"reference": "dc3b523c45e714c70de2ac5113b958223b55dc59",
"url": "https://api.github.com/repos/phpstan/phpstan/zipball/8be50c3992107dc837b17da4d140fbbdf9a5c5bd",
"reference": "8be50c3992107dc837b17da4d140fbbdf9a5c5bd",
"shasum": ""
},
"require": {
@@ -6271,7 +6270,7 @@
"type": "github"
}
],
"time": "2026-04-21T18:22:01+00:00"
"time": "2026-04-29T13:31:09+00:00"
},
{
"name": "phpunit/php-code-coverage",
@@ -8445,7 +8444,7 @@
],
"aliases": [],
"minimum-stability": "dev",
"stability-flags": {},
"stability-flags": [],
"prefer-stable": true,
"prefer-lowest": false,
"platform": {
@@ -8466,5 +8465,5 @@
"platform-dev": {
"ext-fileinfo": "*"
},
"plugin-api-version": "2.9.0"
"plugin-api-version": "2.6.0"
}
-2
View File
@@ -206,8 +206,6 @@ abstract class OAuth2
$code = curl_getinfo($ch, CURLINFO_HTTP_CODE);
\curl_close($ch);
if ($code >= 400) {
throw new Exception($response, $code);
}
+7
View File
@@ -37,6 +37,13 @@ class Authentik extends OAuth2
return 'authentik';
}
public function verifyCredentials(): void
{
if (empty($this->getAuthentikDomain())) {
throw new \Exception('Authentik endpoint is required.');
}
}
/**
* @return string
*/
+7
View File
@@ -37,6 +37,13 @@ class FusionAuth extends OAuth2
return 'fusionauth';
}
public function verifyCredentials(): void
{
if (empty($this->getFusionAuthDomain())) {
throw new \Exception('FusionAuth endpoint is required.');
}
}
/**
* @return string
*/
+11
View File
@@ -37,6 +37,17 @@ class Keycloak extends OAuth2
return 'keycloak';
}
public function verifyCredentials(): void
{
if (empty($this->getKeycloakDomain())) {
throw new \Exception('Keycloak endpoint is required.');
}
if (empty($this->getKeycloakRealm())) {
throw new \Exception('Keycloak realm name is required.');
}
}
/**
* @return string
*/
+9 -2
View File
@@ -36,6 +36,13 @@ class Microsoft extends OAuth2
return 'microsoft';
}
public function verifyCredentials(): void
{
if (empty($this->getTenantID())) {
throw new \Exception('Microsoft tenant is required.');
}
}
/**
* @return string
*/
@@ -201,7 +208,7 @@ class Microsoft extends OAuth2
}
/**
* Extracts the Tenant Id from the JSON stored in appSecret. Defaults to 'common' as a fallback
* Extracts the Tenant Id from the JSON stored in appSecret.
*
* @return string
*/
@@ -209,6 +216,6 @@ class Microsoft extends OAuth2
{
$secret = $this->getAppSecret();
return $secret['tenantID'] ?? 'common';
return $secret['tenantID'] ?? '';
}
}
+2
View File
@@ -9,6 +9,7 @@ use Appwrite\Platform\Modules\Core;
use Appwrite\Platform\Modules\Databases;
use Appwrite\Platform\Modules\Functions;
use Appwrite\Platform\Modules\Health;
use Appwrite\Platform\Modules\Migrations;
use Appwrite\Platform\Modules\Presences;
use Appwrite\Platform\Modules\Project;
use Appwrite\Platform\Modules\Projects;
@@ -41,6 +42,7 @@ class Appwrite extends Platform
$this->addModule(new Storage\Module());
$this->addModule(new VCS\Module());
$this->addModule(new Webhooks\Module());
$this->addModule(new Migrations\Module());
$this->addModule(new Project\Module());
}
}
@@ -62,7 +62,6 @@ class Get extends Action
curl_setopt_array($ch, $options);
curl_exec($ch);
$errno = curl_errno($ch);
curl_close($ch);
return $errno === 0;
}
@@ -85,8 +85,6 @@ class Create extends Action
curl_exec($ch);
curl_close($ch);
$response->chunk('', true);
}
}
@@ -54,9 +54,9 @@ class XList extends Action
$actions = OAuth2Base::getProviderActions();
$providers = [];
foreach ($actions as $providerId => $updateClass) {
$config = $providersConfig[$providerId] ?? null;
if ($config === null) {
foreach ($providersConfig as $providerId => $config) {
$updateClass = $actions[$providerId] ?? null;
if ($updateClass === null) {
continue;
}
if (!($config['enabled'] ?? false)) {
@@ -18,21 +18,21 @@ class XList extends Action
public static function getName(): string
{
return 'listKeyScopes';
return 'listConsoleProjectScopes';
}
public function __construct()
{
$this
->setHttpMethod(Action::HTTP_REQUEST_METHOD_GET)
->setHttpPath('/v1/console/scopes/key')
->desc('List key scopes')
->setHttpPath('/v1/console/scopes/project')
->desc('List project scopes')
->groups(['api'])
->label('scope', 'public')
->label('sdk', new Method(
namespace: 'console',
group: 'console',
name: 'listKeyScopes',
name: 'listProjectScopes',
description: 'List all scopes available for project API keys, along with a description for each scope.',
auth: [AuthType::ADMIN],
responses: [
@@ -56,6 +56,8 @@ class XList extends Action
$scopes[] = new Document([
'$id' => $scopeId,
'description' => $scope['description'] ?? '',
'category' => $scope['category'] ?? '',
'deprecated' => $scope['deprecated'] ?? false,
]);
}
@@ -34,7 +34,7 @@ class Create extends BooleanCreate
->desc('Create boolean column')
->groups(['api', 'database', 'schema'])
->label('event', 'databases.[databaseId].tables.[tableId].columns.[columnId].create')
->label('scope', ['tables.write', 'collections.write'])
->label('scope', ['tables.write', 'collections.write', 'columns.write', 'attributes.write'])
->label('resourceType', RESOURCE_TYPE_DATABASES)
->label('audits.event', 'column.create')
->label('audits.resource', 'database/{request.databaseId}/table/{request.tableId}')
@@ -34,7 +34,7 @@ class Update extends BooleanUpdate
->setHttpPath('/v1/tablesdb/:databaseId/tables/:tableId/columns/boolean/:key')
->desc('Update boolean column')
->groups(['api', 'database', 'schema'])
->label('scope', ['tables.write', 'collections.write'])
->label('scope', ['tables.write', 'collections.write', 'columns.write', 'attributes.write'])
->label('resourceType', RESOURCE_TYPE_DATABASES)
->label('event', 'databases.[databaseId].tables.[tableId].columns.[columnId].update')
->label('audits.event', 'column.update')
@@ -34,7 +34,7 @@ class Create extends DatetimeCreate
->setHttpPath('/v1/tablesdb/:databaseId/tables/:tableId/columns/datetime')
->desc('Create datetime column')
->groups(['api', 'database'])
->label('scope', ['tables.write', 'collections.write'])
->label('scope', ['tables.write', 'collections.write', 'columns.write', 'attributes.write'])
->label('resourceType', RESOURCE_TYPE_DATABASES)
->label('event', 'databases.[databaseId].tables.[tableId].columns.[columnId].create')
->label('audits.event', 'column.create')
@@ -35,7 +35,7 @@ class Update extends DatetimeUpdate
->setHttpPath('/v1/tablesdb/:databaseId/tables/:tableId/columns/datetime/:key')
->desc('Update dateTime column')
->groups(['api', 'database', 'schema'])
->label('scope', ['tables.write', 'collections.write'])
->label('scope', ['tables.write', 'collections.write', 'columns.write', 'attributes.write'])
->label('resourceType', RESOURCE_TYPE_DATABASES)
->label('event', 'databases.[databaseId].tables.[tableId].columns.[columnId].update')
->label('audits.event', 'column.update')
@@ -33,7 +33,7 @@ class Delete extends AttributesDelete
->setHttpPath('/v1/tablesdb/:databaseId/tables/:tableId/columns/:key')
->desc('Delete column')
->groups(['api', 'database', 'schema'])
->label('scope', ['tables.write', 'collections.write'])
->label('scope', ['tables.write', 'collections.write', 'columns.write', 'attributes.write'])
->label('resourceType', RESOURCE_TYPE_DATABASES)
->label('event', 'databases.[databaseId].tables.[tableId].columns.[columnId].update')
->label('audits.event', 'column.delete')
@@ -34,7 +34,7 @@ class Create extends EmailCreate
->setHttpPath('/v1/tablesdb/:databaseId/tables/:tableId/columns/email')
->desc('Create email column')
->groups(['api', 'database', 'schema'])
->label('scope', ['tables.write', 'collections.write'])
->label('scope', ['tables.write', 'collections.write', 'columns.write', 'attributes.write'])
->label('resourceType', RESOURCE_TYPE_DATABASES)
->label('event', 'databases.[databaseId].tables.[tableId].columns.[columnId].create')
->label('audits.event', 'column.create')
@@ -35,7 +35,7 @@ class Update extends EmailUpdate
->setHttpPath('/v1/tablesdb/:databaseId/tables/:tableId/columns/email/:key')
->desc('Update email column')
->groups(['api', 'database', 'schema'])
->label('scope', ['tables.write', 'collections.write'])
->label('scope', ['tables.write', 'collections.write', 'columns.write', 'attributes.write'])
->label('resourceType', RESOURCE_TYPE_DATABASES)
->label('event', 'databases.[databaseId].tables.[tableId].columns.[columnId].update')
->label('audits.event', 'column.update')
@@ -35,7 +35,7 @@ class Create extends EnumCreate
->setHttpPath('/v1/tablesdb/:databaseId/tables/:tableId/columns/enum')
->desc('Create enum column')
->groups(['api', 'database', 'schema'])
->label('scope', ['tables.write', 'collections.write'])
->label('scope', ['tables.write', 'collections.write', 'columns.write', 'attributes.write'])
->label('resourceType', RESOURCE_TYPE_DATABASES)
->label('event', 'databases.[databaseId].tables.[tableId].columns.[columnId].create')
->label('audits.event', 'column.create')
@@ -36,7 +36,7 @@ class Update extends EnumUpdate
->setHttpPath('/v1/tablesdb/:databaseId/tables/:tableId/columns/enum/:key')
->desc('Update enum column')
->groups(['api', 'database', 'schema'])
->label('scope', ['tables.write', 'collections.write'])
->label('scope', ['tables.write', 'collections.write', 'columns.write', 'attributes.write'])
->label('resourceType', RESOURCE_TYPE_DATABASES)
->label('event', 'databases.[databaseId].tables.[tableId].columns.[columnId].update')
->label('audits.event', 'column.update')
@@ -34,7 +34,7 @@ class Create extends FloatCreate
->setHttpPath('/v1/tablesdb/:databaseId/tables/:tableId/columns/float')
->desc('Create float column')
->groups(['api', 'database', 'schema'])
->label('scope', ['tables.write', 'collections.write'])
->label('scope', ['tables.write', 'collections.write', 'columns.write', 'attributes.write'])
->label('resourceType', RESOURCE_TYPE_DATABASES)
->label('event', 'databases.[databaseId].tables.[tableId].columns.[columnId].create')
->label('audits.event', 'column.create')
@@ -35,7 +35,7 @@ class Update extends FloatUpdate
->setHttpPath('/v1/tablesdb/:databaseId/tables/:tableId/columns/float/:key')
->desc('Update float column')
->groups(['api', 'database', 'schema'])
->label('scope', ['tables.write', 'collections.write'])
->label('scope', ['tables.write', 'collections.write', 'columns.write', 'attributes.write'])
->label('resourceType', RESOURCE_TYPE_DATABASES)
->label('event', 'databases.[databaseId].tables.[tableId].columns.[columnId].update')
->label('audits.event', 'column.update')
@@ -42,7 +42,7 @@ class Get extends AttributesGet
->setHttpPath('/v1/tablesdb/:databaseId/tables/:tableId/columns/:key')
->desc('Get column')
->groups(['api', 'database'])
->label('scope', ['tables.read', 'collections.read'])
->label('scope', ['tables.read', 'collections.read', 'columns.read', 'attributes.read'])
->label('resourceType', RESOURCE_TYPE_DATABASES)
->label('sdk', new Method(
namespace: $this->getSDKNamespace(),
@@ -34,7 +34,7 @@ class Create extends IPCreate
->setHttpPath('/v1/tablesdb/:databaseId/tables/:tableId/columns/ip')
->desc('Create IP address column')
->groups(['api', 'database', 'schema'])
->label('scope', ['tables.write', 'collections.write'])
->label('scope', ['tables.write', 'collections.write', 'columns.write', 'attributes.write'])
->label('resourceType', RESOURCE_TYPE_DATABASES)
->label('event', 'databases.[databaseId].tables.[tableId].columns.[columnId].create')
->label('audits.event', 'column.create')
@@ -35,7 +35,7 @@ class Update extends IPUpdate
->setHttpPath('/v1/tablesdb/:databaseId/tables/:tableId/columns/ip/:key')
->desc('Update IP address column')
->groups(['api', 'database', 'schema'])
->label('scope', ['tables.write', 'collections.write'])
->label('scope', ['tables.write', 'collections.write', 'columns.write', 'attributes.write'])
->label('resourceType', RESOURCE_TYPE_DATABASES)
->label('event', 'databases.[databaseId].tables.[tableId].columns.[columnId].update')
->label('audits.event', 'column.update')
@@ -34,7 +34,7 @@ class Create extends IntegerCreate
->setHttpPath('/v1/tablesdb/:databaseId/tables/:tableId/columns/integer')
->desc('Create integer column')
->groups(['api', 'database', 'schema'])
->label('scope', ['tables.write', 'collections.write'])
->label('scope', ['tables.write', 'collections.write', 'columns.write', 'attributes.write'])
->label('resourceType', RESOURCE_TYPE_DATABASES)
->label('event', 'databases.[databaseId].tables.[tableId].columns.[columnId].create')
->label('audits.event', 'column.create')
@@ -35,7 +35,7 @@ class Update extends IntegerUpdate
->setHttpPath('/v1/tablesdb/:databaseId/tables/:tableId/columns/integer/:key')
->desc('Update integer column')
->groups(['api', 'database', 'schema'])
->label('scope', ['tables.write', 'collections.write'])
->label('scope', ['tables.write', 'collections.write', 'columns.write', 'attributes.write'])
->label('resourceType', RESOURCE_TYPE_DATABASES)
->label('event', 'databases.[databaseId].tables.[tableId].columns.[columnId].update')
->label('audits.event', 'column.update')
@@ -35,7 +35,7 @@ class Create extends LineCreate
->desc('Create line column')
->groups(['api', 'database', 'schema'])
->label('event', 'databases.[databaseId].tables.[tableId].columns.[columnId].create')
->label('scope', ['tables.write', 'collections.write'])
->label('scope', ['tables.write', 'collections.write', 'columns.write', 'attributes.write'])
->label('resourceType', RESOURCE_TYPE_DATABASES)
->label('audits.event', 'column.create')
->label('audits.resource', 'database/{request.databaseId}/table/{request.tableId}')
@@ -35,7 +35,7 @@ class Update extends LineUpdate
->setHttpPath('/v1/tablesdb/:databaseId/tables/:tableId/columns/line/:key')
->desc('Update line column')
->groups(['api', 'database', 'schema'])
->label('scope', ['tables.write', 'collections.write'])
->label('scope', ['tables.write', 'collections.write', 'columns.write', 'attributes.write'])
->label('resourceType', RESOURCE_TYPE_DATABASES)
->label('event', 'databases.[databaseId].tables.[tableId].columns.[columnId].update')
->label('audits.event', 'column.update')
@@ -33,7 +33,7 @@ class Create extends LongtextCreate
->setHttpPath('/v1/tablesdb/:databaseId/tables/:tableId/columns/longtext')
->desc('Create longtext column')
->groups(['api', 'database', 'schema'])
->label('scope', ['tables.write', 'collections.write'])
->label('scope', ['tables.write', 'collections.write', 'columns.write', 'attributes.write'])
->label('resourceType', RESOURCE_TYPE_DATABASES)
->label('event', 'databases.[databaseId].tables.[tableId].columns.[columnId].create')
->label('audits.event', 'column.create')
@@ -34,7 +34,7 @@ class Update extends LongtextUpdate
->setHttpPath('/v1/tablesdb/:databaseId/tables/:tableId/columns/longtext/:key')
->desc('Update longtext column')
->groups(['api', 'database', 'schema'])
->label('scope', ['tables.write', 'collections.write'])
->label('scope', ['tables.write', 'collections.write', 'columns.write', 'attributes.write'])
->label('resourceType', RESOURCE_TYPE_DATABASES)
->label('event', 'databases.[databaseId].tables.[tableId].columns.[columnId].update')
->label('audits.event', 'column.update')
@@ -33,7 +33,7 @@ class Create extends MediumtextCreate
->setHttpPath('/v1/tablesdb/:databaseId/tables/:tableId/columns/mediumtext')
->desc('Create mediumtext column')
->groups(['api', 'database', 'schema'])
->label('scope', ['tables.write', 'collections.write'])
->label('scope', ['tables.write', 'collections.write', 'columns.write', 'attributes.write'])
->label('resourceType', RESOURCE_TYPE_DATABASES)
->label('event', 'databases.[databaseId].tables.[tableId].columns.[columnId].create')
->label('audits.event', 'column.create')
@@ -34,7 +34,7 @@ class Update extends MediumtextUpdate
->setHttpPath('/v1/tablesdb/:databaseId/tables/:tableId/columns/mediumtext/:key')
->desc('Update mediumtext column')
->groups(['api', 'database', 'schema'])
->label('scope', ['tables.write', 'collections.write'])
->label('scope', ['tables.write', 'collections.write', 'columns.write', 'attributes.write'])
->label('resourceType', RESOURCE_TYPE_DATABASES)
->label('event', 'databases.[databaseId].tables.[tableId].columns.[columnId].update')
->label('audits.event', 'column.update')
@@ -35,7 +35,7 @@ class Create extends PointCreate
->desc('Create point column')
->groups(['api', 'database', 'schema'])
->label('event', 'databases.[databaseId].tables.[tableId].columns.[columnId].create')
->label('scope', ['tables.write', 'collections.write'])
->label('scope', ['tables.write', 'collections.write', 'columns.write', 'attributes.write'])
->label('resourceType', RESOURCE_TYPE_DATABASES)
->label('audits.event', 'column.create')
->label('audits.resource', 'database/{request.databaseId}/table/{request.tableId}')
@@ -35,7 +35,7 @@ class Update extends PointUpdate
->setHttpPath('/v1/tablesdb/:databaseId/tables/:tableId/columns/point/:key')
->desc('Update point column')
->groups(['api', 'database', 'schema'])
->label('scope', ['tables.write', 'collections.write'])
->label('scope', ['tables.write', 'collections.write', 'columns.write', 'attributes.write'])
->label('resourceType', RESOURCE_TYPE_DATABASES)
->label('event', 'databases.[databaseId].tables.[tableId].columns.[columnId].update')
->label('audits.event', 'column.update')
@@ -35,7 +35,7 @@ class Create extends PolygonCreate
->desc('Create polygon column')
->groups(['api', 'database', 'schema'])
->label('event', 'databases.[databaseId].tables.[tableId].columns.[columnId].create')
->label('scope', ['tables.write', 'collections.write'])
->label('scope', ['tables.write', 'collections.write', 'columns.write', 'attributes.write'])
->label('resourceType', RESOURCE_TYPE_DATABASES)
->label('audits.event', 'column.create')
->label('audits.resource', 'database/{request.databaseId}/table/{request.tableId}')
@@ -35,7 +35,7 @@ class Update extends PolygonUpdate
->setHttpPath('/v1/tablesdb/:databaseId/tables/:tableId/columns/polygon/:key')
->desc('Update polygon column')
->groups(['api', 'database', 'schema'])
->label('scope', ['tables.write', 'collections.write'])
->label('scope', ['tables.write', 'collections.write', 'columns.write', 'attributes.write'])
->label('resourceType', RESOURCE_TYPE_DATABASES)
->label('event', 'databases.[databaseId].tables.[tableId].columns.[columnId].update')
->label('audits.event', 'column.update')
@@ -34,7 +34,7 @@ class Create extends RelationshipCreate
->setHttpPath('/v1/tablesdb/:databaseId/tables/:tableId/columns/relationship')
->desc('Create relationship column')
->groups(['api', 'database'])
->label('scope', ['tables.write', 'collections.write'])
->label('scope', ['tables.write', 'collections.write', 'columns.write', 'attributes.write'])
->label('resourceType', RESOURCE_TYPE_DATABASES)
->label('event', 'databases.[databaseId].tables.[tableId].columns.[columnId].create')
->label('audits.event', 'column.create')
@@ -34,7 +34,7 @@ class Update extends RelationshipUpdate
->setHttpPath('/v1/tablesdb/:databaseId/tables/:tableId/columns/:key/relationship')
->desc('Update relationship column')
->groups(['api', 'database', 'schema'])
->label('scope', ['tables.write', 'collections.write'])
->label('scope', ['tables.write', 'collections.write', 'columns.write', 'attributes.write'])
->label('resourceType', RESOURCE_TYPE_DATABASES)
->label('event', 'databases.[databaseId].tables.[tableId].columns.[columnId].update')
->label('audits.event', 'column.update')
@@ -37,7 +37,7 @@ class Create extends StringCreate
->setHttpPath('/v1/tablesdb/:databaseId/tables/:tableId/columns/string')
->desc('Create string column')
->groups(['api', 'database', 'schema'])
->label('scope', ['tables.write', 'collections.write'])
->label('scope', ['tables.write', 'collections.write', 'columns.write', 'attributes.write'])
->label('resourceType', RESOURCE_TYPE_DATABASES)
->label('event', 'databases.[databaseId].tables.[tableId].columns.[columnId].create')
->label('audits.event', 'column.create')
@@ -37,7 +37,7 @@ class Update extends StringUpdate
->setHttpPath('/v1/tablesdb/:databaseId/tables/:tableId/columns/string/:key')
->desc('Update string column')
->groups(['api', 'database', 'schema'])
->label('scope', ['tables.write', 'collections.write'])
->label('scope', ['tables.write', 'collections.write', 'columns.write', 'attributes.write'])
->label('resourceType', RESOURCE_TYPE_DATABASES)
->label('event', 'databases.[databaseId].tables.[tableId].columns.[columnId].update')
->label('audits.event', 'column.update')
@@ -33,7 +33,7 @@ class Create extends TextCreate
->setHttpPath('/v1/tablesdb/:databaseId/tables/:tableId/columns/text')
->desc('Create text column')
->groups(['api', 'database', 'schema'])
->label('scope', ['tables.write', 'collections.write'])
->label('scope', ['tables.write', 'collections.write', 'columns.write', 'attributes.write'])
->label('resourceType', RESOURCE_TYPE_DATABASES)
->label('event', 'databases.[databaseId].tables.[tableId].columns.[columnId].create')
->label('audits.event', 'column.create')
@@ -34,7 +34,7 @@ class Update extends TextUpdate
->setHttpPath('/v1/tablesdb/:databaseId/tables/:tableId/columns/text/:key')
->desc('Update text column')
->groups(['api', 'database', 'schema'])
->label('scope', ['tables.write', 'collections.write'])
->label('scope', ['tables.write', 'collections.write', 'columns.write', 'attributes.write'])
->label('resourceType', RESOURCE_TYPE_DATABASES)
->label('event', 'databases.[databaseId].tables.[tableId].columns.[columnId].update')
->label('audits.event', 'column.update')
@@ -34,7 +34,7 @@ class Create extends URLCreate
->setHttpPath('/v1/tablesdb/:databaseId/tables/:tableId/columns/url')
->desc('Create URL column')
->groups(['api', 'database', 'schema'])
->label('scope', ['tables.write', 'collections.write'])
->label('scope', ['tables.write', 'collections.write', 'columns.write', 'attributes.write'])
->label('resourceType', RESOURCE_TYPE_DATABASES)
->label('event', 'databases.[databaseId].tables.[tableId].columns.[columnId].create')
->label('audits.event', 'column.create')
@@ -35,7 +35,7 @@ class Update extends URLUpdate
->setHttpPath('/v1/tablesdb/:databaseId/tables/:tableId/columns/url/:key')
->desc('Update URL column')
->groups(['api', 'database', 'schema'])
->label('scope', ['tables.write', 'collections.write'])
->label('scope', ['tables.write', 'collections.write', 'columns.write', 'attributes.write'])
->label('resourceType', RESOURCE_TYPE_DATABASES)
->label('event', 'databases.[databaseId].tables.[tableId].columns.[columnId].update')
->label('audits.event', 'column.update')
@@ -35,7 +35,7 @@ class Create extends VarcharCreate
->setHttpPath('/v1/tablesdb/:databaseId/tables/:tableId/columns/varchar')
->desc('Create varchar column')
->groups(['api', 'database', 'schema'])
->label('scope', ['tables.write', 'collections.write'])
->label('scope', ['tables.write', 'collections.write', 'columns.write', 'attributes.write'])
->label('resourceType', RESOURCE_TYPE_DATABASES)
->label('event', 'databases.[databaseId].tables.[tableId].columns.[columnId].create')
->label('audits.event', 'column.create')
@@ -36,7 +36,7 @@ class Update extends VarcharUpdate
->setHttpPath('/v1/tablesdb/:databaseId/tables/:tableId/columns/varchar/:key')
->desc('Update varchar column')
->groups(['api', 'database', 'schema'])
->label('scope', ['tables.write', 'collections.write'])
->label('scope', ['tables.write', 'collections.write', 'columns.write', 'attributes.write'])
->label('resourceType', RESOURCE_TYPE_DATABASES)
->label('event', 'databases.[databaseId].tables.[tableId].columns.[columnId].update')
->label('audits.event', 'column.update')
@@ -33,7 +33,7 @@ class XList extends AttributesXList
->setHttpPath('/v1/tablesdb/:databaseId/tables/:tableId/columns')
->desc('List columns')
->groups(['api', 'database'])
->label('scope', ['tables.read', 'collections.read'])
->label('scope', ['tables.read', 'collections.read', 'columns.read', 'attributes.read'])
->label('resourceType', RESOURCE_TYPE_DATABASES)
->label('sdk', new Method(
namespace: $this->getSDKNamespace(),
@@ -37,7 +37,7 @@ class Create extends IndexCreate
->desc('Create index')
->groups(['api', 'database'])
->label('event', 'databases.[databaseId].tables.[tableId].indexes.[indexId].create')
->label('scope', ['tables.write', 'collections.write'])
->label('scope', ['tables.write', 'collections.write', 'indexes.write'])
->label('resourceType', RESOURCE_TYPE_DATABASES)
->label('audits.event', 'index.create')
->label('audits.resource', 'database/{request.databaseId}/table/{request.tableId}')
@@ -36,7 +36,7 @@ class Delete extends IndexDelete
->setHttpPath('/v1/tablesdb/:databaseId/tables/:tableId/indexes/:key')
->desc('Delete index')
->groups(['api', 'database'])
->label('scope', ['tables.write', 'collections.write'])
->label('scope', ['tables.write', 'collections.write', 'indexes.write'])
->label('resourceType', RESOURCE_TYPE_DATABASES)
->label('event', 'databases.[databaseId].tables.[tableId].indexes.[indexId].update')
->label('audits.event', 'index.delete')
@@ -32,7 +32,7 @@ class Get extends IndexGet
->setHttpPath('/v1/tablesdb/:databaseId/tables/:tableId/indexes/:key')
->desc('Get index')
->groups(['api', 'database'])
->label('scope', ['tables.read', 'collections.read'])
->label('scope', ['tables.read', 'collections.read', 'indexes.read'])
->label('resourceType', RESOURCE_TYPE_DATABASES)
->label('sdk', new Method(
namespace: $this->getSDKNamespace(),
@@ -33,7 +33,7 @@ class XList extends IndexXList
->setHttpPath('/v1/tablesdb/:databaseId/tables/:tableId/indexes')
->desc('List indexes')
->groups(['api', 'database'])
->label('scope', ['tables.read', 'collections.read'])
->label('scope', ['tables.read', 'collections.read', 'indexes.read'])
->label('resourceType', RESOURCE_TYPE_DATABASES)
->label('sdk', new Method(
namespace: $this->getSDKNamespace(),
@@ -175,15 +175,8 @@ class Create extends Action
throw new Exception(Exception::STORAGE_INVALID_CONTENT_RANGE);
}
// TODO remove the condition that checks `$end === $fileSize` in next breaking version
if ($end === $fileSize - 1 || $end === $fileSize) {
//if it's a last chunks the chunk size might differ, so we set the $chunks and $chunk to notify it's last chunk
$chunks = $chunk = -1;
} else {
// Calculate total number of chunks based on the chunk size i.e ($rangeEnd - $rangeStart)
$chunks = (int) ceil($fileSize / ($end + 1 - $start));
$chunk = (int) ($start / ($end + 1 - $start)) + 1;
}
$chunks = (int) ceil($fileSize / APP_LIMIT_UPLOAD_CHUNK_SIZE);
$chunk = (int) ($start / APP_LIMIT_UPLOAD_CHUNK_SIZE) + 1;
}
if (!$fileSizeValidator->isValid($fileSize) && $functionSizeLimit !== 0) { // Check if file size is exceeding allowed limit
@@ -202,15 +195,14 @@ class Create extends Action
$metadata = ['content_type' => $deviceForLocal->getFileMimeType($fileTmpName)];
if (!$deployment->isEmpty()) {
$chunks = $deployment->getAttribute('sourceChunksTotal', 1);
$uploaded = $deployment->getAttribute('sourceChunksUploaded', 0);
$metadata = $deployment->getAttribute('sourceMetadata', []);
if ($chunk === -1) {
$chunk = $chunks;
}
} else {
// Guard against manually setting range header for single chunk upload
if ($chunks === -1) {
$chunks = 1;
$chunk = 1;
if ($uploaded === $chunks) {
$response
->setStatusCode(Response::STATUS_CODE_ACCEPTED)
->dynamic($deployment, Response::MODEL_DEPLOYMENT);
return;
}
}
@@ -258,6 +250,8 @@ class Create extends Action
'sourcePath' => $path,
'sourceSize' => $fileSize,
'totalSize' => $fileSize,
'sourceChunksTotal' => $chunks,
'sourceChunksUploaded' => $chunksUploaded,
'activate' => $activate,
'sourceMetadata' => $metadata,
'type' => $type
@@ -272,6 +266,7 @@ class Create extends Action
} else {
$deployment = $dbForProject->updateDocument('deployments', $deploymentId, new Document([
'sourceSize' => $fileSize,
'sourceChunksUploaded' => $chunksUploaded,
'sourceMetadata' => $metadata,
]));
}
@@ -60,7 +60,7 @@ class Create extends Base
->setHttpPath('/v1/functions/:functionId/executions')
->desc('Create execution')
->groups(['api', 'functions'])
->label('scope', 'execution.write')
->label('scope', ['executions.write', 'execution.write'])
->label('resourceType', RESOURCE_TYPE_FUNCTIONS)
->label('event', 'functions.[functionId].executions.[executionId].create')
->label('sdk', new Method(
@@ -35,7 +35,7 @@ class Delete extends Base
->setHttpPath('/v1/functions/:functionId/executions/:executionId')
->desc('Delete execution')
->groups(['api', 'functions'])
->label('scope', 'execution.write')
->label('scope', ['executions.write', 'execution.write'])
->label('resourceType', RESOURCE_TYPE_FUNCTIONS)
->label('event', 'functions.[functionId].executions.[executionId].delete')
->label('audits.event', 'executions.delete')
@@ -31,7 +31,7 @@ class Get extends Base
->setHttpPath('/v1/functions/:functionId/executions/:executionId')
->desc('Get execution')
->groups(['api', 'functions'])
->label('scope', 'execution.read')
->label('scope', ['executions.read', 'execution.read'])
->label('resourceType', RESOURCE_TYPE_FUNCTIONS)
->label('sdk', new Method(
namespace: 'functions',
@@ -39,7 +39,7 @@ class XList extends Base
->setHttpPath('/v1/functions/:functionId/executions')
->desc('List executions')
->groups(['api', 'functions'])
->label('scope', 'execution.read')
->label('scope', ['executions.read', 'execution.read'])
->label('resourceType', RESOURCE_TYPE_FUNCTIONS)
->label('sdk', new Method(
namespace: 'functions',
@@ -0,0 +1,110 @@
<?php
namespace Appwrite\Platform\Modules\Migrations\Http\Migrations\Appwrite;
use Appwrite\Event\Event;
use Appwrite\Event\Message\Migration as MigrationMessage;
use Appwrite\Event\Publisher\Migration as MigrationPublisher;
use Appwrite\SDK\AuthType;
use Appwrite\SDK\Method;
use Appwrite\SDK\Response as SDKResponse;
use Appwrite\Utopia\Response;
use Utopia\Database\Database;
use Utopia\Database\Document;
use Utopia\Database\Helpers\ID;
use Utopia\Database\Validator\UID;
use Utopia\Migration\Sources\Appwrite as AppwriteSource;
use Utopia\Platform\Action;
use Utopia\Platform\Scope\HTTP;
use Utopia\Validator\ArrayList;
use Utopia\Validator\Text;
use Utopia\Validator\URL;
use Utopia\Validator\WhiteList;
class Create extends Action
{
use HTTP;
public static function getName(): string
{
return 'createAppwriteMigration';
}
public function __construct()
{
$this
->setHttpMethod(Action::HTTP_REQUEST_METHOD_POST)
->setHttpPath('/v1/migrations/appwrite')
->desc('Create Appwrite migration')
->groups(['api', 'migrations'])
->label('scope', 'migrations.write')
->label('event', 'migrations.[migrationId].create')
->label('audits.event', 'migration.create')
->label('sdk', new Method(
namespace: 'migrations',
group: null,
name: 'createAppwriteMigration',
description: '/docs/references/migrations/migration-appwrite.md',
auth: [AuthType::ADMIN],
responses: [
new SDKResponse(
code: Response::STATUS_CODE_ACCEPTED,
model: Response::MODEL_MIGRATION,
)
]
))
->param('resources', [], new ArrayList(new WhiteList(AppwriteSource::getSupportedResources())), 'List of resources to migrate')
->param('endpoint', '', new URL(), 'Source Appwrite endpoint')
->param('projectId', '', fn (Database $dbForProject) => new UID($dbForProject->getAdapter()->getMaxUIDLength()), 'Source Project ID', false, ['dbForProject'])
->param('apiKey', '', new Text(512), 'Source API Key')
->inject('response')
->inject('dbForProject')
->inject('project')
->inject('platform')
->inject('queueForEvents')
->inject('publisherForMigrations')
->callback($this->action(...));
}
public function action(
array $resources,
string $endpoint,
string $projectId,
string $apiKey,
Response $response,
Database $dbForProject,
Document $project,
array $platform,
Event $queueForEvents,
MigrationPublisher $publisherForMigrations
): void {
$migration = $dbForProject->createDocument('migrations', new Document([
'$id' => ID::unique(),
'status' => 'pending',
'stage' => 'init',
'source' => AppwriteSource::getName(),
'destination' => AppwriteSource::getName(),
'credentials' => [
'endpoint' => $endpoint,
'projectId' => $projectId,
'apiKey' => $apiKey,
],
'resources' => $resources,
'statusCounters' => '{}',
'resourceData' => '{}',
'errors' => [],
]));
$queueForEvents->setParam('migrationId', $migration->getId());
$publisherForMigrations->enqueue(new MigrationMessage(
project: $project,
migration: $migration,
platform: $platform,
));
$response
->setStatusCode(Response::STATUS_CODE_ACCEPTED)
->dynamic($migration, Response::MODEL_MIGRATION);
}
}
@@ -0,0 +1,80 @@
<?php
namespace Appwrite\Platform\Modules\Migrations\Http\Migrations\Appwrite\Report;
use Appwrite\Extend\Exception;
use Appwrite\SDK\AuthType;
use Appwrite\SDK\Method;
use Appwrite\SDK\Response as SDKResponse;
use Appwrite\Utopia\Response;
use Utopia\Database\Document;
use Utopia\Migration\Sources\Appwrite as AppwriteSource;
use Utopia\Platform\Action;
use Utopia\Platform\Scope\HTTP;
use Utopia\Validator\ArrayList;
use Utopia\Validator\Text;
use Utopia\Validator\URL;
use Utopia\Validator\WhiteList;
class Get extends Action
{
use HTTP;
public static function getName(): string
{
return 'getAppwriteReport';
}
public function __construct()
{
$this
->setHttpMethod(Action::HTTP_REQUEST_METHOD_GET)
->setHttpPath('/v1/migrations/appwrite/report')
->desc('Get Appwrite migration report')
->groups(['api', 'migrations'])
->label('scope', 'migrations.write')
->label('sdk', new Method(
namespace: 'migrations',
group: null,
name: 'getAppwriteReport',
description: '/docs/references/migrations/migration-appwrite-report.md',
auth: [AuthType::ADMIN],
responses: [
new SDKResponse(
code: Response::STATUS_CODE_OK,
model: Response::MODEL_MIGRATION_REPORT,
)
]
))
->param('resources', [], new ArrayList(new WhiteList(AppwriteSource::getSupportedResources())), 'List of resources to migrate')
->param('endpoint', '', new URL(), "Source's Appwrite Endpoint")
->param('projectID', '', new Text(512), "Source's Project ID")
->param('key', '', new Text(512), "Source's API Key")
->inject('response')
->inject('getDatabasesDB')
->callback($this->action(...));
}
public function action(
array $resources,
string $endpoint,
string $projectID,
string $key,
Response $response,
callable $getDatabasesDB
): void {
try {
$appwrite = new AppwriteSource($projectID, $endpoint, $key, $getDatabasesDB);
$report = $appwrite->report($resources);
} catch (\Throwable $e) {
throw new Exception(
Exception::MIGRATION_PROVIDER_ERROR,
'Unable to connect to the migration source. Please verify your credentials and ensure the source is reachable from this server. Check for network restrictions such as firewalls, IP allowlists, or outbound connectivity limits.'
);
}
$response
->setStatusCode(Response::STATUS_CODE_OK)
->dynamic(new Document($report), Response::MODEL_MIGRATION_REPORT);
}
}
@@ -0,0 +1,213 @@
<?php
namespace Appwrite\Platform\Modules\Migrations\Http\Migrations\CSV\Exports;
use Appwrite\Event\Event;
use Appwrite\Event\Message\Migration as MigrationMessage;
use Appwrite\Event\Publisher\Migration as MigrationPublisher;
use Appwrite\Extend\Exception;
use Appwrite\SDK\AuthType;
use Appwrite\SDK\Method;
use Appwrite\SDK\Response as SDKResponse;
use Appwrite\Utopia\Database\Validator\CompoundUID;
use Appwrite\Utopia\Response;
use Utopia\Database\Database;
use Utopia\Database\Document;
use Utopia\Database\Exception\Query as QueryException;
use Utopia\Database\Helpers\ID;
use Utopia\Database\Query;
use Utopia\Database\Validator\Authorization;
use Utopia\Database\Validator\Queries\Documents;
use Utopia\Migration\Resource;
use Utopia\Migration\Sources\Appwrite as AppwriteSource;
use Utopia\Migration\Sources\CSV;
use Utopia\Migration\Transfer;
use Utopia\Platform\Action;
use Utopia\Platform\Scope\HTTP;
use Utopia\Validator\ArrayList;
use Utopia\Validator\Boolean;
use Utopia\Validator\Text;
class Create extends Action
{
use HTTP;
public static function getName(): string
{
return 'createCSVExport';
}
public function __construct()
{
$this
->setHttpMethod(Action::HTTP_REQUEST_METHOD_POST)
->setHttpPath('/v1/migrations/csv/exports')
->desc('Export documents to CSV')
->groups(['api', 'migrations'])
->label('scope', 'migrations.write')
->label('event', 'migrations.[migrationId].create')
->label('audits.event', 'migration.create')
->label('sdk', new Method(
namespace: 'migrations',
group: null,
name: 'createCSVExport',
description: '/docs/references/migrations/migration-csv-export.md',
auth: [AuthType::ADMIN],
responses: [
new SDKResponse(
code: Response::STATUS_CODE_ACCEPTED,
model: Response::MODEL_MIGRATION,
)
]
))
->param('resourceId', null, new CompoundUID(), 'Composite ID in the format {databaseId:collectionId}, identifying a collection within a database to export.')
->param('filename', '', new Text(255), 'The name of the file to be created for the export, excluding the .csv extension.')
->param('columns', [], new ArrayList(new Text(Database::LENGTH_KEY)), 'List of attributes to export. If empty, all attributes will be exported. You can use the `*` wildcard to export all attributes from the collection.', true)
->param('queries', [], new ArrayList(new Text(0)), 'Array of query strings generated using the Query class provided by the SDK to filter documents to export. [Learn more about queries](https://appwrite.io/docs/databases#querying-documents). Maximum of ' . APP_LIMIT_ARRAY_PARAMS_SIZE . ' queries are allowed, each ' . APP_LIMIT_ARRAY_ELEMENT_SIZE . ' characters long.', true)
->param('delimiter', ',', new Text(1), 'The character that separates each column value. Default is comma.', true)
->param('enclosure', '"', new Text(1), 'The character that encloses each column value. Default is double quotes.', true)
->param('escape', '"', new Text(1), 'The escape character for the enclosure character. Default is double quotes.', true)
->param('header', true, new Boolean(), 'Whether to include the header row with column names. Default is true.', true)
->param('notify', true, new Boolean(), 'Set to true to receive an email when the export is complete. Default is true.', true)
->inject('user')
->inject('response')
->inject('dbForProject')
->inject('dbForPlatform')
->inject('authorization')
->inject('project')
->inject('platform')
->inject('queueForEvents')
->inject('publisherForMigrations')
->callback($this->action(...));
}
public function action(
string $resourceId,
string $filename,
array $columns,
array $queries,
string $delimiter,
string $enclosure,
string $escape,
bool $header,
bool $notify,
Document $user,
Response $response,
Database $dbForProject,
Database $dbForPlatform,
Authorization $authorization,
Document $project,
array $platform,
Event $queueForEvents,
MigrationPublisher $publisherForMigrations
): void {
try {
$parsedQueries = Query::parseQueries($queries);
} catch (QueryException $e) {
throw new Exception(Exception::GENERAL_QUERY_INVALID, $e->getMessage());
}
$bucket = $authorization->skip(fn () => $dbForPlatform->getDocument('buckets', 'default'));
if ($bucket->isEmpty()) {
throw new Exception(Exception::STORAGE_BUCKET_NOT_FOUND);
}
[$databaseId, $collectionId] = \explode(':', $resourceId, 2);
if (empty($databaseId)) {
throw new Exception(Exception::DATABASE_NOT_FOUND);
}
if (empty($collectionId)) {
throw new Exception(Exception::COLLECTION_NOT_FOUND);
}
$database = $authorization->skip(fn () => $dbForProject->getDocument('databases', $databaseId));
if ($database->isEmpty()) {
throw new Exception(Exception::DATABASE_NOT_FOUND);
}
$collection = $authorization->skip(fn () => $dbForProject->getDocument('database_' . $database->getSequence(), $collectionId));
if ($collection->isEmpty()) {
throw new Exception(Exception::COLLECTION_NOT_FOUND);
}
$databaseType = $database->getAttribute('type');
if (!\in_array($databaseType, CSV_ALLOWED_DATABASE_TYPES)) {
throw new Exception(Exception::MIGRATION_DATABASE_TYPE_UNSUPPORTED, 'Database type not supported for csv');
}
// Schemaless databases (DocumentsDB, VectorsDB) allow queries on dynamic fields
$isSchemaless = \in_array($databaseType, [DATABASE_TYPE_DOCUMENTSDB, DATABASE_TYPE_VECTORSDB]);
$validator = new Documents(
attributes: $collection->getAttribute('attributes', []),
indexes: $collection->getAttribute('indexes', []),
idAttributeType: $dbForProject->getAdapter()->getIdAttributeType(),
supportForAttributes: !$isSchemaless,
);
if (!$validator->isValid($parsedQueries)) {
throw new Exception(Exception::GENERAL_QUERY_INVALID, $validator->getDescription());
}
$resources = Transfer::extractServices([self::transferGroupForDatabaseType($databaseType)]);
$resourceType = self::resourceTypeForDatabaseType($databaseType);
$migration = $dbForProject->createDocument('migrations', new Document([
'$id' => ID::unique(),
'status' => 'pending',
'stage' => 'init',
'source' => AppwriteSource::getName(),
'destination' => CSV::getName(),
'resources' => $resources,
'resourceId' => $resourceId,
'resourceType' => $resourceType,
'statusCounters' => '{}',
'resourceData' => '{}',
'errors' => [],
'options' => [
'bucketId' => 'default', // Always use internal bucket
'filename' => $filename,
'columns' => $columns,
'queries' => $queries,
'delimiter' => $delimiter,
'enclosure' => $enclosure,
'escape' => $escape,
'header' => $header,
'notify' => $notify,
'userInternalId' => $user->getSequence(),
],
]));
$queueForEvents->setParam('migrationId', $migration->getId());
$publisherForMigrations->enqueue(new MigrationMessage(
project: $project,
migration: $migration,
platform: $platform,
));
$response
->setStatusCode(Response::STATUS_CODE_ACCEPTED)
->dynamic($migration, Response::MODEL_MIGRATION);
}
private static function transferGroupForDatabaseType(string $databaseType): string
{
return match ($databaseType) {
DATABASE_TYPE_LEGACY,
DATABASE_TYPE_TABLESDB => Transfer::GROUP_DATABASES_TABLES_DB,
DATABASE_TYPE_VECTORSDB => Transfer::GROUP_DATABASES_VECTOR_DB,
DATABASE_TYPE_DOCUMENTSDB => Transfer::GROUP_DATABASES_DOCUMENTS_DB,
default => throw new \LogicException('Unknown database type: ' . $databaseType),
};
}
private static function resourceTypeForDatabaseType(string $databaseType): string
{
return match ($databaseType) {
DATABASE_TYPE_VECTORSDB => Resource::TYPE_DATABASE_VECTORSDB,
DATABASE_TYPE_DOCUMENTSDB => Resource::TYPE_DATABASE_DOCUMENTSDB,
default => Resource::TYPE_DATABASE,
};
}
}
@@ -0,0 +1,220 @@
<?php
namespace Appwrite\Platform\Modules\Migrations\Http\Migrations\CSV\Imports;
use Appwrite\Event\Event;
use Appwrite\Event\Message\Migration as MigrationMessage;
use Appwrite\Event\Publisher\Migration as MigrationPublisher;
use Appwrite\Extend\Exception;
use Appwrite\OpenSSL\OpenSSL;
use Appwrite\SDK\AuthType;
use Appwrite\SDK\Method;
use Appwrite\SDK\Response as SDKResponse;
use Appwrite\Utopia\Database\Validator\CompoundUID;
use Appwrite\Utopia\Response;
use Utopia\Compression\Algorithms\GZIP;
use Utopia\Compression\Algorithms\Zstd;
use Utopia\Compression\Compression;
use Utopia\Database\Database;
use Utopia\Database\Document;
use Utopia\Database\Helpers\ID;
use Utopia\Database\Validator\Authorization;
use Utopia\Database\Validator\UID;
use Utopia\Migration\Resource;
use Utopia\Migration\Sources\Appwrite as AppwriteSource;
use Utopia\Migration\Sources\CSV;
use Utopia\Migration\Transfer;
use Utopia\Platform\Action;
use Utopia\Platform\Scope\HTTP;
use Utopia\Storage\Device;
use Utopia\System\System;
use Utopia\Validator\Boolean;
class Create extends Action
{
use HTTP;
public static function getName(): string
{
return 'createCSVImport';
}
public function __construct()
{
$this
->setHttpMethod(Action::HTTP_REQUEST_METHOD_POST)
->setHttpPath('/v1/migrations/csv/imports')
->httpAlias('/v1/migrations/csv')
->desc('Import documents from a CSV')
->groups(['api', 'migrations'])
->label('scope', 'migrations.write')
->label('event', 'migrations.[migrationId].create')
->label('audits.event', 'migration.create')
->label('sdk', new Method(
namespace: 'migrations',
group: null,
name: 'createCSVImport',
description: '/docs/references/migrations/migration-csv-import.md',
auth: [AuthType::ADMIN],
responses: [
new SDKResponse(
code: Response::STATUS_CODE_ACCEPTED,
model: Response::MODEL_MIGRATION,
)
]
))
->param('bucketId', '', fn (Database $dbForProject) => new UID($dbForProject->getAdapter()->getMaxUIDLength()), 'Storage bucket unique ID. You can create a new storage bucket using the Storage service [server integration](https://appwrite.io/docs/server/storage#createBucket).', false, ['dbForProject'])
->param('fileId', '', fn (Database $dbForProject) => new UID($dbForProject->getAdapter()->getMaxUIDLength()), 'File ID.', false, ['dbForProject'])
->param('resourceId', null, new CompoundUID(), 'Composite ID in the format {databaseId:collectionId}, identifying a collection within a database.')
->param('internalFile', false, new Boolean(), 'Is the file stored in an internal bucket?', true)
->inject('response')
->inject('dbForProject')
->inject('dbForPlatform')
->inject('authorization')
->inject('project')
->inject('platform')
->inject('deviceForFiles')
->inject('deviceForMigrations')
->inject('queueForEvents')
->inject('publisherForMigrations')
->callback($this->action(...));
}
public function action(
string $bucketId,
string $fileId,
string $resourceId,
bool $internalFile,
Response $response,
Database $dbForProject,
Database $dbForPlatform,
Authorization $authorization,
Document $project,
array $platform,
Device $deviceForFiles,
Device $deviceForMigrations,
Event $queueForEvents,
MigrationPublisher $publisherForMigrations
): void {
$bucket = $authorization->skip(function () use ($internalFile, $dbForPlatform, $dbForProject, $bucketId) {
if ($internalFile) {
return $dbForPlatform->getDocument('buckets', 'default');
}
return $dbForProject->getDocument('buckets', $bucketId);
});
if ($bucket->isEmpty()) {
throw new Exception(Exception::STORAGE_BUCKET_NOT_FOUND);
}
$file = $authorization->skip(fn () => $internalFile ? $dbForPlatform->getDocument('bucket_' . $bucket->getSequence(), $fileId) : $dbForProject->getDocument('bucket_' . $bucket->getSequence(), $fileId));
if ($file->isEmpty()) {
throw new Exception(Exception::STORAGE_FILE_NOT_FOUND);
}
$path = $file->getAttribute('path', '');
if (!$deviceForFiles->exists($path)) {
throw new Exception(Exception::STORAGE_FILE_NOT_FOUND, 'File not found in ' . $path);
}
// No encryption or compression on files above 20MB.
$hasEncryption = !empty($file->getAttribute('openSSLCipher'));
$compression = $file->getAttribute('algorithm', Compression::NONE);
$hasCompression = $compression !== Compression::NONE;
$migrationId = ID::unique();
$newPath = $deviceForMigrations->getPath($migrationId . '_' . $fileId . '.csv');
if ($hasEncryption || $hasCompression) {
$source = $deviceForFiles->read($path);
if ($hasEncryption) {
$source = OpenSSL::decrypt(
$source,
$file->getAttribute('openSSLCipher'),
System::getEnv('_APP_OPENSSL_KEY_V' . $file->getAttribute('openSSLVersion')),
0,
hex2bin($file->getAttribute('openSSLIV')),
hex2bin($file->getAttribute('openSSLTag'))
);
}
if ($hasCompression) {
switch ($compression) {
case Compression::ZSTD:
$source = (new Zstd())->decompress($source);
break;
case Compression::GZIP:
$source = (new GZIP())->decompress($source);
break;
}
}
// Manual write after decryption and/or decompression
if (!$deviceForMigrations->write($newPath, $source, 'text/csv')) {
throw new \Exception('Unable to copy file');
}
} elseif (!$deviceForFiles->transfer($path, $newPath, $deviceForMigrations)) {
throw new \Exception('Unable to copy file');
}
[$databaseId] = \explode(':', $resourceId, 2);
$database = $authorization->skip(fn () => $dbForProject->getDocument('databases', $databaseId));
$databaseType = $database->getAttribute('type');
if (!\in_array($databaseType, CSV_ALLOWED_DATABASE_TYPES)) {
throw new Exception(Exception::MIGRATION_DATABASE_TYPE_UNSUPPORTED, 'Database type not supported for csv');
}
$fileSize = $deviceForMigrations->getFileSize($newPath);
$resources = Transfer::extractServices([self::transferGroupForDatabaseType($databaseType)]);
$resourceType = self::resourceTypeForDatabaseType($databaseType);
$migration = $dbForProject->createDocument('migrations', new Document([
'$id' => $migrationId,
'status' => 'pending',
'stage' => 'init',
'source' => CSV::getName(),
'destination' => AppwriteSource::getName(),
'resources' => $resources,
'resourceId' => $resourceId,
'resourceType' => $resourceType,
'statusCounters' => '{}',
'resourceData' => '{}',
'errors' => [],
'options' => [
'path' => $newPath,
'size' => $fileSize,
],
]));
$queueForEvents->setParam('migrationId', $migration->getId());
$publisherForMigrations->enqueue(new MigrationMessage(
project: $project,
migration: $migration,
));
$response
->setStatusCode(Response::STATUS_CODE_ACCEPTED)
->dynamic($migration, Response::MODEL_MIGRATION);
}
private static function transferGroupForDatabaseType(string $databaseType): string
{
return match ($databaseType) {
DATABASE_TYPE_LEGACY,
DATABASE_TYPE_TABLESDB => Transfer::GROUP_DATABASES_TABLES_DB,
DATABASE_TYPE_VECTORSDB => Transfer::GROUP_DATABASES_VECTOR_DB,
DATABASE_TYPE_DOCUMENTSDB => Transfer::GROUP_DATABASES_DOCUMENTS_DB,
default => throw new \LogicException('Unknown database type: ' . $databaseType),
};
}
private static function resourceTypeForDatabaseType(string $databaseType): string
{
return match ($databaseType) {
DATABASE_TYPE_VECTORSDB => Resource::TYPE_DATABASE_VECTORSDB,
DATABASE_TYPE_DOCUMENTSDB => Resource::TYPE_DATABASE_DOCUMENTSDB,
default => Resource::TYPE_DATABASE,
};
}
}
@@ -0,0 +1,74 @@
<?php
namespace Appwrite\Platform\Modules\Migrations\Http\Migrations;
use Appwrite\Event\Event;
use Appwrite\Extend\Exception;
use Appwrite\SDK\AuthType;
use Appwrite\SDK\ContentType;
use Appwrite\SDK\Method;
use Appwrite\SDK\Response as SDKResponse;
use Appwrite\Utopia\Response;
use Utopia\Database\Database;
use Utopia\Database\Validator\UID;
use Utopia\Platform\Action;
use Utopia\Platform\Scope\HTTP;
class Delete extends Action
{
use HTTP;
public static function getName(): string
{
return 'deleteMigration';
}
public function __construct()
{
$this
->setHttpMethod(Action::HTTP_REQUEST_METHOD_DELETE)
->setHttpPath('/v1/migrations/:migrationId')
->desc('Delete migration')
->groups(['api', 'migrations'])
->label('scope', 'migrations.write')
->label('event', 'migrations.[migrationId].delete')
->label('audits.event', 'migrationId.delete')
->label('audits.resource', 'migrations/{request.migrationId}')
->label('sdk', new Method(
namespace: 'migrations',
group: null,
name: 'delete',
description: '/docs/references/migrations/delete-migration.md',
auth: [AuthType::ADMIN],
responses: [
new SDKResponse(
code: Response::STATUS_CODE_NOCONTENT,
model: Response::MODEL_NONE,
)
],
contentType: ContentType::NONE
))
->param('migrationId', '', fn (Database $dbForProject) => new UID($dbForProject->getAdapter()->getMaxUIDLength()), 'Migration ID.', false, ['dbForProject'])
->inject('response')
->inject('dbForProject')
->inject('queueForEvents')
->callback($this->action(...));
}
public function action(string $migrationId, Response $response, Database $dbForProject, Event $queueForEvents): void
{
$migration = $dbForProject->getDocument('migrations', $migrationId);
if ($migration->isEmpty()) {
throw new Exception(Exception::MIGRATION_NOT_FOUND);
}
if (!$dbForProject->deleteDocument('migrations', $migration->getId())) {
throw new Exception(Exception::GENERAL_SERVER_ERROR, 'Failed to remove migration from DB');
}
$queueForEvents->setParam('migrationId', $migration->getId());
$response->noContent();
}
}
@@ -0,0 +1,114 @@
<?php
namespace Appwrite\Platform\Modules\Migrations\Http\Migrations\Firebase;
use Appwrite\Event\Event;
use Appwrite\Event\Message\Migration as MigrationMessage;
use Appwrite\Event\Publisher\Migration as MigrationPublisher;
use Appwrite\Extend\Exception;
use Appwrite\SDK\AuthType;
use Appwrite\SDK\Method;
use Appwrite\SDK\Response as SDKResponse;
use Appwrite\Utopia\Response;
use Utopia\Database\Database;
use Utopia\Database\Document;
use Utopia\Database\Helpers\ID;
use Utopia\Migration\Sources\Appwrite as AppwriteSource;
use Utopia\Migration\Sources\Firebase;
use Utopia\Platform\Action;
use Utopia\Platform\Scope\HTTP;
use Utopia\Validator\ArrayList;
use Utopia\Validator\Text;
use Utopia\Validator\WhiteList;
class Create extends Action
{
use HTTP;
public static function getName(): string
{
return 'createFirebaseMigration';
}
public function __construct()
{
$this
->setHttpMethod(Action::HTTP_REQUEST_METHOD_POST)
->setHttpPath('/v1/migrations/firebase')
->desc('Create Firebase migration')
->groups(['api', 'migrations'])
->label('scope', 'migrations.write')
->label('event', 'migrations.[migrationId].create')
->label('audits.event', 'migration.create')
->label('sdk', new Method(
namespace: 'migrations',
group: null,
name: 'createFirebaseMigration',
description: '/docs/references/migrations/migration-firebase.md',
auth: [AuthType::ADMIN],
responses: [
new SDKResponse(
code: Response::STATUS_CODE_ACCEPTED,
model: Response::MODEL_MIGRATION,
)
]
))
->param('resources', [], new ArrayList(new WhiteList(Firebase::getSupportedResources())), 'List of resources to migrate')
->param('serviceAccount', '', new Text(65536), 'JSON of the Firebase service account credentials')
->inject('response')
->inject('dbForProject')
->inject('project')
->inject('platform')
->inject('queueForEvents')
->inject('publisherForMigrations')
->callback($this->action(...));
}
public function action(
array $resources,
string $serviceAccount,
Response $response,
Database $dbForProject,
Document $project,
array $platform,
Event $queueForEvents,
MigrationPublisher $publisherForMigrations
): void {
$serviceAccountData = json_decode($serviceAccount, true);
if (empty($serviceAccountData)) {
throw new Exception(Exception::MIGRATION_PROVIDER_ERROR, 'Invalid Service Account JSON');
}
if (!isset($serviceAccountData['project_id']) || !isset($serviceAccountData['client_email']) || !isset($serviceAccountData['private_key'])) {
throw new Exception(Exception::MIGRATION_PROVIDER_ERROR, 'Invalid Service Account JSON');
}
$migration = $dbForProject->createDocument('migrations', new Document([
'$id' => ID::unique(),
'status' => 'pending',
'stage' => 'init',
'source' => Firebase::getName(),
'destination' => AppwriteSource::getName(),
'credentials' => [
'serviceAccount' => $serviceAccount,
],
'resources' => $resources,
'statusCounters' => '{}',
'resourceData' => '{}',
'errors' => [],
]));
$queueForEvents->setParam('migrationId', $migration->getId());
$publisherForMigrations->enqueue(new MigrationMessage(
project: $project,
migration: $migration,
platform: $platform,
));
$response
->setStatusCode(Response::STATUS_CODE_ACCEPTED)
->dynamic($migration, Response::MODEL_MIGRATION);
}
}
@@ -0,0 +1,80 @@
<?php
namespace Appwrite\Platform\Modules\Migrations\Http\Migrations\Firebase\Report;
use Appwrite\Extend\Exception;
use Appwrite\SDK\AuthType;
use Appwrite\SDK\Method;
use Appwrite\SDK\Response as SDKResponse;
use Appwrite\Utopia\Response;
use Utopia\Database\Document;
use Utopia\Migration\Sources\Firebase;
use Utopia\Platform\Action;
use Utopia\Platform\Scope\HTTP;
use Utopia\Validator\ArrayList;
use Utopia\Validator\Text;
use Utopia\Validator\WhiteList;
class Get extends Action
{
use HTTP;
public static function getName(): string
{
return 'getFirebaseReport';
}
public function __construct()
{
$this
->setHttpMethod(Action::HTTP_REQUEST_METHOD_GET)
->setHttpPath('/v1/migrations/firebase/report')
->desc('Get Firebase migration report')
->groups(['api', 'migrations'])
->label('scope', 'migrations.write')
->label('sdk', new Method(
namespace: 'migrations',
group: null,
name: 'getFirebaseReport',
description: '/docs/references/migrations/migration-firebase-report.md',
auth: [AuthType::ADMIN],
responses: [
new SDKResponse(
code: Response::STATUS_CODE_OK,
model: Response::MODEL_MIGRATION_REPORT,
)
]
))
->param('resources', [], new ArrayList(new WhiteList(Firebase::getSupportedResources())), 'List of resources to migrate')
->param('serviceAccount', '', new Text(65536), 'JSON of the Firebase service account credentials')
->inject('response')
->callback($this->action(...));
}
public function action(array $resources, string $serviceAccount, Response $response): void
{
$serviceAccount = json_decode($serviceAccount, true);
if (empty($serviceAccount)) {
throw new Exception(Exception::MIGRATION_PROVIDER_ERROR, 'Invalid Service Account JSON');
}
if (!isset($serviceAccount['project_id']) || !isset($serviceAccount['client_email']) || !isset($serviceAccount['private_key'])) {
throw new Exception(Exception::MIGRATION_PROVIDER_ERROR, 'Invalid Service Account JSON');
}
try {
$firebase = new Firebase($serviceAccount);
$report = $firebase->report($resources);
} catch (\Throwable $e) {
throw new Exception(
Exception::MIGRATION_PROVIDER_ERROR,
'Unable to connect to the migration source. Please verify your credentials and ensure the source is reachable from this server. Check for network restrictions such as firewalls, IP allowlists, or outbound connectivity limits.'
);
}
$response
->setStatusCode(Response::STATUS_CODE_OK)
->dynamic(new Document($report), Response::MODEL_MIGRATION_REPORT);
}
}
@@ -0,0 +1,61 @@
<?php
namespace Appwrite\Platform\Modules\Migrations\Http\Migrations;
use Appwrite\Extend\Exception;
use Appwrite\SDK\AuthType;
use Appwrite\SDK\Method;
use Appwrite\SDK\Response as SDKResponse;
use Appwrite\Utopia\Response;
use Utopia\Database\Database;
use Utopia\Database\Validator\UID;
use Utopia\Platform\Action;
use Utopia\Platform\Scope\HTTP;
class Get extends Action
{
use HTTP;
public static function getName(): string
{
return 'getMigration';
}
public function __construct()
{
$this
->setHttpMethod(Action::HTTP_REQUEST_METHOD_GET)
->setHttpPath('/v1/migrations/:migrationId')
->desc('Get migration')
->groups(['api', 'migrations'])
->label('scope', 'migrations.read')
->label('sdk', new Method(
namespace: 'migrations',
group: null,
name: 'get',
description: '/docs/references/migrations/get-migration.md',
auth: [AuthType::ADMIN],
responses: [
new SDKResponse(
code: Response::STATUS_CODE_OK,
model: Response::MODEL_MIGRATION,
)
]
))
->param('migrationId', '', fn (Database $dbForProject) => new UID($dbForProject->getAdapter()->getMaxUIDLength()), 'Migration unique ID.', false, ['dbForProject'])
->inject('response')
->inject('dbForProject')
->callback($this->action(...));
}
public function action(string $migrationId, Response $response, Database $dbForProject): void
{
$migration = $dbForProject->getDocument('migrations', $migrationId);
if ($migration->isEmpty()) {
throw new Exception(Exception::MIGRATION_NOT_FOUND);
}
$response->dynamic($migration, Response::MODEL_MIGRATION);
}
}
@@ -0,0 +1,198 @@
<?php
namespace Appwrite\Platform\Modules\Migrations\Http\Migrations\JSON\Exports;
use Appwrite\Event\Event;
use Appwrite\Event\Message\Migration as MigrationMessage;
use Appwrite\Event\Publisher\Migration as MigrationPublisher;
use Appwrite\Extend\Exception;
use Appwrite\SDK\AuthType;
use Appwrite\SDK\Method;
use Appwrite\SDK\Response as SDKResponse;
use Appwrite\Utopia\Database\Validator\CompoundUID;
use Appwrite\Utopia\Response;
use Utopia\Database\Database;
use Utopia\Database\Document;
use Utopia\Database\Exception\Query as QueryException;
use Utopia\Database\Helpers\ID;
use Utopia\Database\Query;
use Utopia\Database\Validator\Authorization;
use Utopia\Database\Validator\Queries\Documents;
use Utopia\Migration\Resource;
use Utopia\Migration\Sources\Appwrite as AppwriteSource;
use Utopia\Migration\Sources\JSON as JSONSource;
use Utopia\Migration\Transfer;
use Utopia\Platform\Action;
use Utopia\Platform\Scope\HTTP;
use Utopia\Validator\ArrayList;
use Utopia\Validator\Boolean;
use Utopia\Validator\Text;
class Create extends Action
{
use HTTP;
public static function getName(): string
{
return 'createJSONExport';
}
public function __construct()
{
$this
->setHttpMethod(Action::HTTP_REQUEST_METHOD_POST)
->setHttpPath('/v1/migrations/json/exports')
->desc('Export documents to JSON')
->groups(['api', 'migrations'])
->label('scope', 'migrations.write')
->label('event', 'migrations.[migrationId].create')
->label('audits.event', 'migration.create')
->label('sdk', new Method(
namespace: 'migrations',
group: null,
name: 'createJSONExport',
description: '/docs/references/migrations/migration-json-export.md',
auth: [AuthType::ADMIN],
responses: [
new SDKResponse(
code: Response::STATUS_CODE_ACCEPTED,
model: Response::MODEL_MIGRATION,
)
]
))
->param('resourceId', null, new CompoundUID(), 'Composite ID in the format {databaseId:collectionId}, identifying a collection within a database to export.')
->param('filename', '', new Text(255), 'The name of the file to be created for the export, excluding the .json extension.')
->param('columns', [], new ArrayList(new Text(Database::LENGTH_KEY)), 'List of attributes to export. If empty, all attributes will be exported. You can use the `*` wildcard to export all attributes from the collection.', true)
->param('queries', [], new ArrayList(new Text(0)), 'Array of query strings generated using the Query class provided by the SDK to filter documents to export. [Learn more about queries](https://appwrite.io/docs/databases#querying-documents). Maximum of ' . APP_LIMIT_ARRAY_PARAMS_SIZE . ' queries are allowed, each ' . APP_LIMIT_ARRAY_ELEMENT_SIZE . ' characters long.', true)
->param('notify', true, new Boolean(), 'Set to true to receive an email when the export is complete. Default is true.', true)
->inject('user')
->inject('response')
->inject('dbForProject')
->inject('dbForPlatform')
->inject('authorization')
->inject('project')
->inject('platform')
->inject('queueForEvents')
->inject('publisherForMigrations')
->callback($this->action(...));
}
public function action(
string $resourceId,
string $filename,
array $columns,
array $queries,
bool $notify,
Document $user,
Response $response,
Database $dbForProject,
Database $dbForPlatform,
Authorization $authorization,
Document $project,
array $platform,
Event $queueForEvents,
MigrationPublisher $publisherForMigrations
): void {
try {
$parsedQueries = Query::parseQueries($queries);
} catch (QueryException $e) {
throw new Exception(Exception::GENERAL_QUERY_INVALID, $e->getMessage());
}
$bucket = $authorization->skip(fn () => $dbForPlatform->getDocument('buckets', 'default'));
if ($bucket->isEmpty()) {
throw new Exception(Exception::STORAGE_BUCKET_NOT_FOUND);
}
[$databaseId, $collectionId] = \explode(':', $resourceId, 2);
if (empty($databaseId)) {
throw new Exception(Exception::DATABASE_NOT_FOUND);
}
if (empty($collectionId)) {
throw new Exception(Exception::COLLECTION_NOT_FOUND);
}
$database = $authorization->skip(fn () => $dbForProject->getDocument('databases', $databaseId));
if ($database->isEmpty()) {
throw new Exception(Exception::DATABASE_NOT_FOUND);
}
$collection = $authorization->skip(fn () => $dbForProject->getDocument('database_' . $database->getSequence(), $collectionId));
if ($collection->isEmpty()) {
throw new Exception(Exception::COLLECTION_NOT_FOUND);
}
$databaseType = $database->getAttribute('type');
// Schemaless databases (DocumentsDB, VectorsDB) allow queries on dynamic fields
$isSchemaless = \in_array($databaseType, [DATABASE_TYPE_DOCUMENTSDB, DATABASE_TYPE_VECTORSDB]);
$validator = new Documents(
attributes: $collection->getAttribute('attributes', []),
indexes: $collection->getAttribute('indexes', []),
idAttributeType: $dbForProject->getAdapter()->getIdAttributeType(),
supportForAttributes: !$isSchemaless,
);
if (!$validator->isValid($parsedQueries)) {
throw new Exception(Exception::GENERAL_QUERY_INVALID, $validator->getDescription());
}
$resources = Transfer::extractServices([self::transferGroupForDatabaseType($databaseType)]);
$resourceType = self::resourceTypeForDatabaseType($databaseType);
$migration = $dbForProject->createDocument('migrations', new Document([
'$id' => ID::unique(),
'status' => 'pending',
'stage' => 'init',
'source' => AppwriteSource::getName(),
'destination' => JSONSource::getName(),
'resources' => $resources,
'resourceId' => $resourceId,
'resourceType' => $resourceType,
'statusCounters' => '{}',
'resourceData' => '{}',
'errors' => [],
'options' => [
'bucketId' => 'default', // Always use internal bucket
'filename' => $filename,
'columns' => $columns,
'queries' => $queries,
'notify' => $notify,
'userInternalId' => $user->getSequence(),
],
]));
$queueForEvents->setParam('migrationId', $migration->getId());
$publisherForMigrations->enqueue(new MigrationMessage(
project: $project,
migration: $migration,
platform: $platform,
));
$response
->setStatusCode(Response::STATUS_CODE_ACCEPTED)
->dynamic($migration, Response::MODEL_MIGRATION);
}
private static function transferGroupForDatabaseType(string $databaseType): string
{
return match ($databaseType) {
DATABASE_TYPE_LEGACY,
DATABASE_TYPE_TABLESDB => Transfer::GROUP_DATABASES_TABLES_DB,
DATABASE_TYPE_VECTORSDB => Transfer::GROUP_DATABASES_VECTOR_DB,
DATABASE_TYPE_DOCUMENTSDB => Transfer::GROUP_DATABASES_DOCUMENTS_DB,
default => throw new \LogicException('Unknown database type: ' . $databaseType),
};
}
private static function resourceTypeForDatabaseType(string $databaseType): string
{
return match ($databaseType) {
DATABASE_TYPE_VECTORSDB => Resource::TYPE_DATABASE_VECTORSDB,
DATABASE_TYPE_DOCUMENTSDB => Resource::TYPE_DATABASE_DOCUMENTSDB,
default => Resource::TYPE_DATABASE,
};
}
}
@@ -0,0 +1,221 @@
<?php
namespace Appwrite\Platform\Modules\Migrations\Http\Migrations\JSON\Imports;
use Appwrite\Event\Event;
use Appwrite\Event\Message\Migration as MigrationMessage;
use Appwrite\Event\Publisher\Migration as MigrationPublisher;
use Appwrite\Extend\Exception;
use Appwrite\OpenSSL\OpenSSL;
use Appwrite\SDK\AuthType;
use Appwrite\SDK\Method;
use Appwrite\SDK\Response as SDKResponse;
use Appwrite\Utopia\Database\Validator\CompoundUID;
use Appwrite\Utopia\Response;
use Utopia\Compression\Algorithms\GZIP;
use Utopia\Compression\Algorithms\Zstd;
use Utopia\Compression\Compression;
use Utopia\Database\Database;
use Utopia\Database\Document;
use Utopia\Database\Helpers\ID;
use Utopia\Database\Validator\Authorization;
use Utopia\Database\Validator\UID;
use Utopia\Migration\Resource;
use Utopia\Migration\Sources\Appwrite as AppwriteSource;
use Utopia\Migration\Sources\JSON as JSONSource;
use Utopia\Migration\Transfer;
use Utopia\Platform\Action;
use Utopia\Platform\Scope\HTTP;
use Utopia\Storage\Device;
use Utopia\System\System;
use Utopia\Validator\Boolean;
class Create extends Action
{
use HTTP;
public static function getName(): string
{
return 'createJSONImport';
}
public function __construct()
{
$this
->setHttpMethod(Action::HTTP_REQUEST_METHOD_POST)
->setHttpPath('/v1/migrations/json/imports')
->desc('Import documents from a JSON')
->groups(['api', 'migrations'])
->label('scope', 'migrations.write')
->label('event', 'migrations.[migrationId].create')
->label('audits.event', 'migration.create')
->label('sdk', new Method(
namespace: 'migrations',
group: null,
name: 'createJSONImport',
description: '/docs/references/migrations/migration-json-import.md',
auth: [AuthType::ADMIN],
responses: [
new SDKResponse(
code: Response::STATUS_CODE_ACCEPTED,
model: Response::MODEL_MIGRATION,
)
]
))
->param('bucketId', '', new UID(), 'Storage bucket unique ID. You can create a new storage bucket using the Storage service [server integration](https://appwrite.io/docs/server/storage#createBucket).')
->param('fileId', '', new UID(), 'File ID.')
->param('resourceId', null, new CompoundUID(), 'Composite ID in the format {databaseId:collectionId}, identifying a collection within a database.')
->param('internalFile', false, new Boolean(), 'Is the file stored in an internal bucket?', true)
->inject('response')
->inject('dbForProject')
->inject('dbForPlatform')
->inject('authorization')
->inject('project')
->inject('platform')
->inject('deviceForFiles')
->inject('deviceForMigrations')
->inject('queueForEvents')
->inject('publisherForMigrations')
->callback($this->action(...));
}
public function action(
string $bucketId,
string $fileId,
string $resourceId,
bool $internalFile,
Response $response,
Database $dbForProject,
Database $dbForPlatform,
Authorization $authorization,
Document $project,
array $platform,
Device $deviceForFiles,
Device $deviceForMigrations,
Event $queueForEvents,
MigrationPublisher $publisherForMigrations
): void {
$bucket = $authorization->skip(function () use ($internalFile, $dbForPlatform, $dbForProject, $bucketId) {
if ($internalFile) {
return $dbForPlatform->getDocument('buckets', 'default');
}
return $dbForProject->getDocument('buckets', $bucketId);
});
if ($bucket->isEmpty()) {
throw new Exception(Exception::STORAGE_BUCKET_NOT_FOUND);
}
$file = $authorization->skip(fn () => $internalFile ? $dbForPlatform->getDocument('bucket_' . $bucket->getSequence(), $fileId) : $dbForProject->getDocument('bucket_' . $bucket->getSequence(), $fileId));
if ($file->isEmpty()) {
throw new Exception(Exception::STORAGE_FILE_NOT_FOUND);
}
$path = $file->getAttribute('path', '');
if (!$deviceForFiles->exists($path)) {
throw new Exception(Exception::STORAGE_FILE_NOT_FOUND, 'File not found in ' . $path);
}
// No encryption or compression on files above 20MB.
$hasEncryption = !empty($file->getAttribute('openSSLCipher'));
$compression = $file->getAttribute('algorithm', Compression::NONE);
$hasCompression = $compression !== Compression::NONE;
$migrationId = ID::unique();
$newPath = $deviceForMigrations->getPath($migrationId . '_' . $fileId . '.json');
if ($hasEncryption || $hasCompression) {
$source = $deviceForFiles->read($path);
if ($hasEncryption) {
$source = OpenSSL::decrypt(
$source,
$file->getAttribute('openSSLCipher'),
System::getEnv('_APP_OPENSSL_KEY_V' . $file->getAttribute('openSSLVersion')),
0,
hex2bin($file->getAttribute('openSSLIV')),
hex2bin($file->getAttribute('openSSLTag'))
);
}
if ($hasCompression) {
switch ($compression) {
case Compression::ZSTD:
$source = (new Zstd())->decompress($source);
break;
case Compression::GZIP:
$source = (new GZIP())->decompress($source);
break;
}
}
// Manual write after decryption and/or decompression
if (!$deviceForMigrations->write($newPath, $source, 'application/json')) {
throw new \Exception('Unable to copy file');
}
} elseif (!$deviceForFiles->transfer($path, $newPath, $deviceForMigrations)) {
throw new \Exception('Unable to copy file');
}
$fileSize = $deviceForMigrations->getFileSize($newPath);
[$databaseId] = \explode(':', $resourceId, 2);
$database = $authorization->skip(fn () => $dbForProject->getDocument('databases', $databaseId));
if ($database->isEmpty()) {
throw new Exception(Exception::DATABASE_NOT_FOUND);
}
$databaseType = $database->getAttribute('type');
$resources = Transfer::extractServices([self::transferGroupForDatabaseType($databaseType)]);
$resourceType = self::resourceTypeForDatabaseType($databaseType);
$migration = $dbForProject->createDocument('migrations', new Document([
'$id' => $migrationId,
'status' => 'pending',
'stage' => 'init',
'source' => JSONSource::getName(),
'destination' => AppwriteSource::getName(),
'resources' => $resources,
'resourceId' => $resourceId,
'resourceType' => $resourceType,
'statusCounters' => '{}',
'resourceData' => '{}',
'errors' => [],
'options' => [
'path' => $newPath,
'size' => $fileSize,
],
]));
$queueForEvents->setParam('migrationId', $migration->getId());
$publisherForMigrations->enqueue(new MigrationMessage(
project: $project,
migration: $migration,
platform: $platform,
));
$response
->setStatusCode(Response::STATUS_CODE_ACCEPTED)
->dynamic($migration, Response::MODEL_MIGRATION);
}
private static function transferGroupForDatabaseType(string $databaseType): string
{
return match ($databaseType) {
DATABASE_TYPE_LEGACY,
DATABASE_TYPE_TABLESDB => Transfer::GROUP_DATABASES_TABLES_DB,
DATABASE_TYPE_VECTORSDB => Transfer::GROUP_DATABASES_VECTOR_DB,
DATABASE_TYPE_DOCUMENTSDB => Transfer::GROUP_DATABASES_DOCUMENTS_DB,
default => throw new \LogicException('Unknown database type: ' . $databaseType),
};
}
private static function resourceTypeForDatabaseType(string $databaseType): string
{
return match ($databaseType) {
DATABASE_TYPE_VECTORSDB => Resource::TYPE_DATABASE_VECTORSDB,
DATABASE_TYPE_DOCUMENTSDB => Resource::TYPE_DATABASE_DOCUMENTSDB,
default => Resource::TYPE_DATABASE,
};
}
}
@@ -0,0 +1,122 @@
<?php
namespace Appwrite\Platform\Modules\Migrations\Http\Migrations\NHost;
use Appwrite\Event\Event;
use Appwrite\Event\Message\Migration as MigrationMessage;
use Appwrite\Event\Publisher\Migration as MigrationPublisher;
use Appwrite\SDK\AuthType;
use Appwrite\SDK\Method;
use Appwrite\SDK\Response as SDKResponse;
use Appwrite\Utopia\Response;
use Utopia\Database\Database;
use Utopia\Database\Document;
use Utopia\Database\Helpers\ID;
use Utopia\Migration\Sources\Appwrite as AppwriteSource;
use Utopia\Migration\Sources\NHost;
use Utopia\Platform\Action;
use Utopia\Platform\Scope\HTTP;
use Utopia\Validator\ArrayList;
use Utopia\Validator\Integer;
use Utopia\Validator\Text;
use Utopia\Validator\WhiteList;
class Create extends Action
{
use HTTP;
public static function getName(): string
{
return 'createNHostMigration';
}
public function __construct()
{
$this
->setHttpMethod(Action::HTTP_REQUEST_METHOD_POST)
->setHttpPath('/v1/migrations/nhost')
->desc('Create NHost migration')
->groups(['api', 'migrations'])
->label('scope', 'migrations.write')
->label('event', 'migrations.[migrationId].create')
->label('audits.event', 'migration.create')
->label('sdk', new Method(
namespace: 'migrations',
group: null,
name: 'createNHostMigration',
description: '/docs/references/migrations/migration-nhost.md',
auth: [AuthType::ADMIN],
responses: [
new SDKResponse(
code: Response::STATUS_CODE_ACCEPTED,
model: Response::MODEL_MIGRATION,
)
]
))
->param('resources', [], new ArrayList(new WhiteList(NHost::getSupportedResources())), 'List of resources to migrate')
->param('subdomain', '', new Text(512), 'Source\'s Subdomain')
->param('region', '', new Text(512), 'Source\'s Region')
->param('adminSecret', '', new Text(512), 'Source\'s Admin Secret')
->param('database', '', new Text(512), 'Source\'s Database Name')
->param('username', '', new Text(512), 'Source\'s Database Username')
->param('password', '', new Text(512), 'Source\'s Database Password')
->param('port', 5432, new Integer(true), 'Source\'s Database Port', true)
->inject('response')
->inject('dbForProject')
->inject('project')
->inject('platform')
->inject('queueForEvents')
->inject('publisherForMigrations')
->callback($this->action(...));
}
public function action(
array $resources,
string $subdomain,
string $region,
string $adminSecret,
string $database,
string $username,
string $password,
int $port,
Response $response,
Database $dbForProject,
Document $project,
array $platform,
Event $queueForEvents,
MigrationPublisher $publisherForMigrations
): void {
$migration = $dbForProject->createDocument('migrations', new Document([
'$id' => ID::unique(),
'status' => 'pending',
'stage' => 'init',
'source' => NHost::getName(),
'destination' => AppwriteSource::getName(),
'credentials' => [
'subdomain' => $subdomain,
'region' => $region,
'adminSecret' => $adminSecret,
'database' => $database,
'username' => $username,
'password' => $password,
'port' => $port,
],
'resources' => $resources,
'statusCounters' => '{}',
'resourceData' => '{}',
'errors' => [],
]));
$queueForEvents->setParam('migrationId', $migration->getId());
$publisherForMigrations->enqueue(new MigrationMessage(
project: $project,
migration: $migration,
platform: $platform,
));
$response
->setStatusCode(Response::STATUS_CODE_ACCEPTED)
->dynamic($migration, Response::MODEL_MIGRATION);
}
}
@@ -0,0 +1,86 @@
<?php
namespace Appwrite\Platform\Modules\Migrations\Http\Migrations\NHost\Report;
use Appwrite\Extend\Exception;
use Appwrite\SDK\AuthType;
use Appwrite\SDK\Method;
use Appwrite\SDK\Response as SDKResponse;
use Appwrite\Utopia\Response;
use Utopia\Database\Document;
use Utopia\Migration\Sources\NHost;
use Utopia\Platform\Action;
use Utopia\Platform\Scope\HTTP;
use Utopia\Validator\ArrayList;
use Utopia\Validator\Integer;
use Utopia\Validator\Text;
use Utopia\Validator\WhiteList;
class Get extends Action
{
use HTTP;
public static function getName(): string
{
return 'getNHostReport';
}
public function __construct()
{
$this
->setHttpMethod(Action::HTTP_REQUEST_METHOD_GET)
->setHttpPath('/v1/migrations/nhost/report')
->desc('Get NHost migration report')
->groups(['api', 'migrations'])
->label('scope', 'migrations.write')
->label('sdk', new Method(
namespace: 'migrations',
group: null,
name: 'getNHostReport',
description: '/docs/references/migrations/migration-nhost-report.md',
auth: [AuthType::ADMIN],
responses: [
new SDKResponse(
code: Response::STATUS_CODE_OK,
model: Response::MODEL_MIGRATION_REPORT,
)
]
))
->param('resources', [], new ArrayList(new WhiteList(NHost::getSupportedResources())), 'List of resources to migrate.')
->param('subdomain', '', new Text(512), 'Source\'s Subdomain.')
->param('region', '', new Text(512), 'Source\'s Region.')
->param('adminSecret', '', new Text(512), 'Source\'s Admin Secret.')
->param('database', '', new Text(512), 'Source\'s Database Name.')
->param('username', '', new Text(512), 'Source\'s Database Username.')
->param('password', '', new Text(512), 'Source\'s Database Password.')
->param('port', 5432, new Integer(true), 'Source\'s Database Port.', true)
->inject('response')
->callback($this->action(...));
}
public function action(
array $resources,
string $subdomain,
string $region,
string $adminSecret,
string $database,
string $username,
string $password,
int $port,
Response $response
): void {
try {
$nhost = new NHost($subdomain, $region, $adminSecret, $database, $username, $password, $port);
$report = $nhost->report($resources);
} catch (\Throwable $e) {
throw new Exception(
Exception::MIGRATION_PROVIDER_ERROR,
'Unable to connect to the migration source. Please verify your credentials and ensure the source is reachable from this server. Check for network restrictions such as firewalls, IP allowlists, or outbound connectivity limits.'
);
}
$response
->setStatusCode(Response::STATUS_CODE_OK)
->dynamic(new Document($report), Response::MODEL_MIGRATION_REPORT);
}
}
@@ -0,0 +1,120 @@
<?php
namespace Appwrite\Platform\Modules\Migrations\Http\Migrations\Supabase;
use Appwrite\Event\Event;
use Appwrite\Event\Message\Migration as MigrationMessage;
use Appwrite\Event\Publisher\Migration as MigrationPublisher;
use Appwrite\SDK\AuthType;
use Appwrite\SDK\Method;
use Appwrite\SDK\Response as SDKResponse;
use Appwrite\Utopia\Response;
use Utopia\Database\Database;
use Utopia\Database\Document;
use Utopia\Database\Helpers\ID;
use Utopia\Migration\Sources\Appwrite as AppwriteSource;
use Utopia\Migration\Sources\Supabase;
use Utopia\Platform\Action;
use Utopia\Platform\Scope\HTTP;
use Utopia\Validator\ArrayList;
use Utopia\Validator\Integer;
use Utopia\Validator\Text;
use Utopia\Validator\URL;
use Utopia\Validator\WhiteList;
class Create extends Action
{
use HTTP;
public static function getName(): string
{
return 'createSupabaseMigration';
}
public function __construct()
{
$this
->setHttpMethod(Action::HTTP_REQUEST_METHOD_POST)
->setHttpPath('/v1/migrations/supabase')
->desc('Create Supabase migration')
->groups(['api', 'migrations'])
->label('scope', 'migrations.write')
->label('event', 'migrations.[migrationId].create')
->label('audits.event', 'migration.create')
->label('sdk', new Method(
namespace: 'migrations',
group: null,
name: 'createSupabaseMigration',
description: '/docs/references/migrations/migration-supabase.md',
auth: [AuthType::ADMIN],
responses: [
new SDKResponse(
code: Response::STATUS_CODE_ACCEPTED,
model: Response::MODEL_MIGRATION,
)
]
))
->param('resources', [], new ArrayList(new WhiteList(Supabase::getSupportedResources(), true)), 'List of resources to migrate')
->param('endpoint', '', new URL(), 'Source\'s Supabase Endpoint')
->param('apiKey', '', new Text(512), 'Source\'s API Key')
->param('databaseHost', '', new Text(512), 'Source\'s Database Host')
->param('username', '', new Text(512), 'Source\'s Database Username')
->param('password', '', new Text(512), 'Source\'s Database Password')
->param('port', 5432, new Integer(true), 'Source\'s Database Port', true)
->inject('response')
->inject('dbForProject')
->inject('project')
->inject('platform')
->inject('queueForEvents')
->inject('publisherForMigrations')
->callback($this->action(...));
}
public function action(
array $resources,
string $endpoint,
string $apiKey,
string $databaseHost,
string $username,
string $password,
int $port,
Response $response,
Database $dbForProject,
Document $project,
array $platform,
Event $queueForEvents,
MigrationPublisher $publisherForMigrations
): void {
$migration = $dbForProject->createDocument('migrations', new Document([
'$id' => ID::unique(),
'status' => 'pending',
'stage' => 'init',
'source' => Supabase::getName(),
'destination' => AppwriteSource::getName(),
'credentials' => [
'endpoint' => $endpoint,
'apiKey' => $apiKey,
'databaseHost' => $databaseHost,
'username' => $username,
'password' => $password,
'port' => $port,
],
'resources' => $resources,
'statusCounters' => '{}',
'resourceData' => '{}',
'errors' => [],
]));
$queueForEvents->setParam('migrationId', $migration->getId());
$publisherForMigrations->enqueue(new MigrationMessage(
project: $project,
migration: $migration,
platform: $platform,
));
$response
->setStatusCode(Response::STATUS_CODE_ACCEPTED)
->dynamic($migration, Response::MODEL_MIGRATION);
}
}
@@ -0,0 +1,85 @@
<?php
namespace Appwrite\Platform\Modules\Migrations\Http\Migrations\Supabase\Report;
use Appwrite\Extend\Exception;
use Appwrite\SDK\AuthType;
use Appwrite\SDK\Method;
use Appwrite\SDK\Response as SDKResponse;
use Appwrite\Utopia\Response;
use Utopia\Database\Document;
use Utopia\Migration\Sources\Supabase;
use Utopia\Platform\Action;
use Utopia\Platform\Scope\HTTP;
use Utopia\Validator\ArrayList;
use Utopia\Validator\Integer;
use Utopia\Validator\Text;
use Utopia\Validator\URL;
use Utopia\Validator\WhiteList;
class Get extends Action
{
use HTTP;
public static function getName(): string
{
return 'getSupabaseReport';
}
public function __construct()
{
$this
->setHttpMethod(Action::HTTP_REQUEST_METHOD_GET)
->setHttpPath('/v1/migrations/supabase/report')
->desc('Get Supabase migration report')
->groups(['api', 'migrations'])
->label('scope', 'migrations.write')
->label('sdk', new Method(
namespace: 'migrations',
group: null,
name: 'getSupabaseReport',
description: '/docs/references/migrations/migration-supabase-report.md',
auth: [AuthType::ADMIN],
responses: [
new SDKResponse(
code: Response::STATUS_CODE_OK,
model: Response::MODEL_MIGRATION_REPORT,
)
]
))
->param('resources', [], new ArrayList(new WhiteList(Supabase::getSupportedResources(), true)), 'List of resources to migrate')
->param('endpoint', '', new URL(), 'Source\'s Supabase Endpoint.')
->param('apiKey', '', new Text(512), 'Source\'s API Key.')
->param('databaseHost', '', new Text(512), 'Source\'s Database Host.')
->param('username', '', new Text(512), 'Source\'s Database Username.')
->param('password', '', new Text(512), 'Source\'s Database Password.')
->param('port', 5432, new Integer(true), 'Source\'s Database Port.', true)
->inject('response')
->callback($this->action(...));
}
public function action(
array $resources,
string $endpoint,
string $apiKey,
string $databaseHost,
string $username,
string $password,
int $port,
Response $response
): void {
try {
$supabase = new Supabase($endpoint, $apiKey, $databaseHost, 'postgres', $username, $password, $port);
$report = $supabase->report($resources);
} catch (\Throwable $e) {
throw new Exception(
Exception::MIGRATION_PROVIDER_ERROR,
'Unable to connect to the migration source. Please verify your credentials and ensure the source is reachable from this server. Check for network restrictions such as firewalls, IP allowlists, or outbound connectivity limits.'
);
}
$response
->setStatusCode(Response::STATUS_CODE_OK)
->dynamic(new Document($report), Response::MODEL_MIGRATION_REPORT);
}
}
@@ -0,0 +1,90 @@
<?php
namespace Appwrite\Platform\Modules\Migrations\Http\Migrations;
use Appwrite\Event\Message\Migration as MigrationMessage;
use Appwrite\Event\Publisher\Migration as MigrationPublisher;
use Appwrite\Extend\Exception;
use Appwrite\SDK\AuthType;
use Appwrite\SDK\Method;
use Appwrite\SDK\Response as SDKResponse;
use Appwrite\Utopia\Response;
use Utopia\Database\Database;
use Utopia\Database\Document;
use Utopia\Database\Validator\UID;
use Utopia\Platform\Action;
use Utopia\Platform\Scope\HTTP;
class Update extends Action
{
use HTTP;
public static function getName(): string
{
return 'retryMigration';
}
public function __construct()
{
$this
->setHttpMethod(Action::HTTP_REQUEST_METHOD_PATCH)
->setHttpPath('/v1/migrations/:migrationId')
->desc('Update retry migration')
->groups(['api', 'migrations'])
->label('scope', 'migrations.write')
->label('event', 'migrations.[migrationId].retry')
->label('audits.event', 'migration.retry')
->label('audits.resource', 'migrations/{request.migrationId}')
->label('sdk', new Method(
namespace: 'migrations',
group: null,
name: 'retry',
description: '/docs/references/migrations/retry-migration.md',
auth: [AuthType::ADMIN],
responses: [
new SDKResponse(
code: Response::STATUS_CODE_ACCEPTED,
model: Response::MODEL_MIGRATION,
)
]
))
->param('migrationId', '', fn (Database $dbForProject) => new UID($dbForProject->getAdapter()->getMaxUIDLength()), 'Migration unique ID.', false, ['dbForProject'])
->inject('response')
->inject('dbForProject')
->inject('project')
->inject('platform')
->inject('publisherForMigrations')
->callback($this->action(...));
}
public function action(
string $migrationId,
Response $response,
Database $dbForProject,
Document $project,
array $platform,
MigrationPublisher $publisherForMigrations
): void {
$migration = $dbForProject->getDocument('migrations', $migrationId);
if ($migration->isEmpty()) {
throw new Exception(Exception::MIGRATION_NOT_FOUND);
}
if ($migration->getAttribute('status') !== 'failed') {
throw new Exception(Exception::MIGRATION_IN_PROGRESS, 'Migration not failed yet');
}
$migration
->setAttribute('status', 'pending')
->setAttribute('dateUpdated', \time());
$publisherForMigrations->enqueue(new MigrationMessage(
project: $project,
migration: $migration,
platform: $platform,
));
$response->noContent();
}
}
@@ -0,0 +1,104 @@
<?php
namespace Appwrite\Platform\Modules\Migrations\Http\Migrations;
use Appwrite\Extend\Exception;
use Appwrite\SDK\AuthType;
use Appwrite\SDK\Method;
use Appwrite\SDK\Response as SDKResponse;
use Appwrite\Utopia\Database\Validator\Queries\Migrations;
use Appwrite\Utopia\Response;
use Utopia\Database\Database;
use Utopia\Database\Document;
use Utopia\Database\Exception\Order as OrderException;
use Utopia\Database\Exception\Query as QueryException;
use Utopia\Database\Query;
use Utopia\Database\Validator\Query\Cursor;
use Utopia\Platform\Action;
use Utopia\Platform\Scope\HTTP;
use Utopia\Validator\Boolean;
use Utopia\Validator\Text;
class XList extends Action
{
use HTTP;
public static function getName(): string
{
return 'listMigrations';
}
public function __construct()
{
$this
->setHttpMethod(Action::HTTP_REQUEST_METHOD_GET)
->setHttpPath('/v1/migrations')
->desc('List migrations')
->groups(['api', 'migrations'])
->label('scope', 'migrations.read')
->label('sdk', new Method(
namespace: 'migrations',
group: null,
name: 'list',
description: '/docs/references/migrations/list-migrations.md',
auth: [AuthType::ADMIN],
responses: [
new SDKResponse(
code: Response::STATUS_CODE_OK,
model: Response::MODEL_MIGRATION_LIST,
)
]
))
->param('queries', [], new Migrations(), 'Array of query strings generated using the Query class provided by the SDK. [Learn more about queries](https://appwrite.io/docs/databases#querying-documents). Maximum of ' . APP_LIMIT_ARRAY_PARAMS_SIZE . ' queries are allowed, each ' . APP_LIMIT_ARRAY_ELEMENT_SIZE . ' characters long. You may filter on the following attributes: ' . implode(', ', Migrations::ALLOWED_ATTRIBUTES), true)
->param('search', '', new Text(256), 'Search term to filter your list results. Max length: 256 chars.', true)
->param('total', true, new Boolean(true), 'When set to false, the total count returned will be 0 and will not be calculated.', true)
->inject('response')
->inject('dbForProject')
->callback($this->action(...));
}
public function action(array $queries, string $search, bool $includeTotal, Response $response, Database $dbForProject): void
{
try {
$queries = Query::parseQueries($queries);
} catch (QueryException $e) {
throw new Exception(Exception::GENERAL_QUERY_INVALID, $e->getMessage());
}
if (!empty($search)) {
$queries[] = Query::search('search', $search);
}
$cursor = Query::getCursorQueries($queries, false);
$cursor = \reset($cursor);
if ($cursor !== false) {
$validator = new Cursor();
if (!$validator->isValid($cursor)) {
throw new Exception(Exception::GENERAL_QUERY_INVALID, $validator->getDescription());
}
$migrationId = $cursor->getValue();
$cursorDocument = $dbForProject->getDocument('migrations', $migrationId);
if ($cursorDocument->isEmpty()) {
throw new Exception(Exception::GENERAL_CURSOR_NOT_FOUND, "Migration '{$migrationId}' for the 'cursor' value not found.");
}
$cursor->setValue($cursorDocument);
}
$filterQueries = Query::groupByType($queries)['filters'];
try {
$migrations = $dbForProject->find('migrations', $queries);
$total = $includeTotal ? $dbForProject->count('migrations', $filterQueries, APP_LIMIT_COUNT) : 0;
} catch (OrderException $e) {
throw new Exception(Exception::DATABASE_QUERY_ORDER_NULL, "The order attribute '{$e->getAttribute()}' had a null value. Cursor pagination requires all documents order attribute values are non-null.");
}
$response->dynamic(new Document([
'migrations' => $migrations,
'total' => $total,
]), Response::MODEL_MIGRATION_LIST);
}
}
@@ -0,0 +1,14 @@
<?php
namespace Appwrite\Platform\Modules\Migrations;
use Appwrite\Platform\Modules\Migrations\Services\Http;
use Utopia\Platform;
class Module extends Platform\Module
{
public function __construct()
{
$this->addService('http', new Http());
}
}
@@ -0,0 +1,59 @@
<?php
namespace Appwrite\Platform\Modules\Migrations\Services;
use Appwrite\Platform\Modules\Migrations\Http\Migrations\Appwrite\Create as CreateAppwriteMigration;
use Appwrite\Platform\Modules\Migrations\Http\Migrations\Appwrite\Report\Get as GetAppwriteReport;
use Appwrite\Platform\Modules\Migrations\Http\Migrations\CSV\Exports\Create as CreateCSVExport;
use Appwrite\Platform\Modules\Migrations\Http\Migrations\CSV\Imports\Create as CreateCSVImport;
use Appwrite\Platform\Modules\Migrations\Http\Migrations\Delete as DeleteMigration;
use Appwrite\Platform\Modules\Migrations\Http\Migrations\Firebase\Create as CreateFirebaseMigration;
use Appwrite\Platform\Modules\Migrations\Http\Migrations\Firebase\Report\Get as GetFirebaseReport;
use Appwrite\Platform\Modules\Migrations\Http\Migrations\Get as GetMigration;
use Appwrite\Platform\Modules\Migrations\Http\Migrations\JSON\Exports\Create as CreateJSONExport;
use Appwrite\Platform\Modules\Migrations\Http\Migrations\JSON\Imports\Create as CreateJSONImport;
use Appwrite\Platform\Modules\Migrations\Http\Migrations\NHost\Create as CreateNHostMigration;
use Appwrite\Platform\Modules\Migrations\Http\Migrations\NHost\Report\Get as GetNHostReport;
use Appwrite\Platform\Modules\Migrations\Http\Migrations\Supabase\Create as CreateSupabaseMigration;
use Appwrite\Platform\Modules\Migrations\Http\Migrations\Supabase\Report\Get as GetSupabaseReport;
use Appwrite\Platform\Modules\Migrations\Http\Migrations\Update as UpdateMigration;
use Appwrite\Platform\Modules\Migrations\Http\Migrations\XList as ListMigrations;
use Utopia\Platform\Service;
class Http extends Service
{
public function __construct()
{
$this->type = Service::TYPE_HTTP;
// Migrations
$this->addAction(ListMigrations::getName(), new ListMigrations());
$this->addAction(GetMigration::getName(), new GetMigration());
$this->addAction(UpdateMigration::getName(), new UpdateMigration());
$this->addAction(DeleteMigration::getName(), new DeleteMigration());
// Appwrite source
$this->addAction(CreateAppwriteMigration::getName(), new CreateAppwriteMigration());
$this->addAction(GetAppwriteReport::getName(), new GetAppwriteReport());
// Firebase source
$this->addAction(CreateFirebaseMigration::getName(), new CreateFirebaseMigration());
$this->addAction(GetFirebaseReport::getName(), new GetFirebaseReport());
// Supabase source
$this->addAction(CreateSupabaseMigration::getName(), new CreateSupabaseMigration());
$this->addAction(GetSupabaseReport::getName(), new GetSupabaseReport());
// NHost source
$this->addAction(CreateNHostMigration::getName(), new CreateNHostMigration());
$this->addAction(GetNHostReport::getName(), new GetNHostReport());
// CSV import / export
$this->addAction(CreateCSVImport::getName(), new CreateCSVImport());
$this->addAction(CreateCSVExport::getName(), new CreateCSVExport());
// JSON import / export
$this->addAction(CreateJSONImport::getName(), new CreateJSONImport());
$this->addAction(CreateJSONExport::getName(), new CreateJSONExport());
}
}
@@ -1,6 +1,6 @@
<?php
namespace Appwrite\Platform\Modules\Project\Http\Project\Keys\Standard;
namespace Appwrite\Platform\Modules\Project\Http\Project\Keys;
use Appwrite\Event\Event as QueueEvent;
use Appwrite\Extend\Exception;
@@ -30,17 +30,16 @@ class Create extends Base
public static function getName()
{
return 'createStandardProjectKey';
return 'createProjectKey';
}
public function __construct()
{
$this
->setHttpMethod(Action::HTTP_REQUEST_METHOD_POST)
->setHttpPath('/v1/project/keys/standard')
->httpAlias('/v1/project/keys')
->setHttpPath('/v1/project/keys')
->httpAlias('/v1/projects/:projectId/keys')
->desc('Create standard project key')
->desc('Create project key')
->groups(['api', 'project'])
->label('scope', 'keys.write')
->label('event', 'keys.[keyId].create')
@@ -49,9 +48,9 @@ class Create extends Base
->label('sdk', new Method(
namespace: 'project',
group: 'keys',
name: 'createStandardKey',
name: 'createKey',
description: <<<EOT
Create a new standard API key. It's recommended to have multiple API keys with strict scopes for separate functions within your project.
Create a new API key. It's recommended to have multiple API keys with strict scopes for separate functions within your project.
You can also create an ephemeral API key if you need a short-lived key instead.
EOT,
@@ -59,7 +59,7 @@ class Create extends Base
],
))
->param('scopes', [], new ArrayList(new WhiteList(array_keys(Config::getParam('projectScopes')), true), APP_LIMIT_ARRAY_PARAMS_SIZE), 'Key scopes list. Maximum of ' . APP_LIMIT_ARRAY_PARAMS_SIZE . ' scopes are allowed.', optional: false)
->param('duration', 900, new Range(1, 3600), 'Time in seconds before ephemeral key expires. Default duration is 900 seconds, and maximum is 3600 seconds.', true)
->param('duration', null, new Range(1, 3600), 'Time in seconds before ephemeral key expires. Maximum duration is 3600 seconds.', optional: false)
->inject('response')
->inject('queueForEvents')
->inject('project')
@@ -146,13 +146,14 @@ class Update extends Base
{
$providerId = static::getProviderId();
$oAuthProviders = $project->getAttribute('oAuthProviders', []);
$storedSecret = $this->decodeStoredSecret($project);
return new Document([
'$id' => $providerId,
'enabled' => $oAuthProviders[$providerId . 'Enabled'] ?? false,
static::getClientIdParamName() => $oAuthProviders[$providerId . 'Appid'] ?? '',
'keyId' => '',
'teamId' => '',
'keyId' => $storedSecret['keyID'] ?? '',
'teamId' => $storedSecret['teamID'] ?? '',
'p8File' => '',
]);
}
@@ -105,7 +105,7 @@ class Update extends Base
))
->param(static::getClientIdParamName(), null, new Nullable(new Text(256, 0)), static::getClientIdDescription(), optional: true)
->param(static::getClientSecretParamName(), null, new Nullable(new Text(512, 0)), static::getClientSecretDescription(), optional: true)
->param('endpoint', '', new Text(256, 1), 'Domain of Authentik instance. For example: example.authentik.com', optional: false)
->param('endpoint', null, new Nullable(new Text(256, 0)), 'Domain of Authentik instance. For example: example.authentik.com', optional: true)
->param('enabled', null, new Nullable(new Boolean()), 'OAuth2 sign-in method status. Set to true to enable new session creation. Setting to true will trigger end-to-end credentials validation, and will throw if the credentials are invalid.', true)
->inject('response')
->inject('dbForPlatform')
@@ -138,7 +138,7 @@ class Update extends Base
public function handle(
?string $clientId,
?string $clientSecret,
string $endpoint,
?string $endpoint,
?bool $enabled,
Response $response,
Database $dbForPlatform,
@@ -151,7 +151,7 @@ class Update extends Base
// The secret is stored as JSON `{"clientSecret": "...", "authentikDomain": "..."}`
// to match the shape Authentik's OAuth2 adapter expects (getAuthentikDomain()).
// The `endpoint` param is required on every call, so it's always written.
// The `endpoint` param is optional; if omitted, the existing stored endpoint is preserved.
// `clientSecret` is optional; if omitted, the existing stored secret is preserved.
$storedRaw = $project->getAttribute('oAuthProviders', [])[$providerId . 'Secret'] ?? '';
$existing = [];
@@ -160,7 +160,7 @@ class Update extends Base
}
$encodedSecret = \json_encode([
'clientSecret' => $clientSecret ?? ($existing['clientSecret'] ?? ''),
'authentikDomain' => $endpoint,
'authentikDomain' => $endpoint ?? ($existing['authentikDomain'] ?? ''),
]);
$project = $this->persistCredentials($project, $dbForPlatform, $authorization, $clientId, $encodedSecret, $enabled);
@@ -105,7 +105,7 @@ class Update extends Base
))
->param(static::getClientIdParamName(), null, new Nullable(new Text(256, 0)), static::getClientIdDescription(), optional: true)
->param(static::getClientSecretParamName(), null, new Nullable(new Text(512, 0)), static::getClientSecretDescription(), optional: true)
->param('endpoint', '', new Text(256, 1), 'Domain of FusionAuth instance. For example: example.fusionauth.io', optional: false)
->param('endpoint', null, new Nullable(new Text(256, 0)), 'Domain of FusionAuth instance. For example: example.fusionauth.io', optional: true)
->param('enabled', null, new Nullable(new Boolean()), 'OAuth2 sign-in method status. Set to true to enable new session creation. Setting to true will trigger end-to-end credentials validation, and will throw if the credentials are invalid.', true)
->inject('response')
->inject('dbForPlatform')
@@ -138,7 +138,7 @@ class Update extends Base
public function handle(
?string $clientId,
?string $clientSecret,
string $endpoint,
?string $endpoint,
?bool $enabled,
Response $response,
Database $dbForPlatform,
@@ -151,7 +151,7 @@ class Update extends Base
// The secret is stored as JSON `{"clientSecret": "...", "fusionAuthDomain": "..."}`
// to match the shape FusionAuth's OAuth2 adapter expects (getFusionAuthDomain()).
// The `endpoint` param is required on every call, so it's always written.
// The `endpoint` param is optional; if omitted, the existing stored endpoint is preserved.
// `clientSecret` is optional; if omitted, the existing stored secret is preserved.
$storedRaw = $project->getAttribute('oAuthProviders', [])[$providerId . 'Secret'] ?? '';
$existing = [];
@@ -160,7 +160,7 @@ class Update extends Base
}
$encodedSecret = \json_encode([
'clientSecret' => $clientSecret ?? ($existing['clientSecret'] ?? ''),
'fusionAuthDomain' => $endpoint,
'fusionAuthDomain' => $endpoint ?? ($existing['fusionAuthDomain'] ?? ''),
]);
$project = $this->persistCredentials($project, $dbForPlatform, $authorization, $clientId, $encodedSecret, $enabled);
@@ -35,7 +35,7 @@ class Update extends Base
public static function getClientIdName(): string
{
return 'OAuth 2 app Client ID, or App ID';
return 'OAuth2 app Client ID, or App ID';
}
public static function getClientIdExample(): string

Some files were not shown because too many files have changed in this diff Show More