Compare commits

..
Author SHA1 Message Date
Matej Bačo c1f8fa93d2 Merge branch 'chore-speedup-tests' of https://github.com/appwrite/appwrite into chore-speedup-tests 2026-03-16 09:25:58 +01:00
loks0nandClaude Sonnet 4.6 f9ecd28dd0 Revert composer.lock to 1.8.x
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-15 16:57:39 +00:00
Matej Bačoandloks0n 7e6db9a99b Fix tests missing lock files 2026-03-15 16:39:12 +00:00
Matej Bačoandloks0n c618372d37 linter fix 2026-03-15 16:39:12 +00:00
Matej Bačoandloks0n 975a38bf43 Speed-up deployments in tests 2026-03-15 16:39:12 +00:00
Luke B. SilverandGitHub 6b21968413 Merge pull request #11549 from appwrite/test/split-databases
Split TablesDB e2e tests into a separate CI job
2026-03-15 16:36:14 +00:00
loks0nandClaude Opus 4.6 286e0e5802 Fix import ordering in TablesDB test files
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-15 16:25:05 +00:00
loks0nandClaude Opus 4.6 ac05c9e9ce Split TablesDB e2e tests into a separate CI job
Move all TablesDB* test files from tests/e2e/Services/Databases/ to a
new tests/e2e/Services/TablesDB/ directory, updating namespaces and
adding explicit imports for shared base traits. Add TablesDB as a
separate service in the CI matrix so /v1/databases and /v1/tables tests
run as independent parallel jobs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-15 16:22:01 +00:00
Luke B. SilverandGitHub 153e660274 Merge pull request #11543 from appwrite/fix/ci-before-main-2
Fix devalue vulnerability in astro test sites
2026-03-14 20:13:46 +00:00
loks0nandClaude Opus 4.6 ad97feaf6e Optimize CI matrix job to fetch composer.lock directly
Replace paginated listFiles API call with targeted getContent calls
to avoid timeouts on large PRs with thousands of changed files.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-14 19:25:40 +00:00
loks0nandClaude Opus 4.6 ef618b831d Fix devalue vulnerability by updating from 5.6.3 to 5.6.4
Resolves GHSA-cfw5-2vxh-hr84 (CVSS 6.3) and GHSA-mwv9-gp5h-frr4
(CVSS 2.7) in astro test resource sites.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-14 19:12:10 +00:00
Luke B. SilverandGitHub 021c5fcad6 Merge pull request #11449 from appwrite/refactor/usage-publishing
Refactor usage metrics to stateless publisher pattern
2026-03-13 16:36:04 +00:00
Matej Bačo b028b3142b Fix tests missing lock files 2026-03-13 17:13:30 +01:00
Matej Bačo f890762d14 linter fix 2026-03-13 17:08:18 +01:00
Matej Bačo 3e42b15659 Speed-up deployments in tests 2026-03-13 17:04:49 +01:00
Luke B. SilverandGitHub 4309bd075e Merge pull request #11538 from appwrite/chore/pin-servers
Pin utopia-php/servers to 0.2.5
2026-03-13 14:00:43 +00:00
loks0nandClaude Opus 4.6 236dceae2c Pin utopia-php/servers to 0.2.5
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 13:59:06 +00:00
Matej BačoandGitHub 884b8f10bb Merge pull request #11536 from appwrite/fix-staging-project-creation
Fix: staging env
2026-03-13 12:57:36 +01:00
Matej Bačo 530d681c48 Fix staging env 2026-03-13 11:25:17 +01:00
loks0nandClaude Opus 4.6 0295e27b5d Fix build usage metrics to use actual memory/cpus and fix escapeshellarg double-quoting
Use $memory (which includes minMemory floor) and $cpus instead of raw
spec values in MB-seconds metrics, fixing underreporting for sites and
frameworks bumped to the minimum memory. Also remove redundant double
quotes around escapeshellarg() calls in mv command.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 15:42:41 +00:00
loks0nandClaude Opus 4.6 a804cba999 Refactor usage metrics to stateless publisher pattern
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 15:36:16 +00:00
Luke B. SilverandGitHub d8f733787f Merge pull request #11531 from appwrite/ci/single-pr-workflow
Consolidate PR workflows into single CI workflow
2026-03-12 14:17:23 +00:00
loks0nandClaude Opus 4.6 1abbca9318 Split SARIF uploads with unique categories to fix codeql-action error
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 13:42:28 +00:00
loks0nandClaude Opus 4.6 e99f682cd6 Update trivy-action to v0.35.0
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 13:27:38 +00:00
loks0nandClaude Opus 4.6 26326d05e9 Guard SARIF upload against missing files from failed Trivy scans
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 13:04:33 +00:00
loks0nandClaude Opus 4.6 e67ed2660a Add actions: read permission for osv-scanner reusable workflow
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 13:00:44 +00:00
loks0nandClaude Opus 4.6 8d0a4d7f92 Consolidate remaining PR workflows and simplify Trivy scan
- Move check-dependencies into ci.yml as Checks / Dependencies
  (upgrade to osv-scanner-reusable-pr.yml@v2.3.3, drop merge_group)
- Move pr-scan into ci.yml as Checks / Image
  (upgrade Trivy to 0.33.1, use SARIF + upload-sarif instead of
  custom PR comment logic)
- Rename Setup job to Build
- Fix format job git checkout HEAD^2 to only run on pull_request
- Rename PHPStan step correctly (was mislabeled CodeQL)
- Add Docker Hub login to benchmark job
- Remove no-op pull_request trigger from ai-moderator

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 12:56:41 +00:00
loks0nandClaude Opus 4.6 aecca2f503 Consolidate PR workflows into single CI workflow
Merge linter, static-analysis, tests, and benchmark workflows into ci.yml
with structured job naming (Checks / Format, Tests / E2E / ..., etc.).
Shared Docker image build between tests and benchmark. Update actions to
latest versions.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 12:39:14 +00:00
Luke B. SilverandGitHub 7e4ec9eae0 Merge pull request #11529 from appwrite/ci/focused-testing
ci: focused testing
2026-03-12 11:38:27 +00:00
loks0nandClaude Opus 4.6 edd948557e Refactor matrix job to use GitHub API and clean up test config
Replace shell-based database change detection with github-script using
the GitHub API, eliminating the need for a full checkout. Restructure
matrix generation with guard clauses and no mutation. Remove ciIgnore
exclude group from test command.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 11:01:36 +00:00
loks0nandClaude Opus 4.6 09317f290a Clean up database env setup and improve matrix naming
Hardcode lowercase env vars per database branch instead of using tr.
Use proper casing for database matrix values (MongoDB, MariaDB, PostgreSQL).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 08:56:47 +00:00
loks0nandClaude Opus 4.6 e5f0c2df12 Consolidate CI test matrix with dynamic database and mode dimensions
Merge 6 E2E jobs into 3 by combining dedicated/shared mode variants
into a single matrix dimension. Database adapters and table modes
expand dynamically based on whether utopia-php/database changed.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 08:56:47 +00:00
Eldad A. FuxandGitHub 2afec80a25 Merge pull request #11530 from appwrite/fix-allow-null-branch
Allow null branch
2026-03-12 09:29:25 +01:00
eldadfux 20c65bac37 Allow null branch 2026-03-12 09:02:16 +01:00
Luke B. SilverandGitHub 2c6f37ecad Merge pull request #11526 from appwrite/perf/test-image-pulls
perf: various ci fixes and optimisations
2026-03-12 00:22:06 +00:00
loks0nandClaude Opus 4.6 f09b9a994e Revert .env defaults back to MongoDB
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 23:59:31 +00:00
loks0nandClaude Opus 4.6 9f4ba3a4a2 Reduce php-retry wait time from 300s to 60s
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 23:47:03 +00:00
loks0nandClaude Opus 4.6 8cb36835cc Restore php-retry action for flaky test resilience
Re-add itznotabug/php-retry@v3 wrapping all test steps with
max_attempts: 2 and retry_wait_seconds: 300. Also restore
pull-requests: write permission needed by the action.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 23:41:26 +00:00
loks0nandClaude Opus 4.6 89419344c2 Restore MariaDB defaults, fix remaining timeouts, filter listener span logs
- Change .env defaults back from MongoDB to MariaDB
- Bump all remaining "Load and Start Appwrite" timeouts from 3 to 5 minutes
- Filter listener.* span logs to only export on error

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 23:20:51 +00:00
loks0nandClaude Opus 4.6 7dfe44cb36 Run abuse-enabled tests across entire test suite, not just Projects
The abuseEnabled jobs previously only ran tests in Services/Projects,
missing the Account abuse test and any future abuseEnabled tests in
other services. Also rename jobs to "Abuse" for consistency.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 22:44:42 +00:00
loks0nandClaude Opus 4.6 4cbe50193a Increase Load and Start Appwrite timeout to 5 minutes
The docker compose pull step alone can take over 2.5 minutes on CI
(e.g. openruntimes-executor, traefik), leaving no time for
docker compose up --wait within 3 minutes.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 22:24:52 +00:00
loks0nandClaude Opus 4.6 16929bc420 Remove remaining --debug flags from test commands
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 22:00:00 +00:00
loks0nandClaude Opus 4.6 fd28ad8a66 Remove --debug flag from test commands for quieter CI output
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 21:59:33 +00:00
loks0nandClaude Opus 4.6 38798c8993 Add healthchecks for MariaDB, PostgreSQL, and Redis
Add proper healthchecks to infrastructure services and use
condition: service_healthy for redis in appwrite's depends_on
so it waits for Redis to be ready before starting.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 21:58:30 +00:00
loks0nandClaude Opus 4.6 33ce469ab0 Add Docker healthcheck and use --wait instead of polling loop
Replace the manual shell polling loop (until doctor > /dev/null) with a
proper Docker healthcheck on the appwrite service and `docker compose up
--wait`, which blocks until healthchecks pass. Also reverts the timeout
back to 3 minutes now that image pulls are cached.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 21:44:15 +00:00
loks0nandClaude Opus 4.6 0bbcc3f570 Move dev tools to docker-compose.override.yml
Move adminer, redis-insight, mongo-express, and graphql-explorer to an
override file. CI sets COMPOSE_FILE=docker-compose.yml explicitly so
these are excluded from test runs, reducing the number of images to
pull from 14 to 10. Local docker compose auto-loads both files so dev
workflow is unchanged.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 21:06:44 +00:00
loks0nandClaude Opus 4.6 ecca0d8036 Increase Load and Start Appwrite timeout from 3 to 10 minutes
The 3-minute timeout was too tight with the added docker compose pull
step for downloading third-party images.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 20:48:39 +00:00
loks0nandClaude Opus 4.6 534dc55f17 Remove unnecessary abuse enabled from screenshot tests
The screenshot tests have no abuse-related code. Abuse was only enabled
on these jobs as a side effect of the original sed command applying to
all jobs below e2e_service_test.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 20:35:42 +00:00
loks0nandClaude Opus 4.6 d30df31b82 Move compose-time env vars to docker compose up step
_APP_DATABASE_SHARED_TABLES and _APP_DATABASE_SHARED_TABLES_V1 are
read by the server at boot time via System::getEnv(), not by the test
runner. Passing them via docker compose exec -e had no effect on the
already-running Swoole server. Move them to the Load and Start
Appwrite step so they're set at docker compose up time.

Keep _APP_E2E_RESPONSE_FORMAT on exec since it's read by the test
runner process (tests/e2e/Scopes/Scope.php).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 20:25:47 +00:00
loks0nandClaude Opus 4.6 dc9a1c03d1 Replace shell exports with GitHub Actions env fields
Use step-level env: fields and GHA expressions for conditional values
instead of shell export statements and if/elif blocks.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 20:20:23 +00:00
loks0nandClaude Opus 4.6 3613a645d0 Remove php-retry action and run test commands directly
Replace itznotabug/php-retry with native run steps and timeout-minutes.
Also remove pull-requests: write permission that was only needed by
php-retry to post PR comments.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 20:19:04 +00:00
loks0nandClaude Opus 4.6 6aac7ea6ad Update GitHub Actions to latest versions
- actions/cache: v4 → v5
- docker/setup-buildx-action: v3 → v4

Fixes Node.js 20 deprecation warnings.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 20:17:03 +00:00
loks0nandClaude Opus 4.6 d22642590f Add build directive to all appwrite-dev services for --ignore-buildable
Add x-build YAML anchor and apply it to all services using the
appwrite-dev image so docker compose pull --ignore-buildable correctly
skips them instead of trying to pull from Docker Hub.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 20:11:59 +00:00
loks0nandClaude Opus 4.6 0b416e4475 Skip locally-built images during docker compose pull
Add --ignore-buildable flag so docker compose pull skips images with
build directives (appwrite-dev) instead of trying to pull them from
Docker Hub.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 20:01:25 +00:00
loks0nandClaude Opus 4.6 4e1b710503 Fix Docker Hub login credentials to match repo config
Use `vars.DOCKERHUB_USERNAME` and `secrets.DOCKERHUB_TOKEN` to match
the existing publish and release workflows.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 19:59:23 +00:00
loks0nandClaude Opus 4.6 e557a5fc6e Add Docker Hub login and parallel image pulls to CI
Login to Docker Hub in all test jobs to avoid rate limits, add
`docker compose pull --quiet` to parallelize image downloads before
`docker compose up`, and replace sed-based .env overrides with native
GitHub Actions env fields.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 19:41:49 +00:00
HemachandarandGitHub 7d15d7e1f5 Rename time-travel task (#11523)
* Rename time-travel task

* rename
2026-03-11 16:15:50 +05:30
314 changed files with 9021 additions and 25392 deletions
+4 -13
View File
@@ -39,23 +39,14 @@ _APP_REDIS_HOST=redis
_APP_REDIS_PORT=6379
_APP_REDIS_PASS=
_APP_REDIS_USER=
COMPOSE_PROFILES=mariadb,mongodb,postgresql
_APP_DB_ADAPTER=mariadb
_APP_DB_HOST=mariadb
_APP_DB_PORT=3306
COMPOSE_PROFILES=mongodb
_APP_DB_ADAPTER=mongodb
_APP_DB_HOST=mongodb
_APP_DB_PORT=27017
_APP_DB_SCHEMA=appwrite
_APP_DB_USER=user
_APP_DB_PASS=password
_APP_DB_ROOT_PASS=rootsecretpassword
_APP_DB_ADAPTER_DOCUMENTSDB=mongodb
_APP_DB_HOST_DOCUMENTSDB=mongodb
_APP_DB_PORT_DOCUMENTSDB=27017
_APP_DB_ADAPTER_VECTORSDB=postgresql
_APP_DB_HOST_VECTORSDB=postgresql
_APP_DB_PORT_VECTORSDB=5432
_APP_EMBEDDING_MODELS=embeddinggemma
_APP_EMBEDDING_ENDPOINT='http://ollama:11434/api/embed'
_APP_EMBEDDING_TIMEOUT=30000
_APP_STORAGE_DEVICE=Local
_APP_STORAGE_S3_ACCESS_KEY=
_APP_STORAGE_S3_SECRET=
-2
View File
@@ -5,8 +5,6 @@ on:
types: [opened, edited]
issue_comment:
types: [created, edited]
pull_request:
types: [opened, edited]
pull_request_review:
types: [submitted, edited]
pull_request_review_comment:
-123
View File
@@ -1,123 +0,0 @@
name: Benchmark
concurrency:
group: '${{ github.workflow }}-${{ github.ref }}'
cancel-in-progress: true
env:
COMPOSE_FILE: docker-compose.yml
IMAGE: appwrite-dev
CACHE_KEY: 'appwrite-dev-${{ github.event.pull_request.head.sha }}'
'on':
- pull_request
jobs:
setup:
name: Setup & Build Appwrite Image
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v6
with:
submodules: recursive
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Build Appwrite
uses: docker/build-push-action@v6
with:
context: .
push: false
tags: '${{ env.IMAGE }}'
load: true
cache-from: type=gha
cache-to: 'type=gha,mode=max'
outputs: 'type=docker,dest=/tmp/${{ env.IMAGE }}.tar'
target: development
build-args: |
DEBUG=false
TESTING=true
VERSION=dev
- name: Cache Docker Image
uses: actions/cache@v4
with:
key: '${{ env.CACHE_KEY }}'
path: '/tmp/${{ env.IMAGE }}.tar'
benchmarking:
name: Benchmark
runs-on: ubuntu-latest
needs: setup
permissions:
pull-requests: write
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Load Cache
uses: actions/cache@v4
with:
key: '${{ env.CACHE_KEY }}'
path: '/tmp/${{ env.IMAGE }}.tar'
fail-on-cache-miss: true
- name: Load and Start Appwrite
run: |
sed -i 's/traefik/localhost/g' .env
docker load --input /tmp/${{ env.IMAGE }}.tar
docker compose up -d
sleep 10
- name: Install Oha
run: |
echo "deb [signed-by=/usr/share/keyrings/azlux-archive-keyring.gpg] http://packages.azlux.fr/debian/ stable main" | sudo tee /etc/apt/sources.list.d/azlux.list
sudo wget -O /usr/share/keyrings/azlux-archive-keyring.gpg https://azlux.fr/repo.gpg
sudo apt update
sudo apt install oha
oha --version
- name: Benchmark PR
run: 'oha -z 180s http://localhost/v1/health/version --output-format json > benchmark.json'
- name: Cleaning
run: docker compose down -v
- name: Installing latest version
run: |
rm docker-compose.yml
rm .env
curl https://appwrite.io/install/compose -o docker-compose.yml
curl https://appwrite.io/install/env -o .env
sed -i 's/_APP_OPTIONS_ABUSE=enabled/_APP_OPTIONS_ABUSE=disabled/g' .env
docker compose up -d
sleep 10
- name: Benchmark Latest
run: oha -z 180s http://localhost/v1/health/version --output-format json > benchmark-latest.json
- name: Prepare comment
run: |
echo '## :sparkles: Benchmark results' > benchmark.txt
echo ' ' >> benchmark.txt
echo "- Requests per second: $(jq -r '.summary.requestsPerSec|tonumber?|floor|tostring|[while(length>0;.[:-3])|.[-3:]]|reverse|join(",")' benchmark.json)" >> benchmark.txt
echo "- Requests with 200 status code: $(jq -r '.statusCodeDistribution."200"|tostring|[while(length>0;.[:-3])|.[-3:]]|reverse|join(",")' benchmark.json)" >> benchmark.txt
echo "- P99 latency: $(jq -r '.latencyPercentiles.p99' benchmark.json )" >> benchmark.txt
echo " " >> benchmark.txt
echo " " >> benchmark.txt
echo "## :zap: Benchmark Comparison" >> benchmark.txt
echo " " >> benchmark.txt
echo "| Metric | This PR | Latest version | " >> benchmark.txt
echo "| --- | --- | --- | " >> benchmark.txt
echo "| RPS | $(jq -r '.summary.requestsPerSec|tonumber?|floor|tostring|[while(length>0;.[:-3])|.[-3:]]|reverse|join(",")' benchmark.json) | $(jq -r '.summary.requestsPerSec|tonumber|floor|tostring|[while(length>0;.[:-3])|.[-3:]]|reverse|join(",")' benchmark-latest.json) | " >> benchmark.txt
echo "| 200 | $(jq -r '.statusCodeDistribution."200"|tostring|[while(length>0;.[:-3])|.[-3:]]|reverse|join(",")' benchmark.json) | $(jq -r '.statusCodeDistribution."200"|tostring|[while(length>0;.[:-3])|.[-3:]]|reverse|join(",")' benchmark-latest.json) | " >> benchmark.txt
echo "| P99 | $(jq -r '.latencyPercentiles.p99' benchmark.json ) | $(jq -r '.latencyPercentiles.p99' benchmark-latest.json ) | " >> benchmark.txt
- name: Save results
uses: actions/upload-artifact@v6
if: '${{ !cancelled() }}'
with:
name: benchmark.json
path: benchmark.json
retention-days: 7
- name: Find Comment
if: github.event.pull_request.head.repo.full_name == github.repository
uses: peter-evans/find-comment@v3
id: fc
with:
issue-number: '${{ github.event.pull_request.number }}'
comment-author: 'github-actions[bot]'
body-includes: Benchmark results
- name: Comment on PR
if: github.event.pull_request.head.repo.full_name == github.repository
uses: peter-evans/create-or-update-comment@v4
with:
comment-id: '${{ steps.fc.outputs.comment-id }}'
issue-number: '${{ github.event.pull_request.number }}'
body-path: benchmark.txt
edit-mode: replace
-19
View File
@@ -1,19 +0,0 @@
name: Check dependencies
# Adapted from https://google.github.io/osv-scanner/github-action/#scan-on-pull-request
on:
pull_request:
branches: [main, 1.*.x]
merge_group:
branches: [main, 1.*.x]
permissions:
# Require writing security events to upload SARIF file to security tab
security-events: write
# Only need to read contents
contents: read
jobs:
scan-pr:
uses: "google/osv-scanner-action/.github/workflows/osv-scanner-reusable.yml@v1.7.1"
+660
View File
@@ -0,0 +1,660 @@
name: CI
concurrency:
group: ci-${{ github.ref }}
cancel-in-progress: true
env:
COMPOSE_FILE: docker-compose.yml
IMAGE: appwrite-dev
CACHE_KEY: appwrite-dev-${{ github.event.pull_request.head.sha }}
on:
pull_request:
workflow_dispatch:
inputs:
response_format:
description: 'Response format version to test (e.g., 1.5.0, 1.4.0)'
required: false
type: string
default: ''
jobs:
dependencies:
name: Checks / Dependencies
if: github.event_name == 'pull_request'
permissions:
actions: read
security-events: write
contents: read
uses: "google/osv-scanner-action/.github/workflows/osv-scanner-reusable-pr.yml@v2.3.3"
security:
name: Checks / Image
if: github.event_name == 'pull_request'
runs-on: ubuntu-latest
permissions:
contents: read
security-events: write
steps:
- name: Check out code
uses: actions/checkout@v6
with:
fetch-depth: 0
submodules: 'recursive'
- name: Build the Docker image
uses: docker/build-push-action@v6
with:
context: .
push: false
load: true
tags: pr_image:${{ github.sha }}
target: production
- name: Run Trivy vulnerability scanner on image
uses: aquasecurity/trivy-action@0.35.0
with:
image-ref: 'pr_image:${{ github.sha }}'
format: 'sarif'
output: 'trivy-image-results.sarif'
severity: 'CRITICAL,HIGH'
- name: Run Trivy vulnerability scanner on source code
uses: aquasecurity/trivy-action@0.35.0
with:
scan-type: 'fs'
scan-ref: '.'
format: 'sarif'
output: 'trivy-fs-results.sarif'
severity: 'CRITICAL,HIGH'
skip-setup-trivy: true
- name: Upload image scan results
uses: github/codeql-action/upload-sarif@v4
if: always() && hashFiles('trivy-image-results.sarif') != ''
with:
sarif_file: 'trivy-image-results.sarif'
category: 'trivy-image'
- name: Upload source code scan results
uses: github/codeql-action/upload-sarif@v4
if: always() && hashFiles('trivy-fs-results.sarif') != ''
with:
sarif_file: 'trivy-fs-results.sarif'
category: 'trivy-source'
format:
name: Checks / Format
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v6
with:
fetch-depth: 2
- run: git checkout HEAD^2
if: github.event_name == 'pull_request'
- name: Validate composer.json and composer.lock
run: |
docker run --rm -v $PWD:/app composer:2.8 sh -c \
"composer validate"
- name: Run Linter
run: |
docker run --rm -v $PWD:/app composer:2.8 sh -c \
"composer install --profile --ignore-platform-reqs && composer lint"
analyze:
name: Checks / Analyze
runs-on: ubuntu-latest
steps:
- name: Check out the repo
uses: actions/checkout@v6
- name: Run PHPStan
run: |
docker run --rm -v $PWD:/app composer:2.8 sh -c \
"composer install --profile --ignore-platform-reqs && composer check"
- name: Run Locale check
run: |
docker run --rm -v $PWD:/app node:24-alpine sh -c \
"cd /app/.github/workflows/static-analysis/locale && node index.js"
matrix:
name: Tests / Matrix
runs-on: ubuntu-latest
outputs:
databases: ${{ steps.generate.outputs.databases }}
modes: ${{ steps.generate.outputs.modes }}
steps:
- name: Generate matrix
id: generate
uses: actions/github-script@v8
with:
script: |
const allDatabases = ['MariaDB', 'PostgreSQL', 'MongoDB'];
const allModes = ['dedicated', 'shared_v1', 'shared_v2'];
const defaultDatabases = ['MongoDB'];
const defaultModes = ['dedicated'];
const pr = context.payload.pull_request;
if (!pr) {
core.setOutput('databases', JSON.stringify(allDatabases));
core.setOutput('modes', JSON.stringify(allModes));
return;
}
const getContent = (ref) => github.rest.repos.getContent({
owner: context.repo.owner,
repo: context.repo.repo,
path: 'composer.lock',
ref,
});
const getDbVersion = (lock) => lock.packages?.find(p => p.name === 'utopia-php/database')?.version;
const [{ data: base }, { data: head }] = await Promise.all([
getContent(pr.base.sha),
getContent(pr.head.sha),
]);
const decode = (content) => JSON.parse(Buffer.from(content, 'base64').toString());
const databaseChanged = getDbVersion(decode(base.content)) !== getDbVersion(decode(head.content));
core.setOutput('databases', JSON.stringify(databaseChanged ? allDatabases : defaultDatabases));
core.setOutput('modes', JSON.stringify(databaseChanged ? allModes : defaultModes));
build:
name: Build
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v6
with:
submodules: recursive
- name: Login to Docker Hub
uses: docker/login-action@v4
with:
username: ${{ vars.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v4
- name: Build Appwrite
uses: docker/build-push-action@v6
with:
context: .
push: false
tags: ${{ env.IMAGE }}
load: true
cache-from: type=gha
cache-to: type=gha,mode=max
outputs: type=docker,dest=/tmp/${{ env.IMAGE }}.tar
target: development
build-args: |
DEBUG=false
TESTING=true
VERSION=dev
- name: Cache Docker Image
uses: actions/cache@v5
with:
key: ${{ env.CACHE_KEY }}
path: /tmp/${{ env.IMAGE }}.tar
unit:
name: Tests / Unit
runs-on: ubuntu-latest
needs: build
permissions:
contents: read
pull-requests: write
steps:
- name: checkout
uses: actions/checkout@v6
- name: Load Cache
uses: actions/cache@v5
with:
key: ${{ env.CACHE_KEY }}
path: /tmp/${{ env.IMAGE }}.tar
fail-on-cache-miss: true
- name: Login to Docker Hub
uses: docker/login-action@v4
with:
username: ${{ vars.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Load and Start Appwrite
timeout-minutes: 5
run: |
docker load --input /tmp/${{ env.IMAGE }}.tar
docker compose pull --quiet --ignore-buildable
docker compose up -d --quiet-pull --wait
- name: Environment Variables
run: docker compose exec -T appwrite vars
- name: Run Unit Tests
uses: itznotabug/php-retry@v3
with:
max_attempts: 2
retry_wait_seconds: 60
timeout_minutes: 15
job_id: ${{ job.check_run_id }}
github_token: ${{ secrets.GITHUB_TOKEN }}
test_dir: tests/unit
command: >-
docker compose exec
-e _APP_E2E_RESPONSE_FORMAT="${{ github.event.inputs.response_format }}"
appwrite test /usr/src/code/tests/unit
e2e_general:
name: Tests / E2E / General
runs-on: ubuntu-latest
needs: build
permissions:
contents: read
pull-requests: write
steps:
- name: checkout
uses: actions/checkout@v6
- name: Load Cache
uses: actions/cache@v5
with:
key: ${{ env.CACHE_KEY }}
path: /tmp/${{ env.IMAGE }}.tar
fail-on-cache-miss: true
- name: Login to Docker Hub
uses: docker/login-action@v4
with:
username: ${{ vars.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Load and Start Appwrite
timeout-minutes: 5
run: |
docker load --input /tmp/${{ env.IMAGE }}.tar
docker compose pull --quiet --ignore-buildable
docker compose up -d --quiet-pull --wait
- name: Wait for Open Runtimes
timeout-minutes: 3
run: |
while ! docker compose logs openruntimes-executor | grep -q "Executor is ready."; do
echo "Waiting for Executor to come online"
sleep 1
done
- name: Run General Tests
uses: itznotabug/php-retry@v3
with:
max_attempts: 2
retry_wait_seconds: 60
timeout_minutes: 15
job_id: ${{ job.check_run_id }}
github_token: ${{ secrets.GITHUB_TOKEN }}
test_dir: tests/e2e/General
command: >-
docker compose exec -T
-e _APP_E2E_RESPONSE_FORMAT="${{ github.event.inputs.response_format }}"
appwrite test /usr/src/code/tests/e2e/General
- name: Failure Logs
if: failure()
run: |
echo "=== Appwrite Logs ==="
docker compose logs
e2e_service:
name: Tests / E2E / ${{ matrix.database }} (${{ matrix.mode }}) / ${{ matrix.service }}
runs-on: ubuntu-latest
needs: [build, matrix]
permissions:
contents: read
pull-requests: write
strategy:
fail-fast: false
matrix:
database: ${{ fromJSON(needs.matrix.outputs.databases) }}
mode: ${{ fromJSON(needs.matrix.outputs.modes) }}
service: [
Account,
Avatars,
Console,
Databases,
TablesDB,
Functions,
FunctionsSchedule,
GraphQL,
Health,
Locale,
Projects,
Realtime,
Sites,
Proxy,
Storage,
Tokens,
Teams,
Users,
Webhooks,
VCS,
Messaging,
Migrations
]
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Load Cache
uses: actions/cache@v5
with:
key: ${{ env.CACHE_KEY }}
path: /tmp/${{ env.IMAGE }}.tar
fail-on-cache-miss: true
- name: Set database environment
run: |
if [ "${{ matrix.database }}" = "MariaDB" ]; then
echo "COMPOSE_PROFILES=mariadb" >> $GITHUB_ENV
echo "_APP_DB_ADAPTER=mariadb" >> $GITHUB_ENV
echo "_APP_DB_HOST=mariadb" >> $GITHUB_ENV
echo "_APP_DB_PORT=3306" >> $GITHUB_ENV
elif [ "${{ matrix.database }}" = "MongoDB" ]; then
echo "COMPOSE_PROFILES=mongodb" >> $GITHUB_ENV
echo "_APP_DB_ADAPTER=mongodb" >> $GITHUB_ENV
echo "_APP_DB_HOST=mongodb" >> $GITHUB_ENV
echo "_APP_DB_PORT=27017" >> $GITHUB_ENV
elif [ "${{ matrix.database }}" = "PostgreSQL" ]; then
echo "COMPOSE_PROFILES=postgresql" >> $GITHUB_ENV
echo "_APP_DB_ADAPTER=postgresql" >> $GITHUB_ENV
echo "_APP_DB_HOST=postgresql" >> $GITHUB_ENV
echo "_APP_DB_PORT=5432" >> $GITHUB_ENV
fi
- name: Login to Docker Hub
uses: docker/login-action@v4
with:
username: ${{ vars.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Load and Start Appwrite
timeout-minutes: 5
env:
_APP_BROWSER_HOST: http://invalid-browser/v1
_APP_DATABASE_SHARED_TABLES: ${{ matrix.mode != 'dedicated' && 'database_db_main' || '' }}
_APP_DATABASE_SHARED_TABLES_V1: ${{ matrix.mode == 'shared_v1' && 'database_db_main' || '' }}
run: |
docker load --input /tmp/${{ env.IMAGE }}.tar
docker compose pull --quiet --ignore-buildable
docker compose up -d --quiet-pull --wait
- name: Wait for Open Runtimes
timeout-minutes: 3
run: |
while ! docker compose logs openruntimes-executor | grep -q "Executor is ready."; do
echo "Waiting for Executor to come online"
sleep 1
done
- name: Run tests
uses: itznotabug/php-retry@v3
with:
max_attempts: 2
retry_wait_seconds: 60
timeout_minutes: 20
job_id: ${{ job.check_run_id }}
github_token: ${{ secrets.GITHUB_TOKEN }}
test_dir: tests/e2e/Services/${{ matrix.service }}
command: |
SERVICE_PATH="/usr/src/code/tests/e2e/Services/${{ matrix.service }}"
# Services that rely on sequential test method execution (shared static state)
FUNCTIONAL_FLAG="--functional"
case "${{ matrix.service }}" in
Databases|TablesDB|Functions|Realtime) FUNCTIONAL_FLAG="" ;;
esac
docker compose exec -T \
-e _APP_E2E_RESPONSE_FORMAT="${{ github.event.inputs.response_format }}" \
appwrite vendor/bin/paratest --processes $(nproc) $FUNCTIONAL_FLAG "$SERVICE_PATH" --exclude-group abuseEnabled --exclude-group screenshots --log-junit tests/e2e/Services/${{ matrix.service }}/junit.xml
- name: Failure Logs
if: failure()
run: |
echo "=== Appwrite Logs ==="
docker compose logs
e2e_abuse:
name: Tests / E2E / Abuse (${{ matrix.mode }})
runs-on: ubuntu-latest
needs: [build, matrix]
permissions:
contents: read
pull-requests: write
strategy:
fail-fast: false
matrix:
mode: ${{ fromJSON(needs.matrix.outputs.modes) }}
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Load Cache
uses: actions/cache@v5
with:
key: ${{ env.CACHE_KEY }}
path: /tmp/${{ env.IMAGE }}.tar
fail-on-cache-miss: true
- name: Login to Docker Hub
uses: docker/login-action@v4
with:
username: ${{ vars.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Load and Start Appwrite
timeout-minutes: 5
env:
_APP_OPTIONS_ABUSE: enabled
_APP_DATABASE_SHARED_TABLES: ${{ matrix.mode != 'dedicated' && 'database_db_main' || '' }}
_APP_DATABASE_SHARED_TABLES_V1: ${{ matrix.mode == 'shared_v1' && 'database_db_main' || '' }}
run: |
docker load --input /tmp/${{ env.IMAGE }}.tar
docker compose pull --quiet --ignore-buildable
docker compose up -d --quiet-pull --wait
- name: Run tests
uses: itznotabug/php-retry@v3
with:
max_attempts: 2
retry_wait_seconds: 60
timeout_minutes: 15
job_id: ${{ job.check_run_id }}
github_token: ${{ secrets.GITHUB_TOKEN }}
test_dir: tests/e2e
command: >-
docker compose exec -T
-e _APP_E2E_RESPONSE_FORMAT="${{ github.event.inputs.response_format }}"
appwrite test /usr/src/code/tests/e2e --group=abuseEnabled
- name: Failure Logs
if: failure()
run: |
echo "=== Appwrite Logs ==="
docker compose logs
e2e_screenshots:
name: Tests / E2E / Screenshots (${{ matrix.mode }})
runs-on: ubuntu-latest
needs: [build, matrix]
permissions:
contents: read
pull-requests: write
strategy:
fail-fast: false
matrix:
mode: ${{ fromJSON(needs.matrix.outputs.modes) }}
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Load Cache
uses: actions/cache@v5
with:
key: ${{ env.CACHE_KEY }}
path: /tmp/${{ env.IMAGE }}.tar
fail-on-cache-miss: true
- name: Login to Docker Hub
uses: docker/login-action@v4
with:
username: ${{ vars.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Load and Start Appwrite
timeout-minutes: 5
env:
_APP_DATABASE_SHARED_TABLES: ${{ matrix.mode != 'dedicated' && 'database_db_main' || '' }}
_APP_DATABASE_SHARED_TABLES_V1: ${{ matrix.mode == 'shared_v1' && 'database_db_main' || '' }}
run: |
docker load --input /tmp/${{ env.IMAGE }}.tar
docker compose pull --quiet --ignore-buildable
docker compose up -d --quiet-pull --wait
- name: Wait for Open Runtimes
timeout-minutes: 3
run: |
while ! docker compose logs openruntimes-executor | grep -q "Executor is ready."; do
echo "Waiting for Executor to come online"
sleep 1
done
- name: Run tests
uses: itznotabug/php-retry@v3
with:
max_attempts: 2
retry_wait_seconds: 60
timeout_minutes: 15
job_id: ${{ job.check_run_id }}
github_token: ${{ secrets.GITHUB_TOKEN }}
test_dir: tests/e2e/Services/Sites
command: >-
docker compose exec -T
-e _APP_E2E_RESPONSE_FORMAT="${{ github.event.inputs.response_format }}"
appwrite test /usr/src/code/tests/e2e/Services/Sites --group=screenshots
- name: Failure Logs
if: failure()
run: |
echo "=== Appwrite Logs ==="
docker compose logs
benchmark:
name: Benchmark
runs-on: ubuntu-latest
needs: build
permissions:
pull-requests: write
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Load Cache
uses: actions/cache@v5
with:
key: ${{ env.CACHE_KEY }}
path: /tmp/${{ env.IMAGE }}.tar
fail-on-cache-miss: true
- name: Login to Docker Hub
uses: docker/login-action@v4
with:
username: ${{ vars.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Load and Start Appwrite
run: |
sed -i 's/traefik/localhost/g' .env
docker load --input /tmp/${{ env.IMAGE }}.tar
docker compose up -d
sleep 10
- name: Install Oha
run: |
echo "deb [signed-by=/usr/share/keyrings/azlux-archive-keyring.gpg] http://packages.azlux.fr/debian/ stable main" | sudo tee /etc/apt/sources.list.d/azlux.list
sudo wget -O /usr/share/keyrings/azlux-archive-keyring.gpg https://azlux.fr/repo.gpg
sudo apt update
sudo apt install oha
oha --version
- name: Benchmark PR
run: 'oha -z 180s http://localhost/v1/health/version --output-format json > benchmark.json'
- name: Cleaning
run: docker compose down -v
- name: Installing latest version
run: |
rm docker-compose.yml
rm .env
curl https://appwrite.io/install/compose -o docker-compose.yml
curl https://appwrite.io/install/env -o .env
sed -i 's/_APP_OPTIONS_ABUSE=enabled/_APP_OPTIONS_ABUSE=disabled/g' .env
docker compose up -d
sleep 10
- name: Benchmark Latest
run: oha -z 180s http://localhost/v1/health/version --output-format json > benchmark-latest.json
- name: Prepare comment
run: |
echo '## :sparkles: Benchmark results' > benchmark.txt
echo ' ' >> benchmark.txt
echo "- Requests per second: $(jq -r '.summary.requestsPerSec|tonumber?|floor|tostring|[while(length>0;.[:-3])|.[-3:]]|reverse|join(",")' benchmark.json)" >> benchmark.txt
echo "- Requests with 200 status code: $(jq -r '.statusCodeDistribution."200"|tostring|[while(length>0;.[:-3])|.[-3:]]|reverse|join(",")' benchmark.json)" >> benchmark.txt
echo "- P99 latency: $(jq -r '.latencyPercentiles.p99' benchmark.json )" >> benchmark.txt
echo " " >> benchmark.txt
echo " " >> benchmark.txt
echo "## :zap: Benchmark Comparison" >> benchmark.txt
echo " " >> benchmark.txt
echo "| Metric | This PR | Latest version | " >> benchmark.txt
echo "| --- | --- | --- | " >> benchmark.txt
echo "| RPS | $(jq -r '.summary.requestsPerSec|tonumber?|floor|tostring|[while(length>0;.[:-3])|.[-3:]]|reverse|join(",")' benchmark.json) | $(jq -r '.summary.requestsPerSec|tonumber|floor|tostring|[while(length>0;.[:-3])|.[-3:]]|reverse|join(",")' benchmark-latest.json) | " >> benchmark.txt
echo "| 200 | $(jq -r '.statusCodeDistribution."200"|tostring|[while(length>0;.[:-3])|.[-3:]]|reverse|join(",")' benchmark.json) | $(jq -r '.statusCodeDistribution."200"|tostring|[while(length>0;.[:-3])|.[-3:]]|reverse|join(",")' benchmark-latest.json) | " >> benchmark.txt
echo "| P99 | $(jq -r '.latencyPercentiles.p99' benchmark.json ) | $(jq -r '.latencyPercentiles.p99' benchmark-latest.json ) | " >> benchmark.txt
- name: Save results
uses: actions/upload-artifact@v7
if: ${{ !cancelled() }}
with:
name: benchmark.json
path: benchmark.json
retention-days: 7
- name: Find Comment
if: github.event.pull_request.head.repo.full_name == github.repository
uses: peter-evans/find-comment@v3
id: fc
with:
issue-number: ${{ github.event.pull_request.number }}
comment-author: 'github-actions[bot]'
body-includes: Benchmark results
- name: Comment on PR
if: github.event.pull_request.head.repo.full_name == github.repository
uses: peter-evans/create-or-update-comment@v4
with:
comment-id: ${{ steps.fc.outputs.comment-id }}
issue-number: ${{ github.event.pull_request.number }}
body-path: benchmark.txt
edit-mode: replace
-28
View File
@@ -1,28 +0,0 @@
name: "Linter"
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
on: [pull_request]
jobs:
lint:
name: Linter
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v6
with:
fetch-depth: 2
- run: git checkout HEAD^2
- name: Validate composer.json and composer.lock
run: |
docker run --rm -v $PWD:/app composer:2.8 sh -c \
"composer validate"
- name: Run Linter
run: |
docker run --rm -v $PWD:/app composer:2.8 sh -c \
"composer install --profile --ignore-platform-reqs && composer lint"
-106
View File
@@ -1,106 +0,0 @@
name: PR Security Scan
on:
pull_request_target:
types: [opened, synchronize, reopened]
jobs:
scan:
runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: write
steps:
- name: Check out code
uses: actions/checkout@v6
with:
ref: ${{ github.event.pull_request.head.sha }}
fetch-depth: 0
submodules: 'recursive'
- name: Build the Docker image
uses: docker/build-push-action@v6
with:
context: .
push: false
load: true
tags: pr_image:${{ github.sha }}
target: production
- name: Run Trivy vulnerability scanner on image
uses: aquasecurity/trivy-action@0.20.0
with:
image-ref: 'pr_image:${{ github.sha }}'
format: 'json'
output: 'trivy-image-results.json'
severity: 'CRITICAL,HIGH'
- name: Run Trivy vulnerability scanner on source code
uses: aquasecurity/trivy-action@0.20.0
with:
scan-type: 'fs'
scan-ref: '.'
format: 'json'
output: 'trivy-fs-results.json'
severity: 'CRITICAL,HIGH'
- name: Process Trivy scan results
id: process-results
uses: actions/github-script@v8
with:
script: |
const fs = require('fs');
let commentBody = '## Security Scan Results for PR\n\n';
function processResults(results, title) {
let sectionBody = `### ${title}\n\n`;
if (results.Results && results.Results.some(result => result.Vulnerabilities && result.Vulnerabilities.length > 0)) {
sectionBody += '| Package | Version | Vulnerability | Severity |\n';
sectionBody += '|---------|---------|----------------|----------|\n';
const uniqueVulns = new Set();
results.Results.forEach(result => {
if (result.Vulnerabilities) {
result.Vulnerabilities.forEach(vuln => {
const vulnKey = `${vuln.PkgName}-${vuln.InstalledVersion}-${vuln.VulnerabilityID}`;
if (!uniqueVulns.has(vulnKey)) {
uniqueVulns.add(vulnKey);
sectionBody += `| ${vuln.PkgName} | ${vuln.InstalledVersion} | [${vuln.VulnerabilityID}](https://nvd.nist.gov/vuln/detail/${vuln.VulnerabilityID}) | ${vuln.Severity} |\n`;
}
});
}
});
} else {
sectionBody += '🎉 No vulnerabilities found!\n';
}
return sectionBody;
}
try {
const imageResults = JSON.parse(fs.readFileSync('trivy-image-results.json', 'utf8'));
const fsResults = JSON.parse(fs.readFileSync('trivy-fs-results.json', 'utf8'));
commentBody += processResults(imageResults, "Docker Image Scan Results");
commentBody += '\n';
commentBody += processResults(fsResults, "Source Code Scan Results");
} catch (error) {
commentBody += `There was an error while running the security scan: ${error.message}\n`;
commentBody += 'Please contact the core team for assistance.';
}
core.setOutput('comment-body', commentBody);
- name: Find Comment
uses: peter-evans/find-comment@v3
id: fc
with:
issue-number: ${{ github.event.pull_request.number }}
comment-author: 'github-actions[bot]'
body-includes: Security Scan Results for PR
- name: Create or update comment
uses: peter-evans/create-or-update-comment@v3
with:
issue-number: ${{ github.event.pull_request.number }}
comment-id: ${{ steps.fc.outputs.comment-id }}
body: ${{ steps.process-results.outputs.comment-body }}
edit-mode: replace
+1 -1
View File
@@ -9,7 +9,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/stale@v9
- uses: actions/stale@v10
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}
stale-issue-message: "This issue has been labeled as a 'question', indicating that it requires additional information from the requestor. It has been inactive for 7 days. If no further activity occurs, this issue will be closed in 14 days."
-21
View File
@@ -1,21 +0,0 @@
name: "Static code analysis"
on: [pull_request]
jobs:
lint:
name: CodeQL
runs-on: ubuntu-latest
steps:
- name: Check out the repo
uses: actions/checkout@v6
- name: Run CodeQL
run: |
docker run --rm -v $PWD:/app composer:2.8 sh -c \
"composer install --profile --ignore-platform-reqs && composer check"
- name: Run Locale check
run: |
docker run --rm -v $PWD:/app node:24-alpine sh -c \
"cd /app/.github/workflows/static-analysis/locale && node index.js"
-691
View File
@@ -1,691 +0,0 @@
name: "Tests"
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
env:
COMPOSE_FILE: docker-compose.yml
IMAGE: appwrite-dev
CACHE_KEY: appwrite-dev-${{ github.event.pull_request.head.sha }}
on:
pull_request:
workflow_dispatch:
inputs:
response_format:
description: 'Response format version to test (e.g., 1.5.0, 1.4.0)'
required: false
type: string
default: ''
jobs:
check_database_changes:
name: Check if utopia-php/database changed
runs-on: ubuntu-latest
outputs:
database_changed: ${{ steps.check.outputs.database_changed }}
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Fetch base branch
run: git fetch origin ${{ github.event.pull_request.base.ref }}
- name: Check for utopia-php/database changes
id: check
run: |
if git diff origin/${{ github.event.pull_request.base.ref }} HEAD -- composer.lock | grep -q '"name": "utopia-php/database"'; then
echo "Database version changed, going to run all mode tests."
echo "database_changed=true" >> "$GITHUB_ENV"
echo "database_changed=true" >> "$GITHUB_OUTPUT"
else
echo "database_changed=false" >> "$GITHUB_ENV"
echo "database_changed=false" >> "$GITHUB_OUTPUT"
fi
setup:
name: Setup & Build Appwrite Image
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v6
with:
submodules: recursive
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Build Appwrite
uses: docker/build-push-action@v6
with:
context: .
push: false
tags: ${{ env.IMAGE }}
load: true
cache-from: type=gha
cache-to: type=gha,mode=max
outputs: type=docker,dest=/tmp/${{ env.IMAGE }}.tar
target: development
build-args: |
DEBUG=false
TESTING=true
VERSION=dev
- name: Cache Docker Image
uses: actions/cache@v4
with:
key: ${{ env.CACHE_KEY }}
path: /tmp/${{ env.IMAGE }}.tar
unit_test:
name: Unit Test
runs-on: ubuntu-latest
needs: setup
permissions:
contents: read
pull-requests: write
steps:
- name: checkout
uses: actions/checkout@v6
- name: Load Cache
uses: actions/cache@v4
with:
key: ${{ env.CACHE_KEY }}
path: /tmp/${{ env.IMAGE }}.tar
fail-on-cache-miss: true
- name: Load and Start Appwrite
timeout-minutes: 3
run: |
docker load --input /tmp/${{ env.IMAGE }}.tar
docker compose up -d
until docker compose exec -T appwrite doctor > /dev/null 2>&1; do
echo "Waiting for Appwrite to be ready..."
sleep 2
done
- name: Environment Variables
run: docker compose exec -T appwrite vars
- name: Run Unit Tests
uses: itznotabug/php-retry@v3
with:
max_attempts: 2
retry_wait_seconds: 300
timeout_minutes: 15
job_id: ${{ job.check_run_id }}
github_token: ${{ secrets.GITHUB_TOKEN }}
test_dir: tests/unit
command: >-
docker compose exec
-e _APP_E2E_RESPONSE_FORMAT="${{ github.event.inputs.response_format }}"
appwrite test /usr/src/code/tests/unit
e2e_general_test:
name: E2E General Test
runs-on: ubuntu-latest
needs: setup
permissions:
contents: read
pull-requests: write
steps:
- name: checkout
uses: actions/checkout@v6
- name: Load Cache
uses: actions/cache@v4
with:
key: ${{ env.CACHE_KEY }}
path: /tmp/${{ env.IMAGE }}.tar
fail-on-cache-miss: true
- name: Load and Start Appwrite
timeout-minutes: 3
run: |
docker load --input /tmp/${{ env.IMAGE }}.tar
docker compose up -d
until docker compose exec -T appwrite doctor > /dev/null 2>&1; do
echo "Waiting for Appwrite to be ready..."
sleep 2
done
- name: Wait for Open Runtimes
timeout-minutes: 3
run: |
while ! docker compose logs openruntimes-executor | grep -q "Executor is ready."; do
echo "Waiting for Executor to come online"
sleep 1
done
- name: Run General Tests
uses: itznotabug/php-retry@v3
with:
max_attempts: 2
retry_wait_seconds: 300
timeout_minutes: 15
job_id: ${{ job.check_run_id }}
github_token: ${{ secrets.GITHUB_TOKEN }}
test_dir: tests/e2e/General
command: >-
docker compose exec -T
-e _APP_E2E_RESPONSE_FORMAT="${{ github.event.inputs.response_format }}"
appwrite test /usr/src/code/tests/e2e/General --debug
- name: Failure Logs
if: failure()
run: |
echo "=== Appwrite Logs ==="
docker compose logs
e2e_service_test:
name: E2E Service Test
runs-on: ubuntu-latest
needs: setup
permissions:
contents: read
pull-requests: write
strategy:
fail-fast: false
matrix:
db_adapter: [
MARIADB,
POSTGRESQL,
MONGODB
]
service: [
Account,
Avatars,
Console,
Databases,
Functions,
FunctionsSchedule,
GraphQL,
Health,
Locale,
Projects,
Realtime,
Sites,
Proxy,
Storage,
Tokens,
Teams,
Users,
Webhooks,
VCS,
Messaging,
Migrations
]
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Load Cache
uses: actions/cache@v4
with:
key: ${{ env.CACHE_KEY }}
path: /tmp/${{ env.IMAGE }}.tar
fail-on-cache-miss: true
- name: Set DB Adapter environment
id: set-db-env
run: |
DB_ADAPTER_LOWER=$(echo "${{ matrix.db_adapter }}" | tr 'A-Z' 'a-z')
echo "COMPOSE_PROFILES=${DB_ADAPTER_LOWER}" >> $GITHUB_ENV
if [ "${{ matrix.db_adapter }}" = "MARIADB" ]; then
echo "_APP_DB_ADAPTER=mariadb" >> $GITHUB_ENV
echo "_APP_DB_HOST=mariadb" >> $GITHUB_ENV
echo "_APP_DB_PORT=3306" >> $GITHUB_ENV
elif [ "${{ matrix.db_adapter }}" = "MONGODB" ]; then
echo "_APP_DB_ADAPTER=mongodb" >> $GITHUB_ENV
echo "_APP_DB_HOST=mongodb" >> $GITHUB_ENV
echo "_APP_DB_PORT=27017" >> $GITHUB_ENV
elif [ "${{ matrix.db_adapter }}" = "POSTGRESQL" ]; then
echo "_APP_DB_ADAPTER=postgresql" >> $GITHUB_ENV
echo "_APP_DB_HOST=postgresql" >> $GITHUB_ENV
echo "_APP_DB_PORT=5432" >> $GITHUB_ENV
fi
- name: Load and Start Appwrite
timeout-minutes: 3
env:
_APP_BROWSER_HOST: http://invalid-browser/v1
run: |
docker load --input /tmp/${{ env.IMAGE }}.tar
docker compose up -d
until docker compose exec -T appwrite doctor > /dev/null 2>&1; do
echo "Waiting for Appwrite to be ready..."
sleep 2
done
- name: Wait for Open Runtimes
timeout-minutes: 3
run: |
while ! docker compose logs openruntimes-executor | grep -q "Executor is ready."; do
echo "Waiting for Executor to come online"
sleep 1
done
- name: Run ${{ matrix.service }} tests with Project table mode
uses: itznotabug/php-retry@v3
with:
max_attempts: 2
retry_wait_seconds: 300
timeout_minutes: 20
job_id: ${{ job.check_run_id }}
github_token: ${{ secrets.GITHUB_TOKEN }}
test_dir: tests/e2e/Services/${{ matrix.service }}
command: |
echo "Using project tables"
SERVICE_PATH="/usr/src/code/tests/e2e/Services/${{ matrix.service }}"
# Services that rely on sequential test method execution (shared static state)
FUNCTIONAL_FLAG="--functional"
case "${{ matrix.service }}" in
Databases|Functions|Realtime) FUNCTIONAL_FLAG="" ;;
esac
echo "Running with paratest (parallel) for: ${{ matrix.service }} ${FUNCTIONAL_FLAG:+(functional)}"
docker compose exec -T \
-e _APP_DATABASE_SHARED_TABLES="" \
-e _APP_DATABASE_SHARED_TABLES_V1="" \
-e _APP_DB_ADAPTER="${{ env._APP_DB_ADAPTER }}" \
-e _APP_DB_HOST="${{ env._APP_DB_HOST }}" \
-e _APP_DB_PORT="${{ env._APP_DB_PORT }}" \
-e _APP_DB_SCHEMA=appwrite \
-e _APP_E2E_RESPONSE_FORMAT="${{ github.event.inputs.response_format }}" \
appwrite vendor/bin/paratest --processes $(nproc) $FUNCTIONAL_FLAG "$SERVICE_PATH" --exclude-group abuseEnabled --exclude-group screenshots --exclude-group ciIgnore --log-junit tests/e2e/Services/${{ matrix.service }}/junit.xml
- name: Failure Logs
if: failure()
run: |
echo "=== Appwrite Logs ==="
docker compose logs
e2e_shared_mode_test:
name: E2E Shared Mode Service Test
runs-on: ubuntu-latest
needs: [ setup, check_database_changes ]
if: needs.check_database_changes.outputs.database_changed == 'true'
permissions:
contents: read
pull-requests: write
strategy:
fail-fast: false
matrix:
service:
[
Account,
Avatars,
Console,
Databases,
Functions,
FunctionsSchedule,
GraphQL,
Health,
Locale,
Projects,
Realtime,
Sites,
Proxy,
Storage,
Teams,
Users,
Webhooks,
VCS,
Messaging,
Migrations,
Tokens
]
tables-mode: [
'Shared V1',
'Shared V2',
]
steps:
- name: checkout
uses: actions/checkout@v6
- name: Load Cache
uses: actions/cache@v4
with:
key: ${{ env.CACHE_KEY }}
path: /tmp/${{ env.IMAGE }}.tar
fail-on-cache-miss: true
- name: Load and Start Appwrite
timeout-minutes: 3
run: |
docker load --input /tmp/${{ env.IMAGE }}.tar
docker compose up -d
until docker compose exec -T appwrite doctor > /dev/null 2>&1; do
echo "Waiting for Appwrite to be ready..."
sleep 2
done
- name: Wait for Open Runtimes
timeout-minutes: 3
run: |
while ! docker compose logs openruntimes-executor | grep -q "Executor is ready."; do
echo "Waiting for Executor to come online"
sleep 1
done
- name: Run ${{ matrix.service }} tests with ${{ matrix.tables-mode }} table mode
uses: itznotabug/php-retry@v3
with:
max_attempts: 2
retry_wait_seconds: 300
timeout_minutes: 20
job_id: ${{ job.check_run_id }}
github_token: ${{ secrets.GITHUB_TOKEN }}
test_dir: tests/e2e/Services/${{ matrix.service }}
command: |
if [ "${{ matrix.tables-mode }}" == "Shared V1" ]; then
echo "Using shared tables V1"
export _APP_DATABASE_SHARED_TABLES=database_db_main
export _APP_DATABASE_SHARED_TABLES_V1=database_db_main
elif [ "${{ matrix.tables-mode }}" == "Shared V2" ]; then
echo "Using shared tables V2"
export _APP_DATABASE_SHARED_TABLES=database_db_main
export _APP_DATABASE_SHARED_TABLES_V1=
fi
SERVICE_PATH="/usr/src/code/tests/e2e/Services/${{ matrix.service }}"
# Services that rely on sequential test method execution (shared static state)
FUNCTIONAL_FLAG="--functional"
case "${{ matrix.service }}" in
Databases|Functions|Realtime) FUNCTIONAL_FLAG="" ;;
esac
docker compose exec -T \
-e _APP_DATABASE_SHARED_TABLES \
-e _APP_DATABASE_SHARED_TABLES_V1 \
-e _APP_E2E_RESPONSE_FORMAT="${{ github.event.inputs.response_format }}" \
appwrite vendor/bin/paratest --processes $(nproc) $FUNCTIONAL_FLAG "$SERVICE_PATH" --exclude-group abuseEnabled --exclude-group screenshots --exclude-group ciIgnore --log-junit tests/e2e/Services/${{ matrix.service }}/junit.xml
- name: Failure Logs
if: failure()
run: |
echo "=== Appwrite Worker Builds Logs ==="
docker compose logs appwrite-worker-builds
echo "=== OpenRuntimes Executor Logs ==="
docker compose logs openruntimes-executor
e2e_abuse_enabled:
name: E2E Service Test (Abuse enabled)
runs-on: ubuntu-latest
needs: setup
permissions:
contents: read
pull-requests: write
steps:
- name: checkout
uses: actions/checkout@v6
- name: Load Cache
uses: actions/cache@v4
with:
key: ${{ env.CACHE_KEY }}
path: /tmp/${{ env.IMAGE }}.tar
fail-on-cache-miss: true
- name: Load and Start Appwrite
timeout-minutes: 3
run: |
docker load --input /tmp/${{ env.IMAGE }}.tar
sed -i 's/_APP_OPTIONS_ABUSE=disabled/_APP_OPTIONS_ABUSE=enabled/' .env
docker compose up -d
until docker compose exec -T appwrite doctor > /dev/null 2>&1; do
echo "Waiting for Appwrite to be ready..."
sleep 2
done
- name: Run Projects tests in dedicated table mode
uses: itznotabug/php-retry@v3
with:
max_attempts: 2
retry_wait_seconds: 300
timeout_minutes: 15
job_id: ${{ job.check_run_id }}
github_token: ${{ secrets.GITHUB_TOKEN }}
test_dir: tests/e2e/Services/Projects
command: |
echo "Using project tables"
export _APP_DATABASE_SHARED_TABLES=
export _APP_DATABASE_SHARED_TABLES_V1=
docker compose exec -T \
-e _APP_DATABASE_SHARED_TABLES \
-e _APP_DATABASE_SHARED_TABLES_V1 \
-e _APP_E2E_RESPONSE_FORMAT="${{ github.event.inputs.response_format }}" \
appwrite test /usr/src/code/tests/e2e/Services/Projects --debug --group=abuseEnabled
- name: Failure Logs
if: failure()
run: |
echo "=== Appwrite Worker Builds Logs ==="
docker compose logs appwrite-worker-builds
echo "=== OpenRuntimes Executor Logs ==="
docker compose logs openruntimes-executor
e2e_abuse_enabled_shared_mode:
name: E2E Shared Mode Service Test (Abuse enabled)
runs-on: ubuntu-latest
needs: [ setup, check_database_changes ]
if: needs.check_database_changes.outputs.database_changed == 'true'
permissions:
contents: read
pull-requests: write
strategy:
fail-fast: false
matrix:
tables-mode: [
'Shared V1',
'Shared V2',
]
steps:
- name: checkout
uses: actions/checkout@v6
- name: Load Cache
uses: actions/cache@v4
with:
key: ${{ env.CACHE_KEY }}
path: /tmp/${{ env.IMAGE }}.tar
fail-on-cache-miss: true
- name: Load and Start Appwrite
timeout-minutes: 3
run: |
docker load --input /tmp/${{ env.IMAGE }}.tar
sed -i 's/_APP_OPTIONS_ABUSE=disabled/_APP_OPTIONS_ABUSE=enabled/' .env
docker compose up -d
until docker compose exec -T appwrite doctor > /dev/null 2>&1; do
echo "Waiting for Appwrite to be ready..."
sleep 2
done
- name: Run Projects tests in ${{ matrix.tables-mode }} table mode
uses: itznotabug/php-retry@v3
with:
max_attempts: 2
retry_wait_seconds: 300
timeout_minutes: 15
job_id: ${{ job.check_run_id }}
github_token: ${{ secrets.GITHUB_TOKEN }}
test_dir: tests/e2e/Services/Projects
command: |
if [ "${{ matrix.tables-mode }}" == "Shared V1" ]; then
echo "Using shared tables V1"
export _APP_DATABASE_SHARED_TABLES=database_db_main
export _APP_DATABASE_SHARED_TABLES_V1=database_db_main
elif [ "${{ matrix.tables-mode }}" == "Shared V2" ]; then
echo "Using shared tables V2"
export _APP_DATABASE_SHARED_TABLES=database_db_main
export _APP_DATABASE_SHARED_TABLES_V1=
fi
docker compose exec -T \
-e _APP_DATABASE_SHARED_TABLES \
-e _APP_DATABASE_SHARED_TABLES_V1 \
-e _APP_E2E_RESPONSE_FORMAT="${{ github.event.inputs.response_format }}" \
appwrite test /usr/src/code/tests/e2e/Services/Projects --debug --group=abuseEnabled
- name: Failure Logs
if: failure()
run: |
echo "=== Appwrite Worker Builds Logs ==="
docker compose logs appwrite-worker-builds
echo "=== OpenRuntimes Executor Logs ==="
docker compose logs openruntimes-executor
e2e_screenshots:
name: E2E Service Test (Site Screenshots)
runs-on: ubuntu-latest
needs: setup
permissions:
contents: read
pull-requests: write
steps:
- name: checkout
uses: actions/checkout@v6
- name: Load Cache
uses: actions/cache@v4
with:
key: ${{ env.CACHE_KEY }}
path: /tmp/${{ env.IMAGE }}.tar
fail-on-cache-miss: true
- name: Load and Start Appwrite
timeout-minutes: 3
run: |
docker load --input /tmp/${{ env.IMAGE }}.tar
sed -i 's/_APP_OPTIONS_ABUSE=disabled/_APP_OPTIONS_ABUSE=enabled/' .env
docker compose up -d
until docker compose exec -T appwrite doctor > /dev/null 2>&1; do
echo "Waiting for Appwrite to be ready..."
sleep 2
done
- name: Wait for Open Runtimes
timeout-minutes: 3
run: |
while ! docker compose logs openruntimes-executor | grep -q "Executor is ready."; do
echo "Waiting for Executor to come online"
sleep 1
done
- name: Run Site tests with browser connected in dedicated table mode
uses: itznotabug/php-retry@v3
with:
max_attempts: 2
retry_wait_seconds: 300
timeout_minutes: 15
job_id: ${{ job.check_run_id }}
github_token: ${{ secrets.GITHUB_TOKEN }}
test_dir: tests/e2e/Services/Sites
command: |
echo "Keeping original value of _APP_BROWSER_HOST"
echo "Using project tables"
export _APP_DATABASE_SHARED_TABLES=
export _APP_DATABASE_SHARED_TABLES_V1=
docker compose exec -T \
-e _APP_DATABASE_SHARED_TABLES \
-e _APP_DATABASE_SHARED_TABLES_V1 \
-e _APP_E2E_RESPONSE_FORMAT="${{ github.event.inputs.response_format }}" \
appwrite test /usr/src/code/tests/e2e/Services/Sites --debug --group=screenshots
- name: Failure Logs
if: failure()
run: |
echo "=== Appwrite Worker Builds Logs ==="
docker compose logs appwrite-worker-builds
echo "=== OpenRuntimes Executor Logs ==="
docker compose logs openruntimes-executor
e2e_screenshots_shared_mode:
name: E2E Shared Mode Service Test (Site Screenshots)
runs-on: ubuntu-latest
needs: [ setup, check_database_changes ]
if: needs.check_database_changes.outputs.database_changed == 'true'
permissions:
contents: read
pull-requests: write
strategy:
fail-fast: false
matrix:
tables-mode: [
'Shared V1',
'Shared V2',
]
steps:
- name: checkout
uses: actions/checkout@v6
- name: Load Cache
uses: actions/cache@v4
with:
key: ${{ env.CACHE_KEY }}
path: /tmp/${{ env.IMAGE }}.tar
fail-on-cache-miss: true
- name: Load and Start Appwrite
timeout-minutes: 3
run: |
docker load --input /tmp/${{ env.IMAGE }}.tar
sed -i 's/_APP_OPTIONS_ABUSE=disabled/_APP_OPTIONS_ABUSE=enabled/' .env
docker compose up -d
until docker compose exec -T appwrite doctor > /dev/null 2>&1; do
echo "Waiting for Appwrite to be ready..."
sleep 2
done
- name: Wait for Open Runtimes
timeout-minutes: 3
run: |
while ! docker compose logs openruntimes-executor | grep -q "Executor is ready."; do
echo "Waiting for Executor to come online"
sleep 1
done
- name: Run Site tests with browser connected in ${{ matrix.tables-mode }} table mode
uses: itznotabug/php-retry@v3
with:
max_attempts: 2
retry_wait_seconds: 300
timeout_minutes: 15
job_id: ${{ job.check_run_id }}
github_token: ${{ secrets.GITHUB_TOKEN }}
test_dir: tests/e2e/Services/Sites
command: |
echo "Keeping original value of _APP_BROWSER_HOST"
if [ "${{ matrix.tables-mode }}" == "Shared V1" ]; then
echo "Using shared tables V1"
export _APP_DATABASE_SHARED_TABLES=database_db_main
export _APP_DATABASE_SHARED_TABLES_V1=database_db_main
elif [ "${{ matrix.tables-mode }}" == "Shared V2" ]; then
echo "Using shared tables V2"
export _APP_DATABASE_SHARED_TABLES=database_db_main
export _APP_DATABASE_SHARED_TABLES_V1=
fi
docker compose exec -T \
-e _APP_DATABASE_SHARED_TABLES \
-e _APP_DATABASE_SHARED_TABLES_V1 \
-e _APP_E2E_RESPONSE_FORMAT="${{ github.event.inputs.response_format }}" \
appwrite test /usr/src/code/tests/e2e/Services/Sites --debug --group=screenshots
- name: Failure Logs
if: failure()
run: |
echo "=== Appwrite Worker Builds Logs ==="
docker compose logs appwrite-worker-builds
echo "=== OpenRuntimes Executor Logs ==="
docker compose logs openruntimes-executor
+29 -16
View File
@@ -409,14 +409,16 @@ Next follow the appropriate steps below depending on whether you're adding the m
**API**
In file `app/controllers/shared/api.php` On the database listener, add to an existing or create a new switch case. Add a call to the usage worker with your new metric const like so:
In file `app/controllers/shared/api.php` On the database listener, add to an existing or create a new switch case. Accumulate metrics in the usage context like so:
```php
case $document->getCollection() === 'teams':
$queueForStatsUsage
->addMetric(METRIC_TEAMS, $value); // per project
$usage->addMetric(METRIC_TEAMS, $value); // per project
break;
```
The metrics will be automatically published by the shutdown hook at the end of the request. There is no need to manually trigger or publish.
There are cases when you need to handle metric that has a parent entity, like buckets.
Files are linked to a parent bucket, you should verify you remove the files stats when you delete a bucket.
@@ -425,14 +427,13 @@ In that case you need also to handle children removal using addReduce() method c
```php
case $document->getCollection() === 'buckets': //buckets
$queueForStatsUsage
->addMetric(METRIC_BUCKETS, $value); // per project
$usage->addMetric(METRIC_BUCKETS, $value); // per project
if ($event === Database::EVENT_DOCUMENT_DELETE) {
$queueForStatsUsage
$usage
->addReduce($document);
}
break;
```
In addition, you will also need to add some logic to the `reduce()` method of the Usage worker located in `/src/Appwrite/Platform/Workers/Usage.php`, like so:
@@ -460,8 +461,12 @@ case $document->getCollection() === 'buckets':
**Background worker**
You need to inject the usage queue in the desired worker on the constructor method
You need to inject the usage context and publisher in the desired worker on the constructor method
```php
use Appwrite\Usage\Context;
use Appwrite\Event\Publisher\Usage as UsagePublisher;
use Appwrite\Event\Message\Usage as UsageMessage;
/**
* @throws Exception
*/
@@ -474,24 +479,32 @@ public function __construct()
->inject('dbForProject')
->inject('queueForFunctions')
->inject('queueForEvents')
->inject('queueForStatsUsage')
->inject('usage')
->inject('publisherForUsage')
->inject('log')
->callback(fn (Message $message, Database $dbForProject, Func $queueForFunctions, Event $queueForEvents, StatsUsage $queueForStatsUsage, Log $log) => $this->action($message, $dbForProject, $queueForFunctions, $queueForEvents, $queueForStatsUsage, $log));
->callback(fn (Message $message, Database $dbForProject, Func $queueForFunctions, Event $queueForEvents, Context $usage, UsagePublisher $publisherForUsage, Log $log) => $this->action($message, $dbForProject, $queueForFunctions, $queueForEvents, $usage, $publisherForUsage, $log));
}
```
and then trigger the queue with the new metric like so:
and then accumulate metrics, create a message, and publish like so:
```php
$queueForStatsUsage
$usage
->addMetric(METRIC_BUILDS, 1)
->addMetric(METRIC_BUILDS_STORAGE, $build->getAttribute('size', 0))
->addMetric(METRIC_BUILDS_COMPUTE, (int)$build->getAttribute('duration', 0) * 1000)
->addMetric(str_replace('{functionInternalId}', $function->getSequence(), METRIC_FUNCTION_ID_BUILDS), 1)
->addMetric(str_replace('{functionInternalId}', $function->getSequence(), METRIC_FUNCTION_ID_BUILDS), 1)
->addMetric(str_replace('{functionInternalId}', $function->getSequence(), METRIC_FUNCTION_ID_BUILDS_STORAGE), $build->getAttribute('size', 0))
->addMetric(str_replace('{functionInternalId}', $function->getSequence(), METRIC_FUNCTION_ID_BUILDS_COMPUTE), (int)$build->getAttribute('duration', 0) * 1000)
->setProject($project)
->trigger();
->addMetric(str_replace('{functionInternalId}', $function->getSequence(), METRIC_FUNCTION_ID_BUILDS_COMPUTE), (int)$build->getAttribute('duration', 0) * 1000);
// Publish the accumulated metrics (workers don't have shutdown hooks)
$message = new UsageMessage(
project: $project,
metrics: $usage->getMetrics(),
reduce: $usage->getReduce()
);
$publisherForUsage->enqueue($message);
$usage->reset();
```
+1 -1
View File
@@ -70,7 +70,7 @@ RUN chmod +x /usr/local/bin/doctor && \
chmod +x /usr/local/bin/sdks && \
chmod +x /usr/local/bin/specs && \
chmod +x /usr/local/bin/ssl && \
chmod +x /usr/local/bin/time-travel && \
chmod +x /usr/local/bin/task-time-travel && \
chmod +x /usr/local/bin/screenshot && \
chmod +x /usr/local/bin/test && \
chmod +x /usr/local/bin/upgrade && \
+24 -18
View File
@@ -4,11 +4,13 @@ require_once __DIR__ . '/init.php';
use Appwrite\Event\Certificate;
use Appwrite\Event\Delete;
use Appwrite\Event\Event;
use Appwrite\Event\Func;
use Appwrite\Event\Publisher\Usage as UsagePublisher;
use Appwrite\Event\StatsResources;
use Appwrite\Event\StatsUsage;
use Appwrite\Platform\Appwrite;
use Appwrite\Runtimes\Runtimes;
use Appwrite\Usage\Context as UsageContext;
use Appwrite\Utopia\Database\Documents\User;
use Executor\Executor;
use Swoole\Runtime;
@@ -29,6 +31,7 @@ use Utopia\Platform\Service;
use Utopia\Pools\Group;
use Utopia\Queue\Broker\Pool as BrokerPool;
use Utopia\Queue\Publisher;
use Utopia\Queue\Queue;
use Utopia\Registry\Registry;
use Utopia\System\System;
use Utopia\Telemetry\Adapter\None as NoTelemetry;
@@ -47,7 +50,7 @@ $platform = new Appwrite();
$args = $platform->getEnv('argv');
\array_shift($args);
if (!isset($args[0])) {
if (! isset($args[0])) {
Console::error('Missing task name');
Console::exit(1);
}
@@ -85,6 +88,7 @@ $setResource('pools', function (Registry $register) {
$setResource('authorization', function () {
$authorization = new Authorization();
$authorization->disable();
return $authorization;
}, []);
@@ -113,7 +117,7 @@ $setResource('dbForPlatform', function ($pools, $cache, $authorization) {
$collections = Config::getParam('collections', [])['console'];
$last = \array_key_last($collections);
if (!($dbForPlatform->exists($dbForPlatform->getDatabase(), $last))) { /** TODO cache ready variable using registry */
if (! ($dbForPlatform->exists($dbForPlatform->getDatabase(), $last))) { /** TODO cache ready variable using registry */
throw new Exception('Tables not ready yet.');
}
@@ -122,10 +126,10 @@ $setResource('dbForPlatform', function ($pools, $cache, $authorization) {
Console::warning($err->getMessage());
sleep($sleep);
}
} while ($attempts < $maxAttempts && !$ready);
} while ($attempts < $maxAttempts && ! $ready);
if (!$ready) {
throw new Exception("Console is not ready yet. Please try again later.");
if (! $ready) {
throw new Exception('Console is not ready yet. Please try again later.');
}
return $dbForPlatform;
@@ -163,7 +167,7 @@ $setResource('getProjectDB', function (Group $pools, Database $dbForPlatform, $c
if (\in_array($dsn->getHost(), $sharedTables)) {
$database
->setSharedTables(true)
->setTenant((int)$project->getSequence())
->setTenant((int) $project->getSequence())
->setNamespace($dsn->getParam('namespace'));
} else {
$database
@@ -184,7 +188,7 @@ $setResource('getProjectDB', function (Group $pools, Database $dbForPlatform, $c
if (\in_array($dsn->getHost(), $sharedTables)) {
$database
->setSharedTables(true)
->setTenant((int)$project->getSequence())
->setTenant((int) $project->getSequence())
->setNamespace($dsn->getParam('namespace'));
} else {
$database
@@ -207,8 +211,9 @@ $setResource('getLogsDB', function (Group $pools, Cache $cache, Authorization $a
$database = null;
return function (?Document $project = null) use ($pools, $cache, $database, $authorization) {
if ($database !== null && $project !== null && !$project->isEmpty() && $project->getId() !== 'console') {
$database->setTenant((int)$project->getSequence());
if ($database !== null && $project !== null && ! $project->isEmpty() && $project->getId() !== 'console') {
$database->setTenant((int) $project->getSequence());
return $database;
}
@@ -224,8 +229,8 @@ $setResource('getLogsDB', function (Group $pools, Cache $cache, Authorization $a
->setMaxQueryValues(APP_DATABASE_QUERY_MAX_VALUES);
// set tenant
if ($project !== null && !$project->isEmpty() && $project->getId() !== 'console') {
$database->setTenant((int)$project->getSequence());
if ($project !== null && ! $project->isEmpty() && $project->getId() !== 'console') {
$database->setTenant((int) $project->getSequence());
}
return $database;
@@ -243,15 +248,16 @@ $setResource('publisherFunctions', function (BrokerPool $publisher) {
$setResource('publisherMigrations', function (BrokerPool $publisher) {
return $publisher;
}, ['publisher']);
$setResource('publisherStatsUsage', function (BrokerPool $publisher) {
return $publisher;
}, ['publisher']);
$setResource('publisherMessaging', function (BrokerPool $publisher) {
return $publisher;
}, ['publisher']);
$setResource('queueForStatsUsage', function (Publisher $publisher) {
return new StatsUsage($publisher);
}, ['publisher']);
$setResource('usage', function () {
return new UsageContext();
}, []);
$setResource('publisherForUsage', fn (Publisher $publisher) => new UsagePublisher(
$publisher,
new Queue(System::getEnv('_APP_STATS_USAGE_QUEUE_NAME', Event::STATS_USAGE_QUEUE_NAME))
), ['publisher']);
$setResource('queueForStatsResources', function (Publisher $publisher) {
return new StatsResources($publisher);
}, ['publisher']);
-2
View File
@@ -4,7 +4,6 @@
$common = include __DIR__ . '/collections/common.php';
$projects = include __DIR__ . '/collections/projects.php';
$databases = include __DIR__ . '/collections/databases.php';
$vectorsdb = include __DIR__ . '/collections/vectorsdb.php';
$platform = include __DIR__ . '/collections/platform.php';
$logs = include __DIR__ . '/collections/logs.php';
@@ -27,7 +26,6 @@ unset($common['files']);
$collections = [
'buckets' => $buckets,
'databases' => $databases,
'vectorsdb' => $vectorsdb,
'projects' => array_merge_recursive($projects, $common),
'console' => array_merge_recursive($platform, $common),
'logs' => $logs,
+22 -9
View File
@@ -61,15 +61,6 @@ return [
'array' => false,
'filters' => [],
],
[
'$id' => ID::custom('database'),
'type' => Database::VAR_STRING,
'size' => 128,
'required' => true,
'signed' => true,
'array' => false,
'filters' => [],
]
],
'indexes' => [
[
@@ -797,6 +788,17 @@ return [
'default' => null,
'filters' => [],
],
[
'array' => false,
'$id' => ID::custom('specification'),
'type' => Database::VAR_STRING,
'format' => '',
'size' => 128,
'signed' => false,
'required' => false,
'default' => APP_COMPUTE_SPECIFICATION_DEFAULT,
'filters' => [],
],
[
'array' => false,
'$id' => ID::custom('buildSpecification'),
@@ -1254,6 +1256,17 @@ return [
'array' => false,
'filters' => [],
],
[
'array' => false,
'$id' => ID::custom('specification'),
'type' => Database::VAR_STRING,
'format' => '',
'size' => 128,
'signed' => false,
'required' => false,
'default' => APP_COMPUTE_SPECIFICATION_DEFAULT,
'filters' => [],
],
[
'array' => false,
'$id' => ID::custom('buildSpecification'),
-165
View File
@@ -1,165 +0,0 @@
<?php
use Utopia\Database\Database;
use Utopia\Database\Helpers\ID;
return [
'collections' => [
'$collection' => ID::custom('databases'),
'$id' => ID::custom('collections'),
'name' => 'Collections',
'attributes' => [
[
'$id' => ID::custom('databaseInternalId'),
'type' => Database::VAR_STRING,
'format' => '',
'size' => Database::LENGTH_KEY,
'signed' => true,
'required' => true,
'default' => null,
'array' => false,
'filters' => [],
],
[
'$id' => ID::custom('databaseId'),
'type' => Database::VAR_STRING,
'signed' => true,
'size' => Database::LENGTH_KEY,
'format' => '',
'filters' => [],
'required' => true,
'default' => null,
'array' => false,
],
[
'$id' => ID::custom('name'),
'type' => Database::VAR_STRING,
'size' => 256,
'required' => true,
'signed' => true,
'array' => false,
'filters' => [],
],
[
'$id' => ID::custom('dimension'),
'type' => Database::VAR_INTEGER,
'size' => 0,
'required' => true,
'signed' => false,
'array' => false,
'filters' => [],
],
[
'$id' => ID::custom('enabled'),
'type' => Database::VAR_BOOLEAN,
'signed' => true,
'size' => 0,
'format' => '',
'filters' => [],
'required' => true,
'default' => null,
'array' => false,
],
[
'$id' => ID::custom('documentSecurity'),
'type' => Database::VAR_BOOLEAN,
'signed' => true,
'size' => 0,
'format' => '',
'filters' => [],
'required' => true,
'default' => null,
'array' => false,
],
[
'$id' => ID::custom('attributes'),
'type' => Database::VAR_STRING,
'size' => 1000000,
'required' => false,
'signed' => true,
'array' => false,
'filters' => ['subQueryAttributes'],
],
[
'$id' => ID::custom('indexes'),
'type' => Database::VAR_STRING,
'size' => 1000000,
'required' => false,
'signed' => true,
'array' => false,
'filters' => ['subQueryIndexes'],
],
[
'$id' => ID::custom('search'),
'type' => Database::VAR_STRING,
'format' => '',
'size' => 16384,
'signed' => true,
'required' => false,
'default' => null,
'array' => false,
'filters' => [],
],
],
'defaultAttributes' => [
[
'$id' => ID::custom('embeddings'),
'type' => Database::VAR_VECTOR,
'required' => true,
'signed' => false,
'array' => false,
'filters' => [],
],
[
'$id' => ID::custom('metadata'),
'type' => Database::VAR_OBJECT,
'default' => [],
'required' => false,
'size' => 0,
'signed' => false,
'array' => false,
'filters' => [],
],
],
'indexes' => [
[
'$id' => ID::custom('_fulltext_search'),
'type' => Database::INDEX_FULLTEXT,
'attributes' => ['search'],
'lengths' => [],
'orders' => [],
],
[
'$id' => ID::custom('_key_name'),
'type' => Database::INDEX_KEY,
'attributes' => ['name'],
'lengths' => [256],
'orders' => [Database::ORDER_ASC],
],
[
'$id' => ID::custom('_key_enabled'),
'type' => Database::INDEX_KEY,
'attributes' => ['enabled'],
'lengths' => [],
'orders' => [Database::ORDER_ASC],
],
[
'$id' => ID::custom('_key_documentSecurity'),
'type' => Database::INDEX_KEY,
'attributes' => ['documentSecurity'],
'lengths' => [],
'orders' => [Database::ORDER_ASC],
],
],
'defaultIndexes' => [
// not creating default indexes on the embeddings as it depends on the type of query users using the most
[
'$id' => ID::custom('_key_metadata'),
'type' => Database::INDEX_OBJECT,
'attributes' => ['metadata'],
'lengths' => [],
'orders' => [],
],
]
]
];
-5
View File
@@ -1201,11 +1201,6 @@ return [
'description' => 'Migration is already in progress. You can check the status of the migration in your Appwrite Console\'s "Settings" > "Migrations".',
'code' => 409,
],
Exception::MIGRATION_DATABASE_TYPE_UNSUPPORTED => [
'name' => Exception::MIGRATION_DATABASE_TYPE_UNSUPPORTED,
'description' => 'The specified database type is not supported for CSV import or export operations.',
'code' => 400,
],
/** Realtime */
Exception::REALTIME_MESSAGE_FORMAT_INVALID => [
-9
View File
@@ -1,9 +0,0 @@
<?php
// Resource types available for console project migration keys
return [
'platforms' => 'platforms.read',
'devKeys' => 'devKeys.read',
'devKeysWrite' => 'devKeys.write',
];
+9 -17
View File
@@ -14,7 +14,6 @@ use Appwrite\Event\Delete;
use Appwrite\Event\Event;
use Appwrite\Event\Mail;
use Appwrite\Event\Messaging;
use Appwrite\Event\StatsUsage;
use Appwrite\Extend\Exception;
use Appwrite\Hooks\Hooks;
use Appwrite\Network\Validator\Email as EmailValidator;
@@ -28,6 +27,7 @@ use Appwrite\SDK\MethodType;
use Appwrite\SDK\Response as SDKResponse;
use Appwrite\Template\Template;
use Appwrite\URL\URL as URLParser;
use Appwrite\Usage\Context;
use Appwrite\Utopia\Database\Documents\User;
use Appwrite\Utopia\Database\Validator\CustomId;
use Appwrite\Utopia\Database\Validator\Queries\Identities;
@@ -2801,12 +2801,12 @@ Http::post('/v1/account/tokens/phone')
->inject('queueForMessaging')
->inject('locale')
->inject('timelimit')
->inject('queueForStatsUsage')
->inject('usage')
->inject('plan')
->inject('store')
->inject('proofForCode')
->inject('authorization')
->action(function (string $userId, string $phone, Request $request, Response $response, User $user, Document $project, array $platform, Database $dbForProject, Event $queueForEvents, Messaging $queueForMessaging, Locale $locale, callable $timelimit, StatsUsage $queueForStatsUsage, array $plan, Store $store, ProofsCode $proofForCode, Authorization $authorization) {
->action(function (string $userId, string $phone, Request $request, Response $response, User $user, Document $project, array $platform, Database $dbForProject, Event $queueForEvents, Messaging $queueForMessaging, Locale $locale, callable $timelimit, Context $usage, array $plan, Store $store, ProofsCode $proofForCode, Authorization $authorization) {
if (empty(System::getEnv('_APP_SMS_PROVIDER'))) {
throw new Exception(Exception::GENERAL_PHONE_DISABLED, 'Phone provider not configured');
}
@@ -2955,16 +2955,12 @@ Http::post('/v1/account/tokens/phone')
$countryCode = $helper->parse($phone)->getCountryCode();
if (!empty($countryCode)) {
$queueForStatsUsage
->addMetric(str_replace('{countryCode}', $countryCode, METRIC_AUTH_METHOD_PHONE_COUNTRY_CODE), 1);
$usage->addMetric(str_replace('{countryCode}', $countryCode, METRIC_AUTH_METHOD_PHONE_COUNTRY_CODE), 1);
}
} catch (NumberParseException $e) {
// Ignore invalid phone number for country code stats
}
$queueForStatsUsage
->addMetric(METRIC_AUTH_METHOD_PHONE, 1)
->setProject($project)
->trigger();
$usage->addMetric(METRIC_AUTH_METHOD_PHONE, 1);
}
$token->setAttribute('secret', $secret);
@@ -4199,11 +4195,11 @@ Http::post('/v1/account/verifications/phone')
->inject('project')
->inject('locale')
->inject('timelimit')
->inject('queueForStatsUsage')
->inject('usage')
->inject('plan')
->inject('proofForCode')
->inject('authorization')
->action(function (Request $request, Response $response, User $user, Database $dbForProject, Event $queueForEvents, Messaging $queueForMessaging, Document $project, Locale $locale, callable $timelimit, StatsUsage $queueForStatsUsage, array $plan, ProofsCode $proofForCode, Authorization $authorization) {
->action(function (Request $request, Response $response, User $user, Database $dbForProject, Event $queueForEvents, Messaging $queueForMessaging, Document $project, Locale $locale, callable $timelimit, Context $usage, array $plan, ProofsCode $proofForCode, Authorization $authorization) {
if (empty(System::getEnv('_APP_SMS_PROVIDER'))) {
throw new Exception(Exception::GENERAL_PHONE_DISABLED, 'Phone provider not configured');
}
@@ -4288,16 +4284,12 @@ Http::post('/v1/account/verifications/phone')
$countryCode = $helper->parse($phone)->getCountryCode();
if (!empty($countryCode)) {
$queueForStatsUsage
->addMetric(str_replace('{countryCode}', $countryCode, METRIC_AUTH_METHOD_PHONE_COUNTRY_CODE), 1);
$usage->addMetric(str_replace('{countryCode}', $countryCode, METRIC_AUTH_METHOD_PHONE_COUNTRY_CODE), 1);
}
} catch (NumberParseException $e) {
// Ignore invalid phone number for country code stats
}
$queueForStatsUsage
->addMetric(METRIC_AUTH_METHOD_PHONE, 1)
->setProject($project)
->trigger();
$usage->addMetric(METRIC_AUTH_METHOD_PHONE, 1);
}
$verification->setAttribute('secret', $secret);
+7 -90
View File
@@ -1,6 +1,5 @@
<?php
use Ahc\Jwt\JWT;
use Appwrite\Event\Event;
use Appwrite\Event\Migration;
use Appwrite\Extend\Exception;
@@ -15,9 +14,7 @@ use Appwrite\Utopia\Response;
use Utopia\Compression\Algorithms\GZIP;
use Utopia\Compression\Algorithms\Zstd;
use Utopia\Compression\Compression;
use Utopia\Config\Config;
use Utopia\Database\Database;
use Utopia\Database\DateTime;
use Utopia\Database\Document;
use Utopia\Database\Exception\Order as OrderException;
use Utopia\Database\Exception\Query as QueryException;
@@ -46,16 +43,6 @@ use Utopia\Validator\WhiteList;
include_once __DIR__ . '/../shared/api.php';
function getDatabaseTransferResourceServices(string $databaseType)
{
return match($databaseType) {
DATABASE_TYPE_LEGACY,
DATABASE_TYPE_TABLESDB => Transfer::GROUP_DATABASES_TABLES_DB,
DATABASE_TYPE_VECTORSDB => Transfer::GROUP_DATABASES_VECTOR_DB,
DATABASE_TYPE_DOCUMENTSDB => Transfer::GROUP_DATABASES_DOCUMENTS_DB
};
}
Http::post('/v1/migrations/appwrite')
->groups(['api', 'migrations'])
->desc('Create Appwrite migration')
@@ -440,16 +427,8 @@ Http::post('/v1/migrations/csv/imports')
throw new \Exception('Unable to copy file');
}
// getting databasetype
$resources = explode(':', $resourceId);
$databaseId = $resources[0];
$database = $authorization->skip(fn () => $dbForProject->getDocument('databases', $databaseId));
$databaseType = $database->getAttribute('type');
if (!in_array($databaseType, CSV_ALLOWED_DATABASE_TYPES)) {
throw new Exception(Exception::MIGRATION_DATABASE_TYPE_UNSUPPORTED, 'Database type not supported for csv');
}
$fileSize = $deviceForMigrations->getFileSize($newPath);
$resources = Transfer::extractServices([getDatabaseTransferResourceServices($databaseType)]);
$resources = Transfer::extractServices([Transfer::GROUP_DATABASES]);
$migration = $dbForProject->createDocument('migrations', new Document([
'$id' => $migrationId,
@@ -578,23 +557,13 @@ Http::post('/v1/migrations/csv/exports')
throw new Exception(Exception::GENERAL_QUERY_INVALID, $validator->getDescription());
}
// getting databasetype
$resources = explode(':', $resourceId);
$databaseId = $resources[0];
$database = $authorization->skip(fn () => $dbForProject->getDocument('databases', $databaseId));
$databaseType = $database->getAttribute('type');
if (!in_array($databaseType, CSV_ALLOWED_DATABASE_TYPES)) {
throw new Exception(Exception::MIGRATION_DATABASE_TYPE_UNSUPPORTED, 'Database type not supported for csv');
}
$resources = Transfer::extractServices([getDatabaseTransferResourceServices($databaseType)]);
$migration = $dbForProject->createDocument('migrations', new Document([
'$id' => ID::unique(),
'status' => 'pending',
'stage' => 'init',
'source' => Appwrite::getName(),
'destination' => CSV::getName(),
'resources' => $resources,
'resources' => Transfer::extractServices([Transfer::GROUP_DATABASES]),
'resourceId' => $resourceId,
'resourceType' => Resource::TYPE_DATABASE,
'statusCounters' => '{}',
@@ -722,59 +691,6 @@ Http::get('/v1/migrations/:migrationId')
$response->dynamic($migration, Response::MODEL_MIGRATION);
});
Http::post('/v1/migrations/appwrite/console-key')
->groups(['api', 'migrations'])
->desc('Create console API key for migration')
->label('scope', 'migrations.write')
->label('sdk', new Method(
namespace: 'migrations',
group: null,
name: 'createAppwriteConsoleKey',
description: '/docs/references/migrations/migration-appwrite-console-key.md',
auth: [AuthType::KEY],
responses: [
new SDKResponse(
code: Response::STATUS_CODE_OK,
model: Response::MODEL_MIGRATION_KEY,
)
]
))
->param('resources', [], new ArrayList(new WhiteList(\array_keys(Config::getParam('consoleProjectScopes')))), 'List of resource types to request access for.', true)
->inject('response')
->inject('project')
->action(function (array $resources, Response $response, Document $project) {
$consoleProjectScopes = Config::getParam('consoleProjectScopes');
$scopes = empty($resources)
? \array_values($consoleProjectScopes)
: \array_values(\array_intersect_key($consoleProjectScopes, \array_flip($resources)));
if (empty($scopes)) {
throw new Exception(Exception::GENERAL_UNAUTHORIZED_SCOPE);
}
$jwt = new JWT(System::getEnv('_APP_OPENSSL_KEY_V1'), 'HS256', APP_CONSOLE_KEY_TTL, 0);
$consoleKey = $jwt->encode([
'projectId' => 'console',
'name' => 'Migration Settings Key',
'source' => KEY_SOURCE_MIGRATION,
'scopes' => $scopes,
'disabledMetrics' => [
METRIC_DATABASES_OPERATIONS_READS,
METRIC_DATABASES_OPERATIONS_WRITES,
METRIC_NETWORK_REQUESTS,
METRIC_NETWORK_INBOUND,
METRIC_NETWORK_OUTBOUND,
],
'scopedProjectId' => $project->getId(),
]);
$response->dynamic(new Document([
'key' => API_KEY_DYNAMIC . '_' . $consoleKey,
'expire' => DateTime::addSeconds(new \DateTime(), APP_CONSOLE_KEY_TTL),
]), Response::MODEL_MIGRATION_KEY);
});
Http::get('/v1/migrations/appwrite/report')
->groups(['api', 'migrations'])
->desc('Get Appwrite migration report')
@@ -797,11 +713,12 @@ Http::get('/v1/migrations/appwrite/report')
->param('projectID', '', new Text(512), "Source's Project ID")
->param('key', '', new Text(512), "Source's API Key")
->inject('response')
->inject('getDatabasesDB')
->action(function (array $resources, string $endpoint, string $projectID, string $key, Response $response, callable $getDatabasesDB) {
->inject('dbForProject')
->inject('project')
->inject('user')
->action(function (array $resources, string $endpoint, string $projectID, string $key, Response $response) {
try {
$appwrite = new Appwrite($projectID, $endpoint, $key, $getDatabasesDB);
$appwrite = new Appwrite($projectID, $endpoint, $key);
$report = $appwrite->report($resources);
} catch (\Throwable $e) {
throw new Exception(
-60
View File
@@ -62,33 +62,16 @@ Http::get('/v1/project/usage')
METRIC_EXECUTIONS_MB_SECONDS,
METRIC_BUILDS_MB_SECONDS,
METRIC_DOCUMENTS,
METRIC_DOCUMENTS_DOCUMENTSDB,
METRIC_DATABASES,
METRIC_DATABASES_DOCUMENTSDB,
METRIC_USERS,
METRIC_BUCKETS,
METRIC_FILES_STORAGE,
METRIC_DATABASES_STORAGE,
METRIC_DATABASES_STORAGE_DOCUMENTSDB,
METRIC_DEPLOYMENTS_STORAGE,
METRIC_BUILDS_STORAGE,
METRIC_DATABASES_OPERATIONS_READS,
METRIC_DATABASES_OPERATIONS_READS_DOCUMENTSDB,
METRIC_DATABASES_OPERATIONS_WRITES,
METRIC_DATABASES_OPERATIONS_WRITES_DOCUMENTSDB,
METRIC_FILES_IMAGES_TRANSFORMED,
// VectorsDB totals
METRIC_DATABASES_VECTORSDB,
METRIC_COLLECTIONS_VECTORSDB,
METRIC_DOCUMENTS_VECTORSDB,
METRIC_DATABASES_STORAGE_VECTORSDB,
METRIC_DATABASES_OPERATIONS_READS_VECTORSDB,
METRIC_DATABASES_OPERATIONS_WRITES_VECTORSDB,
// Embeddings totals
METRIC_EMBEDDINGS_TEXT,
METRIC_EMBEDDINGS_TEXT_TOTAL_TOKENS,
METRIC_EMBEDDINGS_TEXT_TOTAL_DURATION,
METRIC_EMBEDDINGS_TEXT_TOTAL_ERROR
],
'period' => [
METRIC_NETWORK_REQUESTS,
@@ -97,26 +80,11 @@ Http::get('/v1/project/usage')
METRIC_USERS,
METRIC_EXECUTIONS,
METRIC_DATABASES_STORAGE,
METRIC_DATABASES_STORAGE_DOCUMENTSDB,
METRIC_EXECUTIONS_MB_SECONDS,
METRIC_BUILDS_MB_SECONDS,
METRIC_DATABASES_OPERATIONS_READS,
METRIC_DATABASES_OPERATIONS_READS_DOCUMENTSDB,
METRIC_DATABASES_OPERATIONS_WRITES,
METRIC_DATABASES_OPERATIONS_WRITES_DOCUMENTSDB,
METRIC_FILES_IMAGES_TRANSFORMED,
// VectorsDB time series
METRIC_DATABASES_VECTORSDB,
METRIC_COLLECTIONS_VECTORSDB,
METRIC_DOCUMENTS_VECTORSDB,
METRIC_DATABASES_STORAGE_VECTORSDB,
METRIC_DATABASES_OPERATIONS_READS_VECTORSDB,
METRIC_DATABASES_OPERATIONS_WRITES_VECTORSDB,
// Embeddings time series
METRIC_EMBEDDINGS_TEXT,
METRIC_EMBEDDINGS_TEXT_TOTAL_TOKENS,
METRIC_EMBEDDINGS_TEXT_TOTAL_DURATION,
METRIC_EMBEDDINGS_TEXT_TOTAL_ERROR
]
];
@@ -389,11 +357,8 @@ Http::get('/v1/project/usage')
'buildsMbSecondsTotal' => $total[METRIC_BUILDS_MB_SECONDS],
'documentsTotal' => $total[METRIC_DOCUMENTS],
'rowsTotal' => $total[METRIC_DOCUMENTS],
'documentsdbDocumentsTotal' => $total[METRIC_DOCUMENTS_DOCUMENTSDB],
'databasesTotal' => $total[METRIC_DATABASES],
'documentsdbTotal' => $total[METRIC_DATABASES_DOCUMENTSDB],
'databasesStorageTotal' => $total[METRIC_DATABASES_STORAGE],
'documentsdbDatabasesStorageTotal' => $total[METRIC_DATABASES_STORAGE_DOCUMENTSDB],
'usersTotal' => $total[METRIC_USERS],
'bucketsTotal' => $total[METRIC_BUCKETS],
'filesStorageTotal' => $total[METRIC_FILES_STORAGE],
@@ -402,27 +367,10 @@ Http::get('/v1/project/usage')
'deploymentsStorageTotal' => $total[METRIC_DEPLOYMENTS_STORAGE],
'databasesReadsTotal' => $total[METRIC_DATABASES_OPERATIONS_READS],
'databasesWritesTotal' => $total[METRIC_DATABASES_OPERATIONS_WRITES],
'documentsdbDatabasesReadsTotal' => $total[METRIC_DATABASES_OPERATIONS_READS_DOCUMENTSDB],
'documentsdbDatabasesWritesTotal' => $total[METRIC_DATABASES_OPERATIONS_WRITES_DOCUMENTSDB],
'vectorsdbDatabasesTotal' => $total[METRIC_DATABASES_VECTORSDB] ?? 0,
'vectorsdbCollectionsTotal' => $total[METRIC_COLLECTIONS_VECTORSDB] ?? 0,
'vectorsdbDocumentsTotal' => $total[METRIC_DOCUMENTS_VECTORSDB] ?? 0,
'vectorsdbDatabasesStorageTotal' => $total[METRIC_DATABASES_STORAGE_VECTORSDB] ?? 0,
'vectorsdbDatabasesReadsTotal' => $total[METRIC_DATABASES_OPERATIONS_READS_VECTORSDB] ?? 0,
'vectorsdbDatabasesWritesTotal' => $total[METRIC_DATABASES_OPERATIONS_WRITES_VECTORSDB] ?? 0,
'executionsBreakdown' => $executionsBreakdown,
'bucketsBreakdown' => $bucketsBreakdown,
'databasesReads' => $usage[METRIC_DATABASES_OPERATIONS_READS],
'databasesWrites' => $usage[METRIC_DATABASES_OPERATIONS_WRITES],
'documentsdbDatabasesReads' => $usage[METRIC_DATABASES_OPERATIONS_READS_DOCUMENTSDB],
'documentsdbDatabasesWrites' => $usage[METRIC_DATABASES_OPERATIONS_WRITES_DOCUMENTSDB],
'documentsdbDatabasesStorage' => $usage[METRIC_DATABASES_STORAGE_DOCUMENTSDB],
'vectorsdbDatabases' => $usage[METRIC_DATABASES_VECTORSDB] ?? [],
'vectorsdbCollections' => $usage[METRIC_COLLECTIONS_VECTORSDB] ?? [],
'vectorsdbDocuments' => $usage[METRIC_DOCUMENTS_VECTORSDB] ?? [],
'vectorsdbDatabasesStorage' => $usage[METRIC_DATABASES_STORAGE_VECTORSDB] ?? [],
'vectorsdbDatabasesReads' => $usage[METRIC_DATABASES_OPERATIONS_READS_VECTORSDB] ?? [],
'vectorsdbDatabasesWrites' => $usage[METRIC_DATABASES_OPERATIONS_WRITES_VECTORSDB] ?? [],
'databasesStorageBreakdown' => $databasesStorageBreakdown,
'executionsMbSecondsBreakdown' => $executionsMbSecondsBreakdown,
'buildsMbSecondsBreakdown' => $buildsMbSecondsBreakdown,
@@ -432,14 +380,6 @@ Http::get('/v1/project/usage')
'authPhoneCountryBreakdown' => $authPhoneCountryBreakdown,
'imageTransformations' => $usage[METRIC_FILES_IMAGES_TRANSFORMED],
'imageTransformationsTotal' => $total[METRIC_FILES_IMAGES_TRANSFORMED],
'embeddingsText' => $usage[METRIC_EMBEDDINGS_TEXT] ?? [],
'embeddingsTextTokens' => $usage[METRIC_EMBEDDINGS_TEXT_TOTAL_TOKENS] ?? [],
'embeddingsTextDuration' => $usage[METRIC_EMBEDDINGS_TEXT_TOTAL_DURATION] ?? [],
'embeddingsTextErrors' => $usage[METRIC_EMBEDDINGS_TEXT_TOTAL_ERROR] ?? [],
'embeddingsTextTotal' => $total[METRIC_EMBEDDINGS_TEXT] ?? 0,
'embeddingsTextTokensTotal' => $total[METRIC_EMBEDDINGS_TEXT_TOTAL_TOKENS] ?? 0,
'embeddingsTextDurationTotal' => $total[METRIC_EMBEDDINGS_TEXT_TOTAL_DURATION] ?? 0,
'embeddingsTextErrorsTotal' => $total[METRIC_EMBEDDINGS_TEXT_TOTAL_ERROR] ?? 0,
]), Response::MODEL_USAGE_PROJECT);
});
+105 -114
View File
@@ -10,14 +10,16 @@ use Appwrite\Event\Delete;
use Appwrite\Event\Event;
use Appwrite\Event\Func;
use Appwrite\Event\Mail;
use Appwrite\Event\Message\Usage as UsageMessage;
use Appwrite\Event\Messaging;
use Appwrite\Event\Publisher\Usage as UsagePublisher;
use Appwrite\Event\Realtime;
use Appwrite\Event\StatsUsage;
use Appwrite\Event\Webhook;
use Appwrite\Extend\Exception;
use Appwrite\Extend\Exception as AppwriteException;
use Appwrite\Functions\EventProcessor;
use Appwrite\SDK\Method;
use Appwrite\Usage\Context;
use Appwrite\Utopia\Database\Documents\User;
use Appwrite\Utopia\Request;
use Appwrite\Utopia\Response;
@@ -53,7 +55,7 @@ $parseLabel = function (string $label, array $responsePayload, array $requestPar
$replace = $parts[1] ?? '';
$params = match ($namespace) {
'user' => (array)$user,
'user' => (array) $user,
'request' => $requestParams,
default => $responsePayload,
};
@@ -61,13 +63,13 @@ $parseLabel = function (string $label, array $responsePayload, array $requestPar
if (array_key_exists($replace, $params)) {
$replacement = $params[$replace];
// Convert to string if it's not already a string
if (!is_string($replacement)) {
if (! is_string($replacement)) {
if (is_array($replacement)) {
$replacement = json_encode($replacement);
} elseif (is_object($replacement) && method_exists($replacement, '__toString')) {
$replacement = (string)$replacement;
$replacement = (string) $replacement;
} elseif (is_scalar($replacement)) {
$replacement = (string)$replacement;
$replacement = (string) $replacement;
} else {
throw new Exception(Exception::GENERAL_SERVER_ERROR, "The server encountered an error while parsing the label: $label. Please create an issue on GitHub to allow us to investigate further https://github.com/appwrite/appwrite/issues/new/choose");
}
@@ -75,6 +77,7 @@ $parseLabel = function (string $label, array $responsePayload, array $requestPar
$label = \str_replace($find, $replacement, $label);
}
}
return $label;
};
@@ -160,7 +163,7 @@ Http::init()
$scopes = $roles[$role]['scopes'];
// Step 5: API Key Authentication
if (!empty($apiKey)) {
if (! empty($apiKey)) {
// Check if key is expired
if ($apiKey->isExpired()) {
throw new Exception(Exception::PROJECT_KEY_EXPIRED);
@@ -170,7 +173,6 @@ Http::init()
$role = $apiKey->getRole();
$scopes = $apiKey->getScopes();
// Handle special app role case
if ($apiKey->getRole() === User::ROLE_APPS) {
// Disable authorization checks for project API keys
@@ -193,19 +195,19 @@ Http::init()
// For standard keys, update last accessed time
if (\in_array($apiKey->getType(), [API_KEY_STANDARD, API_KEY_ORGANIZATION, API_KEY_ACCOUNT])) {
$dbKey = null;
if (!empty($apiKey->getProjectId())) {
if (! empty($apiKey->getProjectId())) {
$dbKey = $project->find(
key: 'secret',
find: $request->getHeader('x-appwrite-key', ''),
subject: 'keys'
);
} elseif (!empty($apiKey->getUserId())) {
} elseif (! empty($apiKey->getUserId())) {
$dbKey = $user->find(
key: 'secret',
find: $request->getHeader('x-appwrite-key', ''),
subject: 'keys'
);
} elseif (!empty($apiKey->getTeamId())) {
} elseif (! empty($apiKey->getTeamId())) {
$dbKey = $team->find(
key: 'secret',
find: $request->getHeader('x-appwrite-key', ''),
@@ -213,9 +215,7 @@ Http::init()
);
}
if (!$dbKey) {
\var_dump($apiKey);
\var_dump($request->getHeader('x-appwrite-key', ''));
if (! $dbKey) {
throw new Exception(Exception::USER_UNAUTHORIZED);
}
@@ -233,7 +233,7 @@ Http::init()
if ($sdk !== 'UNKNOWN' && $sdkValidator->isValid($sdk)) {
$sdks = $dbKey->getAttribute('sdks', []);
if (!in_array($sdk, $sdks)) {
if (! in_array($sdk, $sdks)) {
$sdks[] = $sdk;
$updates->setAttribute('sdks', $sdks);
@@ -241,14 +241,14 @@ Http::init()
}
}
if (!$updates->isEmpty()) {
if (! $updates->isEmpty()) {
$dbForPlatform->getAuthorization()->skip(fn () => $dbForPlatform->updateDocument('keys', $dbKey->getId(), $updates));
if (!empty($apiKey->getProjectId())) {
if (! empty($apiKey->getProjectId())) {
$dbForPlatform->getAuthorization()->skip(fn () => $dbForPlatform->purgeCachedDocument('projects', $project->getId()));
} elseif (!empty($apiKey->getUserId())) {
} elseif (! empty($apiKey->getUserId())) {
$dbForPlatform->getAuthorization()->skip(fn () => $dbForPlatform->purgeCachedDocument('users', $user->getId()));
} elseif (!empty($apiKey->getTeamId())) {
} elseif (! empty($apiKey->getTeamId())) {
$dbForPlatform->getAuthorization()->skip(fn () => $dbForPlatform->purgeCachedDocument('teams', $team->getId()));
}
}
@@ -285,7 +285,7 @@ Http::init()
}
}
} // Admin User Authentication
elseif (($project->getId() === 'console' && !$team->isEmpty() && !$user->isEmpty()) || ($project->getId() !== 'console' && !$user->isEmpty() && $mode === APP_MODE_ADMIN)) {
elseif (($project->getId() === 'console' && ! $team->isEmpty() && ! $user->isEmpty()) || ($project->getId() !== 'console' && ! $user->isEmpty() && $mode === APP_MODE_ADMIN)) {
$teamId = $team->getId();
$adminRoles = [];
$memberships = $user->getAttribute('memberships', []);
@@ -310,7 +310,7 @@ Http::init()
// Useful for those who have project-specific roles but don't have team-wide role.
$scopes = ['teams.read', 'projects.read'];
foreach ($adminRoles as $adminRole) {
$isTeamWideRole = !str_starts_with($adminRole, 'project-');
$isTeamWideRole = ! str_starts_with($adminRole, 'project-');
$isProjectSpecificRole = $projectId !== 'console' && str_starts_with($adminRole, 'project-' . $projectId);
if ($isTeamWideRole || $isProjectSpecificRole) {
@@ -338,23 +338,6 @@ Http::init()
$scopes = \array_unique($scopes);
// Migration-sourced keys are scoped to a single project's console
// endpoints (e.g. /v1/projects/:projectId/platforms). Verify the
// URL :projectId matches the token's scopedProjectId.
if (!empty($apiKey) && $apiKey->getSource() === KEY_SOURCE_MIGRATION) {
$scopedProjectId = $apiKey->getScopedProjectId();
if (empty($scopedProjectId)) {
throw new Exception(Exception::GENERAL_UNAUTHORIZED_SCOPE);
}
$pathValues = $route->getPathValues($request);
if (($pathValues['projectId'] ?? '') !== $scopedProjectId) {
throw new Exception(Exception::GENERAL_UNAUTHORIZED_SCOPE);
}
}
$authorization->addRole($role);
foreach ($user->getRoles($authorization) as $authRole) {
$authorization->addRole($authRole);
@@ -365,18 +348,18 @@ Http::init()
* But, for actions on resources (sites, functions, etc.) in a non-console project, we explicitly check
* whether the admin user has necessary permission on the project (sites, functions, etc. don't have permissions associated to them).
*/
if (empty($apiKey) && !$user->isEmpty() && $project->getId() !== 'console' && $mode === APP_MODE_ADMIN) {
if (empty($apiKey) && ! $user->isEmpty() && $project->getId() !== 'console' && $mode === APP_MODE_ADMIN) {
$input = new Input(Database::PERMISSION_READ, $project->getPermissionsByType(Database::PERMISSION_READ));
$initialStatus = $authorization->getStatus();
$authorization->enable();
if (!$authorization->isValid($input)) {
if (! $authorization->isValid($input)) {
throw new Exception(Exception::PROJECT_NOT_FOUND);
}
$authorization->setStatus($initialStatus);
}
// Step 6: Update project and user last activity
if (!$project->isEmpty() && $project->getId() !== 'console') {
if (! $project->isEmpty() && $project->getId() !== 'console') {
$accessedAt = $project->getAttribute('accessedAt', 0);
if (DateTime::formatTz(DateTime::addSeconds(new \DateTime(), -APP_PROJECT_ACCESS)) > $accessedAt) {
$authorization->skip(fn () => $dbForPlatform->updateDocument('projects', $project->getId(), new Document([
@@ -385,12 +368,12 @@ Http::init()
}
}
if (!empty($user->getId())) {
if (! empty($user->getId())) {
$accessedAt = $user->getAttribute('accessedAt', 0);
if (DateTime::formatTz(DateTime::addSeconds(new \DateTime(), -APP_USER_ACCESS)) > $accessedAt) {
$user->setAttribute('accessedAt', DateTime::now());
if ($project->getId() !== 'console' && APP_MODE_ADMIN !== $mode) {
if ($project->getId() !== 'console' && $mode !== APP_MODE_ADMIN) {
$dbForProject->updateDocument('users', $user->getId(), new Document([
'accessedAt' => $user->getAttribute('accessedAt')
]));
@@ -414,26 +397,26 @@ Http::init()
$method = $method[0];
}
if (!empty($method)) {
if (! empty($method)) {
$namespace = $method->getNamespace();
if (
array_key_exists($namespace, $project->getAttribute('services', []))
&& !$project->getAttribute('services', [])[$namespace]
&& !(User::isPrivileged($authorization->getRoles()) || User::isApp($authorization->getRoles()))
&& ! $project->getAttribute('services', [])[$namespace]
&& ! (User::isPrivileged($authorization->getRoles()) || User::isApp($authorization->getRoles()))
) {
throw new Exception(Exception::GENERAL_SERVICE_DISABLED);
}
}
// Step 9: Validate scope permissions
$allowed = (array)$route->getLabel('scope', 'none');
$allowed = (array) $route->getLabel('scope', 'none');
if (empty(\array_intersect($allowed, $scopes))) {
throw new Exception(Exception::GENERAL_UNAUTHORIZED_SCOPE, $user->getAttribute('email', 'User') . ' (role: ' . \strtolower($roles[$role]['label']) . ') missing scopes (' . \json_encode($allowed) . ')');
}
// Step 10: Check if user is blocked
if (false === $user->getAttribute('status')) { // Account is blocked
if ($user->getAttribute('status') === false) { // Account is blocked
throw new Exception(Exception::USER_BLOCKED);
}
@@ -451,7 +434,7 @@ Http::init()
$minimumFactors = ($mfaEnabled && $hasMoreFactors) ? 2 : 1;
// Step 13: Handle Multi-Factor Authentication
if (!in_array('mfa', $route->getGroups())) {
if (! in_array('mfa', $route->getGroups())) {
if ($session && \count($session->getAttribute('factors', [])) < $minimumFactors) {
throw new Exception(Exception::USER_MORE_FACTORS_REQUIRED);
}
@@ -471,7 +454,7 @@ Http::init()
->inject('queueForDeletes')
->inject('queueForDatabase')
->inject('queueForBuilds')
->inject('queueForStatsUsage')
->inject('usage')
->inject('queueForFunctions')
->inject('queueForMails')
->inject('dbForProject')
@@ -484,20 +467,14 @@ Http::init()
->inject('telemetry')
->inject('platform')
->inject('authorization')
->action(function (Http $utopia, Request $request, Response $response, Document $project, Document $user, Event $queueForEvents, Messaging $queueForMessaging, Audit $queueForAudits, Delete $queueForDeletes, EventDatabase $queueForDatabase, Build $queueForBuilds, StatsUsage $queueForStatsUsage, Func $queueForFunctions, Mail $queueForMails, Database $dbForProject, callable $timelimit, Document $resourceToken, string $mode, ?Key $apiKey, array $plan, Document $devKey, Telemetry $telemetry, array $platform, Authorization $authorization) {
->action(function (Http $utopia, Request $request, Response $response, Document $project, Document $user, Event $queueForEvents, Messaging $queueForMessaging, Audit $queueForAudits, Delete $queueForDeletes, EventDatabase $queueForDatabase, Build $queueForBuilds, Context $usage, Func $queueForFunctions, Mail $queueForMails, Database $dbForProject, callable $timelimit, Document $resourceToken, string $mode, ?Key $apiKey, array $plan, Document $devKey, Telemetry $telemetry, array $platform, Authorization $authorization) {
$route = $utopia->getRoute();
$path = $route->getMatchedPath();
$databaseType = match (true) {
str_contains($path, '/documentsdb') => DATABASE_TYPE_DOCUMENTSDB,
str_contains($path, '/vectorsdb') => DATABASE_TYPE_VECTORSDB,
default => '',
};
if (
array_key_exists('rest', $project->getAttribute('apis', []))
&& !$project->getAttribute('apis', [])['rest']
&& !(User::isPrivileged($authorization->getRoles()) || User::isApp($authorization->getRoles()))
&& ! $project->getAttribute('apis', [])['rest']
&& ! (User::isPrivileged($authorization->getRoles()) || User::isApp($authorization->getRoles()))
) {
throw new AppwriteException(AppwriteException::GENERAL_API_DISABLED);
}
@@ -509,7 +486,7 @@ Http::init()
$abuseKeyLabel = $route->getLabel('abuse-key', 'url:{url},ip:{ip}');
$timeLimitArray = [];
$abuseKeyLabel = (!is_array($abuseKeyLabel)) ? [$abuseKeyLabel] : $abuseKeyLabel;
$abuseKeyLabel = (! is_array($abuseKeyLabel)) ? [$abuseKeyLabel] : $abuseKeyLabel;
foreach ($abuseKeyLabel as $abuseKey) {
$start = $request->getContentRangeStart();
@@ -522,7 +499,7 @@ Http::init()
->setParam('{ip}', $request->getIP())
->setParam('{url}', $request->getHostname() . $route->getPath())
->setParam('{method}', $request->getMethod())
->setParam('{chunkId}', (int)($start / ($end + 1 - $start)));
->setParam('{chunkId}', (int) ($start / ($end + 1 - $start)));
$timeLimitArray[] = $timeLimit;
}
@@ -534,7 +511,7 @@ Http::init()
foreach ($timeLimitArray as $timeLimit) {
foreach ($request->getParams() as $key => $value) { // Set request params as potential abuse keys
if (!empty($value)) {
if (! empty($value)) {
$timeLimit->setParam('{param-' . $key . '}', (\is_array($value)) ? \json_encode($value) : $value);
}
}
@@ -557,8 +534,8 @@ Http::init()
if (
$enabled // Abuse is enabled
&& !$isAppUser // User is not API key
&& !$isPrivilegedUser // User is not an admin
&& ! $isAppUser // User is not API key
&& ! $isPrivilegedUser // User is not an admin
&& $devKey->isEmpty() // request doesn't not contain development key
&& $abuse->check() // Route is rate-limited
) {
@@ -587,19 +564,13 @@ Http::init()
->setProject($project);
/* If a session exists, use the user associated with the session */
if (!$user->isEmpty()) {
if (! $user->isEmpty()) {
$userClone = clone $user;
// $user doesn't support `type` and can cause unintended effects.
$userClone->setAttribute('type', ACTIVITY_TYPE_USER);
$queueForAudits->setUser($userClone);
}
if (!empty($apiKey) && !empty($apiKey->getDisabledMetrics())) {
foreach ($apiKey->getDisabledMetrics() as $key) {
$queueForStatsUsage->disableMetric($key);
}
}
/* Auto-set projects */
$queueForDeletes->setProject($project);
$queueForDatabase->setProject($project);
@@ -613,69 +584,64 @@ Http::init()
$queueForBuilds->setPlatform($platform);
$queueForMails->setPlatform($platform);
$useCache = $route->getLabel('cache', false);
$storageCacheOperationsCounter = $telemetry->createCounter('storage.cache.operations.load');
if ($useCache) {
$route = $utopia->match($request);
$isImageTransformation = $route->getPath() === '/v1/storage/buckets/:bucketId/files/:fileId/preview';
$isDisabled = isset($plan['imageTransformations']) && $plan['imageTransformations'] === -1 && !User::isPrivileged($authorization->getRoles());
$isDisabled = isset($plan['imageTransformations']) && $plan['imageTransformations'] === -1 && ! User::isPrivileged($authorization->getRoles());
$key = $request->cacheIdentifier();
$cacheLog = $authorization->skip(fn () => $dbForProject->getDocument('cache', $key));
$cacheLog = $authorization->skip(fn () => $dbForProject->getDocument('cache', $key));
$cache = new Cache(
new Filesystem(APP_STORAGE_CACHE . DIRECTORY_SEPARATOR . 'app-' . $project->getId())
);
$timestamp = 60 * 60 * 24 * 180; // Temporarily increase the TTL to 180 day to ensure files in the cache are still fetched.
$data = $cache->load($key, $timestamp);
if (!empty($data) && !$cacheLog->isEmpty()) {
$usageMetric = $route->getLabel('usage.metric', null);
if ($usageMetric === METRIC_AVATARS_SCREENSHOTS_GENERATED) {
$queueForStatsUsage->disableMetric(METRIC_AVATARS_SCREENSHOTS_GENERATED);
}
if (! empty($data) && ! $cacheLog->isEmpty()) {
$parts = explode('/', $cacheLog->getAttribute('resourceType', ''));
$type = $parts[0] ?? null;
if ($type === 'bucket' && (!$isImageTransformation || !$isDisabled)) {
if ($type === 'bucket' && (! $isImageTransformation || ! $isDisabled)) {
$bucketId = $parts[1] ?? null;
$bucket = $authorization->skip(fn () => $dbForProject->getDocument('buckets', $bucketId));
$isToken = !$resourceToken->isEmpty() && $resourceToken->getAttribute('bucketInternalId') === $bucket->getSequence();
$isToken = ! $resourceToken->isEmpty() && $resourceToken->getAttribute('bucketInternalId') === $bucket->getSequence();
$isPrivilegedUser = User::isPrivileged($authorization->getRoles());
if ($bucket->isEmpty() || (!$bucket->getAttribute('enabled') && !$isAppUser && !$isPrivilegedUser)) {
if ($bucket->isEmpty() || (! $bucket->getAttribute('enabled') && ! $isAppUser && ! $isPrivilegedUser)) {
throw new Exception(Exception::STORAGE_BUCKET_NOT_FOUND);
}
if (!$bucket->getAttribute('transformations', true) && !$isAppUser && !$isPrivilegedUser) {
if (! $bucket->getAttribute('transformations', true) && ! $isAppUser && ! $isPrivilegedUser) {
throw new Exception(Exception::STORAGE_BUCKET_TRANSFORMATIONS_DISABLED);
}
$fileSecurity = $bucket->getAttribute('fileSecurity', false);
$valid = $authorization->isValid(new Input(Database::PERMISSION_READ, $bucket->getRead()));
if (!$fileSecurity && !$valid && !$isToken) {
if (! $fileSecurity && ! $valid && ! $isToken) {
throw new Exception(Exception::USER_UNAUTHORIZED);
}
$parts = explode('/', $cacheLog->getAttribute('resource'));
$fileId = $parts[1] ?? null;
if ($fileSecurity && !$valid && !$isToken) {
if ($fileSecurity && ! $valid && ! $isToken) {
$file = $dbForProject->getDocument('bucket_' . $bucket->getSequence(), $fileId);
} else {
$file = $authorization->skip(fn () => $dbForProject->getDocument('bucket_' . $bucket->getSequence(), $fileId));
}
if (!$resourceToken->isEmpty() && $resourceToken->getAttribute('fileInternalId') !== $file->getSequence()) {
if (! $resourceToken->isEmpty() && $resourceToken->getAttribute('fileInternalId') !== $file->getSequence()) {
throw new Exception(Exception::USER_UNAUTHORIZED);
}
if ($file->isEmpty()) {
throw new Exception(Exception::STORAGE_FILE_NOT_FOUND);
}
//Do not update transformedAt if it's a console user
if (!User::isPrivileged($authorization->getRoles())) {
// Do not update transformedAt if it's a console user
if (! User::isPrivileged($authorization->getRoles())) {
$transformedAt = $file->getAttribute('transformedAt', '');
if (DateTime::formatTz(DateTime::addSeconds(new \DateTime(), -APP_PROJECT_ACCESS)) > $transformedAt) {
$file->setAttribute('transformedAt', DateTime::now());
@@ -691,7 +657,7 @@ Http::init()
->addHeader('X-Appwrite-Cache', 'hit')
->setContentType($cacheLog->getAttribute('mimeType'));
$storageCacheOperationsCounter->add(1, ['result' => 'hit']);
if (!$isImageTransformation || !$isDisabled) {
if (! $isImageTransformation || ! $isDisabled) {
$response->send($data);
}
} else {
@@ -714,7 +680,7 @@ Http::init()
return;
}
if (!$user->isEmpty()) {
if (! $user->isEmpty()) {
throw new Exception(Exception::USER_SESSION_ALREADY_EXISTS);
}
});
@@ -768,7 +734,8 @@ Http::shutdown()
->inject('user')
->inject('queueForEvents')
->inject('queueForAudits')
->inject('queueForStatsUsage')
->inject('usage')
->inject('publisherForUsage')
->inject('queueForDeletes')
->inject('queueForDatabase')
->inject('queueForBuilds')
@@ -781,11 +748,12 @@ Http::shutdown()
->inject('timelimit')
->inject('eventProcessor')
->inject('bus')
->action(function (Http $utopia, Request $request, Response $response, Document $project, User $user, Event $queueForEvents, Audit $queueForAudits, StatsUsage $queueForStatsUsage, Delete $queueForDeletes, EventDatabase $queueForDatabase, Build $queueForBuilds, Messaging $queueForMessaging, Func $queueForFunctions, Event $queueForWebhooks, Realtime $queueForRealtime, Database $dbForProject, Authorization $authorization, callable $timelimit, EventProcessor $eventProcessor, Bus $bus) use ($parseLabel) {
->inject('apiKey')
->action(function (Http $utopia, Request $request, Response $response, Document $project, User $user, Event $queueForEvents, Audit $queueForAudits, Context $usage, UsagePublisher $publisherForUsage, Delete $queueForDeletes, EventDatabase $queueForDatabase, Build $queueForBuilds, Messaging $queueForMessaging, Func $queueForFunctions, Event $queueForWebhooks, Realtime $queueForRealtime, Database $dbForProject, Authorization $authorization, callable $timelimit, EventProcessor $eventProcessor, Bus $bus, ?Key $apiKey) use ($parseLabel) {
$responsePayload = $response->getPayload();
if (!empty($queueForEvents->getEvent())) {
if (! empty($queueForEvents->getEvent())) {
if (empty($queueForEvents->getPayload())) {
$queueForEvents->setPayload($responsePayload);
}
@@ -807,7 +775,7 @@ Http::shutdown()
}
// Only trigger functions if there are matching function events
if (!empty($functionsEvents)) {
if (! empty($functionsEvents)) {
foreach ($generatedEvents as $event) {
if (isset($functionsEvents[$event])) {
$queueForFunctions
@@ -819,7 +787,7 @@ Http::shutdown()
}
// Only trigger webhooks if there are matching webhook events
if (!empty($webhooksEvents)) {
if (! empty($webhooksEvents)) {
foreach ($generatedEvents as $event) {
if (isset($webhooksEvents[$event])) {
$queueForWebhooks
@@ -843,7 +811,7 @@ Http::shutdown()
if ($abuseEnabled && \count($abuseResetCode) > 0 && \in_array($response->getStatusCode(), $abuseResetCode)) {
$abuseKeyLabel = $route->getLabel('abuse-key', 'url:{url},ip:{ip}');
$abuseKeyLabel = (!is_array($abuseKeyLabel)) ? [$abuseKeyLabel] : $abuseKeyLabel;
$abuseKeyLabel = (! is_array($abuseKeyLabel)) ? [$abuseKeyLabel] : $abuseKeyLabel;
foreach ($abuseKeyLabel as $abuseKey) {
$start = $request->getContentRangeStart();
@@ -856,10 +824,10 @@ Http::shutdown()
->setParam('{ip}', $request->getIP())
->setParam('{url}', $request->getHostname() . $route->getPath())
->setParam('{method}', $request->getMethod())
->setParam('{chunkId}', (int)($start / ($end + 1 - $start)));
->setParam('{chunkId}', (int) ($start / ($end + 1 - $start)));
foreach ($request->getParams() as $key => $value) { // Set request params as potential abuse keys
if (!empty($value)) {
if (! empty($value)) {
$timeLimit->setParam('{param-' . $key . '}', (\is_array($value)) ? \json_encode($value) : $value);
}
}
@@ -873,14 +841,14 @@ Http::shutdown()
* Audit labels
*/
$pattern = $route->getLabel('audits.resource', null);
if (!empty($pattern)) {
if (! empty($pattern)) {
$resource = $parseLabel($pattern, $responsePayload, $requestParams, $user);
if (!empty($resource) && $resource !== $pattern) {
if (! empty($resource) && $resource !== $pattern) {
$queueForAudits->setResource($resource);
}
}
if (!$user->isEmpty()) {
if (! $user->isEmpty()) {
$userClone = clone $user;
// $user doesn't support `type` and can cause unintended effects.
$userClone->setAttribute('type', ACTIVITY_TYPE_USER);
@@ -906,13 +874,13 @@ Http::shutdown()
$queueForAudits->setUser($user);
}
if (!empty($queueForAudits->getResource()) && !$queueForAudits->getUser()->isEmpty()) {
if (! empty($queueForAudits->getResource()) && ! $queueForAudits->getUser()->isEmpty()) {
/**
* audits.payload is switched to default true
* in order to auto audit payload for all endpoints
*/
$pattern = $route->getLabel('audits.payload', true);
if (!empty($pattern)) {
if (! empty($pattern)) {
$queueForAudits->setPayload($responsePayload);
}
@@ -923,19 +891,19 @@ Http::shutdown()
$queueForAudits->trigger();
}
if (!empty($queueForDeletes->getType())) {
if (! empty($queueForDeletes->getType())) {
$queueForDeletes->trigger();
}
if (!empty($queueForDatabase->getType())) {
if (! empty($queueForDatabase->getType())) {
$queueForDatabase->trigger();
}
if (!empty($queueForBuilds->getType())) {
if (! empty($queueForBuilds->getType())) {
$queueForBuilds->trigger();
}
if (!empty($queueForMessaging->getType())) {
if (! empty($queueForMessaging->getType())) {
$queueForMessaging->trigger();
}
@@ -944,14 +912,14 @@ Http::shutdown()
if ($useCache) {
$resource = $resourceType = null;
$data = $response->getPayload();
if (!empty($data['payload'])) {
if (! empty($data['payload'])) {
$pattern = $route->getLabel('cache.resource', null);
if (!empty($pattern)) {
if (! empty($pattern)) {
$resource = $parseLabel($pattern, $responsePayload, $requestParams, $user);
}
$pattern = $route->getLabel('cache.resourceType', null);
if (!empty($pattern)) {
if (! empty($pattern)) {
$resourceType = $parseLabel($pattern, $responsePayload, $requestParams, $user);
}
@@ -961,7 +929,7 @@ Http::shutdown()
$key = $request->cacheIdentifier();
$signature = md5($data['payload']);
$cacheLog = $authorization->skip(fn () => $dbForProject->getDocument('cache', $key));
$cacheLog = $authorization->skip(fn () => $dbForProject->getDocument('cache', $key));
$accessedAt = $cacheLog->getAttribute('accessedAt', 0);
$now = DateTime::now();
if ($cacheLog->isEmpty()) {
@@ -994,7 +962,7 @@ Http::shutdown()
}
if ($project->getId() !== 'console') {
if (!User::isPrivileged($authorization->getRoles())) {
if (! User::isPrivileged($authorization->getRoles())) {
$bus->dispatch(new RequestCompleted(
project: $project->getArrayCopy(),
request: $request,
@@ -1002,9 +970,32 @@ Http::shutdown()
));
}
$queueForStatsUsage
->setProject($project)
->trigger();
// Publish usage metrics if context has data
if (! $usage->isEmpty()) {
$metrics = $usage->getMetrics();
// Filter out API key disabled metrics using suffix pattern matching
$disabledMetrics = $apiKey?->getDisabledMetrics() ?? [];
if (! empty($disabledMetrics)) {
$metrics = array_values(array_filter($metrics, function ($metric) use ($disabledMetrics) {
foreach ($disabledMetrics as $pattern) {
if (str_ends_with($metric['key'], $pattern)) {
return false;
}
}
return true;
}));
}
$message = new UsageMessage(
project: $project,
metrics: $metrics,
reduce: $usage->getReduce()
);
$publisherForUsage->enqueue($message);
}
}
});
+1 -19
View File
@@ -196,8 +196,6 @@ include __DIR__ . '/controllers/general.php';
function createDatabase(Http $app, string $resourceKey, string $dbName, array $collections, mixed $pools, ?callable $extraSetup = null): void
{
$max = 15;
$sleep = 2;
$max = 15;
$sleep = 2;
$attempts = 0;
@@ -411,29 +409,13 @@ $http->on(Constant::EVENT_START, function (Server $http) use ($payloadSize, $tot
});
$projectCollections = $collections['projects'];
$sharedTables = \explode(',', System::getEnv('_APP_DATABASE_SHARED_TABLES', ''));
$sharedTablesV1 = \explode(',', System::getEnv('_APP_DATABASE_SHARED_TABLES_V1', ''));
$sharedTablesV2 = \array_diff($sharedTables, $sharedTablesV1);
$documentsSharedTables = \explode(',', System::getEnv('_APP_DATABASE_DOCUMENTSDB_SHARED_TABLES', ''));
$documentsSharedTablesV1 = \explode(',', System::getEnv('_APP_DATABASE_DOCUMENTSDB_SHARED_TABLES_V1', ''));
$documentsSharedTablesV2 = \array_diff($documentsSharedTables, $documentsSharedTablesV1);
$vectorSharedTables = \explode(',', System::getEnv('_APP_DATABASE_VECTORSDB_SHARED_TABLES', ''));
$vectorSharedTablesV1 = \explode(',', System::getEnv('_APP_DATABASE_VECTORSDB_SHARED_TABLES_V1', ''));
$vectorSharedTablesV2 = \array_diff($vectorSharedTables, $vectorSharedTablesV1);
$cache = $app->getResource('cache');
// All shared tables V2 pools that need project metadata collections
$sharedTablesV2All = \array_values(\array_unique(\array_filter([
...$sharedTablesV2,
...$documentsSharedTablesV2,
...$vectorSharedTablesV2,
])));
foreach ($sharedTablesV2All as $hostname) {
foreach ($sharedTablesV2 as $hostname) {
Span::init('database.setup');
Span::add('database.hostname', $hostname);
-1
View File
@@ -25,7 +25,6 @@ Config::load('roles', __DIR__ . '/../config/roles.php', $configAdapter); // Use
Config::load('projectScopes', __DIR__ . '/../config/scopes/project.php', $configAdapter);
Config::load('organizationScopes', __DIR__ . '/../config/scopes/organization.php', $configAdapter);
Config::load('accountScopes', __DIR__ . '/../config/scopes/account.php', $configAdapter);
Config::load('consoleProjectScopes', __DIR__ . '/../config/scopes/consoleProject.php', $configAdapter);
Config::load('services', __DIR__ . '/../config/services.php', $configAdapter); // List of services
Config::load('variables', __DIR__ . '/../config/variables.php', $configAdapter); // List of env variables
Config::load('regions', __DIR__ . '/../config/regions.php', $configAdapter); // List of available regions
-56
View File
@@ -41,7 +41,6 @@ const APP_LIMIT_LIST_DEFAULT = 25; // Default maximum number of items to return
const APP_LIMIT_DATABASE_BATCH = 100; // Default maximum batch size for database operations
const APP_LIMIT_DATABASE_TRANSACTION = 100; // Default maximum operations per transaction
const APP_KEY_ACCESS = 24 * 60 * 60; // 24 hours
const APP_CONSOLE_KEY_TTL = 120; // 2 minutes
const APP_USER_ACCESS = 24 * 60 * 60; // 24 hours
const APP_PROJECT_ACCESS = 24 * 60 * 60; // 24 hours
const APP_RESOURCE_TOKEN_ACCESS = 24 * 60 * 60; // 24 hours
@@ -257,8 +256,6 @@ const API_KEY_STANDARD = 'standard';
const API_KEY_DYNAMIC = 'dynamic';
const API_KEY_ORGANIZATION = 'organization';
const API_KEY_ACCOUNT = 'account';
// API key source identifiers
const KEY_SOURCE_MIGRATION = 'migration';
// Usage metrics
const METRIC_TEAMS = 'teams';
const METRIC_USERS = 'users';
@@ -291,45 +288,6 @@ const METRIC_DATABASES_OPERATIONS_READS = 'databases.operations.reads';
const METRIC_DATABASE_ID_OPERATIONS_READS = '{databaseInternalId}.databases.operations.reads';
const METRIC_DATABASES_OPERATIONS_WRITES = 'databases.operations.writes';
const METRIC_DATABASE_ID_OPERATIONS_WRITES = '{databaseInternalId}.databases.operations.writes';
// documentsdb
const METRIC_DATABASES_DOCUMENTSDB = 'documentsdb.databases';
const METRIC_COLLECTIONS_DOCUMENTSDB = 'documentsdb.collections';
const METRIC_DATABASES_STORAGE_DOCUMENTSDB = 'documentsdb.databases.storage';
const METRIC_DATABASE_ID_COLLECTIONS_DOCUMENTSDB = 'documentsdb.{databaseInternalId}.collections';
const METRIC_DATABASE_ID_STORAGE_DOCUMENTSDB = 'documentsdb.{databaseInternalId}.databases.storage';
const METRIC_DOCUMENTS_DOCUMENTSDB = 'documentsdb.documents';
const METRIC_DATABASE_ID_DOCUMENTS_DOCUMENTSDB = 'documentsdb.{databaseInternalId}.documents';
const METRIC_DATABASE_ID_COLLECTION_ID_DOCUMENTS_DOCUMENTSDB = 'documentsdb.{databaseInternalId}.{collectionInternalId}.documents';
const METRIC_DATABASE_ID_COLLECTION_ID_STORAGE_DOCUMENTSDB = 'documentsdb.{databaseInternalId}.{collectionInternalId}.databases.storage';
const METRIC_DATABASES_OPERATIONS_READS_DOCUMENTSDB = 'documentsdb.databases.operations.reads';
const METRIC_DATABASE_ID_OPERATIONS_READS_DOCUMENTSDB = 'documentsdb.{databaseInternalId}.databases.operations.reads';
const METRIC_DATABASES_OPERATIONS_WRITES_DOCUMENTSDB = 'documentsdb.databases.operations.writes';
const METRIC_DATABASE_ID_OPERATIONS_WRITES_DOCUMENTSDB = 'documentsdb.{databaseInternalId}.databases.operations.writes';
// vectorsdb
const METRIC_DATABASES_VECTORSDB = 'vectorsdb.databases';
const METRIC_COLLECTIONS_VECTORSDB = 'vectorsdb.collections';
const METRIC_DATABASES_STORAGE_VECTORSDB = 'vectorsdb.databases.storage';
const METRIC_DATABASE_ID_COLLECTIONS_VECTORSDB = 'vectorsdb.{databaseInternalId}.collections';
const METRIC_DATABASE_ID_STORAGE_VECTORSDB = 'vectorsdb.{databaseInternalId}.databases.storage';
const METRIC_DOCUMENTS_VECTORSDB = 'vectorsdb.documents';
const METRIC_DATABASE_ID_DOCUMENTS_VECTORSDB = 'vectorsdb.{databaseInternalId}.documents';
const METRIC_DATABASE_ID_COLLECTION_ID_DOCUMENTS_VECTORSDB = 'vectorsdb.{databaseInternalId}.{collectionInternalId}.documents';
const METRIC_DATABASE_ID_COLLECTION_ID_STORAGE_VECTORSDB = 'vectorsdb.{databaseInternalId}.{collectionInternalId}.databases.storage';
const METRIC_DATABASES_OPERATIONS_READS_VECTORSDB = 'vectorsdb.databases.operations.reads';
const METRIC_DATABASE_ID_OPERATIONS_READS_VECTORSDB = 'vectorsdb.{databaseInternalId}.databases.operations.reads';
const METRIC_DATABASES_OPERATIONS_WRITES_VECTORSDB = 'vectorsdb.databases.operations.writes';
const METRIC_DATABASE_ID_OPERATIONS_WRITES_VECTORSDB = 'vectorsdb.{databaseInternalId}.databases.operations.writes';
const METRIC_EMBEDDINGS_TEXT = 'embeddings.text';
const METRIC_EMBEDDINGS_MODEL_TEXT = 'embeddings.text.{embeddingModel}';
const METRIC_EMBEDDINGS_TEXT_TOTAL_ERROR = 'embeddings.text.totalErrors';
const METRIC_EMBEDDINGS_MODEL_TEXT_TOTAL_ERROR = 'embeddings.text.{embeddingModel}.totalErrors';
const METRIC_EMBEDDINGS_TEXT_TOTAL_DURATION = 'embeddings.text.totalDuration';
const METRIC_EMBEDDINGS_MODEL_TEXT_TOTAL_DURATION = 'embeddings.text.{embeddingModel}.totalDuration';
const METRIC_EMBEDDINGS_TEXT_TOTAL_TOKENS = 'embeddings.text.totalTokens';
const METRIC_EMBEDDINGS_MODEL_TEXT_TOTAL_TOKENS = 'embeddings.text.{embeddingModel}.totalTokens';
const METRIC_BUCKETS = 'buckets';
const METRIC_FILES = 'files';
const METRIC_FILES_STORAGE = 'files.storage';
@@ -422,7 +380,6 @@ const RESOURCE_TYPE_SUBSCRIBERS = 'subscribers';
const RESOURCE_TYPE_MESSAGES = 'messages';
const RESOURCE_TYPE_EXECUTIONS = 'executions';
const RESOURCE_TYPE_VCS = 'vcs';
const RESOURCE_TYPE_EMBEDDINGS_TEXT = 'embeddingsText';
// Resource types for Tokens
const TOKENS_RESOURCE_TYPE_FILES = 'files';
@@ -444,16 +401,3 @@ const CACHE_RECONNECT_RETRY_DELAY = 1000;
// Project status
const PROJECT_STATUS_ACTIVE = 'active';
// Database types
const DATABASE_TYPE_LEGACY = 'legacy';
const DATABASE_TYPE_TABLESDB = 'tablesdb';
const DATABASE_TYPE_DOCUMENTSDB = 'documentsdb';
const DATABASE_TYPE_VECTORSDB = 'vectorsdb';
// CSV import/export allowed database types
const CSV_ALLOWED_DATABASE_TYPES = [
DATABASE_TYPE_LEGACY,
DATABASE_TYPE_TABLESDB,
DATABASE_TYPE_VECTORSDB
];
-24
View File
@@ -22,7 +22,6 @@ use Appwrite\Utopia\Response\Model\AttributeLine;
use Appwrite\Utopia\Response\Model\AttributeList;
use Appwrite\Utopia\Response\Model\AttributeLongtext;
use Appwrite\Utopia\Response\Model\AttributeMediumtext;
use Appwrite\Utopia\Response\Model\AttributeObject;
use Appwrite\Utopia\Response\Model\AttributePoint;
use Appwrite\Utopia\Response\Model\AttributePolygon;
use Appwrite\Utopia\Response\Model\AttributeRelationship;
@@ -30,7 +29,6 @@ use Appwrite\Utopia\Response\Model\AttributeString;
use Appwrite\Utopia\Response\Model\AttributeText;
use Appwrite\Utopia\Response\Model\AttributeURL;
use Appwrite\Utopia\Response\Model\AttributeVarchar;
use Appwrite\Utopia\Response\Model\AttributeVector;
use Appwrite\Utopia\Response\Model\AuthProvider;
use Appwrite\Utopia\Response\Model\BaseList;
use Appwrite\Utopia\Response\Model\Branch;
@@ -67,7 +65,6 @@ use Appwrite\Utopia\Response\Model\DetectionRuntime;
use Appwrite\Utopia\Response\Model\DetectionVariable;
use Appwrite\Utopia\Response\Model\DevKey;
use Appwrite\Utopia\Response\Model\Document as ModelDocument;
use Appwrite\Utopia\Response\Model\Embedding;
use Appwrite\Utopia\Response\Model\Error;
use Appwrite\Utopia\Response\Model\ErrorDev;
use Appwrite\Utopia\Response\Model\Execution;
@@ -101,7 +98,6 @@ use Appwrite\Utopia\Response\Model\MFARecoveryCodes;
use Appwrite\Utopia\Response\Model\MFAType;
use Appwrite\Utopia\Response\Model\Migration;
use Appwrite\Utopia\Response\Model\MigrationFirebaseProject;
use Appwrite\Utopia\Response\Model\MigrationKey;
use Appwrite\Utopia\Response\Model\MigrationReport;
use Appwrite\Utopia\Response\Model\Mock;
use Appwrite\Utopia\Response\Model\MockNumber;
@@ -140,8 +136,6 @@ use Appwrite\Utopia\Response\Model\UsageBuckets;
use Appwrite\Utopia\Response\Model\UsageCollection;
use Appwrite\Utopia\Response\Model\UsageDatabase;
use Appwrite\Utopia\Response\Model\UsageDatabases;
use Appwrite\Utopia\Response\Model\UsageDocumentsDB;
use Appwrite\Utopia\Response\Model\UsageDocumentsDBs;
use Appwrite\Utopia\Response\Model\UsageFunction;
use Appwrite\Utopia\Response\Model\UsageFunctions;
use Appwrite\Utopia\Response\Model\UsageProject;
@@ -150,12 +144,9 @@ use Appwrite\Utopia\Response\Model\UsageSites;
use Appwrite\Utopia\Response\Model\UsageStorage;
use Appwrite\Utopia\Response\Model\UsageTable;
use Appwrite\Utopia\Response\Model\UsageUsers;
use Appwrite\Utopia\Response\Model\UsageVectorsDB;
use Appwrite\Utopia\Response\Model\UsageVectorsDBs;
use Appwrite\Utopia\Response\Model\User;
use Appwrite\Utopia\Response\Model\Variable;
use Appwrite\Utopia\Response\Model\VcsContent;
use Appwrite\Utopia\Response\Model\VectorsDBCollection;
use Appwrite\Utopia\Response\Model\Webhook;
// General
@@ -220,12 +211,9 @@ Response::setModel(new BaseList('Migrations List', Response::MODEL_MIGRATION_LIS
Response::setModel(new BaseList('Migrations Firebase Projects List', Response::MODEL_MIGRATION_FIREBASE_PROJECT_LIST, 'projects', Response::MODEL_MIGRATION_FIREBASE_PROJECT));
Response::setModel(new BaseList('Specifications List', Response::MODEL_SPECIFICATION_LIST, 'specifications', Response::MODEL_SPECIFICATION));
Response::setModel(new BaseList('VCS Content List', Response::MODEL_VCS_CONTENT_LIST, 'contents', Response::MODEL_VCS_CONTENT));
Response::setModel(new BaseList('VectorsDB Collections List', Response::MODEL_VECTORSDB_COLLECTION_LIST, 'collections', Response::MODEL_VECTORSDB_COLLECTION));
Response::setModel(new BaseList('Embedding list', Response::MODEL_EMBEDDING_LIST, 'embeddings', Response::MODEL_EMBEDDING));
// Entities
Response::setModel(new Database());
Response::setModel(new Embedding());
// Collection API Models
Response::setModel(new Collection());
@@ -249,17 +237,6 @@ Response::setModel(new AttributeText());
Response::setModel(new AttributeMediumtext());
Response::setModel(new AttributeLongtext());
// DocumentsDB API Models
Response::setModel(new UsageDocumentsDBs());
Response::setModel(new UsageDocumentsDB());
// VectorsDB API Models
Response::setModel(new VectorsDBCollection());
Response::setModel(new AttributeObject());
Response::setModel(new AttributeVector());
Response::setModel(new UsageVectorsDBs());
Response::setModel(new UsageVectorsDB());
// Table API Models
Response::setModel(new Table());
Response::setModel(new Column());
@@ -380,7 +357,6 @@ Response::setModel(new Subscriber());
Response::setModel(new Target());
Response::setModel(new Migration());
Response::setModel(new MigrationReport());
Response::setModel(new MigrationKey());
Response::setModel(new MigrationFirebaseProject());
// Tests (keep last)
+3 -31
View File
@@ -160,6 +160,7 @@ $register->set('pools', function () {
'pass' => System::getEnv('_APP_DB_PASS', ''),
'path' => System::getEnv('_APP_DB_SCHEMA', ''),
]);
$fallbackForRedis = 'redis_main=' . AppwriteURL::unparse([
'scheme' => 'redis',
'host' => System::getEnv('_APP_REDIS_HOST', 'redis'),
@@ -168,23 +169,6 @@ $register->set('pools', function () {
'pass' => System::getEnv('_APP_REDIS_PASS', ''),
]);
$fallbackForDocumentsDB = 'db_main=' . AppwriteURL::unparse([
'scheme' => System::getEnv('_APP_DB_ADAPTER_DOCUMENTSDB', 'mongodb'),
'host' => System::getEnv('_APP_DB_HOST_DOCUMENTSDB', 'mongodb'),
'port' => System::getEnv('_APP_DB_PORT_DOCUMENTSDB', '27017'),
'user' => System::getEnv('_APP_DB_USER', ''),
'pass' => System::getEnv('_APP_DB_PASS', ''),
'path' => System::getEnv('_APP_DB_SCHEMA', ''),
]);
$fallbackForVectorsDB = 'db_main=' . AppwriteURL::unparse([
'scheme' => System::getEnv('_APP_DB_ADAPTER_VECTORSDB', 'postgresql'),
'host' => System::getEnv('_APP_DB_HOST_VECTORSDB', 'postgresql'),
'port' => System::getEnv('_APP_DB_PORT_VECTORSDB', '5432'),
'user' => System::getEnv('_APP_DB_USER', ''),
'pass' => System::getEnv('_APP_DB_PASS', ''),
'path' => System::getEnv('_APP_DB_SCHEMA', ''),
]);
$connections = [
'console' => [
'type' => 'database',
@@ -196,25 +180,13 @@ $register->set('pools', function () {
'type' => 'database',
'dsns' => $fallbackForDB,
'multiple' => true,
'schemes' => ['mongodb','mariadb', 'mysql','postgresql'],
],
'documentsdb' => [
'type' => 'database',
'dsns' => System::getEnv('_APP_CONNECTIONS_DATABASE_DOCUMENTSDB', $fallbackForDocumentsDB),
'multiple' => true,
'schemes' => ['mongodb'],
],
'vectorsdb' => [
'type' => 'database',
'dsns' => System::getEnv('_APP_CONNECTIONS_DATABASE_VECTORSDB', $fallbackForVectorsDB),
'multiple' => true,
'schemes' => ['postgresql'],
'schemes' => ['mariadb', 'mongodb', 'mysql', 'postgresql'],
],
'logs' => [
'type' => 'database',
'dsns' => System::getEnv('_APP_CONNECTIONS_DB_LOGS', $fallbackForDB),
'multiple' => false,
'schemes' => ['mongodb','mariadb', 'mysql','postgresql'],
'schemes' => ['mariadb', 'mongodb', 'mysql', 'postgresql'],
],
'publisher' => [
'type' => 'publisher',
+128 -293
View File
@@ -14,10 +14,10 @@ use Appwrite\Event\Func;
use Appwrite\Event\Mail;
use Appwrite\Event\Messaging;
use Appwrite\Event\Migration;
use Appwrite\Event\Publisher\Usage as UsagePublisher;
use Appwrite\Event\Realtime;
use Appwrite\Event\Screenshot;
use Appwrite\Event\StatsResources;
use Appwrite\Event\StatsUsage;
use Appwrite\Event\Webhook;
use Appwrite\Extend\Exception;
use Appwrite\Functions\EventProcessor;
@@ -26,13 +26,12 @@ use Appwrite\Network\Cors;
use Appwrite\Network\Platform;
use Appwrite\Network\Validator\Origin;
use Appwrite\Network\Validator\Redirect;
use Appwrite\Usage\Context as UsageContext;
use Appwrite\Utopia\Database\Documents\User;
use Appwrite\Utopia\Request;
use Appwrite\Utopia\Response;
use Executor\Executor;
use Utopia\Abuse\Adapters\TimeLimit\Redis as TimeLimitRedis;
use Utopia\Agents\Adapters\Ollama;
use Utopia\Agents\Agent;
use Utopia\Audit\Adapter\Database as AdapterDatabase;
use Utopia\Audit\Audit;
use Utopia\Auth\Hashes\Argon2;
@@ -59,6 +58,7 @@ use Utopia\Logger\Log;
use Utopia\Pools\Group;
use Utopia\Queue\Broker\Pool as BrokerPool;
use Utopia\Queue\Publisher;
use Utopia\Queue\Queue;
use Utopia\Storage\Device;
use Utopia\Storage\Device\AWS;
use Utopia\Storage\Device\Backblaze;
@@ -90,6 +90,7 @@ Http::setResource('register', fn () => $register);
Http::setResource('locale', function () {
$locale = new Locale(System::getEnv('_APP_LOCALE', 'en'));
$locale->setFallback(System::getEnv('_APP_LOCALE', 'en'));
return $locale;
});
@@ -110,9 +111,6 @@ Http::setResource('publisherFunctions', function (Publisher $publisher) {
Http::setResource('publisherMigrations', function (Publisher $publisher) {
return $publisher;
}, ['publisher']);
Http::setResource('publisherStatsUsage', function (Publisher $publisher) {
return $publisher;
}, ['publisher']);
Http::setResource('publisherMails', function (Publisher $publisher) {
return $publisher;
}, ['publisher']);
@@ -152,9 +150,13 @@ Http::setResource('queueForWebhooks', function (Publisher $publisher) {
Http::setResource('queueForRealtime', function () {
return new Realtime();
}, []);
Http::setResource('queueForStatsUsage', function (Publisher $publisher) {
return new StatsUsage($publisher);
}, ['publisher']);
Http::setResource('usage', function () {
return new UsageContext();
}, []);
Http::setResource('publisherForUsage', fn (Publisher $publisher) => new UsagePublisher(
$publisher,
new Queue(System::getEnv('_APP_STATS_USAGE_QUEUE_NAME', Event::STATS_USAGE_QUEUE_NAME))
), ['publisher']);
Http::setResource('queueForAudits', function (Publisher $publisher) {
return new AuditEvent($publisher);
}, ['publisher']);
@@ -188,14 +190,14 @@ Http::setResource('allowedHostnames', function (array $platform, Document $proje
$allowed = [...($platform['hostnames'] ?? [])];
/* Add platform configured hostnames */
if (!$project->isEmpty() && $project->getId() !== 'console') {
if (! $project->isEmpty() && $project->getId() !== 'console') {
$platforms = $project->getAttribute('platforms', []);
$hostnames = Platform::getHostnames($platforms);
$allowed = [...$allowed, ...$hostnames];
}
/* Add the request hostname if a dev key is found */
if (!$devKey->isEmpty()) {
if (! $devKey->isEmpty()) {
$allowed[] = $request->getHostname();
}
@@ -213,12 +215,12 @@ Http::setResource('allowedHostnames', function (array $platform, Document $proje
}
/* Allow the request origin of rule */
if (!$rule->isEmpty() && !empty($rule->getAttribute('domain', ''))) {
if (! $rule->isEmpty() && ! empty($rule->getAttribute('domain', ''))) {
$allowed[] = $rule->getAttribute('domain', '');
}
/* Allow the request origin if a dev key is found */
if (!$devKey->isEmpty() && !empty($hostname)) {
if (! $devKey->isEmpty() && ! empty($hostname)) {
$allowed[] = $hostname;
}
@@ -231,7 +233,7 @@ Http::setResource('allowedHostnames', function (array $platform, Document $proje
Http::setResource('allowedSchemes', function (array $platform, Document $project) {
$allowed = [...($platform['schemas'] ?? [])];
if (!$project->isEmpty() && $project->getId() !== 'console') {
if (! $project->isEmpty() && $project->getId() !== 'console') {
/* Add hardcoded schemes */
$allowed[] = 'exp';
$allowed[] = 'appwrite-callback-' . $project->getId();
@@ -275,7 +277,7 @@ Http::setResource('rule', function (Request $request, Database $dbForPlatform, D
// Temporary implementation until custom wildcard domains are an official feature
// Allow trusted projects; Used for Console (website) previews
if (!$permitsCurrentProject && !$rule->isEmpty() && !empty($rule->getAttribute('projectId', ''))) {
if (! $permitsCurrentProject && ! $rule->isEmpty() && ! empty($rule->getAttribute('projectId', ''))) {
$trustedProjects = [];
foreach (\explode(',', System::getEnv('_APP_CONSOLE_TRUSTED_PROJECTS', '')) as $trustedProject) {
if (empty($trustedProject)) {
@@ -288,7 +290,7 @@ Http::setResource('rule', function (Request $request, Database $dbForPlatform, D
}
}
if (!$permitsCurrentProject) {
if (! $permitsCurrentProject) {
return new Document();
}
@@ -311,16 +313,18 @@ Http::setResource('cors', function (array $allowedHostnames) {
}, ['allowedHostnames']);
Http::setResource('originValidator', function (Document $devKey, array $allowedHostnames, array $allowedSchemes) {
if (!$devKey->isEmpty()) {
if (! $devKey->isEmpty()) {
return new URL();
}
return new Origin($allowedHostnames, $allowedSchemes);
}, ['devKey', 'allowedHostnames', 'allowedSchemes']);
Http::setResource('redirectValidator', function (Document $devKey, array $allowedHostnames, array $allowedSchemes) {
if (!$devKey->isEmpty()) {
if (! $devKey->isEmpty()) {
return new URL();
}
return new Redirect($allowedHostnames, $allowedSchemes);
}, ['devKey', 'allowedHostnames', 'allowedSchemes']);
@@ -344,12 +348,11 @@ Http::setResource('user', function (string $mode, Document $project, Document $c
* overwriting the previous value.
* 7. If account API key is passed, use user of the account API key as long as user ID header matches too
*/
$authorization->setDefaultStatus(true);
$store->setKey('a_session_' . $project->getId());
if (APP_MODE_ADMIN === $mode) {
if ($mode === APP_MODE_ADMIN) {
$store->setKey('a_session_' . $console->getId());
}
@@ -364,7 +367,7 @@ Http::setResource('user', function (string $mode, Document $project, Document $c
if (empty($store->getProperty('id', '')) && empty($store->getProperty('secret', ''))) {
$sessionHeader = $request->getHeader('x-appwrite-session', '');
if (!empty($sessionHeader)) {
if (! empty($sessionHeader)) {
$store->decode($sessionHeader);
}
}
@@ -384,14 +387,14 @@ Http::setResource('user', function (string $mode, Document $project, Document $c
}
$user = null;
if (APP_MODE_ADMIN === $mode) {
if ($mode === APP_MODE_ADMIN) {
/** @var User $user */
$user = $dbForPlatform->getDocument('users', $store->getProperty('id', ''));
} else {
if ($project->isEmpty()) {
$user = new User([]);
} else {
if (!empty($store->getProperty('id', ''))) {
if (! empty($store->getProperty('id', ''))) {
if ($project->getId() === 'console') {
/** @var User $user */
$user = $dbForPlatform->getDocument('users', $store->getProperty('id', ''));
@@ -404,16 +407,16 @@ Http::setResource('user', function (string $mode, Document $project, Document $c
}
if (
!$user ||
! $user ||
$user->isEmpty() // Check a document has been found in the DB
|| !$user->sessionVerify($store->getProperty('secret', ''), $proofForToken)
|| ! $user->sessionVerify($store->getProperty('secret', ''), $proofForToken)
) { // Validate user has valid login token
$user = new User([]);
}
$authJWT = $request->getHeader('x-appwrite-jwt', '');
if (!empty($authJWT) && !$project->isEmpty()) { // JWT authentication
if (!$user->isEmpty()) {
if (! empty($authJWT) && ! $project->isEmpty()) { // JWT authentication
if (! $user->isEmpty()) {
throw new Exception(Exception::USER_JWT_AND_COOKIE_SET);
}
@@ -425,7 +428,7 @@ Http::setResource('user', function (string $mode, Document $project, Document $c
}
$jwtUserId = $payload['userId'] ?? '';
if (!empty($jwtUserId)) {
if (! empty($jwtUserId)) {
if ($mode === APP_MODE_ADMIN) {
$user = $dbForPlatform->getDocument('users', $jwtUserId);
} else {
@@ -433,7 +436,7 @@ Http::setResource('user', function (string $mode, Document $project, Document $c
}
}
$jwtSessionId = $payload['sessionId'] ?? '';
if (!empty($jwtSessionId)) {
if (! empty($jwtSessionId)) {
if (empty($user->find('$id', $jwtSessionId, 'sessions'))) { // Match JWT to active token
$user = new User([]);
}
@@ -443,22 +446,22 @@ Http::setResource('user', function (string $mode, Document $project, Document $c
// Account based on account API key
$accountKey = $request->getHeader('x-appwrite-key', '');
$accountKeyUserId = $request->getHeader('x-appwrite-user', '');
if (!empty($accountKeyUserId) && !empty($accountKey)) {
if (!$user->isEmpty()) {
if (! empty($accountKeyUserId) && ! empty($accountKey)) {
if (! $user->isEmpty()) {
throw new Exception(Exception::USER_API_KEY_AND_SESSION_SET);
}
$accountKeyUser = $dbForPlatform->getAuthorization()->skip(fn () => $dbForPlatform->getDocument('users', $accountKeyUserId));
if (!$accountKeyUser->isEmpty()) {
if (! $accountKeyUser->isEmpty()) {
$key = $accountKeyUser->find(
key: 'secret',
find: $accountKey,
subject: 'keys'
);
if (!empty($key)) {
if (! empty($key)) {
$expire = $key->getAttribute('expire');
if (!empty($expire) && $expire < DatabaseDateTime::formatTz(DatabaseDateTime::now())) {
if (! empty($expire) && $expire < DatabaseDateTime::formatTz(DatabaseDateTime::now())) {
throw new Exception(Exception::ACCOUNT_KEY_EXPIRED);
}
@@ -477,10 +480,9 @@ Http::setResource('project', function ($dbForPlatform, $request, $console, $auth
/** @var Appwrite\Utopia\Request $request */
/** @var Utopia\Database\Database $dbForPlatform */
/** @var Utopia\Database\Document $console */
$projectId = $request->getParam('project', $request->getHeader('x-appwrite-project', ''));
// Realtime channel "project" can send project=Query array
if (!\is_string($projectId)) {
if (! \is_string($projectId)) {
$projectId = $request->getHeader('x-appwrite-project', '');
}
@@ -501,7 +503,7 @@ Http::setResource('session', function (User $user, Store $store, Token $proofFor
$sessions = $user->getAttribute('sessions', []);
$sessionId = $user->sessionVerify($store->getProperty('secret', ''), $proofForToken);
if (!$sessionId) {
if (! $sessionId) {
return;
}
foreach ($sessions as $session) {
@@ -511,7 +513,6 @@ Http::setResource('session', function (User $user, Store $store, Token $proofFor
}
}
return;
}, ['user', 'store', 'proofForToken']);
Http::setResource('store', function (): Store {
@@ -535,12 +536,14 @@ Http::setResource('proofForPassword', function (): Password {
Http::setResource('proofForToken', function (): Token {
$token = new Token();
$token->setHash(new Sha());
return $token;
});
Http::setResource('proofForCode', function (): Code {
$code = new Code();
$code->setHash(new Sha());
return $code;
});
@@ -552,7 +555,7 @@ Http::setResource('authorization', function () {
return new Authorization();
}, []);
Http::setResource('dbForProject', function (Group $pools, Database $dbForPlatform, Cache $cache, Document $project, Response $response, Publisher $publisher, Publisher $publisherFunctions, Publisher $publisherWebhooks, Event $queueForEvents, Func $queueForFunctions, Webhook $queueForWebhooks, Realtime $queueForRealtime, StatsUsage $queueForStatsUsage, Authorization $authorization, Request $request) {
Http::setResource('dbForProject', function (Group $pools, Database $dbForPlatform, Cache $cache, Document $project, Response $response, Publisher $publisher, Publisher $publisherFunctions, Publisher $publisherWebhooks, Event $queueForEvents, Func $queueForFunctions, Webhook $queueForWebhooks, Realtime $queueForRealtime, UsageContext $usage, Authorization $authorization) {
if ($project->isEmpty() || $project->getId() === 'console') {
return $dbForPlatform;
}
@@ -617,9 +620,8 @@ Http::setResource('dbForProject', function (Group $pools, Database $dbForPlatfor
->from($queueForEvents)
->trigger();
/** Trigger webhooks events only if a project has them enabled */
if (!empty($project->getAttribute('webhooks'))) {
if (! empty($project->getAttribute('webhooks'))) {
$queueForWebhooks
->from($queueForEvents)
->trigger();
@@ -638,7 +640,6 @@ Http::setResource('dbForProject', function (Group $pools, Database $dbForPlatfor
*/
$functionsEventsCacheListener = function (string $event, Document $document, Document $project, Database $dbForProject) {
if ($document->getCollection() !== 'functions') {
return;
}
@@ -660,31 +661,7 @@ Http::setResource('dbForProject', function (Group $pools, Database $dbForPlatfor
$dbForProject->getCache()->purge($cacheKey);
};
/**
* Prefix metrics with database type when applicable.
* Avoids prefixing for legacy and tablesdb types to preserve historical metrics.
*/
$getDatabaseTypePrefixedMetric = function (string $databaseType, string $metric): string {
if (
$databaseType === '' ||
$databaseType === DATABASE_TYPE_LEGACY ||
$databaseType === DATABASE_TYPE_TABLESDB
) {
return $metric;
}
return $databaseType . '.' . $metric;
};
// Determine database type from request path, similar to api.php
$path = $request->getURI();
$databaseType = match (true) {
str_contains($path, '/documentsdb') => DATABASE_TYPE_DOCUMENTSDB,
str_contains($path, '/vectorsdb') => DATABASE_TYPE_VECTORSDB,
default => '',
};
$usageDatabaseListener = function (string $event, Document $document, StatsUsage $queueForStatsUsage) use ($getDatabaseTypePrefixedMetric, $databaseType) {
$usageDatabaseListener = function (string $event, Document $document, UsageContext $usage) {
$value = 1;
switch ($event) {
@@ -704,87 +681,78 @@ Http::setResource('dbForProject', function (Group $pools, Database $dbForPlatfor
switch (true) {
case $document->getCollection() === 'teams':
$queueForStatsUsage->addMetric(METRIC_TEAMS, $value); // per project
$usage->addMetric(METRIC_TEAMS, $value); // per project
break;
case $document->getCollection() === 'users':
$queueForStatsUsage->addMetric(METRIC_USERS, $value); // per project
$usage->addMetric(METRIC_USERS, $value); // per project
if ($event === Database::EVENT_DOCUMENT_DELETE) {
$queueForStatsUsage->addReduce($document);
$usage->addReduce($document);
}
break;
case $document->getCollection() === 'sessions': // sessions
$queueForStatsUsage->addMetric(METRIC_SESSIONS, $value); //per project
$usage->addMetric(METRIC_SESSIONS, $value); // per project
break;
case $document->getCollection() === 'databases': // databases
$metric = $getDatabaseTypePrefixedMetric($databaseType, METRIC_DATABASES);
$queueForStatsUsage->addMetric($metric, $value); // per project
$usage->addMetric(METRIC_DATABASES, $value); // per project
if ($event === Database::EVENT_DOCUMENT_DELETE) {
$queueForStatsUsage->addReduce($document);
$usage->addReduce($document);
}
break;
case str_starts_with($document->getCollection(), 'database_') && !str_contains($document->getCollection(), 'collection'): //collections
case str_starts_with($document->getCollection(), 'database_') && ! str_contains($document->getCollection(), 'collection'): // collections
$parts = explode('_', $document->getCollection());
$databaseInternalId = $parts[1] ?? 0;
$collectionMetric = $getDatabaseTypePrefixedMetric($databaseType, METRIC_COLLECTIONS);
$databaseIdCollectionMetric = $getDatabaseTypePrefixedMetric($databaseType, METRIC_DATABASE_ID_COLLECTIONS);
$queueForStatsUsage
->addMetric($collectionMetric, $value) // per project
->addMetric(str_replace('{databaseInternalId}', $databaseInternalId, $databaseIdCollectionMetric), $value);
$usage
->addMetric(METRIC_COLLECTIONS, $value) // per project
->addMetric(str_replace('{databaseInternalId}', $databaseInternalId, METRIC_DATABASE_ID_COLLECTIONS), $value);
if ($event === Database::EVENT_DOCUMENT_DELETE) {
$queueForStatsUsage->addReduce($document);
$usage->addReduce($document);
}
break;
case str_starts_with($document->getCollection(), 'database_') && str_contains($document->getCollection(), '_collection_'): //documents
case str_starts_with($document->getCollection(), 'database_') && str_contains($document->getCollection(), '_collection_'): // documents
$parts = explode('_', $document->getCollection());
$databaseInternalId = $parts[1] ?? 0;
$databaseInternalId = $parts[1] ?? 0;
$collectionInternalId = $parts[3] ?? 0;
$documentsMetric = $getDatabaseTypePrefixedMetric($databaseType, METRIC_DOCUMENTS);
$databaseIdDocumentsMetric = $getDatabaseTypePrefixedMetric($databaseType, METRIC_DATABASE_ID_DOCUMENTS);
$databaseIdCollectionIdDocumentsMetric = $getDatabaseTypePrefixedMetric($databaseType, METRIC_DATABASE_ID_COLLECTION_ID_DOCUMENTS);
$queueForStatsUsage
->addMetric($documentsMetric, $value) // per project
->addMetric(str_replace('{databaseInternalId}', $databaseInternalId, $databaseIdDocumentsMetric), $value) // per database
->addMetric(str_replace(['{databaseInternalId}', '{collectionInternalId}'], [$databaseInternalId, $collectionInternalId], $databaseIdCollectionIdDocumentsMetric), $value); // per collection
$usage
->addMetric(METRIC_DOCUMENTS, $value) // per project
->addMetric(str_replace('{databaseInternalId}', $databaseInternalId, METRIC_DATABASE_ID_DOCUMENTS), $value) // per database
->addMetric(str_replace(['{databaseInternalId}', '{collectionInternalId}'], [$databaseInternalId, $collectionInternalId], METRIC_DATABASE_ID_COLLECTION_ID_DOCUMENTS), $value); // per collection
break;
case $document->getCollection() === 'buckets': //buckets
$queueForStatsUsage
->addMetric(METRIC_BUCKETS, $value); // per project
case $document->getCollection() === 'buckets': // buckets
$usage->addMetric(METRIC_BUCKETS, $value); // per project
if ($event === Database::EVENT_DOCUMENT_DELETE) {
$queueForStatsUsage
$usage
->addReduce($document);
}
break;
case str_starts_with($document->getCollection(), 'bucket_'): // files
$parts = explode('_', $document->getCollection());
$bucketInternalId = $parts[1];
$queueForStatsUsage
$bucketInternalId = $parts[1];
$usage
->addMetric(METRIC_FILES, $value) // per project
->addMetric(METRIC_FILES_STORAGE, $document->getAttribute('sizeOriginal') * $value) // per project
->addMetric(str_replace('{bucketInternalId}', $bucketInternalId, METRIC_BUCKET_ID_FILES), $value) // per bucket
->addMetric(str_replace('{bucketInternalId}', $bucketInternalId, METRIC_BUCKET_ID_FILES_STORAGE), $document->getAttribute('sizeOriginal') * $value); // per bucket
break;
case $document->getCollection() === 'functions':
$queueForStatsUsage
->addMetric(METRIC_FUNCTIONS, $value); // per project
$usage->addMetric(METRIC_FUNCTIONS, $value); // per project
if ($event === Database::EVENT_DOCUMENT_DELETE) {
$queueForStatsUsage
$usage
->addReduce($document);
}
break;
case $document->getCollection() === 'sites':
$queueForStatsUsage
->addMetric(METRIC_SITES, $value); // per project
$usage->addMetric(METRIC_SITES, $value); // per project
if ($event === Database::EVENT_DOCUMENT_DELETE) {
$queueForStatsUsage
$usage
->addReduce($document);
}
break;
case $document->getCollection() === 'deployments':
$queueForStatsUsage
$usage
->addMetric(METRIC_DEPLOYMENTS, $value) // per project
->addMetric(METRIC_DEPLOYMENTS_STORAGE, $document->getAttribute('size') * $value) // per project
->addMetric(str_replace(['{resourceType}'], [$document->getAttribute('resourceType')], METRIC_RESOURCE_TYPE_DEPLOYMENTS), $value) // per function
@@ -804,33 +772,27 @@ Http::setResource('dbForProject', function (Group $pools, Database $dbForPlatfor
$queueForWebhooks = new Webhook($publisherWebhooks);
$queueForRealtime = new Realtime();
// to get consumed in the stats worker without calling to database
$queueForStatsUsage->setContext('database', new Document(['type' => $databaseType]));
$database
->on(Database::EVENT_DOCUMENT_CREATE, 'calculate-usage', fn ($event, $document) => $usageDatabaseListener($event, $document, $queueForStatsUsage))
->on(Database::EVENT_DOCUMENT_DELETE, 'calculate-usage', fn ($event, $document) => $usageDatabaseListener($event, $document, $queueForStatsUsage))
->on(Database::EVENT_DOCUMENTS_CREATE, 'calculate-usage', fn ($event, $document) => $usageDatabaseListener($event, $document, $queueForStatsUsage))
->on(Database::EVENT_DOCUMENTS_DELETE, 'calculate-usage', fn ($event, $document) => $usageDatabaseListener($event, $document, $queueForStatsUsage))
->on(Database::EVENT_DOCUMENTS_UPSERT, 'calculate-usage', fn ($event, $document) => $usageDatabaseListener($event, $document, $queueForStatsUsage))
->on(Database::EVENT_DOCUMENT_CREATE, 'create-trigger-events', fn ($event, $document) => $eventDatabaseListener(
$project,
$document,
$response,
$queueForEventsClone->from($queueForEvents),
$queueForFunctions->from($queueForEvents),
$queueForWebhooks->from($queueForEvents),
$queueForRealtime->from($queueForEvents)
))
->on(Database::EVENT_DOCUMENT_CREATE, 'purge-function-events-cache', fn ($event, $document) => $functionsEventsCacheListener($event, $document, $project, $database))
->on(Database::EVENT_DOCUMENT_UPDATE, 'purge-function-events-cache', fn ($event, $document) => $functionsEventsCacheListener($event, $document, $project, $database))
->on(Database::EVENT_DOCUMENT_DELETE, 'purge-function-events-cache', fn ($event, $document) => $functionsEventsCacheListener($event, $document, $project, $database))
;
->on(Database::EVENT_DOCUMENT_CREATE, 'calculate-usage', fn ($event, $document) => $usageDatabaseListener($event, $document, $usage))
->on(Database::EVENT_DOCUMENT_DELETE, 'calculate-usage', fn ($event, $document) => $usageDatabaseListener($event, $document, $usage))
->on(Database::EVENT_DOCUMENTS_CREATE, 'calculate-usage', fn ($event, $document) => $usageDatabaseListener($event, $document, $usage))
->on(Database::EVENT_DOCUMENTS_DELETE, 'calculate-usage', fn ($event, $document) => $usageDatabaseListener($event, $document, $usage))
->on(Database::EVENT_DOCUMENTS_UPSERT, 'calculate-usage', fn ($event, $document) => $usageDatabaseListener($event, $document, $usage))
->on(Database::EVENT_DOCUMENT_CREATE, 'create-trigger-events', fn ($event, $document) => $eventDatabaseListener(
$project,
$document,
$response,
$queueForEventsClone->from($queueForEvents),
$queueForFunctions->from($queueForEvents),
$queueForWebhooks->from($queueForEvents),
$queueForRealtime->from($queueForEvents)
))
->on(Database::EVENT_DOCUMENT_CREATE, 'purge-function-events-cache', fn ($event, $document) => $functionsEventsCacheListener($event, $document, $project, $database))
->on(Database::EVENT_DOCUMENT_UPDATE, 'purge-function-events-cache', fn ($event, $document) => $functionsEventsCacheListener($event, $document, $project, $database))
->on(Database::EVENT_DOCUMENT_DELETE, 'purge-function-events-cache', fn ($event, $document) => $functionsEventsCacheListener($event, $document, $project, $database));
return $database;
}, ['pools', 'dbForPlatform', 'cache', 'project', 'response', 'publisher', 'publisherFunctions', 'publisherWebhooks', 'queueForEvents', 'queueForFunctions', 'queueForWebhooks', 'queueForRealtime', 'queueForStatsUsage', 'authorization', 'request']);
}, ['pools', 'dbForPlatform', 'cache', 'project', 'response', 'publisher', 'publisherFunctions', 'publisherWebhooks', 'queueForEvents', 'queueForFunctions', 'queueForWebhooks', 'queueForRealtime', 'usage', 'authorization']);
Http::setResource('dbForPlatform', function (Group $pools, Cache $cache, Authorization $authorization) {
@@ -851,138 +813,6 @@ Http::setResource('dbForPlatform', function (Group $pools, Cache $cache, Authori
return $database;
}, ['pools', 'cache', 'authorization']);
Http::setResource('getDatabasesDB', function (Group $pools, Cache $cache, Document $project, Request $request, StatsUsage $queueForStatsUsage, Authorization $authorization) {
return function (Document $database) use ($pools, $cache, $project, $request, $queueForStatsUsage, $authorization): Database {
$databaseDSN = $database->getAttribute('database', '');
$databaseType = $database->getAttribute('type', '');
try {
$databaseDSN = new DSN($databaseDSN);
} catch (\InvalidArgumentException) {
// for old databases migrated through patch script
// databaseDSN determines the adapter
$databaseDSN = new DSN('mysql://'.$databaseDSN);
}
try {
$dsn = new DSN($project->getAttribute('database'));
} catch (\InvalidArgumentException) {
// TODO: Temporary until all projects are using shared tables
$dsn = new DSN('mysql://' . $project->getAttribute('database'));
}
$pool = $pools->get($databaseDSN->getHost());
$adapter = new DatabasePool($pool);
$database = new Database($adapter, $cache);
$sharedTables = \explode(',', System::getEnv('_APP_DATABASE_SHARED_TABLES', ''));
$database
->setDatabase(APP_DATABASE)
->setAuthorization($authorization)
->setMetadata('host', \gethostname())
->setMetadata('project', $project->getId())
->setTimeout(APP_DATABASE_TIMEOUT_MILLISECONDS_API)
->setMaxQueryValues(APP_DATABASE_QUERY_MAX_VALUES);
// inside pools authorization needs to be set first
$database->getAdapter()->setSupportForAttributes($databaseType !== DOCUMENTSDB);
if (\in_array($dsn->getHost(), $sharedTables)) {
$database
->setSharedTables(true)
->setTenant((int)$project->getSequence())
->setNamespace($dsn->getParam('namespace'));
} else {
$database
->setSharedTables(false)
->setTenant(null)
->setNamespace('_' . $project->getSequence());
}
$timeout = \intval($request->getHeader('x-appwrite-timeout'));
if (!empty($timeout) && Http::isDevelopment()) {
$database->setTimeout($timeout);
}
// Register database event listeners for usage stats collection
$documentsMetric = METRIC_DOCUMENTS;
$databaseIdDocumentsMetric = METRIC_DATABASE_ID_DOCUMENTS;
$databaseIdCollectionIdDocumentsMetric = METRIC_DATABASE_ID_COLLECTION_ID_DOCUMENTS;
if ($databaseType !== DATABASE_TYPE_LEGACY && $databaseType !== DATABASE_TYPE_TABLESDB) {
$documentsMetric = $databaseType. '.' .$documentsMetric;
$databaseIdDocumentsMetric = $databaseType. '.' .$databaseIdDocumentsMetric;
$databaseIdCollectionIdDocumentsMetric = $databaseType . '.' .$databaseIdCollectionIdDocumentsMetric;
}
$database
->on(Database::EVENT_DOCUMENT_CREATE, 'calculate-usage', function ($event, $document) use ($queueForStatsUsage, $documentsMetric, $databaseIdDocumentsMetric, $databaseIdCollectionIdDocumentsMetric) {
$value = 1;
if (str_starts_with($document->getCollection(), 'database_') && str_contains($document->getCollection(), '_collection_')) {
$parts = explode('_', $document->getCollection());
$databaseInternalId = $parts[1] ?? 0;
$collectionInternalId = $parts[3] ?? 0;
$queueForStatsUsage
->addMetric($documentsMetric, $value) // per project
->addMetric(str_replace('{databaseInternalId}', $databaseInternalId, $databaseIdDocumentsMetric), $value) // per database
->addMetric(str_replace(['{databaseInternalId}', '{collectionInternalId}'], [$databaseInternalId, $collectionInternalId], $databaseIdCollectionIdDocumentsMetric), $value); // per collection
}
})
->on(Database::EVENT_DOCUMENT_DELETE, 'calculate-usage', function ($event, $document) use ($queueForStatsUsage, $documentsMetric, $databaseIdDocumentsMetric, $databaseIdCollectionIdDocumentsMetric) {
$value = -1;
if (str_starts_with($document->getCollection(), 'database_') && str_contains($document->getCollection(), '_collection_')) {
$parts = explode('_', $document->getCollection());
$databaseInternalId = $parts[1] ?? 0;
$collectionInternalId = $parts[3] ?? 0;
$queueForStatsUsage
->addMetric($documentsMetric, $value) // per project
->addMetric(str_replace('{databaseInternalId}', $databaseInternalId, $databaseIdDocumentsMetric), $value) // per database
->addMetric(str_replace(['{databaseInternalId}', '{collectionInternalId}'], [$databaseInternalId, $collectionInternalId], $databaseIdCollectionIdDocumentsMetric), $value); // per collection
}
})
->on(Database::EVENT_DOCUMENTS_CREATE, 'calculate-usage', function ($event, $document) use ($queueForStatsUsage, $documentsMetric, $databaseIdDocumentsMetric, $databaseIdCollectionIdDocumentsMetric) {
$value = $document->getAttribute('modified', 0);
if (str_starts_with($document->getCollection(), 'database_') && str_contains($document->getCollection(), '_collection_')) {
$parts = explode('_', $document->getCollection());
$databaseInternalId = $parts[1] ?? 0;
$collectionInternalId = $parts[3] ?? 0;
$queueForStatsUsage
->addMetric($documentsMetric, $value) // per project
->addMetric(str_replace('{databaseInternalId}', $databaseInternalId, $databaseIdDocumentsMetric), $value) // per database
->addMetric(str_replace(['{databaseInternalId}', '{collectionInternalId}'], [$databaseInternalId, $collectionInternalId], $databaseIdCollectionIdDocumentsMetric), $value); // per collection
}
})
->on(Database::EVENT_DOCUMENTS_DELETE, 'calculate-usage', function ($event, $document) use ($queueForStatsUsage, $documentsMetric, $databaseIdDocumentsMetric, $databaseIdCollectionIdDocumentsMetric) {
$value = -1 * $document->getAttribute('modified', 0);
if (str_starts_with($document->getCollection(), 'database_') && str_contains($document->getCollection(), '_collection_')) {
$parts = explode('_', $document->getCollection());
$databaseInternalId = $parts[1] ?? 0;
$collectionInternalId = $parts[3] ?? 0;
$queueForStatsUsage
->addMetric($documentsMetric, $value) // per project
->addMetric(str_replace('{databaseInternalId}', $databaseInternalId, $databaseIdDocumentsMetric), $value) // per database
->addMetric(str_replace(['{databaseInternalId}', '{collectionInternalId}'], [$databaseInternalId, $collectionInternalId], $databaseIdCollectionIdDocumentsMetric), $value); // per collection
}
})
->on(Database::EVENT_DOCUMENTS_UPSERT, 'calculate-usage', function ($event, $document) use ($queueForStatsUsage, $documentsMetric, $databaseIdDocumentsMetric, $databaseIdCollectionIdDocumentsMetric) {
$value = $document->getAttribute('created', 0);
if (str_starts_with($document->getCollection(), 'database_') && str_contains($document->getCollection(), '_collection_')) {
$parts = explode('_', $document->getCollection());
$databaseInternalId = $parts[1] ?? 0;
$collectionInternalId = $parts[3] ?? 0;
$queueForStatsUsage
->addMetric($documentsMetric, $value) // per project
->addMetric(str_replace('{databaseInternalId}', $databaseInternalId, $databaseIdDocumentsMetric), $value) // per database
->addMetric(str_replace(['{databaseInternalId}', '{collectionInternalId}'], [$databaseInternalId, $collectionInternalId], $databaseIdCollectionIdDocumentsMetric), $value); // per collection
}
});
return $database;
};
}, ['pools','cache','project','request','queueForStatsUsage','authorization']);
Http::setResource('getProjectDB', function (Group $pools, Database $dbForPlatform, $cache, Authorization $authorization) {
$databases = [];
@@ -1011,8 +841,7 @@ Http::setResource('getProjectDB', function (Group $pools, Database $dbForPlatfor
->setMetadata('project', $project->getId())
->setTimeout(APP_DATABASE_TIMEOUT_MILLISECONDS_API)
->setMaxQueryValues(APP_DATABASE_QUERY_MAX_VALUES)
->setDocumentType('users', User::class)
;
->setDocumentType('users', User::class);
$sharedTables = \explode(',', System::getEnv('_APP_DATABASE_SHARED_TABLES', ''));
@@ -1032,6 +861,7 @@ Http::setResource('getProjectDB', function (Group $pools, Database $dbForPlatfor
if (isset($databases[$dsn->getHost()])) {
$database = $databases[$dsn->getHost()];
$configure($database);
return $database;
}
@@ -1048,8 +878,9 @@ Http::setResource('getLogsDB', function (Group $pools, Cache $cache, Authorizati
$database = null;
return function (?Document $project = null) use ($pools, $cache, $authorization, &$database) {
if ($database !== null && $project !== null && !$project->isEmpty() && $project->getId() !== 'console') {
if ($database !== null && $project !== null && ! $project->isEmpty() && $project->getId() !== 'console') {
$database->setTenant((int) $project->getSequence());
return $database;
}
@@ -1065,7 +896,7 @@ Http::setResource('getLogsDB', function (Group $pools, Cache $cache, Authorizati
->setMaxQueryValues(APP_DATABASE_QUERY_MAX_VALUES);
// set tenant
if ($project !== null && !$project->isEmpty() && $project->getId() !== 'console') {
if ($project !== null && ! $project->isEmpty() && $project->getId() !== 'console') {
$database->setTenant((int) $project->getSequence());
}
@@ -1075,6 +906,7 @@ Http::setResource('getLogsDB', function (Group $pools, Cache $cache, Authorizati
Http::setResource('audit', function ($dbForProject) {
$adapter = new AdapterDatabase($dbForProject);
return new Audit($adapter);
}, ['dbForProject']);
@@ -1090,6 +922,7 @@ Http::setResource('cache', function (Group $pools, Telemetry $telemetry) {
$cache = new Cache(new Sharding($adapters));
$cache->setTelemetry($telemetry);
return $cache;
}, ['pools', 'telemetry']);
@@ -1135,9 +968,9 @@ Http::setResource('deviceForBuilds', function ($project, Telemetry $telemetry) {
function getDevice(string $root, string $connection = ''): Device
{
$connection = !empty($connection) ? $connection : System::getEnv('_APP_CONNECTIONS_STORAGE', '');
$connection = ! empty($connection) ? $connection : System::getEnv('_APP_CONNECTIONS_STORAGE', '');
if (!empty($connection)) {
if (! empty($connection)) {
$acl = 'private';
$device = Storage::DEVICE_LOCAL;
$accessKey = '';
@@ -1159,8 +992,9 @@ function getDevice(string $root, string $connection = ''): Device
switch ($device) {
case Storage::DEVICE_S3:
if (!empty($url)) {
$bucketRoot = (!empty($bucket) ? $bucket . '/' : '') . \ltrim($root, '/');
if (! empty($url)) {
$bucketRoot = (! empty($bucket) ? $bucket . '/' : '') . \ltrim($root, '/');
return new S3($bucketRoot, $accessKey, $accessSecret, $url, $region, $acl);
} else {
return new AWS($root, $accessKey, $accessSecret, $bucket, $region, $acl);
@@ -1169,6 +1003,7 @@ function getDevice(string $root, string $connection = ''): Device
case STORAGE::DEVICE_DO_SPACES:
$device = new DOSpaces($root, $accessKey, $accessSecret, $bucket, $region, $acl);
$device->setHttpVersion(S3::HTTP_VERSION_1_1);
return $device;
case Storage::DEVICE_BACKBLAZE:
return new Backblaze($root, $accessKey, $accessSecret, $bucket, $region, $acl);
@@ -1192,8 +1027,9 @@ function getDevice(string $root, string $connection = ''): Device
$s3Bucket = System::getEnv('_APP_STORAGE_S3_BUCKET', '');
$s3Acl = 'private';
$s3EndpointUrl = System::getEnv('_APP_STORAGE_S3_ENDPOINT', '');
if (!empty($s3EndpointUrl)) {
$bucketRoot = (!empty($s3Bucket) ? $s3Bucket . '/' : '') . \ltrim($root, '/');
if (! empty($s3EndpointUrl)) {
$bucketRoot = (! empty($s3Bucket) ? $s3Bucket . '/' : '') . \ltrim($root, '/');
return new S3($bucketRoot, $s3AccessKey, $s3SecretKey, $s3EndpointUrl, $s3Region, $s3Acl);
} else {
return new AWS($root, $s3AccessKey, $s3SecretKey, $s3Bucket, $s3Region, $s3Acl);
@@ -1207,6 +1043,7 @@ function getDevice(string $root, string $connection = ''): Device
$doSpacesAcl = 'private';
$device = new DOSpaces($root, $doSpacesAccessKey, $doSpacesSecretKey, $doSpacesBucket, $doSpacesRegion, $doSpacesAcl);
$device->setHttpVersion(S3::HTTP_VERSION_1_1);
return $device;
case Storage::DEVICE_BACKBLAZE:
$backblazeAccessKey = System::getEnv('_APP_STORAGE_BACKBLAZE_ACCESS_KEY', '');
@@ -1214,6 +1051,7 @@ function getDevice(string $root, string $connection = ''): Device
$backblazeRegion = System::getEnv('_APP_STORAGE_BACKBLAZE_REGION', '');
$backblazeBucket = System::getEnv('_APP_STORAGE_BACKBLAZE_BUCKET', '');
$backblazeAcl = 'private';
return new Backblaze($root, $backblazeAccessKey, $backblazeSecretKey, $backblazeBucket, $backblazeRegion, $backblazeAcl);
case Storage::DEVICE_LINODE:
$linodeAccessKey = System::getEnv('_APP_STORAGE_LINODE_ACCESS_KEY', '');
@@ -1221,6 +1059,7 @@ function getDevice(string $root, string $connection = ''): Device
$linodeRegion = System::getEnv('_APP_STORAGE_LINODE_REGION', '');
$linodeBucket = System::getEnv('_APP_STORAGE_LINODE_BUCKET', '');
$linodeAcl = 'private';
return new Linode($root, $linodeAccessKey, $linodeSecretKey, $linodeBucket, $linodeRegion, $linodeAcl);
case Storage::DEVICE_WASABI:
$wasabiAccessKey = System::getEnv('_APP_STORAGE_WASABI_ACCESS_KEY', '');
@@ -1228,6 +1067,7 @@ function getDevice(string $root, string $connection = ''): Device
$wasabiRegion = System::getEnv('_APP_STORAGE_WASABI_REGION', '');
$wasabiBucket = System::getEnv('_APP_STORAGE_WASABI_BUCKET', '');
$wasabiAcl = 'private';
return new Wasabi($root, $wasabiAccessKey, $wasabiSecretKey, $wasabiBucket, $wasabiRegion, $wasabiAcl);
}
}
@@ -1254,7 +1094,6 @@ Http::setResource('passwordsDictionary', function ($register) {
return $register->get('passwordsDictionary');
}, ['register']);
Http::setResource('servers', function () {
$platforms = Config::getParam('sdks');
$server = $platforms[APP_SDK_PLATFORM_SERVER];
@@ -1362,16 +1201,17 @@ Http::setResource('gitHub', function (Cache $cache) {
}, ['cache']);
Http::setResource('requestTimestamp', function ($request) {
//TODO: Move this to the Request class itself
// TODO: Move this to the Request class itself
$timestampHeader = $request->getHeader('x-appwrite-timestamp');
$requestTimestamp = null;
if (!empty($timestampHeader)) {
if (! empty($timestampHeader)) {
try {
$requestTimestamp = new \DateTime($timestampHeader);
} catch (\Throwable $e) {
throw new Exception(Exception::GENERAL_ARGUMENT_INVALID, 'Invalid X-Appwrite-Timestamp header value');
}
}
return $requestTimestamp;
}, ['request']);
@@ -1388,13 +1228,13 @@ Http::setResource('devKey', function (Request $request, Document $project, array
// Check if given key match project's development keys
$key = $project->find('secret', $devKey, 'devKeys');
if (!$key) {
if (! $key) {
return new Document([]);
}
// check expiration
$expire = $key->getAttribute('expire');
if (!empty($expire) && $expire < DatabaseDateTime::formatTz(DatabaseDateTime::now())) {
if (! empty($expire) && $expire < DatabaseDateTime::formatTz(DatabaseDateTime::now())) {
return new Document([]);
}
@@ -1415,7 +1255,7 @@ Http::setResource('devKey', function (Request $request, Document $project, array
if ($sdk !== 'UNKNOWN' && $sdkValidator->isValid($sdk)) {
$sdks = $key->getAttribute('sdks', []);
if (!in_array($sdk, $sdks)) {
if (! in_array($sdk, $sdks)) {
$sdks[] = $sdk;
$key->setAttribute('sdks', $sdks);
@@ -1438,7 +1278,7 @@ Http::setResource('team', function (Document $project, Database $dbForPlatform,
$teamInternalId = $project->getAttribute('teamInternalId', '');
} else {
$route = $utopia->match($request);
$path = !empty($route) ? $route->getPath() : $request->getURI();
$path = ! empty($route) ? $route->getPath() : $request->getURI();
$orgHeader = $request->getHeader('x-appwrite-organization', '');
if (str_starts_with($path, '/v1/projects/:projectId')) {
$uri = $request->getURI();
@@ -1453,8 +1293,9 @@ Http::setResource('team', function (Document $project, Database $dbForPlatform,
}
$team = $authorization->skip(fn () => $dbForPlatform->getDocument('teams', $teamId));
return $team;
} elseif (!empty($orgHeader)) {
} elseif (! empty($orgHeader)) {
return $authorization->skip(fn () => $dbForPlatform->getDocument('teams', $orgHeader));
}
}
@@ -1484,13 +1325,13 @@ Http::setResource('previewHostname', function (Request $request, ?Key $apiKey) {
if (Http::isDevelopment()) {
$allowed = true;
} elseif (!\is_null($apiKey) && $apiKey->getHostnameOverride() === true) {
} elseif (! \is_null($apiKey) && $apiKey->getHostnameOverride() === true) {
$allowed = true;
}
if ($allowed) {
$host = $request->getQuery('appwrite-hostname', $request->getHeader('x-appwrite-hostname', '')) ?? '';
if (!empty($host)) {
if (! empty($host)) {
return $host;
}
}
@@ -1511,19 +1352,19 @@ Http::setResource('apiKey', function (Request $request, Document $project, Docum
$organizationHeader = $request->getHeader('x-appwrite-organization');
$projectHeader = $request->getHeader('x-appwrite-project');
if (!empty($key->getProjectId())) {
if (! empty($key->getProjectId())) {
if (empty($projectHeader) || $projectHeader !== $key->getProjectId()) {
throw new Exception(Exception::PROJECT_ID_MISSING);
}
}
if (!empty($key->getUserId())) {
if (! empty($key->getUserId())) {
if (empty($userHeader) || $userHeader !== $key->getUserId()) {
throw new Exception(Exception::USER_ID_MISSING);
}
}
if (!empty($key->getTeamId())) {
if (! empty($key->getTeamId())) {
if (empty($organizationHeader) || $organizationHeader !== $key->getTeamId()) {
throw new Exception(Exception::ORGANIZATION_ID_MISSING);
}
@@ -1537,7 +1378,7 @@ Http::setResource('executor', fn () => new Executor());
Http::setResource('resourceToken', function ($project, $dbForProject, $request, Authorization $authorization) {
$tokenJWT = $request->getParam('token');
if (!empty($tokenJWT) && !$project->isEmpty()) { // JWT authentication
if (! empty($tokenJWT) && ! $project->isEmpty()) { // JWT authentication
// Use a large but reasonable maxAge to avoid auto-exp when token has no expiry
$jwt = new JWT(System::getEnv('_APP_OPENSSL_KEY_V1'), RESOURCE_TOKEN_ALGORITHM, RESOURCE_TOKEN_MAX_AGE, RESOURCE_TOKEN_LEEWAY); // Instantiate with key, algo, maxAge and leeway.
@@ -1597,12 +1438,13 @@ Http::setResource('resourceToken', function ($project, $dbForProject, $request,
default => throw new Exception(Exception::TOKEN_RESOURCE_TYPE_INVALID),
};
}
return new Document([]);
}, ['project', 'dbForProject', 'request', 'authorization']);
Http::setResource('transactionState', function (Database $dbForProject, Authorization $authorization, callable $getDatabasesDB) {
return new TransactionState($dbForProject, $authorization, $getDatabasesDB);
}, ['dbForProject', 'authorization', 'getDatabasesDB']);
Http::setResource('transactionState', function (Database $dbForProject, Authorization $authorization) {
return new TransactionState($dbForProject, $authorization);
}, ['dbForProject', 'authorization']);
Http::setResource('executionsRetentionCount', function (Document $project, array $plan) {
if ($project->getId() === 'console' || empty($plan)) {
@@ -1611,10 +1453,3 @@ Http::setResource('executionsRetentionCount', function (Document $project, array
return (int) ($plan['executionsRetentionCount'] ?? 100);
}, ['project', 'plan']);
Http::setResource('embeddingAgent', function ($register) {
$adapter = new Ollama();
$adapter->setEndpoint(System::getEnv('_APP_EMBEDDING_ENDPOINT', 'http://ollama:11434/api/embed'));
$adapter->setTimeout((int) System::getEnv('_APP_EMBEDDING_TIMEOUT', '30000'));
return new Agent($adapter);
}, ['register']);
+6 -1
View File
@@ -5,4 +5,9 @@ use Utopia\Span\Span;
use Utopia\Span\Storage;
Span::setStorage(new Storage\Coroutine());
Span::addExporter(new Exporter\Pretty());
Span::addExporter(new Exporter\Pretty(), function (Span $span): bool {
if (\str_starts_with($span->getAction(), 'listener.')) {
return $span->getError() !== null;
}
return true;
});
+31 -81
View File
@@ -13,11 +13,12 @@ use Appwrite\Event\Func;
use Appwrite\Event\Mail;
use Appwrite\Event\Messaging;
use Appwrite\Event\Migration;
use Appwrite\Event\Publisher\Usage as UsagePublisher;
use Appwrite\Event\Realtime;
use Appwrite\Event\Screenshot;
use Appwrite\Event\StatsUsage;
use Appwrite\Event\Webhook;
use Appwrite\Platform\Appwrite;
use Appwrite\Usage\Context;
use Appwrite\Utopia\Database\Documents\User;
use Executor\Executor;
use Swoole\Runtime;
@@ -42,6 +43,7 @@ use Utopia\Pools\Group;
use Utopia\Queue\Broker\Pool as BrokerPool;
use Utopia\Queue\Message;
use Utopia\Queue\Publisher;
use Utopia\Queue\Queue;
use Utopia\Queue\Server;
use Utopia\Registry\Registry;
use Utopia\Storage\Device\Telemetry as TelemetryDevice;
@@ -58,7 +60,8 @@ Server::setResource('register', fn () => $register);
Server::setResource('authorization', function () {
$authorization = new Authorization();
$authorization->disable();
return $authorization;
return $authorization;
}, []);
Server::setResource('dbForPlatform', function (Cache $cache, Registry $register, Authorization $authorization) {
@@ -70,9 +73,7 @@ Server::setResource('dbForPlatform', function (Cache $cache, Registry $register,
->setDatabase(APP_DATABASE)
->setAuthorization($authorization)
->setNamespace('_console')
->setDocumentType('users', User::class)
;
->setDocumentType('users', User::class);
return $dbForPlatform;
}, ['cache', 'register', 'authorization']);
@@ -111,7 +112,7 @@ Server::setResource('dbForProject', function (Cache $cache, Registry $register,
if (\in_array($dsn->getHost(), $sharedTables)) {
$database
->setSharedTables(true)
->setTenant((int)$project->getSequence())
->setTenant((int) $project->getSequence())
->setNamespace($dsn->getParam('namespace'));
} else {
$database
@@ -151,7 +152,7 @@ Server::setResource('getProjectDB', function (Group $pools, Database $dbForPlatf
if (\in_array($dsn->getHost(), $sharedTables)) {
$database
->setSharedTables(true)
->setTenant((int)$project->getSequence())
->setTenant((int) $project->getSequence())
->setNamespace($dsn->getParam('namespace'));
} else {
$database
@@ -173,7 +174,7 @@ Server::setResource('getProjectDB', function (Group $pools, Database $dbForPlatf
if (\in_array($dsn->getHost(), $sharedTables)) {
$database
->setSharedTables(true)
->setTenant((int)$project->getSequence())
->setTenant((int) $project->getSequence())
->setNamespace($dsn->getParam('namespace'));
} else {
$database
@@ -193,9 +194,11 @@ Server::setResource('getProjectDB', function (Group $pools, Database $dbForPlatf
Server::setResource('getLogsDB', function (Group $pools, Cache $cache, Authorization $authorization) {
$database = null;
return function (?Document $project = null) use ($pools, $cache, $database, $authorization) {
if ($database !== null && $project !== null && !$project->isEmpty() && $project->getId() !== 'console') {
$database->setTenant((int)$project->getSequence());
if ($database !== null && $project !== null && ! $project->isEmpty() && $project->getId() !== 'console') {
$database->setTenant((int) $project->getSequence());
return $database;
}
@@ -211,68 +214,14 @@ Server::setResource('getLogsDB', function (Group $pools, Cache $cache, Authoriza
->setMaxQueryValues(APP_DATABASE_QUERY_MAX_VALUES_WORKER);
// set tenant
if ($project !== null && !$project->isEmpty() && $project->getId() !== 'console') {
$database->setTenant((int)$project->getSequence());
if ($project !== null && ! $project->isEmpty() && $project->getId() !== 'console') {
$database->setTenant((int) $project->getSequence());
}
return $database;
};
}, ['pools', 'cache', 'authorization']);
Server::setResource('getDatabasesDB', function (Cache $cache, Registry $register, Document $project, Authorization $authorization) {
return function (Document $database, ?Document $projectDocument = null) use ($cache, $register, $project, $authorization): Database {
$projectDocument ??= $project;
$databaseDSN = $database->getAttribute('database', '');
$databaseType = $database->getAttribute('type', '');
// Backwardscompatibility: older or seeded legacy databases may not have a DSN stored
// in the "database" attribute. In that case, fall back to the project's database DSN.
if ($databaseDSN === '') {
$databaseDSN = $projectDocument->getAttribute('database', '');
}
try {
$databaseDSN = new DSN($databaseDSN);
} catch (\InvalidArgumentException) {
$databaseDSN = new DSN('mysql://'.$databaseDSN);
}
try {
$dsn = new DSN($projectDocument->getAttribute('database'));
} catch (\InvalidArgumentException) {
// Temporary fallback until all projects use shared tables
$dsn = new DSN('mysql://' . $projectDocument->getAttribute('database'));
}
$pools = $register->get('pools');
$pool = $pools->get($databaseDSN->getHost());
$adapter = new DatabasePool($pool);
$database = new Database($adapter, $cache);
$database
->setDatabase(APP_DATABASE)
->setAuthorization($authorization);
$database->getAdapter()->setSupportForAttributes($databaseType !== DOCUMENTSDB);
$sharedTables = \explode(',', System::getEnv('_APP_DATABASE_SHARED_TABLES', ''));
if (\in_array($dsn->getHost(), $sharedTables, true)) {
$database
->setSharedTables(true)
->setTenant((int) $projectDocument->getSequence())
->setNamespace($dsn->getParam('namespace'));
} else {
$database
->setSharedTables(false)
->setTenant(null)
->setNamespace('_' . $projectDocument->getSequence());
}
$database->setTimeout(APP_DATABASE_TIMEOUT_MILLISECONDS_WORKER);
return $database;
};
}, ['cache', 'register', 'project', 'authorization']);
Server::setResource('abuseRetention', function () {
return time() - (int) System::getEnv('_APP_MAINTENANCE_RETENTION_ABUSE', 86400); // 1 day
});
@@ -281,6 +230,7 @@ Server::setResource('auditRetention', function (Document $project) {
if ($project->getId() === 'console') {
return DateTime::addSeconds(new \DateTime(), -1 * System::getEnv('_APP_MAINTENANCE_RETENTION_AUDIT_CONSOLE', 15778800)); // 6 months
}
return DateTime::addSeconds(new \DateTime(), -1 * System::getEnv('_APP_MAINTENANCE_RETENTION_AUDIT', 1209600)); // 14 days
}, ['project']);
@@ -306,7 +256,7 @@ Server::setResource('redis', function () {
$pass = System::getEnv('_APP_REDIS_PASS', '');
$redis = new \Redis();
@$redis->pconnect($host, (int)$port);
@$redis->pconnect($host, (int) $port);
if ($pass) {
$redis->auth($pass);
}
@@ -323,7 +273,6 @@ Server::setResource('timelimit', function (\Redis $redis) {
Server::setResource('log', fn () => new Log());
Server::setResource('publisher', function (Group $pools) {
return new BrokerPool(publisher: $pools->get('publisher'));
}, ['pools']);
@@ -340,10 +289,6 @@ Server::setResource('publisherMigrations', function (BrokerPool $publisher) {
return $publisher;
}, ['publisher']);
Server::setResource('publisherStatsUsage', function (BrokerPool $publisher) {
return $publisher;
}, ['publisher']);
Server::setResource('publisherMessaging', function (BrokerPool $publisher) {
return $publisher;
}, ['publisher']);
@@ -364,9 +309,13 @@ Server::setResource('consumerStatsUsage', function (BrokerPool $consumer) {
return $consumer;
}, ['consumer']);
Server::setResource('queueForStatsUsage', function (Publisher $publisher) {
return new StatsUsage($publisher);
}, ['publisher']);
Server::setResource('usage', function () {
return new Context();
}, []);
Server::setResource('publisherForUsage', fn (Publisher $publisher) => new UsagePublisher(
$publisher,
new Queue(System::getEnv('_APP_STATS_USAGE_QUEUE_NAME', Event::STATS_USAGE_QUEUE_NAME))
), ['publisher']);
Server::setResource('queueForDatabase', function (Publisher $publisher) {
return new EventDatabase($publisher);
@@ -408,7 +357,6 @@ Server::setResource('queueForFunctions', function (Publisher $publisher) {
return new Func($publisher);
}, ['publisher']);
Server::setResource('queueForRealtime', function () {
return new Realtime();
}, []);
@@ -538,11 +486,13 @@ Server::setResource('getAudit', function (Database $dbForPlatform, callable $get
return function (Document $project) use ($dbForPlatform, $getProjectDB) {
if ($project->isEmpty() || $project->getId() === 'console') {
$adapter = new AdapterDatabase($dbForPlatform);
return new UtopiaAudit($adapter);
}
$dbForProject = $getProjectDB($project);
$adapter = new AdapterDatabase($dbForProject);
return new UtopiaAudit($adapter);
};
}, ['dbForPlatform', 'getProjectDB']);
@@ -559,7 +509,7 @@ $pools = $register->get('pools');
$platform = new Appwrite();
$args = $platform->getEnv('argv');
if (!isset($args[1])) {
if (! isset($args[1])) {
Console::error('Missing worker name');
Console::exit(1);
}
@@ -584,10 +534,10 @@ try {
'workersNum' => System::getEnv('_APP_WORKERS_NUM', 1),
'connection' => $pools->get('consumer')->pop()->getResource(),
'workerName' => strtolower($workerName) ?? null,
'queueName' => $queueName
'queueName' => $queueName,
]);
} catch (\Throwable $e) {
Console::error($e->getMessage() . ', File: ' . $e->getFile() . ', Line: ' . $e->getLine());
Console::error($e->getMessage() . ', File: ' . $e->getFile() . ', Line: ' . $e->getLine());
}
$worker = $platform->getWorker();
@@ -604,11 +554,11 @@ $worker
->inject('pools')
->inject('project')
->inject('authorization')
->action(function (Throwable $error, ?Logger $logger, Log $log, Group $pools, Document $project, Authorization $authorization) use ($worker, $queueName) {
->action(function (Throwable $error, ?Logger $logger, Log $log, Group $pools, Document $project, Authorization $authorization) use ($queueName) {
$version = System::getEnv('_APP_VERSION', 'UNKNOWN');
if ($logger) {
$log->setNamespace("appwrite-worker");
$log->setNamespace('appwrite-worker');
$log->setServer(System::getEnv('_APP_LOGGING_SERVICE_IDENTIFIER', \gethostname()));
$log->setVersion($version);
$log->setType(Log::TYPE_ERROR);
+4 -2
View File
@@ -49,6 +49,7 @@
"appwrite/php-runtimes": "0.19.*",
"appwrite/php-clamav": "2.0.*",
"utopia-php/abuse": "1.2.*",
"utopia-php/agents": "1.2.*",
"utopia-php/analytics": "0.15.*",
"utopia-php/audit": "2.2.*",
"utopia-php/auth": "0.5.*",
@@ -58,7 +59,6 @@
"utopia-php/config": "1.*",
"utopia-php/console": "0.1.*",
"utopia-php/database": "5.*",
"utopia-php/agents": "1.*",
"utopia-php/detector": "0.2.*",
"utopia-php/domains": "1.*",
"utopia-php/emails": "0.6.*",
@@ -70,12 +70,13 @@
"utopia-php/locale": "0.8.*",
"utopia-php/logger": "0.6.*",
"utopia-php/messaging": "0.20.*",
"utopia-php/migration": "dev-add-dev-key-migration as 1.8.0",
"utopia-php/migration": "1.7.*",
"utopia-php/platform": "0.7.*",
"utopia-php/pools": "1.*",
"utopia-php/span": "1.1.*",
"utopia-php/preloader": "0.2.*",
"utopia-php/queue": "0.15.*",
"utopia-php/servers": "0.2.5",
"utopia-php/registry": "0.5.*",
"utopia-php/storage": "1.0.*",
"utopia-php/system": "0.10.*",
@@ -109,6 +110,7 @@
},
"config": {
"platform": {
"php": "8.3"
},
"allow-plugins": {
"php-http/discovery": true,
Generated
+112 -117
View File
@@ -4,7 +4,7 @@
"Read more about it at https://getcomposer.org/doc/01-basic-usage.md#installing-dependencies",
"This file is @generated automatically"
],
"content-hash": "a487482e621dfe0514f1fdcd78e6b70a",
"content-hash": "02892029fbc4800e4b7eb6953121efd5",
"packages": [
{
"name": "adhocore/jwt",
@@ -3606,16 +3606,16 @@
},
{
"name": "utopia-php/cache",
"version": "1.0.0",
"version": "1.0.1",
"source": {
"type": "git",
"url": "https://github.com/utopia-php/cache.git",
"reference": "7068870c086a6aea16173563a26b93ef3e408439"
"reference": "05ceba981436a4022553f7aaa2a05fa049d0f71c"
},
"dist": {
"type": "zip",
"url": "https://api.github.com/repos/utopia-php/cache/zipball/7068870c086a6aea16173563a26b93ef3e408439",
"reference": "7068870c086a6aea16173563a26b93ef3e408439",
"url": "https://api.github.com/repos/utopia-php/cache/zipball/05ceba981436a4022553f7aaa2a05fa049d0f71c",
"reference": "05ceba981436a4022553f7aaa2a05fa049d0f71c",
"shasum": ""
},
"require": {
@@ -3652,9 +3652,9 @@
],
"support": {
"issues": "https://github.com/utopia-php/cache/issues",
"source": "https://github.com/utopia-php/cache/tree/1.0.0"
"source": "https://github.com/utopia-php/cache/tree/1.0.1"
},
"time": "2026-01-28T10:55:44+00:00"
"time": "2026-03-12T03:39:09+00:00"
},
{
"name": "utopia-php/cli",
@@ -3850,16 +3850,16 @@
},
{
"name": "utopia-php/database",
"version": "5.3.8",
"version": "5.3.12",
"source": {
"type": "git",
"url": "https://github.com/utopia-php/database.git",
"reference": "4920bb60afb98d4bd81f4d331765716ae1d40255"
"reference": "7e843989e67b1de3bfb27212334c28694bda4e95"
},
"dist": {
"type": "zip",
"url": "https://api.github.com/repos/utopia-php/database/zipball/4920bb60afb98d4bd81f4d331765716ae1d40255",
"reference": "4920bb60afb98d4bd81f4d331765716ae1d40255",
"url": "https://api.github.com/repos/utopia-php/database/zipball/7e843989e67b1de3bfb27212334c28694bda4e95",
"reference": "7e843989e67b1de3bfb27212334c28694bda4e95",
"shasum": ""
},
"require": {
@@ -3902,9 +3902,9 @@
],
"support": {
"issues": "https://github.com/utopia-php/database/issues",
"source": "https://github.com/utopia-php/database/tree/5.3.8"
"source": "https://github.com/utopia-php/database/tree/5.3.12"
},
"time": "2026-03-11T01:03:34+00:00"
"time": "2026-03-13T12:16:26+00:00"
},
{
"name": "utopia-php/detector",
@@ -4058,16 +4058,16 @@
},
{
"name": "utopia-php/domains",
"version": "1.0.2",
"version": "1.0.5",
"source": {
"type": "git",
"url": "https://github.com/utopia-php/domains.git",
"reference": "b4896a6746f0fbe29dfd5e32f7790bd94c1af1e6"
"reference": "0edf6bb2b07f30db849a267027077bf5abb994c6"
},
"dist": {
"type": "zip",
"url": "https://api.github.com/repos/utopia-php/domains/zipball/b4896a6746f0fbe29dfd5e32f7790bd94c1af1e6",
"reference": "b4896a6746f0fbe29dfd5e32f7790bd94c1af1e6",
"url": "https://api.github.com/repos/utopia-php/domains/zipball/0edf6bb2b07f30db849a267027077bf5abb994c6",
"reference": "0edf6bb2b07f30db849a267027077bf5abb994c6",
"shasum": ""
},
"require": {
@@ -4114,9 +4114,9 @@
],
"support": {
"issues": "https://github.com/utopia-php/domains/issues",
"source": "https://github.com/utopia-php/domains/tree/1.0.2"
"source": "https://github.com/utopia-php/domains/tree/1.0.5"
},
"time": "2026-02-25T08:18:25+00:00"
"time": "2026-03-03T09:20:50+00:00"
},
{
"name": "utopia-php/dsn",
@@ -4517,16 +4517,16 @@
},
{
"name": "utopia-php/migration",
"version": "dev-add-dev-key-migration",
"version": "1.7.0",
"source": {
"type": "git",
"url": "https://github.com/utopia-php/migration.git",
"reference": "8577c62a77415ced8b70713901d00750c83495a9"
"reference": "97583ae502e40621ea91a71de19d053c5ae2e558"
},
"dist": {
"type": "zip",
"url": "https://api.github.com/repos/utopia-php/migration/zipball/8577c62a77415ced8b70713901d00750c83495a9",
"reference": "8577c62a77415ced8b70713901d00750c83495a9",
"url": "https://api.github.com/repos/utopia-php/migration/zipball/97583ae502e40621ea91a71de19d053c5ae2e558",
"reference": "97583ae502e40621ea91a71de19d053c5ae2e558",
"shasum": ""
},
"require": {
@@ -4566,22 +4566,22 @@
],
"support": {
"issues": "https://github.com/utopia-php/migration/issues",
"source": "https://github.com/utopia-php/migration/tree/add-dev-key-migration"
"source": "https://github.com/utopia-php/migration/tree/1.7.0"
},
"time": "2026-03-16T21:06:31+00:00"
"time": "2026-03-10T06:36:27+00:00"
},
{
"name": "utopia-php/mongo",
"version": "1.0.0",
"version": "1.0.1",
"source": {
"type": "git",
"url": "https://github.com/utopia-php/mongo.git",
"reference": "45bedf36c2c946ec7a0a3e59b9f12f772de0b01d"
"reference": "83dbcde768d5fb40241f5ca8aa5ed8ca140a7469"
},
"dist": {
"type": "zip",
"url": "https://api.github.com/repos/utopia-php/mongo/zipball/45bedf36c2c946ec7a0a3e59b9f12f772de0b01d",
"reference": "45bedf36c2c946ec7a0a3e59b9f12f772de0b01d",
"url": "https://api.github.com/repos/utopia-php/mongo/zipball/83dbcde768d5fb40241f5ca8aa5ed8ca140a7469",
"reference": "83dbcde768d5fb40241f5ca8aa5ed8ca140a7469",
"shasum": ""
},
"require": {
@@ -4627,9 +4627,9 @@
],
"support": {
"issues": "https://github.com/utopia-php/mongo/issues",
"source": "https://github.com/utopia-php/mongo/tree/1.0.0"
"source": "https://github.com/utopia-php/mongo/tree/1.0.1"
},
"time": "2026-02-12T05:54:06+00:00"
"time": "2026-03-13T07:29:24+00:00"
},
{
"name": "utopia-php/platform",
@@ -5215,29 +5215,28 @@
},
{
"name": "utopia-php/vcs",
"version": "2.0.0",
"version": "2.0.2",
"source": {
"type": "git",
"url": "https://github.com/utopia-php/vcs.git",
"reference": "058049326e04a2a0c2f0ce8ad00c7e84825aba14"
"reference": "5769679308bad498f2777547d48ab332166c4c0b"
},
"dist": {
"type": "zip",
"url": "https://api.github.com/repos/utopia-php/vcs/zipball/058049326e04a2a0c2f0ce8ad00c7e84825aba14",
"reference": "058049326e04a2a0c2f0ce8ad00c7e84825aba14",
"url": "https://api.github.com/repos/utopia-php/vcs/zipball/5769679308bad498f2777547d48ab332166c4c0b",
"reference": "5769679308bad498f2777547d48ab332166c4c0b",
"shasum": ""
},
"require": {
"adhocore/jwt": "^1.1",
"php": ">=8.0",
"utopia-php/cache": "1.0.*",
"utopia-php/framework": "0.*.*",
"utopia-php/system": "0.10.*"
"utopia-php/cache": "1.0.*"
},
"require-dev": {
"laravel/pint": "1.*.*",
"phpstan/phpstan": "1.*.*",
"phpunit/phpunit": "^9.4"
"phpunit/phpunit": "^9.4",
"utopia-php/system": "0.10.*"
},
"type": "library",
"autoload": {
@@ -5258,9 +5257,9 @@
],
"support": {
"issues": "https://github.com/utopia-php/vcs/issues",
"source": "https://github.com/utopia-php/vcs/tree/2.0.0"
"source": "https://github.com/utopia-php/vcs/tree/2.0.2"
},
"time": "2026-02-25T11:36:45+00:00"
"time": "2026-03-13T15:25:16+00:00"
},
{
"name": "utopia-php/websocket",
@@ -5438,16 +5437,16 @@
"packages-dev": [
{
"name": "appwrite/sdk-generator",
"version": "1.11.1",
"version": "1.11.6",
"source": {
"type": "git",
"url": "https://github.com/appwrite/sdk-generator.git",
"reference": "6ff411f26f2750eea05c7598c14bb3a2ada898cb"
"reference": "f80e302d000cdc2f98b4bb5ff2fc3bd0bdff7b38"
},
"dist": {
"type": "zip",
"url": "https://api.github.com/repos/appwrite/sdk-generator/zipball/6ff411f26f2750eea05c7598c14bb3a2ada898cb",
"reference": "6ff411f26f2750eea05c7598c14bb3a2ada898cb",
"url": "https://api.github.com/repos/appwrite/sdk-generator/zipball/f80e302d000cdc2f98b4bb5ff2fc3bd0bdff7b38",
"reference": "f80e302d000cdc2f98b4bb5ff2fc3bd0bdff7b38",
"shasum": ""
},
"require": {
@@ -5483,22 +5482,22 @@
"description": "Appwrite PHP library for generating API SDKs for multiple programming languages and platforms",
"support": {
"issues": "https://github.com/appwrite/sdk-generator/issues",
"source": "https://github.com/appwrite/sdk-generator/tree/1.11.1"
"source": "https://github.com/appwrite/sdk-generator/tree/1.11.6"
},
"time": "2026-02-25T07:15:19+00:00"
"time": "2026-03-09T07:12:51+00:00"
},
{
"name": "brianium/paratest",
"version": "v7.19.0",
"version": "v7.19.2",
"source": {
"type": "git",
"url": "https://github.com/paratestphp/paratest.git",
"reference": "7c6c29af7c4b406b49ce0c6b0a3a81d3684474e6"
"reference": "66e4f7910cecf67736bccf2b8bd53a2e3eb98bd9"
},
"dist": {
"type": "zip",
"url": "https://api.github.com/repos/paratestphp/paratest/zipball/7c6c29af7c4b406b49ce0c6b0a3a81d3684474e6",
"reference": "7c6c29af7c4b406b49ce0c6b0a3a81d3684474e6",
"url": "https://api.github.com/repos/paratestphp/paratest/zipball/66e4f7910cecf67736bccf2b8bd53a2e3eb98bd9",
"reference": "66e4f7910cecf67736bccf2b8bd53a2e3eb98bd9",
"shasum": ""
},
"require": {
@@ -5512,9 +5511,9 @@
"phpunit/php-code-coverage": "^12.5.3 || ^13.0.1",
"phpunit/php-file-iterator": "^6.0.1 || ^7",
"phpunit/php-timer": "^8 || ^9",
"phpunit/phpunit": "^12.5.9 || ^13",
"phpunit/phpunit": "^12.5.14 || ^13.0.5",
"sebastian/environment": "^8.0.3 || ^9",
"symfony/console": "^7.4.4 || ^8.0.4",
"symfony/console": "^7.4.7 || ^8.0.7",
"symfony/process": "^7.4.5 || ^8.0.5"
},
"require-dev": {
@@ -5522,11 +5521,11 @@
"ext-pcntl": "*",
"ext-pcov": "*",
"ext-posix": "*",
"phpstan/phpstan": "^2.1.38",
"phpstan/phpstan-deprecation-rules": "^2.0.3",
"phpstan/phpstan-phpunit": "^2.0.12",
"phpstan/phpstan-strict-rules": "^2.0.8",
"symfony/filesystem": "^7.4.0 || ^8.0.1"
"phpstan/phpstan": "^2.1.40",
"phpstan/phpstan-deprecation-rules": "^2.0.4",
"phpstan/phpstan-phpunit": "^2.0.16",
"phpstan/phpstan-strict-rules": "^2.0.10",
"symfony/filesystem": "^7.4.6 || ^8.0.6"
},
"bin": [
"bin/paratest",
@@ -5566,7 +5565,7 @@
],
"support": {
"issues": "https://github.com/paratestphp/paratest/issues",
"source": "https://github.com/paratestphp/paratest/tree/v7.19.0"
"source": "https://github.com/paratestphp/paratest/tree/v7.19.2"
},
"funding": [
{
@@ -5578,7 +5577,7 @@
"type": "paypal"
}
],
"time": "2026-02-06T10:53:26+00:00"
"time": "2026-03-09T14:33:17+00:00"
},
{
"name": "czproject/git-php",
@@ -5921,16 +5920,16 @@
},
{
"name": "laravel/pint",
"version": "v1.27.1",
"version": "v1.29.0",
"source": {
"type": "git",
"url": "https://github.com/laravel/pint.git",
"reference": "54cca2de13790570c7b6f0f94f37896bee4abcb5"
"reference": "bdec963f53172c5e36330f3a400604c69bf02d39"
},
"dist": {
"type": "zip",
"url": "https://api.github.com/repos/laravel/pint/zipball/54cca2de13790570c7b6f0f94f37896bee4abcb5",
"reference": "54cca2de13790570c7b6f0f94f37896bee4abcb5",
"url": "https://api.github.com/repos/laravel/pint/zipball/bdec963f53172c5e36330f3a400604c69bf02d39",
"reference": "bdec963f53172c5e36330f3a400604c69bf02d39",
"shasum": ""
},
"require": {
@@ -5941,13 +5940,14 @@
"php": "^8.2.0"
},
"require-dev": {
"friendsofphp/php-cs-fixer": "^3.93.1",
"illuminate/view": "^12.51.0",
"larastan/larastan": "^3.9.2",
"friendsofphp/php-cs-fixer": "^3.94.2",
"illuminate/view": "^12.54.1",
"larastan/larastan": "^3.9.3",
"laravel-zero/framework": "^12.0.5",
"mockery/mockery": "^1.6.12",
"nunomaduro/termwind": "^2.3.3",
"pestphp/pest": "^3.8.5"
"nunomaduro/termwind": "^2.4.0",
"pestphp/pest": "^3.8.6",
"shipfastlabs/agent-detector": "^1.1.0"
},
"bin": [
"builds/pint"
@@ -5984,7 +5984,7 @@
"issues": "https://github.com/laravel/pint/issues",
"source": "https://github.com/laravel/pint"
},
"time": "2026-02-10T20:00:20+00:00"
"time": "2026-03-12T15:51:39+00:00"
},
{
"name": "matthiasmullie/minify",
@@ -6398,16 +6398,16 @@
},
{
"name": "phpbench/phpbench",
"version": "1.4.3",
"version": "1.5.1",
"source": {
"type": "git",
"url": "https://github.com/phpbench/phpbench.git",
"reference": "b641dde59d969ea42eed70a39f9b51950bc96878"
"reference": "9a28fd0833f11171b949843c6fd663eb69b6d14c"
},
"dist": {
"type": "zip",
"url": "https://api.github.com/repos/phpbench/phpbench/zipball/b641dde59d969ea42eed70a39f9b51950bc96878",
"reference": "b641dde59d969ea42eed70a39f9b51950bc96878",
"url": "https://api.github.com/repos/phpbench/phpbench/zipball/9a28fd0833f11171b949843c6fd663eb69b6d14c",
"reference": "9a28fd0833f11171b949843c6fd663eb69b6d14c",
"shasum": ""
},
"require": {
@@ -6418,7 +6418,7 @@
"ext-reflection": "*",
"ext-spl": "*",
"ext-tokenizer": "*",
"php": "^8.1",
"php": "^8.2",
"phpbench/container": "^2.2",
"psr/log": "^1.1 || ^2.0 || ^3.0",
"seld/jsonlint": "^1.1",
@@ -6438,8 +6438,9 @@
"phpstan/extension-installer": "^1.1",
"phpstan/phpstan": "^1.0",
"phpstan/phpstan-phpunit": "^1.0",
"phpunit/phpunit": "^10.4 || ^11.0",
"phpunit/phpunit": "^11.5",
"rector/rector": "^1.2",
"sebastian/exporter": "^6.3.2",
"symfony/error-handler": "^6.1 || ^7.0 || ^8.0",
"symfony/var-dumper": "^6.1 || ^7.0 || ^8.0"
},
@@ -6484,7 +6485,7 @@
],
"support": {
"issues": "https://github.com/phpbench/phpbench/issues",
"source": "https://github.com/phpbench/phpbench/tree/1.4.3"
"source": "https://github.com/phpbench/phpbench/tree/1.5.1"
},
"funding": [
{
@@ -6492,15 +6493,15 @@
"type": "github"
}
],
"time": "2025-11-06T19:07:31+00:00"
"time": "2026-03-05T08:18:58+00:00"
},
{
"name": "phpstan/phpstan",
"version": "1.12.32",
"version": "1.12.33",
"dist": {
"type": "zip",
"url": "https://api.github.com/repos/phpstan/phpstan/zipball/2770dcdf5078d0b0d53f94317e06affe88419aa8",
"reference": "2770dcdf5078d0b0d53f94317e06affe88419aa8",
"url": "https://api.github.com/repos/phpstan/phpstan/zipball/37982d6fc7cbb746dda7773530cda557cdf119e1",
"reference": "37982d6fc7cbb746dda7773530cda557cdf119e1",
"shasum": ""
},
"require": {
@@ -6545,7 +6546,7 @@
"type": "github"
}
],
"time": "2025-09-30T10:16:31+00:00"
"time": "2026-02-28T20:30:03+00:00"
},
{
"name": "phpunit/php-code-coverage",
@@ -8095,16 +8096,16 @@
},
{
"name": "symfony/console",
"version": "v8.0.4",
"version": "v8.0.7",
"source": {
"type": "git",
"url": "https://github.com/symfony/console.git",
"reference": "ace03c4cf9805080ff40cbeec69fca180c339a3b"
"reference": "15ed9008a4ebe2d6a78e4937f74e0c13ef2e618a"
},
"dist": {
"type": "zip",
"url": "https://api.github.com/repos/symfony/console/zipball/ace03c4cf9805080ff40cbeec69fca180c339a3b",
"reference": "ace03c4cf9805080ff40cbeec69fca180c339a3b",
"url": "https://api.github.com/repos/symfony/console/zipball/15ed9008a4ebe2d6a78e4937f74e0c13ef2e618a",
"reference": "15ed9008a4ebe2d6a78e4937f74e0c13ef2e618a",
"shasum": ""
},
"require": {
@@ -8161,7 +8162,7 @@
"terminal"
],
"support": {
"source": "https://github.com/symfony/console/tree/v8.0.4"
"source": "https://github.com/symfony/console/tree/v8.0.7"
},
"funding": [
{
@@ -8181,20 +8182,20 @@
"type": "tidelift"
}
],
"time": "2026-01-13T13:06:50+00:00"
"time": "2026-03-06T14:06:22+00:00"
},
{
"name": "symfony/filesystem",
"version": "v8.0.1",
"version": "v8.0.6",
"source": {
"type": "git",
"url": "https://github.com/symfony/filesystem.git",
"reference": "d937d400b980523dc9ee946bb69972b5e619058d"
"reference": "7bf9162d7a0dff98d079b72948508fa48018a770"
},
"dist": {
"type": "zip",
"url": "https://api.github.com/repos/symfony/filesystem/zipball/d937d400b980523dc9ee946bb69972b5e619058d",
"reference": "d937d400b980523dc9ee946bb69972b5e619058d",
"url": "https://api.github.com/repos/symfony/filesystem/zipball/7bf9162d7a0dff98d079b72948508fa48018a770",
"reference": "7bf9162d7a0dff98d079b72948508fa48018a770",
"shasum": ""
},
"require": {
@@ -8231,7 +8232,7 @@
"description": "Provides basic utilities for the filesystem",
"homepage": "https://symfony.com",
"support": {
"source": "https://github.com/symfony/filesystem/tree/v8.0.1"
"source": "https://github.com/symfony/filesystem/tree/v8.0.6"
},
"funding": [
{
@@ -8251,20 +8252,20 @@
"type": "tidelift"
}
],
"time": "2025-12-01T09:13:36+00:00"
"time": "2026-02-25T16:59:43+00:00"
},
{
"name": "symfony/finder",
"version": "v8.0.5",
"version": "v8.0.6",
"source": {
"type": "git",
"url": "https://github.com/symfony/finder.git",
"reference": "8bd576e97c67d45941365bf824e18dc8538e6eb0"
"reference": "441404f09a54de6d1bd6ad219e088cdf4c91f97c"
},
"dist": {
"type": "zip",
"url": "https://api.github.com/repos/symfony/finder/zipball/8bd576e97c67d45941365bf824e18dc8538e6eb0",
"reference": "8bd576e97c67d45941365bf824e18dc8538e6eb0",
"url": "https://api.github.com/repos/symfony/finder/zipball/441404f09a54de6d1bd6ad219e088cdf4c91f97c",
"reference": "441404f09a54de6d1bd6ad219e088cdf4c91f97c",
"shasum": ""
},
"require": {
@@ -8299,7 +8300,7 @@
"description": "Finds files and directories via an intuitive fluent interface",
"homepage": "https://symfony.com",
"support": {
"source": "https://github.com/symfony/finder/tree/v8.0.5"
"source": "https://github.com/symfony/finder/tree/v8.0.6"
},
"funding": [
{
@@ -8319,7 +8320,7 @@
"type": "tidelift"
}
],
"time": "2026-01-26T15:08:38+00:00"
"time": "2026-01-29T09:41:02+00:00"
},
{
"name": "symfony/options-resolver",
@@ -8789,16 +8790,16 @@
},
{
"name": "symfony/string",
"version": "v8.0.4",
"version": "v8.0.6",
"source": {
"type": "git",
"url": "https://github.com/symfony/string.git",
"reference": "758b372d6882506821ed666032e43020c4f57194"
"reference": "6c9e1108041b5dce21a9a4984b531c4923aa9ec4"
},
"dist": {
"type": "zip",
"url": "https://api.github.com/repos/symfony/string/zipball/758b372d6882506821ed666032e43020c4f57194",
"reference": "758b372d6882506821ed666032e43020c4f57194",
"url": "https://api.github.com/repos/symfony/string/zipball/6c9e1108041b5dce21a9a4984b531c4923aa9ec4",
"reference": "6c9e1108041b5dce21a9a4984b531c4923aa9ec4",
"shasum": ""
},
"require": {
@@ -8855,7 +8856,7 @@
"utf8"
],
"support": {
"source": "https://github.com/symfony/string/tree/v8.0.4"
"source": "https://github.com/symfony/string/tree/v8.0.6"
},
"funding": [
{
@@ -8875,7 +8876,7 @@
"type": "tidelift"
}
],
"time": "2026-01-12T12:37:40+00:00"
"time": "2026-02-09T10:14:57+00:00"
},
{
"name": "textalk/websocket",
@@ -9105,18 +9106,9 @@
"time": "2024-03-07T20:33:40+00:00"
}
],
"aliases": [
{
"package": "utopia-php/migration",
"version": "dev-add-dev-key-migration",
"alias": "1.8.0",
"alias_normalized": "1.8.0.0"
}
],
"aliases": [],
"minimum-stability": "dev",
"stability-flags": {
"utopia-php/migration": 20
},
"stability-flags": {},
"prefer-stable": true,
"prefer-lowest": false,
"platform": {
@@ -9137,5 +9129,8 @@
"platform-dev": {
"ext-fileinfo": "*"
},
"platform-overrides": {
"php": "8.3"
},
"plugin-api-version": "2.9.0"
}
+144
View File
@@ -0,0 +1,144 @@
# Dev tools for local development only.
# This file is automatically loaded by `docker compose` alongside docker-compose.yml.
# CI sets COMPOSE_FILE=docker-compose.yml explicitly, so these services are excluded from test runs.
services:
appwrite-mongo-express:
profiles: ["mongodb"]
image: mongo-express
container_name: appwrite-mongo-express
networks:
- appwrite
ports:
- "8082:8081"
environment:
ME_CONFIG_MONGODB_URL: "mongodb://root:${_APP_DB_ROOT_PASS}@appwrite-mongodb:27017/?replicaSet=rs0&directConnection=true"
ME_CONFIG_BASICAUTH_USERNAME: ${_APP_DB_USER}
ME_CONFIG_BASICAUTH_PASSWORD: ${_APP_DB_PASS}
depends_on:
- mongodb
adminer:
image: adminer
container_name: appwrite-adminer
restart: always
ports:
- 9506:8080
networks:
- appwrite
- gateway
environment:
- ADMINER_DESIGN=pepa-linha
- ADMINER_DEFAULT_SERVER=mariadb
- ADMINER_DEFAULT_USERNAME=root
- ADMINER_DEFAULT_PASSWORD=rootsecretpassword
- ADMINER_DEFAULT_DB=appwrite
configs:
- source: adminer-index.php
target: /var/www/html/index.php
mode: 0755
labels:
- "traefik.enable=true"
- "traefik.constraint-label-stack=appwrite"
- "traefik.docker.network=gateway"
- "traefik.http.services.appwrite_adminer.loadbalancer.server.port=8080"
- "traefik.http.routers.appwrite_adminer_http.entrypoints=appwrite_web"
- "traefik.http.routers.appwrite_adminer_http.rule=Host(`mysql.localhost`)"
- "traefik.http.routers.appwrite_adminer_http.service=appwrite_adminer"
- "traefik.http.routers.appwrite_adminer_https.entrypoints=appwrite_websecure"
- "traefik.http.routers.appwrite_adminer_https.rule=Host(`mysql.localhost`)"
- "traefik.http.routers.appwrite_adminer_https.service=appwrite_adminer"
- "traefik.http.routers.appwrite_adminer_https.tls=true"
redis-insight:
image: redis/redisinsight:latest
restart: unless-stopped
networks:
- appwrite
- gateway
environment:
- RI_PRE_SETUP_DATABASES_PATH=/mnt/connections.json
configs:
- source: redisinsight-connections.json
target: /mnt/connections.json
mode: 0755
labels:
- "traefik.enable=true"
- "traefik.constraint-label-stack=appwrite"
- "traefik.docker.network=gateway"
- "traefik.http.services.appwrite_redisinsight.loadbalancer.server.port=5540"
- "traefik.http.routers.appwrite_redisinsight_http.entrypoints=appwrite_web"
- "traefik.http.routers.appwrite_redisinsight_http.rule=Host(`redis.localhost`)"
- "traefik.http.routers.appwrite_redisinsight_http.service=appwrite_redisinsight"
- "traefik.http.routers.appwrite_redisinsight_https.entrypoints=appwrite_websecure"
- "traefik.http.routers.appwrite_redisinsight_https.rule=Host(`redis.localhost`)"
- "traefik.http.routers.appwrite_redisinsight_https.service=appwrite_redisinsight"
- "traefik.http.routers.appwrite_redisinsight_https.tls=true"
ports:
- "8081:5540"
graphql-explorer:
container_name: appwrite-graphql-explorer
image: appwrite/altair:0.3.0
restart: unless-stopped
networks:
- appwrite
ports:
- "9509:3000"
environment:
- SERVER_URL=http://localhost/v1/graphql
configs:
redisinsight-connections.json:
content: |
[
{
"compressor": "NONE",
"id": "104dc90a-21ef-4d5e-8912-b30baabb152f",
"host": "redis",
"port": 6379,
"name": "redis:6379",
"db": 0,
"username": "default",
"password": null,
"connectionType": "STANDALONE",
"nameFromProvider": null,
"provider": "REDIS",
"lastConnection": "2025-10-16T09:22:02.591Z",
"modules": [
{
"name": "ReJSON",
"version": 20808,
"semanticVersion": "2.8.8"
},
{
"name": "search",
"version": 21015,
"semanticVersion": "2.10.15"
}
],
"tls": false,
"tlsServername": null,
"verifyServerCert": null,
"caCert": null,
"clientCert": null,
"ssh": false,
"sshOptions": null,
"forceStandalone": false,
"tags": []
}
]
adminer-index.php:
content: |
<?php
if(!count($$_GET)) {
$$_POST['auth'] = [
'server' => $$_ENV['ADMINER_DEFAULT_SERVER'],
'driver' => 'server', /* seems to autodetect the driver from server settings */
'username' => $$_ENV['ADMINER_DEFAULT_USERNAME'],
'password' => $$_ENV['ADMINER_DEFAULT_PASSWORD'],
'db' => $$_ENV['ADMINER_DEFAULT_DB'],
];
}
include './adminer.php';
+64 -225
View File
@@ -10,6 +10,15 @@ x-logging: &x-logging
max-file: "5"
max-size: "10m"
x-build: &x-build
build:
context: .
target: development
args:
DEBUG: false
TESTING: true
VERSION: dev
services:
traefik:
image: traefik:3.6
@@ -50,15 +59,13 @@ services:
appwrite:
container_name: appwrite
<<: *x-logging
<<: [*x-logging, *x-build]
image: appwrite-dev
build:
context: .
target: development
args:
DEBUG: false
TESTING: true
VERSION: dev
healthcheck:
test: ["CMD", "doctor"]
interval: 5s
timeout: 5s
retries: 12
ports:
- 9501:80
networks:
@@ -101,11 +108,10 @@ services:
- ./dev:/usr/src/code/dev
depends_on:
- ${_APP_DB_HOST:-mongodb}
- redis
- coredns
- ollama
# - clamav
redis:
condition: service_healthy
coredns:
condition: service_started
entrypoint:
- php
- -e
@@ -153,12 +159,6 @@ services:
- _APP_DB_SCHEMA
- _APP_DB_USER
- _APP_DB_PASS
- _APP_DB_ADAPTER_VECTORSDB
- _APP_DB_HOST_VECTORSDB
- _APP_DB_PORT_VECTORSDB
- _APP_DB_SCHEMA_VECTORSDB
- _APP_DB_USER_VECTORSDB
- _APP_DB_PASS_VECTORSDB
- _APP_SMTP_HOST
- _APP_SMTP_PORT
- _APP_SMTP_SECURE
@@ -267,7 +267,7 @@ services:
appwrite-realtime:
entrypoint: realtime
<<: *x-logging
<<: [*x-logging, *x-build]
container_name: appwrite-realtime
image: appwrite-dev
restart: unless-stopped
@@ -295,7 +295,6 @@ services:
depends_on:
- ${_APP_DB_HOST:-mongodb}
- redis
- ollama
environment:
- _APP_ENV
- _APP_WORKER_PER_CORE
@@ -312,12 +311,6 @@ services:
- _APP_DB_SCHEMA
- _APP_DB_USER
- _APP_DB_PASS
- _APP_DB_ADAPTER_VECTORSDB
- _APP_DB_HOST_VECTORSDB
- _APP_DB_PORT_VECTORSDB
- _APP_DB_SCHEMA_VECTORSDB
- _APP_DB_USER_VECTORSDB
- _APP_DB_PASS_VECTORSDB
- _APP_USAGE_STATS
- _APP_LOGGING_CONFIG
- _APP_LOGGING_CONFIG_REALTIME
@@ -326,7 +319,7 @@ services:
appwrite-worker-audits:
entrypoint: worker-audits
<<: *x-logging
<<: [*x-logging, *x-build]
container_name: appwrite-worker-audits
image: appwrite-dev
networks:
@@ -337,7 +330,6 @@ services:
depends_on:
- redis
- ${_APP_DB_HOST:-mongodb}
- ollama
environment:
- _APP_ENV
- _APP_WORKER_PER_CORE
@@ -358,7 +350,7 @@ services:
appwrite-worker-webhooks:
entrypoint: worker-webhooks
<<: *x-logging
<<: [*x-logging, *x-build]
container_name: appwrite-worker-webhooks
image: appwrite-dev
networks:
@@ -371,7 +363,6 @@ services:
- ${_APP_DB_HOST:-mongodb}
- request-catcher-sms
- request-catcher-webhook
- ollama
environment:
- _APP_ENV
- _APP_WORKER_PER_CORE
@@ -394,7 +385,7 @@ services:
appwrite-worker-deletes:
entrypoint: worker-deletes
<<: *x-logging
<<: [*x-logging, *x-build]
container_name: appwrite-worker-deletes
image: appwrite-dev
networks:
@@ -402,7 +393,6 @@ services:
depends_on:
- redis
- ${_APP_DB_HOST:-mongodb}
- ollama
volumes:
- appwrite-uploads:/storage/uploads:rw
- appwrite-cache:/storage/cache:rw
@@ -412,7 +402,6 @@ services:
- appwrite-certificates:/storage/certificates:rw
- ./app:/usr/src/code/app
- ./src:/usr/src/code/src
environment:
- _APP_ENV
- _APP_WORKER_PER_CORE
@@ -461,7 +450,7 @@ services:
appwrite-worker-databases:
entrypoint: worker-databases
<<: *x-logging
<<: [*x-logging, *x-build]
container_name: appwrite-worker-databases
image: appwrite-dev
networks:
@@ -469,11 +458,9 @@ services:
volumes:
- ./app:/usr/src/code/app
- ./src:/usr/src/code/src
depends_on:
- redis
- ${_APP_DB_HOST:-mongodb}
- ollama
environment:
- _APP_ENV
- _APP_WORKER_PER_CORE
@@ -489,12 +476,6 @@ services:
- _APP_DB_SCHEMA
- _APP_DB_USER
- _APP_DB_PASS
- _APP_DB_ADAPTER_VECTORSDB
- _APP_DB_HOST_VECTORSDB
- _APP_DB_PORT_VECTORSDB
- _APP_DB_SCHEMA_VECTORSDB
- _APP_DB_USER_VECTORSDB
- _APP_DB_PASS_VECTORSDB
- _APP_LOGGING_CONFIG
- _APP_WORKERS_NUM
- _APP_QUEUE_NAME
@@ -502,7 +483,7 @@ services:
appwrite-worker-builds:
entrypoint: worker-builds
<<: *x-logging
<<: [*x-logging, *x-build]
container_name: appwrite-worker-builds
image: appwrite-dev
networks:
@@ -516,7 +497,6 @@ services:
depends_on:
- redis
- ${_APP_DB_HOST:-mongodb}
- ollama
environment:
- _APP_ENV
- _APP_WORKER_PER_CORE
@@ -578,7 +558,7 @@ services:
appwrite-worker-screenshots:
entrypoint: worker-screenshots
<<: *x-logging
<<: [*x-logging, *x-build]
container_name: appwrite-worker-screenshots
image: appwrite-dev
networks:
@@ -641,7 +621,7 @@ services:
appwrite-worker-certificates:
entrypoint: worker-certificates
<<: *x-logging
<<: [*x-logging, *x-build]
container_name: appwrite-worker-certificates
image: appwrite-dev
networks:
@@ -649,7 +629,6 @@ services:
depends_on:
- redis
- ${_APP_DB_HOST:-mongodb}
- ollama
volumes:
- appwrite-config:/storage/config:rw
- appwrite-certificates:/storage/certificates:rw
@@ -684,7 +663,7 @@ services:
appwrite-worker-executions:
entrypoint: worker-executions
<<: *x-logging
<<: [*x-logging, *x-build]
container_name: appwrite-worker-executions
image: appwrite-dev
networks:
@@ -714,7 +693,7 @@ services:
appwrite-worker-functions:
entrypoint: worker-functions
<<: *x-logging
<<: [*x-logging, *x-build]
container_name: appwrite-worker-functions
image: appwrite-dev
networks:
@@ -758,7 +737,7 @@ services:
appwrite-worker-mails:
entrypoint: worker-mails
<<: *x-logging
<<: [*x-logging, *x-build]
container_name: appwrite-worker-mails
image: appwrite-dev
networks:
@@ -800,7 +779,7 @@ services:
appwrite-worker-messaging:
entrypoint: worker-messaging
<<: *x-logging
<<: [*x-logging, *x-build]
container_name: appwrite-worker-messaging
restart: unless-stopped
image: appwrite-dev
@@ -857,7 +836,7 @@ services:
appwrite-worker-migrations:
entrypoint: worker-migrations
<<: *x-logging
<<: [*x-logging, *x-build]
container_name: appwrite-worker-migrations
restart: unless-stopped
image: appwrite-dev
@@ -869,7 +848,6 @@ services:
- ./app:/usr/src/code/app
- ./src:/usr/src/code/src
- ./tests:/usr/src/code/tests
depends_on:
- ${_APP_DB_HOST:-mongodb}
environment:
@@ -903,7 +881,7 @@ services:
appwrite-task-maintenance:
entrypoint: maintenance
<<: *x-logging
<<: [*x-logging, *x-build]
container_name: appwrite-task-maintenance
image: appwrite-dev
networks:
@@ -949,7 +927,7 @@ services:
appwrite-task-interval:
entrypoint: interval
<<: *x-logging
<<: [*x-logging, *x-build]
container_name: appwrite-task-interval
image: appwrite-dev
networks:
@@ -990,7 +968,7 @@ services:
appwrite-task-stats-resources:
container_name: appwrite-task-stats-resources
entrypoint: stats-resources
<<: *x-logging
<<: [*x-logging, *x-build]
image: appwrite-dev
networks:
- appwrite
@@ -1022,7 +1000,7 @@ services:
appwrite-worker-stats-resources:
entrypoint: worker-stats-resources
<<: *x-logging
<<: [*x-logging, *x-build]
container_name: appwrite-worker-stats-resources
image: appwrite-dev
networks:
@@ -1055,7 +1033,7 @@ services:
appwrite-worker-stats-usage:
entrypoint: worker-stats-usage
<<: *x-logging
<<: [*x-logging, *x-build]
container_name: appwrite-worker-stats-usage
image: appwrite-dev
networks:
@@ -1066,7 +1044,6 @@ services:
depends_on:
- redis
- ${_APP_DB_HOST:-mongodb}
- ollama
environment:
- _APP_ENV
- _APP_WORKER_PER_CORE
@@ -1089,7 +1066,7 @@ services:
appwrite-task-scheduler-functions:
entrypoint: schedule-functions
<<: *x-logging
<<: [*x-logging, *x-build]
container_name: appwrite-task-scheduler-functions
image: appwrite-dev
networks:
@@ -1100,7 +1077,6 @@ services:
depends_on:
- ${_APP_DB_HOST:-mongodb}
- redis
- ollama
environment:
- _APP_ENV
- _APP_WORKER_PER_CORE
@@ -1120,7 +1096,7 @@ services:
appwrite-task-scheduler-executions:
entrypoint: schedule-executions
<<: *x-logging
<<: [*x-logging, *x-build]
container_name: appwrite-task-scheduler-executions
image: appwrite-dev
networks:
@@ -1131,7 +1107,6 @@ services:
depends_on:
- ${_APP_DB_HOST:-mongodb}
- redis
- ollama
environment:
- _APP_ENV
- _APP_WORKER_PER_CORE
@@ -1150,7 +1125,7 @@ services:
appwrite-task-scheduler-messages:
entrypoint: schedule-messages
<<: *x-logging
<<: [*x-logging, *x-build]
container_name: appwrite-task-scheduler-messages
image: appwrite-dev
networks:
@@ -1161,7 +1136,6 @@ services:
depends_on:
- ${_APP_DB_HOST:-mongodb}
- redis
- ollama
environment:
- _APP_ENV
- _APP_WORKER_PER_CORE
@@ -1253,6 +1227,7 @@ services:
start_period: 5s
mariadb:
profiles: ["mariadb"]
image: mariadb:10.11 # fix issues when upgrading using: mysql_upgrade -u root -p
container_name: appwrite-mariadb
<<: *x-logging
@@ -1269,8 +1244,14 @@ services:
- MYSQL_PASSWORD=${_APP_DB_PASS}
- MARIADB_AUTO_UPGRADE=1
command: "mysqld --innodb-flush-method=fsync"
healthcheck:
test: ["CMD", "healthcheck.sh", "--connect", "--innodb_initialized"]
interval: 5s
timeout: 5s
retries: 12
mongodb:
profiles: ["mongodb"]
image: mongo:8.2.5
container_name: appwrite-mongodb
<<: *x-logging
@@ -1306,55 +1287,32 @@ services:
retries: 10
start_period: 30s
appwrite-mongo-express:
image: mongo-express
container_name: appwrite-mongo-express
networks:
- appwrite
ports:
- "8082:8081"
environment:
ME_CONFIG_MONGODB_URL: "mongodb://root:${_APP_DB_ROOT_PASS}@appwrite-mongodb:27017/?replicaSet=rs0&directConnection=true"
ME_CONFIG_BASICAUTH_USERNAME: ${_APP_DB_USER}
ME_CONFIG_BASICAUTH_PASSWORD: ${_APP_DB_PASS}
depends_on:
- mongodb
postgresql:
image: appwrite/postgres:0.1.0
profiles: ["postgresql"]
build:
context: ./tests/resources/postgresql
args:
POSTGRES_VERSION: 17
container_name: appwrite-postgresql
<<: *x-logging
networks:
- appwrite
volumes:
- appwrite-postgresql:/var/lib/postgresql/18/data:rw
- appwrite-postgresql:/var/lib/postgresql/data:rw
ports:
- "5432:5432"
environment:
- POSTGRES_DB=${_APP_DB_SCHEMA}
- POSTGRES_USER=${_APP_DB_USER}
- POSTGRES_PASSWORD=${_APP_DB_PASS}
command: "postgres"
healthcheck:
test: ["CMD-SHELL", "pg_isready -U ${_APP_DB_USER} -d ${_APP_DB_SCHEMA}"]
test: ["CMD-SHELL", "pg_isready -U ${_APP_DB_USER}"]
interval: 5s
timeout: 5s
retries: 10
start_period: 10s
command: "postgres"
ollama:
image: appwrite/ollama:0.1.1
container_name: ollama
ports:
- "11434:11434"
restart: unless-stopped
environment:
MODELS: ${_APP_EMBEDDING_MODELS:-embeddinggemma}
OLLAMA_KEEP_ALIVE: 24h
volumes:
- appwrite-models:/root/.ollama
networks:
- appwrite
retries: 12
redis:
image: redis:7.4.7-alpine
@@ -1371,6 +1329,11 @@ services:
- appwrite
volumes:
- appwrite-redis:/data:rw
healthcheck:
test: ["CMD", "redis-cli", "ping"]
interval: 5s
timeout: 5s
retries: 12
coredns: # DNS server for testing purposes (Proxy APIs)
image: coredns/coredns:1.12.4
@@ -1442,78 +1405,8 @@ services:
networks:
- appwrite
adminer:
image: adminer
container_name: appwrite-adminer
<<: *x-logging
restart: always
ports:
- 9506:8080
networks:
- appwrite
- gateway
environment:
- ADMINER_DESIGN=pepa-linha
- ADMINER_DEFAULT_SERVER=mariadb
- ADMINER_DEFAULT_USERNAME=root
- ADMINER_DEFAULT_PASSWORD=rootsecretpassword
- ADMINER_DEFAULT_DB=appwrite
configs:
- source: adminer-index.php
target: /var/www/html/index.php
mode: 0755
labels:
- "traefik.enable=true"
- "traefik.constraint-label-stack=appwrite"
- "traefik.docker.network=gateway"
- "traefik.http.services.appwrite_adminer.loadbalancer.server.port=8080"
- "traefik.http.routers.appwrite_adminer_http.entrypoints=appwrite_web"
- "traefik.http.routers.appwrite_adminer_http.rule=Host(`mysql.localhost`)"
- "traefik.http.routers.appwrite_adminer_http.service=appwrite_adminer"
- "traefik.http.routers.appwrite_adminer_https.entrypoints=appwrite_websecure"
- "traefik.http.routers.appwrite_adminer_https.rule=Host(`mysql.localhost`)"
- "traefik.http.routers.appwrite_adminer_https.service=appwrite_adminer"
- "traefik.http.routers.appwrite_adminer_https.tls=true"
redis-insight:
image: redis/redisinsight:latest
restart: unless-stopped
networks:
- appwrite
- gateway
environment:
- RI_PRE_SETUP_DATABASES_PATH=/mnt/connections.json
configs:
- source: redisinsight-connections.json
target: /mnt/connections.json
mode: 0755
labels:
- "traefik.enable=true"
- "traefik.constraint-label-stack=appwrite"
- "traefik.docker.network=gateway"
- "traefik.http.services.appwrite_redisinsight.loadbalancer.server.port=5540"
- "traefik.http.routers.appwrite_redisinsight_http.entrypoints=appwrite_web"
- "traefik.http.routers.appwrite_redisinsight_http.rule=Host(`redis.localhost`)"
- "traefik.http.routers.appwrite_redisinsight_http.service=appwrite_redisinsight"
- "traefik.http.routers.appwrite_redisinsight_https.entrypoints=appwrite_websecure"
- "traefik.http.routers.appwrite_redisinsight_https.rule=Host(`redis.localhost`)"
- "traefik.http.routers.appwrite_redisinsight_https.service=appwrite_redisinsight"
- "traefik.http.routers.appwrite_redisinsight_https.tls=true"
ports:
- "8081:5540"
graphql-explorer:
container_name: appwrite-graphql-explorer
image: appwrite/altair:0.3.0
restart: unless-stopped
networks:
- appwrite
ports:
- "9509:3000"
environment:
- SERVER_URL=http://localhost/v1/graphql
# Dev Tools End ------------------------------------------------------------------------------------------
# Dev tools (adminer, redis-insight, mongo-express, graphql-explorer)
# are defined in docker-compose.override.yml
networks:
gateway:
@@ -1526,60 +1419,7 @@ networks:
runtimes:
name: runtimes
configs:
redisinsight-connections.json:
content: |
[
{
"compressor": "NONE",
"id": "104dc90a-21ef-4d5e-8912-b30baabb152f",
"host": "redis",
"port": 6379,
"name": "redis:6379",
"db": 0,
"username": "default",
"password": null,
"connectionType": "STANDALONE",
"nameFromProvider": null,
"provider": "REDIS",
"lastConnection": "2025-10-16T09:22:02.591Z",
"modules": [
{
"name": "ReJSON",
"version": 20808,
"semanticVersion": "2.8.8"
},
{
"name": "search",
"version": 21015,
"semanticVersion": "2.10.15"
}
],
"tls": false,
"tlsServername": null,
"verifyServerCert": null,
"caCert": null,
"clientCert": null,
"ssh": false,
"sshOptions": null,
"forceStandalone": false,
"tags": []
}
]
adminer-index.php:
content: |
<?php
if(!count($$_GET)) {
$$_POST['auth'] = [
'server' => $$_ENV['ADMINER_DEFAULT_SERVER'],
'driver' => 'server', /* seems to autodetect the driver from server settings */
'username' => $$_ENV['ADMINER_DEFAULT_USERNAME'],
'password' => $$_ENV['ADMINER_DEFAULT_PASSWORD'],
'db' => $$_ENV['ADMINER_DEFAULT_DB'],
];
}
include './adminer.php';
volumes:
appwrite-mariadb:
@@ -1595,4 +1435,3 @@ volumes:
appwrite-sites:
appwrite-builds:
appwrite-config:
appwrite-models:
@@ -1 +0,0 @@
Create a new Collection. Before using this route, you should create a new database resource using either a [server integration](https://appwrite.io/docs/server/databases#documentsDBCreateCollection) API or directly from your database console.
@@ -1 +0,0 @@
Create a new Document. Before using this route, you should create a new collection resource using either a [server integration](https://appwrite.io/docs/server/databases#documentsDBCreateCollection) API or directly from your database console.
@@ -1 +0,0 @@
Create new Documents. Before using this route, you should create a new collection resource using either a [server integration](https://appwrite.io/docs/server/databases#documentsDBCreateCollection) API or directly from your database console.
@@ -1,2 +0,0 @@
Creates an index on the attributes listed. Your index should include all the attributes you will query in a single request.
Attributes can be `key`, `fulltext`, and `unique`.
-1
View File
@@ -1 +0,0 @@
Create a new Database.
@@ -1 +0,0 @@
Decrement a specific column of a row by a given value.
@@ -1 +0,0 @@
Delete a collection by its unique ID. Only users with write permissions have access to delete this resource.
@@ -1 +0,0 @@
Delete a document by its unique ID.
@@ -1 +0,0 @@
Bulk delete documents using queries, if no queries are passed then all documents are deleted.
@@ -1 +0,0 @@
Delete an index.
-1
View File
@@ -1 +0,0 @@
Delete a database by its unique ID. Only API keys with with databases.write scope can delete a database.
@@ -1 +0,0 @@
Get the collection activity logs list by its unique ID.
@@ -1 +0,0 @@
Get usage metrics and statistics for a collection. Returning the total number of documents. The response includes both current totals and historical data over time. Use the optional range parameter to specify the time window for historical data: 24h (last 24 hours), 30d (last 30 days), or 90d (last 90 days). If not specified, range defaults to 30 days.
@@ -1 +0,0 @@
Get a collection by its unique ID. This endpoint response returns a JSON object with the collection metadata.
@@ -1 +0,0 @@
Get usage metrics and statistics for a database. You can view the total number of collections, documents, and storage usage. The response includes both current totals and historical data over time. Use the optional range parameter to specify the time window for historical data: 24h (last 24 hours), 30d (last 30 days), or 90d (last 90 days). If not specified, range defaults to 30 days.
@@ -1 +0,0 @@
Get the document activity logs list by its unique ID.
@@ -1 +0,0 @@
Get a document by its unique ID. This endpoint response returns a JSON object with the document data.
-1
View File
@@ -1 +0,0 @@
Get index by ID.
-1
View File
@@ -1 +0,0 @@
Get the database activity logs list by its unique ID.
-1
View File
@@ -1 +0,0 @@
Get a database by its unique ID. This endpoint response returns a JSON object with the database metadata.
@@ -1 +0,0 @@
Increment a specific column of a row by a given value.
@@ -1 +0,0 @@
List attributes in the collection.
@@ -1 +0,0 @@
Get a list of all collections that belong to the provided databaseId. You can use the search parameter to filter your results.
@@ -1 +0,0 @@
Get a list of all the user's documents in a given collection. You can use the query params to filter your results.
@@ -1 +0,0 @@
List indexes in the collection.
@@ -1 +0,0 @@
List usage metrics and statistics for all databases in the project. You can view the total number of databases, collections, documents, and storage usage. The response includes both current totals and historical data over time. Use the optional range parameter to specify the time window for historical data: 24h (last 24 hours), 30d (last 30 days), or 90d (last 90 days). If not specified, range defaults to 30 days.
-1
View File
@@ -1 +0,0 @@
Get a list of all databases from the current Appwrite project. You can use the search parameter to filter your results.
@@ -1 +0,0 @@
Update a collection by its unique ID.
@@ -1 +0,0 @@
Update a document by its unique ID. Using the patch method you can pass only specific fields that will get updated.
@@ -1 +0,0 @@
Update all documents that match your queries, if no queries are submitted then all documents are updated. You can pass only specific fields to be updated.
-1
View File
@@ -1 +0,0 @@
Update a database by its unique ID.
@@ -1 +0,0 @@
Create or update a Document. Before using this route, you should create a new collection resource using either a [server integration](https://appwrite.io/docs/server/databases#documentsDBCreateCollection) API or directly from your database console.
@@ -1 +0,0 @@
Create or update Documents. Before using this route, you should create a new collection resource using either a [server integration](https://appwrite.io/docs/server/databases#documentsDBCreateCollection) API or directly from your database console.
@@ -1 +0,0 @@
Generate a temporary console-scoped API key for migrating project settings (platforms, keys, schedules).
Regular → Executable
View File
Regular → Executable
View File
+1 -1
View File
@@ -15,4 +15,4 @@ adminDb.createUser({
roles: [
{ role: 'readWrite', db: database }
]
});
});
+10
View File
@@ -0,0 +1,10 @@
{
"name": "@appwrite.io/repo",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "@appwrite.io/repo"
}
}
}
+2 -24
View File
@@ -28,8 +28,6 @@ class Key
protected bool $projectCheckDisabled = false,
protected bool $previewAuthDisabled = false,
protected bool $deploymentStatusIgnored = false,
protected string $scopedProjectId = '',
protected string $source = '',
) {
}
@@ -105,16 +103,6 @@ class Key
return $this->projectCheckDisabled;
}
public function getScopedProjectId(): string
{
return $this->scopedProjectId;
}
public function getSource(): string
{
return $this->source;
}
/**
* Decode the given secret key into a Key object, containing the project ID, type, role, scopes, and name.
* Can be a stored API key or a dynamic key (JWT).
@@ -173,15 +161,7 @@ class Key
$projectCheckDisabled = $payload['projectCheckDisabled'] ?? false;
$previewAuthDisabled = $payload['previewAuthDisabled'] ?? false;
$deploymentStatusIgnored = $payload['deploymentStatusIgnored'] ?? false;
$scopedProjectId = $payload['scopedProjectId'] ?? '';
$source = $payload['source'] ?? '';
// Keys with a scoped project are restricted — only use explicit JWT scopes
if (!empty($scopedProjectId)) {
$scopes = $payload['scopes'] ?? [];
} else {
$scopes = \array_merge($payload['scopes'] ?? [], $scopes);
}
$scopes = \array_merge($payload['scopes'] ?? [], $scopes);
if (!$projectCheckDisabled && $projectId !== $project->getId()) {
return $guestKey;
@@ -201,9 +181,7 @@ class Key
$bannerDisabled,
$projectCheckDisabled,
$previewAuthDisabled,
$deploymentStatusIgnored,
$scopedProjectId,
$source
$deploymentStatusIgnored
);
case API_KEY_STANDARD:
$key = $project->find(
+22 -21
View File
@@ -4,11 +4,12 @@ namespace Appwrite\Bus\Listeners;
use Appwrite\Bus\Events\ExecutionCompleted;
use Appwrite\Bus\Events\RequestCompleted;
use Appwrite\Event\StatsUsage;
use Appwrite\Event\Message\Usage as UsageMessage;
use Appwrite\Event\Publisher\Usage as Publisher;
use Appwrite\Usage\Context;
use Utopia\Bus\Event;
use Utopia\Bus\Listener;
use Utopia\Database\Document;
use Utopia\Queue\Publisher;
class Usage extends Listener
{
@@ -29,20 +30,21 @@ class Usage extends Listener
{
$this
->desc('Records usage metrics')
->inject('publisherStatsUsage')
->inject('publisherForUsage')
->inject('usage')
->callback($this->handle(...));
}
public function handle(Event $event, Publisher $publisher): void
public function handle(Event $event, Publisher $publisherForUsage, Context $usage): void
{
match (true) {
$event instanceof ExecutionCompleted => $this->handleExecutionCompleted($event, $publisher),
$event instanceof RequestCompleted => $this->handleRequestCompleted($event, $publisher),
$event instanceof ExecutionCompleted => $this->handleExecutionCompleted($event, $publisherForUsage),
$event instanceof RequestCompleted => $this->handleRequestCompleted($event, $usage),
default => null,
};
}
private function handleExecutionCompleted(ExecutionCompleted $event, Publisher $publisher): void
private function handleExecutionCompleted(ExecutionCompleted $event, Publisher $publisherForUsage): void
{
$execution = new Document($event->execution);
$resource = new Document($event->resource);
@@ -61,9 +63,7 @@ class Usage extends Listener
$compute = (int)($duration * 1000);
$mbSeconds = (int)(($spec['memory'] ?? APP_COMPUTE_MEMORY_DEFAULT) * $duration * ($spec['cpus'] ?? APP_COMPUTE_CPUS_DEFAULT));
$queueForStatsUsage = new StatsUsage($publisher);
$queueForStatsUsage
->setProject($project)
$context = (new Context())
->addMetric(METRIC_EXECUTIONS, 1)
->addMetric(str_replace(['{resourceType}'], [$resourceType], METRIC_RESOURCE_TYPE_EXECUTIONS), 1)
->addMetric(str_replace(['{resourceType}', '{resourceInternalId}'], [$resourceType, $resourceInternalId], METRIC_RESOURCE_TYPE_ID_EXECUTIONS), 1)
@@ -72,11 +72,18 @@ class Usage extends Listener
->addMetric(str_replace(['{resourceType}', '{resourceInternalId}'], [$resourceType, $resourceInternalId], METRIC_RESOURCE_TYPE_ID_EXECUTIONS_COMPUTE), $compute)
->addMetric(METRIC_EXECUTIONS_MB_SECONDS, $mbSeconds)
->addMetric(str_replace(['{resourceType}'], [$resourceType], METRIC_RESOURCE_TYPE_EXECUTIONS_MB_SECONDS), $mbSeconds)
->addMetric(str_replace(['{resourceType}', '{resourceInternalId}'], [$resourceType, $resourceInternalId], METRIC_RESOURCE_TYPE_ID_EXECUTIONS_MB_SECONDS), $mbSeconds)
->trigger();
->addMetric(str_replace(['{resourceType}', '{resourceInternalId}'], [$resourceType, $resourceInternalId], METRIC_RESOURCE_TYPE_ID_EXECUTIONS_MB_SECONDS), $mbSeconds);
$message = new UsageMessage(
project: $project,
metrics: $context->getMetrics(),
reduce: $context->getReduce()
);
$publisherForUsage->enqueue($message);
}
private function handleRequestCompleted(RequestCompleted $event, Publisher $publisher): void
private function handleRequestCompleted(RequestCompleted $event, Context $usage): void
{
$fileSize = 0;
$file = $event->request->getFiles('file');
@@ -84,18 +91,14 @@ class Usage extends Listener
$fileSize = (\is_array($file['size']) && isset($file['size'][0])) ? $file['size'][0] : $file['size'];
}
$project = new Document($event->project);
$deployment = new Document($event->deployment);
$queueForStatsUsage = new StatsUsage($publisher);
$inbound = $event->request->getSize() + $fileSize;
$outbound = $event->response->getSize();
$queueForStatsUsage->setProject($project);
if ($deployment->getAttribute('resourceType') === 'sites') {
$siteInternalId = $deployment->getAttribute('resourceInternalId', '');
$queueForStatsUsage
$usage
->addMetric(METRIC_SITES_REQUESTS, 1)
->addMetric(METRIC_SITES_INBOUND, $inbound)
->addMetric(METRIC_SITES_OUTBOUND, $outbound)
@@ -103,12 +106,10 @@ class Usage extends Listener
->addMetric(str_replace('{siteInternalId}', $siteInternalId, METRIC_SITES_ID_INBOUND), $inbound)
->addMetric(str_replace('{siteInternalId}', $siteInternalId, METRIC_SITES_ID_OUTBOUND), $outbound);
} else {
$queueForStatsUsage
$usage
->addMetric(METRIC_NETWORK_REQUESTS, 1)
->addMetric(METRIC_NETWORK_INBOUND, $inbound)
->addMetric(METRIC_NETWORK_OUTBOUND, $outbound);
}
$queueForStatsUsage->trigger();
}
}
+14 -27
View File
@@ -21,23 +21,17 @@ class TransactionState
{
private Database $dbForProject;
private Authorization $authorization;
/**
* @var callable(Document $database): Database
*/
private mixed $getDatabasesDB;
public function __construct(Database $dbForProject, Authorization $authorization, callable $getDatabasesDB)
/** @var Authorization $authorization */
public function __construct(Database $dbForProject, Authorization $authorization)
{
$this->dbForProject = $dbForProject;
$this->authorization = $authorization;
$this->getDatabasesDB = $getDatabasesDB;
}
/**
* Get a document with transaction-aware logic
*
* @param Document $database Target database document
* @param string $collectionId Collection ID
* @param string $documentId Document ID
* @param string|null $transactionId Optional transaction ID
@@ -48,15 +42,13 @@ class TransactionState
* @throws Timeout
*/
public function getDocument(
Document $database,
string $collectionId,
string $documentId,
?string $transactionId = null,
array $queries = []
): Document {
$dbForDatabases = ($this->getDatabasesDB)($database);
if ($transactionId === null) {
return $dbForDatabases->getDocument($collectionId, $documentId, $queries);
return $this->dbForProject->getDocument($collectionId, $documentId, $queries);
}
$state = $this->getTransactionState($transactionId);
@@ -74,7 +66,7 @@ class TransactionState
if ($docState['action'] === 'update' || $docState['action'] === 'upsert') {
// Merge with committed version
$committedDoc = $dbForDatabases->getDocument($collectionId, $documentId, $queries);
$committedDoc = $this->dbForProject->getDocument($collectionId, $documentId, $queries);
if (!$committedDoc->isEmpty()) {
foreach ($docState['document']->getAttributes() as $key => $value) {
if ($key !== '$id') {
@@ -88,13 +80,13 @@ class TransactionState
}
}
}
return $dbForDatabases->getDocument($collectionId, $documentId, $queries);
return $this->dbForProject->getDocument($collectionId, $documentId, $queries);
}
/**
* List documents with transaction-aware logic
*
* @param Document $database Target database document
* @param string $collectionId Collection ID
* @param string|null $transactionId Optional transaction ID
* @param array $queries Optional query filters
@@ -104,19 +96,17 @@ class TransactionState
* @throws Timeout
*/
public function listDocuments(
Document $database,
string $collectionId,
?string $transactionId = null,
array $queries = []
): array {
$dbForDatabases = ($this->getDatabasesDB)($database);
// If no transaction, use normal database retrieval
if ($transactionId === null) {
return $dbForDatabases->find($collectionId, $queries);
return $this->dbForProject->find($collectionId, $queries);
}
$state = $this->getTransactionState($transactionId);
$committedDocs = $dbForDatabases->find($collectionId, $queries);
$committedDocs = $this->dbForProject->find($collectionId, $queries);
$documentMap = [];
// Build map of committed documents
@@ -157,7 +147,6 @@ class TransactionState
/**
* Count documents with transaction-aware logic
*
* @param Document $database Target database document
* @param string $collectionId Collection ID
* @param string|null $transactionId Optional transaction ID
* @param array $queries Optional query filters
@@ -167,23 +156,23 @@ class TransactionState
* @throws Timeout
*/
public function countDocuments(
Document $database,
string $collectionId,
?string $transactionId = null,
array $queries = []
): int {
$dbForDatabases = ($this->getDatabasesDB)($database);
if ($transactionId === null) {
return $dbForDatabases->count($collectionId, $queries, APP_LIMIT_COUNT);
return $this->dbForProject->count($collectionId, $queries, APP_LIMIT_COUNT);
}
$state = $this->getTransactionState($transactionId);
$baseCount = $dbForDatabases->count($collectionId, $queries, APP_LIMIT_COUNT);
$baseCount = $this->dbForProject->count($collectionId, $queries, APP_LIMIT_COUNT);
if (!isset($state[$collectionId])) {
return $baseCount;
}
$committedDocs = $dbForDatabases->find($collectionId, $queries);
$committedDocs = $this->dbForProject->find($collectionId, $queries);
$committedDocIds = [];
foreach ($committedDocs as $doc) {
$committedDocIds[$doc->getId()] = true;
@@ -225,19 +214,17 @@ class TransactionState
/**
* Check if a document exists with transaction-aware logic
*
* @param Document $database Target database document
* @param string $collectionId Collection ID
* @param string $documentId Document ID
* @param string|null $transactionId Optional transaction ID
* @return bool True if document exists
*/
public function documentExists(
Document $database,
string $collectionId,
string $documentId,
?string $transactionId = null
): bool {
$doc = $this->getDocument($database, $collectionId, $documentId, $transactionId);
$doc = $this->getDocument($collectionId, $documentId, $transactionId);
return !$doc->isEmpty();
}
+2 -9
View File
@@ -519,7 +519,6 @@ class Event
* @param string $pattern
* @param array $params
* @param ?Document $database
* @param ?Document $database
* @return array
* @throws \InvalidArgumentException
*/
@@ -534,7 +533,7 @@ class Event
$parsed = self::parseEventPattern($pattern);
// to switch the resource types from databases to the required prefix
// eg; all databases events get fired with databases. prefix which mainly depicts legacy type
// so a projection from databases to the actual prefix(documentsdb, vectorsdb,etc)
// so a projection from databases to the actual prefix
if ((str_contains($pattern, 'databases.') && $database && $database->getAttribute('type') !== 'legacy')) {
$parsed = self::getDatabaseTypeEvents($database, $parsed);
}
@@ -696,6 +695,7 @@ class Event
)
) {
$pairedEvents = [];
foreach ($events as $event) {
$pairedEvents[] = $event;
// tablesdb needs databases event with tables and collections
@@ -745,13 +745,6 @@ class Event
'attributes' => 'columns',
];
break;
case 'documentsdb':
case 'vectorsdb':
// sending the type itself(eg: documentsdb, vectorsdb)
$eventMap = [
'databases' => $database->getAttribute('type')
];
break;
}
foreach ($event as $eventKey => $eventValue) {
if (isset($eventMap[$eventValue])) {
+21
View File
@@ -0,0 +1,21 @@
<?php
namespace Appwrite\Event\Message;
abstract class Base
{
/**
* Serialize message to array for queue
*
* @return array
*/
abstract public function toArray(): array;
/**
* Deserialize message from array
*
* @param array $data
* @return static
*/
abstract public static function fromArray(array $data): static;
}
+49
View File
@@ -0,0 +1,49 @@
<?php
namespace Appwrite\Event\Message;
use Utopia\Database\Document;
class Usage extends Base
{
/**
* @param Document $project
* @param array<array{key: string, value: int}> $metrics
* @param array<Document> $reduce
*/
public function __construct(
public readonly Document $project,
public readonly array $metrics,
public readonly array $reduce = [],
) {
}
/**
* @return array
*/
public function toArray(): array
{
return [
'project' => [
'$id' => $this->project->getId(),
'$sequence' => $this->project->getSequence(),
'database' => $this->project->getAttribute('database', ''),
],
'metrics' => $this->metrics,
'reduce' => array_map(fn (Document $doc) => $doc->getArrayCopy(), $this->reduce),
];
}
/**
* @param array $data
* @return static
*/
public static function fromArray(array $data): static
{
return new self(
project: new Document($data['project'] ?? []),
metrics: $data['metrics'] ?? [],
reduce: array_map(fn (array $doc) => new Document($doc), $data['reduce'] ?? []),
);
}
}
+33
View File
@@ -0,0 +1,33 @@
<?php
namespace Appwrite\Event\Publisher;
use Appwrite\Event\Message\Base as BaseMessage;
use Utopia\Queue\Publisher;
use Utopia\Queue\Queue;
readonly class Base
{
public function __construct(
protected Publisher $publisher
) {
}
/**
* Publish a message to the queue
*/
public function publish(Queue $queue, BaseMessage $message): string|bool
{
$payload = $message->toArray();
return $this->publisher->enqueue($queue, $payload);
}
/**
* Get the size of a queue
*/
public function getQueueSize(Queue $queue, bool $failed = false): int
{
return $this->publisher->getQueueSize($queue, $failed);
}
}
+39
View File
@@ -0,0 +1,39 @@
<?php
namespace Appwrite\Event\Publisher;
use Appwrite\Event\Message\Usage as UsageMessage;
use Utopia\Console;
use Utopia\Queue\Publisher;
use Utopia\Queue\Queue;
readonly class Usage extends Base
{
public function __construct(
Publisher $publisher,
protected Queue $queue
) {
parent::__construct($publisher);
}
/**
* Enqueue a usage message
*/
public function enqueue(UsageMessage $message): string|bool
{
try {
return $this->publish($this->queue, $message);
} catch (\Throwable $th) {
Console::error('[Usage] Failed to publish usage message: ' . $th->getMessage());
return false;
}
}
/**
* Get the size of the usage queue
*/
public function getSize(bool $failed = false): int
{
return $this->getQueueSize($this->queue, $failed);
}
}
-97
View File
@@ -1,97 +0,0 @@
<?php
namespace Appwrite\Event;
use Utopia\Database\Document;
use Utopia\Queue\Publisher;
use Utopia\System\System;
class StatsUsage extends Event
{
protected array $metrics = [];
protected array $reduce = [];
protected array $disabled = [];
protected bool $critical = false;
public function __construct(protected Publisher $publisher)
{
parent::__construct($publisher);
$this
->setQueue(System::getEnv('_APP_STATS_USAGE_QUEUE_NAME', Event::STATS_USAGE_QUEUE_NAME))
->setClass(System::getEnv('_APP_STATS_USAGE_CLASS_NAME', Event::STATS_USAGE_CLASS_NAME));
}
/**
* Add reduce.
*
* @param Document $document
* @return self
*/
public function addReduce(Document $document): self
{
$this->reduce[] = $document;
return $this;
}
/**
* Add metric.
*
* @param string $key
* @param int $value
* @return self
*/
public function addMetric(string $key, int $value): self
{
$this->metrics[] = [
'key' => $key,
'value' => $value,
];
return $this;
}
/**
* Set disabled metrics.
*
* @param string $key
* @return self
*/
public function disableMetric(string $key): self
{
$this->disabled[] = $key;
return $this;
}
/**
* Prepare the payload for the event
*
* @return array
*/
protected function preparePayload(): array
{
return [
'project' => $this->getProject(),
'reduce' => $this->reduce,
'metrics' => \array_filter($this->metrics, function ($metric) {
foreach ($this->disabled as $disabledMetric) {
if (\str_ends_with($metric['key'], $disabledMetric)) {
return false;
}
}
return true;
}),
'context' => $this->context,
];
}
public function reset(): Event
{
$this->metrics = [];
parent::reset();
return $this;
}
}
-1
View File
@@ -339,7 +339,6 @@ class Exception extends \Exception
public const string MIGRATION_ALREADY_EXISTS = 'migration_already_exists';
public const string MIGRATION_IN_PROGRESS = 'migration_in_progress';
public const string MIGRATION_PROVIDER_ERROR = 'migration_provider_error';
public const string MIGRATION_DATABASE_TYPE_UNSUPPORTED = 'migration_database_type_unsupported';
/** Realtime */
public const string REALTIME_MESSAGE_FORMAT_INVALID = 'realtime_message_format_invalid';
+6 -25
View File
@@ -492,8 +492,6 @@ class Realtime extends MessagingAdapter
break;
case 'databases':
case 'tablesdb':
case 'documentsdb':
case 'vectorsdb':
$resource = $parts[4] ?? '';
if (in_array($resource, ['columns', 'attributes', 'indexes'])) {
$channels[] = 'console';
@@ -513,20 +511,12 @@ class Realtime extends MessagingAdapter
$resourceId = $tableId ?: $collectionId;
$channels = [];
switch ($parts[0]) {
case 'databases':
case 'tablesdb':
// sending legacy + tablesdb events to both legacy and tablesdb
$channels = array_values(array_unique(array_merge(
self::getDatabaseChannels('legacy', $database->getId(), $resourceId, $payload->getId(), 'databases'),
self::getDatabaseChannels('tablesdb', $database->getId(), $resourceId, $payload->getId(), 'databases'),
self::getDatabaseChannels('tablesdb', $database->getId(), $resourceId, $payload->getId())
)));
break;
default:
// only prefixed events
$channels = array_values(self::getDatabaseChannels($parts[0], $database->getId(), $resourceId, $payload->getId()));
}
// sending legacy + tablesdb events to both legacy and tablesdb
$channels = array_values(array_unique(array_merge(
self::getDatabaseChannels('legacy', $database->getId(), $resourceId, $payload->getId(), 'databases'),
self::getDatabaseChannels('tablesdb', $database->getId(), $resourceId, $payload->getId(), 'databases'),
self::getDatabaseChannels('tablesdb', $database->getId(), $resourceId, $payload->getId())
)));
$roles = $collection->getAttribute('documentSecurity', false)
? \array_merge($collection->getRead(), $payload->getRead())
@@ -592,7 +582,6 @@ class Realtime extends MessagingAdapter
* @param string $resourceId The collection/table ID
* @param string $payloadId The document/row ID
* @param string $prefixOverride Override the channel prefix when different API types share the same terminology but need different prefixes
* (e.g., 'databases' and 'documentsdb' use same terminology but need different prefixes)
* @return array Array of channel names
*/
private static function getDatabaseChannels(
@@ -626,13 +615,6 @@ class Realtime extends MessagingAdapter
$channels[] = "{$basePrefix}.{$databaseId}.tables.{$resourceId}.rows.{$payloadId}";
break;
case 'documentsdb':
case 'vectorsdb':
$channels[] = 'documents';
$channels[] = "{$basePrefix}.{$databaseId}.collections.{$resourceId}.documents";
$channels[] = "{$basePrefix}.{$databaseId}.collections.{$resourceId}.documents.{$payloadId}";
break;
default:
$basePrefix = 'databases';
$channels[] = 'documents';
@@ -641,7 +623,6 @@ class Realtime extends MessagingAdapter
break;
}
return $channels;
}
}
@@ -7,7 +7,6 @@ use Appwrite\Detector\Detector;
use Appwrite\Event\Event;
use Appwrite\Event\Mail;
use Appwrite\Event\Messaging;
use Appwrite\Event\StatsUsage;
use Appwrite\Extend\Exception;
use Appwrite\SDK\AuthType;
use Appwrite\SDK\ContentType;
@@ -15,6 +14,7 @@ use Appwrite\SDK\Deprecated;
use Appwrite\SDK\Method;
use Appwrite\SDK\Response as SDKResponse;
use Appwrite\Template\Template;
use Appwrite\Usage\Context;
use Appwrite\Utopia\Request;
use Appwrite\Utopia\Response;
use libphonenumber\NumberParseException;
@@ -104,7 +104,7 @@ class Create extends Action
->inject('queueForMessaging')
->inject('queueForMails')
->inject('timelimit')
->inject('queueForStatsUsage')
->inject('usage')
->inject('plan')
->inject('proofForToken')
->inject('proofForCode')
@@ -124,7 +124,7 @@ class Create extends Action
Messaging $queueForMessaging,
Mail $queueForMails,
callable $timelimit,
StatsUsage $queueForStatsUsage,
Context $usage,
array $plan,
ProofsToken $proofForToken,
ProofsCode $proofForCode
@@ -201,16 +201,12 @@ class Create extends Action
$countryCode = $helper->parse($phone)->getCountryCode();
if (!empty($countryCode)) {
$queueForStatsUsage
->addMetric(str_replace('{countryCode}', $countryCode, METRIC_AUTH_METHOD_PHONE_COUNTRY_CODE), 1);
$usage->addMetric(str_replace('{countryCode}', $countryCode, METRIC_AUTH_METHOD_PHONE_COUNTRY_CODE), 1);
}
} catch (NumberParseException $e) {
// Ignore invalid phone number for country code stats
}
$queueForStatsUsage
->addMetric(METRIC_AUTH_METHOD_PHONE, 1)
->setProject($project)
->trigger();
$usage->addMetric(METRIC_AUTH_METHOD_PHONE, 1);
break;
case Type::EMAIL:
if (empty(System::getEnv('_APP_SMTP_HOST'))) {
@@ -2,7 +2,6 @@
namespace Appwrite\Platform\Modules\Avatars\Http\Screenshots;
use Appwrite\Event\StatsUsage;
use Appwrite\Extend\Exception;
use Appwrite\Platform\Modules\Avatars\Http\Action;
use Appwrite\SDK\AuthType;
@@ -10,6 +9,7 @@ use Appwrite\SDK\ContentType;
use Appwrite\SDK\Method;
use Appwrite\SDK\MethodType;
use Appwrite\SDK\Response as SDKResponse;
use Appwrite\Usage\Context;
use Appwrite\Utopia\Response;
use Utopia\Config\Config;
use Utopia\Domains\Domain;
@@ -84,11 +84,11 @@ class Get extends Action
->param('quality', -1, new Range(-1, 100), 'Screenshot quality. Pass an integer between 0 to 100. Defaults to keep existing image quality.', true, example: '85')
->param('output', '', new WhiteList(\array_keys(Config::getParam('storage-outputs')), true), 'Output format type (jpeg, jpg, png, gif and webp).', true, example: 'jpeg')
->inject('response')
->inject('queueForStatsUsage')
->inject('usage')
->callback($this->action(...));
}
public function action(string $url, array $headers, int $viewportWidth, int $viewportHeight, float $scale, string $theme, string $userAgent, bool $fullpage, string $locale, string $timezone, float $latitude, float $longitude, float $accuracy, bool $touch, array $permissions, int $sleep, int $width, int $height, int $quality, string $output, Response $response, StatsUsage $queueForStatsUsage)
public function action(string $url, array $headers, int $viewportWidth, int $viewportHeight, float $scale, string $theme, string $userAgent, bool $fullpage, string $locale, string $timezone, float $latitude, float $longitude, float $accuracy, bool $touch, array $permissions, int $sleep, int $width, int $height, int $quality, string $output, Response $response, Context $usage)
{
if (!\extension_loaded('imagick')) {
throw new Exception(Exception::GENERAL_SERVER_ERROR, 'Imagick extension is missing');
@@ -210,7 +210,7 @@ class Get extends Action
$outputs = Config::getParam('storage-outputs');
$contentType = $outputs[$output] ?? $outputs['png'];
$queueForStatsUsage->addMetric(METRIC_AVATARS_SCREENSHOTS_GENERATED, 1);
$usage->addMetric(METRIC_AVATARS_SCREENSHOTS_GENERATED, 1);
$response
->addHeader('Cache-Control', 'private, max-age=2592000') // 30 days
@@ -22,11 +22,3 @@ const INDEX = 'index';
const DOCUMENTS = 'document';
const ATTRIBUTES = 'attribute';
const COLLECTIONS = 'collection';
const LEGACY = 'legacy';
const TABLESDB = 'tablesdb';
const DOCUMENTSDB = 'documentsdb';
const VECTORSDB = 'vectorsdb';
const MIN_VECTOR_DIMENSION = 1;
const MAX_VECTOR_DIMENSION = 16000;
@@ -10,7 +10,7 @@ use Utopia\Database\Operator;
class Action extends AppwriteAction
{
private string $context = DATABASE_TYPE_LEGACY;
private string $context = 'legacy';
public function getDatabaseType(): string
{
@@ -20,13 +20,7 @@ class Action extends AppwriteAction
public function setHttpPath(string $path): AppwriteAction
{
if (\str_contains($path, '/tablesdb')) {
$this->context = DATABASE_TYPE_TABLESDB;
}
if (\str_contains($path, '/documentsdb')) {
$this->context = DATABASE_TYPE_DOCUMENTSDB;
}
if (\str_contains($path, '/vectorsdb')) {
$this->context = DATABASE_TYPE_VECTORSDB;
$this->context = 'tablesdb';
}
return parent::setHttpPath($path);
}
@@ -15,8 +15,6 @@ abstract class Action extends UtopiaAction
*/
private ?string $context = COLLECTIONS;
private ?string $databaseType = LEGACY;
/**
* Get the response model used in the SDK and HTTP responses.
*/
@@ -26,9 +24,6 @@ abstract class Action extends UtopiaAction
{
if (\str_contains($path, '/tablesdb')) {
$this->context = TABLES;
$this->databaseType = TABLESDB;
} elseif (\str_contains($path, '/vectorsdb')) {
$this->databaseType = VECTORSDB;
}
return parent::setHttpPath($path);
}
@@ -41,14 +36,6 @@ abstract class Action extends UtopiaAction
return $this->context;
}
/**
* Get the current API database type.
*/
protected function getDatabaseType(): string
{
return $this->databaseType;
}
/**
* Get the key used in event parameters (e.g., 'collectionId' or 'tableId').
*/
@@ -84,13 +84,12 @@ class Create extends Action
->param('indexes', [], new ArrayList(new JSON(), APP_LIMIT_ARRAY_PARAMS_SIZE), 'Array of index definitions to create. Each index should contain: key (string), type (string: key, fulltext, unique, spatial), attributes (array of attribute keys), orders (array of ASC/DESC, optional), and lengths (array of integers, optional).', true)
->inject('response')
->inject('dbForProject')
->inject('getDatabasesDB')
->inject('queueForEvents')
->inject('authorization')
->callback($this->action(...));
}
public function action(string $databaseId, string $collectionId, string $name, ?array $permissions, bool $documentSecurity, bool $enabled, array $attributes, array $indexes, UtopiaResponse $response, Database $dbForProject, callable $getDatabasesDB, Event $queueForEvents, Authorization $authorization): void
public function action(string $databaseId, string $collectionId, string $name, ?array $permissions, bool $documentSecurity, bool $enabled, array $attributes, array $indexes, UtopiaResponse $response, Database $dbForProject, Event $queueForEvents, Authorization $authorization): void
{
$database = $authorization->skip(fn () => $dbForProject->getDocument('databases', $databaseId));
@@ -122,18 +121,12 @@ class Create extends Action
throw new Exception(Exception::DATABASE_NOT_FOUND, params: [$databaseId]);
}
/**
* @var Database $dbForDatabases
*/
$dbForDatabases = $getDatabasesDB($database);
$collectionKey = 'database_' . $database->getSequence() . '_collection_' . $collection->getSequence();
$databaseKey = 'database_' . $database->getSequence();
$attributesValidator = new AttributesValidator(
APP_LIMIT_ARRAY_PARAMS_SIZE,
$dbForDatabases->getAdapter()->getSupportForSpatialAttributes(),
$dbForDatabases->getAdapter()->getSupportForAttributes()
$dbForProject->getAdapter()->getSupportForSpatialAttributes()
);
if (!$attributesValidator->isValid($attributes)) {
@@ -162,7 +155,7 @@ class Create extends Action
}
// Validate indexes
$indexesValidator = new IndexesValidator($dbForDatabases->getLimitForIndexes());
$indexesValidator = new IndexesValidator($dbForProject->getLimitForIndexes());
if (!$indexesValidator->isValid($indexes)) {
$dbForProject->deleteDocument($databaseKey, $collection->getId());
throw new Exception(Exception::GENERAL_ARGUMENT_INVALID, $indexesValidator->getDescription());
@@ -185,23 +178,21 @@ class Create extends Action
$indexValidator = new IndexValidator(
$collectionAttributes,
[],
$dbForDatabases->getAdapter()->getMaxIndexLength(),
$dbForDatabases->getAdapter()->getInternalIndexesKeys(),
$dbForDatabases->getAdapter()->getSupportForIndexArray(),
$dbForDatabases->getAdapter()->getSupportForSpatialIndexNull(),
$dbForDatabases->getAdapter()->getSupportForSpatialIndexOrder(),
$dbForDatabases->getAdapter()->getSupportForVectors(),
$dbForDatabases->getAdapter()->getSupportForAttributes(),
$dbForDatabases->getAdapter()->getSupportForMultipleFulltextIndexes(),
$dbForDatabases->getAdapter()->getSupportForIdenticalIndexes(),
$dbForDatabases->getAdapter()->getSupportForObjectIndexes(),
$dbForDatabases->getAdapter()->getSupportForTrigramIndex(),
$dbForDatabases->getAdapter()->getSupportForSpatialAttributes(),
$dbForDatabases->getAdapter()->getSupportForIndex(),
$dbForDatabases->getAdapter()->getSupportForUniqueIndex(),
$dbForDatabases->getAdapter()->getSupportForFulltextIndex(),
$dbForDatabases->getAdapter()->getSupportForTTLIndexes(),
$dbForDatabases->getAdapter()->getSupportForObject(),
$dbForProject->getAdapter()->getMaxIndexLength(),
$dbForProject->getAdapter()->getInternalIndexesKeys(),
$dbForProject->getAdapter()->getSupportForIndexArray(),
$dbForProject->getAdapter()->getSupportForSpatialIndexNull(),
$dbForProject->getAdapter()->getSupportForSpatialIndexOrder(),
$dbForProject->getAdapter()->getSupportForVectors(),
$dbForProject->getAdapter()->getSupportForAttributes(),
$dbForProject->getAdapter()->getSupportForMultipleFulltextIndexes(),
$dbForProject->getAdapter()->getSupportForIdenticalIndexes(),
$dbForProject->getAdapter()->getSupportForObjectIndexes(),
$dbForProject->getAdapter()->getSupportForTrigramIndex(),
$dbForProject->getAdapter()->getSupportForSpatialAttributes(),
$dbForProject->getAdapter()->getSupportForIndex(),
$dbForProject->getAdapter()->getSupportForUniqueIndex(),
$dbForProject->getAdapter()->getSupportForFulltextIndex(),
);
foreach ($collectionIndexes as $indexDoc) {
@@ -212,7 +203,7 @@ class Create extends Action
}
try {
$dbForDatabases->createCollection(
$dbForProject->createCollection(
id: $collectionKey,
attributes: $collectionAttributes,
indexes: $collectionIndexes,
@@ -62,14 +62,13 @@ class Delete extends Action
->param('collectionId', '', fn (Database $dbForProject) => new UID($dbForProject->getAdapter()->getMaxUIDLength()), 'Collection ID.', false, ['dbForProject'])
->inject('response')
->inject('dbForProject')
->inject('getDatabasesDB')
->inject('queueForDatabase')
->inject('queueForEvents')
->inject('authorization')
->callback($this->action(...));
}
public function action(string $databaseId, string $collectionId, UtopiaResponse $response, Database $dbForProject, callable $getDatabasesDB, EventDatabase $queueForDatabase, Event $queueForEvents, Authorization $authorization): void
public function action(string $databaseId, string $collectionId, UtopiaResponse $response, Database $dbForProject, EventDatabase $queueForDatabase, Event $queueForEvents, Authorization $authorization): void
{
$database = $authorization->skip(fn () => $dbForProject->getDocument('databases', $databaseId));
if ($database->isEmpty()) {
@@ -86,8 +85,7 @@ class Delete extends Action
throw new Exception(Exception::GENERAL_SERVER_ERROR, "Failed to remove $type from DB");
}
$dbForDatabases = $getDatabasesDB($database);
$dbForDatabases->purgeCachedCollection('database_' . $database->getSequence() . '_collection_' . $collection->getSequence());
$dbForProject->purgeCachedCollection('database_' . $database->getSequence() . '_collection_' . $collection->getSequence());
$queueForDatabase
->setType(DATABASE_TYPE_DELETE_COLLECTION)
@@ -17,7 +17,6 @@ abstract class Action extends DatabasesAction
* @var string|null The current context (either 'row' or 'document')
*/
private ?string $context = DOCUMENTS;
private ?string $databaseType = DATABASE_TYPE_LEGACY;
/**
* Get the response model used in the SDK and HTTP responses.
@@ -28,10 +27,6 @@ abstract class Action extends DatabasesAction
{
if (str_contains($path, '/tablesdb/')) {
$this->context = ROWS;
} elseif (str_contains($path, '/documentsdb/')) {
$this->databaseType = DATABASE_TYPE_DOCUMENTSDB;
} elseif (str_contains($path, '/vectorsdb/')) {
$this->databaseType = DATABASE_TYPE_VECTORSDB;
}
$contextId = '$' . $this->getCollectionsEventsContext() . 'Id';
@@ -50,39 +45,6 @@ abstract class Action extends DatabasesAction
return parent::setHttpPath($path);
}
protected function getDatabasesOperationReadMetric(): string
{
if ($this->databaseType === DATABASE_TYPE_LEGACY || $this->databaseType === DATABASE_TYPE_TABLESDB) {
return METRIC_DATABASES_OPERATIONS_READS;
}
return $this->databaseType.'.'.METRIC_DATABASES_OPERATIONS_READS;
}
protected function getDatabasesIdOperationReadMetric(): string
{
if ($this->databaseType === DATABASE_TYPE_LEGACY || $this->databaseType === DATABASE_TYPE_TABLESDB) {
return METRIC_DATABASE_ID_OPERATIONS_READS;
}
return $this->databaseType.'.'.METRIC_DATABASE_ID_OPERATIONS_READS;
}
protected function getDatabasesOperationWriteMetric(): string
{
if ($this->databaseType === DATABASE_TYPE_LEGACY || $this->databaseType === DATABASE_TYPE_TABLESDB) {
return METRIC_DATABASES_OPERATIONS_WRITES;
}
return $this->databaseType.'.'.METRIC_DATABASES_OPERATIONS_WRITES;
}
protected function getDatabasesIdOperationWriteMetric(): string
{
if ($this->databaseType === DATABASE_TYPE_LEGACY || $this->databaseType === DATABASE_TYPE_TABLESDB) {
return METRIC_DATABASE_ID_OPERATIONS_WRITES;
}
return $this->databaseType.'.'.METRIC_DATABASE_ID_OPERATIONS_WRITES;
}
/**
* Get the plural of the given name.
*
@@ -3,7 +3,6 @@
namespace Appwrite\Platform\Modules\Databases\Http\Databases\Collections\Documents\Attribute;
use Appwrite\Event\Event;
use Appwrite\Event\StatsUsage;
use Appwrite\Extend\Exception;
use Appwrite\Platform\Modules\Databases\Http\Databases\Collections\Documents\Action;
use Appwrite\SDK\AuthType;
@@ -11,6 +10,7 @@ use Appwrite\SDK\ContentType;
use Appwrite\SDK\Deprecated;
use Appwrite\SDK\Method;
use Appwrite\SDK\Response as SDKResponse;
use Appwrite\Usage\Context;
use Appwrite\Utopia\Database\Documents\User;
use Appwrite\Utopia\Response as UtopiaResponse;
use InvalidArgumentException;
@@ -82,15 +82,14 @@ class Decrement extends Action
->param('transactionId', null, fn (Database $dbForProject) => new Nullable(new UID($dbForProject->getAdapter()->getMaxUIDLength())), 'Transaction ID for staging the operation.', true, ['dbForProject'])
->inject('response')
->inject('dbForProject')
->inject('getDatabasesDB')
->inject('queueForEvents')
->inject('queueForStatsUsage')
->inject('usage')
->inject('plan')
->inject('authorization')
->callback($this->action(...));
}
public function action(string $databaseId, string $collectionId, string $documentId, string $attribute, int|float $value, int|float|null $min, ?string $transactionId, UtopiaResponse $response, Database $dbForProject, callable $getDatabasesDB, Event $queueForEvents, StatsUsage $queueForStatsUsage, array $plan, Authorization $authorization): void
public function action(string $databaseId, string $collectionId, string $documentId, string $attribute, int|float $value, int|float|null $min, ?string $transactionId, UtopiaResponse $response, Database $dbForProject, Event $queueForEvents, Context $usage, array $plan, Authorization $authorization): void
{
$isAPIKey = User::isApp($authorization->getRoles());
$isPrivilegedUser = User::isPrivileged($authorization->getRoles());
@@ -171,9 +170,8 @@ class Decrement extends Action
return;
}
$dbForDatabases = $getDatabasesDB($database);
try {
$document = $dbForDatabases->decreaseDocumentAttribute(
$document = $dbForProject->decreaseDocumentAttribute(
collection: 'database_' . $database->getSequence() . '_collection_' . $collection->getSequence(),
id: $documentId,
attribute: $attribute,
@@ -202,9 +200,9 @@ class Decrement extends Action
)
);
$queueForStatsUsage
->addMetric($this->getDatabasesOperationWriteMetric(), 1)
->addMetric(str_replace('{databaseInternalId}', $database->getSequence(), $this->getDatabasesIdOperationWriteMetric()), 1);
$usage
->addMetric(METRIC_DATABASES_OPERATIONS_WRITES, 1)
->addMetric(str_replace('{databaseInternalId}', $database->getSequence(), METRIC_DATABASE_ID_OPERATIONS_WRITES), 1);
$queueForEvents
->setParam('databaseId', $databaseId)
@@ -3,7 +3,6 @@
namespace Appwrite\Platform\Modules\Databases\Http\Databases\Collections\Documents\Attribute;
use Appwrite\Event\Event;
use Appwrite\Event\StatsUsage;
use Appwrite\Extend\Exception;
use Appwrite\Platform\Modules\Databases\Http\Databases\Collections\Documents\Action;
use Appwrite\SDK\AuthType;
@@ -11,6 +10,7 @@ use Appwrite\SDK\ContentType;
use Appwrite\SDK\Deprecated;
use Appwrite\SDK\Method;
use Appwrite\SDK\Response as SDKResponse;
use Appwrite\Usage\Context;
use Appwrite\Utopia\Database\Documents\User;
use Appwrite\Utopia\Response as UtopiaResponse;
use InvalidArgumentException;
@@ -82,15 +82,14 @@ class Increment extends Action
->param('transactionId', null, fn (Database $dbForProject) => new Nullable(new UID($dbForProject->getAdapter()->getMaxUIDLength())), 'Transaction ID for staging the operation.', true, ['dbForProject'])
->inject('response')
->inject('dbForProject')
->inject('getDatabasesDB')
->inject('queueForEvents')
->inject('queueForStatsUsage')
->inject('usage')
->inject('plan')
->inject('authorization')
->callback($this->action(...));
}
public function action(string $databaseId, string $collectionId, string $documentId, string $attribute, int|float $value, int|float|null $max, ?string $transactionId, UtopiaResponse $response, Database $dbForProject, callable $getDatabasesDB, Event $queueForEvents, StatsUsage $queueForStatsUsage, array $plan, Authorization $authorization): void
public function action(string $databaseId, string $collectionId, string $documentId, string $attribute, int|float $value, int|float|null $max, ?string $transactionId, UtopiaResponse $response, Database $dbForProject, Event $queueForEvents, Context $usage, array $plan, Authorization $authorization): void
{
$isAPIKey = User::isApp($authorization->getRoles());
$isPrivilegedUser = User::isPrivileged($authorization->getRoles());
@@ -171,9 +170,8 @@ class Increment extends Action
return;
}
$dbForDatabases = $getDatabasesDB($database);
try {
$document = $dbForDatabases->increaseDocumentAttribute(
$document = $dbForProject->increaseDocumentAttribute(
collection: 'database_' . $database->getSequence() . '_collection_' . $collection->getSequence(),
id: $documentId,
attribute: $attribute,
@@ -202,9 +200,9 @@ class Increment extends Action
)
);
$queueForStatsUsage
->addMetric($this->getDatabasesOperationWriteMetric(), 1)
->addMetric(str_replace('{databaseInternalId}', $database->getSequence(), $this->getDatabasesIdOperationWriteMetric()), 1);
$usage
->addMetric(METRIC_DATABASES_OPERATIONS_WRITES, 1)
->addMetric(str_replace('{databaseInternalId}', $database->getSequence(), METRIC_DATABASE_ID_OPERATIONS_WRITES), 1);
$queueForEvents
->setParam('databaseId', $databaseId)
@@ -3,7 +3,6 @@
namespace Appwrite\Platform\Modules\Databases\Http\Databases\Collections\Documents\Bulk;
use Appwrite\Event\Event;
use Appwrite\Event\StatsUsage;
use Appwrite\Extend\Exception;
use Appwrite\Functions\EventProcessor;
use Appwrite\Platform\Modules\Databases\Http\Databases\Collections\Documents\Action;
@@ -12,6 +11,7 @@ use Appwrite\SDK\ContentType;
use Appwrite\SDK\Deprecated;
use Appwrite\SDK\Method;
use Appwrite\SDK\Response as SDKResponse;
use Appwrite\Usage\Context;
use Appwrite\Utopia\Response as UtopiaResponse;
use Utopia\Database\Database;
use Utopia\Database\Document;
@@ -76,8 +76,7 @@ class Delete extends Action
->param('transactionId', null, fn (Database $dbForProject) => new Nullable(new UID($dbForProject->getAdapter()->getMaxUIDLength())), 'Transaction ID for staging the operation.', true, ['dbForProject'])
->inject('response')
->inject('dbForProject')
->inject('getDatabasesDB')
->inject('queueForStatsUsage')
->inject('usage')
->inject('queueForEvents')
->inject('queueForRealtime')
->inject('queueForFunctions')
@@ -87,7 +86,7 @@ class Delete extends Action
->callback($this->action(...));
}
public function action(string $databaseId, string $collectionId, array $queries, ?string $transactionId, UtopiaResponse $response, Database $dbForProject, callable $getDatabasesDB, StatsUsage $queueForStatsUsage, Event $queueForEvents, Event $queueForRealtime, Event $queueForFunctions, Event $queueForWebhooks, array $plan, EventProcessor $eventProcessor): void
public function action(string $databaseId, string $collectionId, array $queries, ?string $transactionId, UtopiaResponse $response, Database $dbForProject, Context $usage, Event $queueForEvents, Event $queueForRealtime, Event $queueForFunctions, Event $queueForWebhooks, array $plan, EventProcessor $eventProcessor): void
{
$database = $dbForProject->getDocument('databases', $databaseId);
if ($database->isEmpty()) {
@@ -164,11 +163,10 @@ class Delete extends Action
return;
}
$dbForDatabases = $getDatabasesDB($database);
$documents = [];
try {
$modified = $dbForDatabases->deleteDocuments(
$modified = $dbForProject->deleteDocuments(
'database_' . $database->getSequence() . '_collection_' . $collection->getSequence(),
$queries,
onNext: function (Document $document) use ($plan, &$documents) {
@@ -187,16 +185,16 @@ class Delete extends Action
foreach ($documents as $document) {
$document->setAttribute('$databaseId', $database->getId());
$document->setAttribute('$'.$this->getCollectionsEventsContext().'Id', $collection->getId());
$document->setAttribute('$' . $this->getCollectionsEventsContext() . 'Id', $collection->getId());
}
$queueForStatsUsage
->addMetric($this->getDatabasesOperationWriteMetric(), \max(1, $modified))
->addMetric(str_replace('{databaseInternalId}', $database->getSequence(), $this->getDatabasesIdOperationWriteMetric()), \max(1, $modified));
$usage
->addMetric(METRIC_DATABASES_OPERATIONS_WRITES, \max(1, $modified))
->addMetric(str_replace('{databaseInternalId}', $database->getSequence(), METRIC_DATABASE_ID_OPERATIONS_WRITES), \max(1, $modified));
$response->dynamic(new Document([
'total' => $modified,
$this->getSDKGroup() => $documents
$this->getSDKGroup() => $documents,
]), $this->getResponseModel());
$this->triggerBulk(
@@ -3,7 +3,6 @@
namespace Appwrite\Platform\Modules\Databases\Http\Databases\Collections\Documents\Bulk;
use Appwrite\Event\Event;
use Appwrite\Event\StatsUsage;
use Appwrite\Extend\Exception;
use Appwrite\Functions\EventProcessor;
use Appwrite\Platform\Modules\Databases\Http\Databases\Collections\Documents\Action;
@@ -12,6 +11,7 @@ use Appwrite\SDK\ContentType;
use Appwrite\SDK\Deprecated;
use Appwrite\SDK\Method;
use Appwrite\SDK\Response as SDKResponse;
use Appwrite\Usage\Context;
use Appwrite\Utopia\Response as UtopiaResponse;
use Utopia\Database\Database;
use Utopia\Database\Document;
@@ -80,8 +80,7 @@ class Update extends Action
->param('transactionId', null, fn (Database $dbForProject) => new Nullable(new UID($dbForProject->getAdapter()->getMaxUIDLength())), 'Transaction ID for staging the operation.', true, ['dbForProject'])
->inject('response')
->inject('dbForProject')
->inject('getDatabasesDB')
->inject('queueForStatsUsage')
->inject('usage')
->inject('queueForEvents')
->inject('queueForRealtime')
->inject('queueForFunctions')
@@ -91,7 +90,7 @@ class Update extends Action
->callback($this->action(...));
}
public function action(string $databaseId, string $collectionId, string|array $data, array $queries, ?string $transactionId, UtopiaResponse $response, Database $dbForProject, callable $getDatabasesDB, StatsUsage $queueForStatsUsage, Event $queueForEvents, Event $queueForRealtime, Event $queueForFunctions, Event $queueForWebhooks, array $plan, EventProcessor $eventProcessor): void
public function action(string $databaseId, string $collectionId, string|array $data, array $queries, ?string $transactionId, UtopiaResponse $response, Database $dbForProject, Context $usage, Event $queueForEvents, Event $queueForRealtime, Event $queueForFunctions, Event $queueForWebhooks, array $plan, EventProcessor $eventProcessor): void
{
$data = \is_string($data)
? \json_decode($data, true)
@@ -190,12 +189,11 @@ class Update extends Action
return;
}
$dbForDatabases = $getDatabasesDB($database);
$documents = [];
try {
$modified = $dbForDatabases->withPreserveDates(function () use ($plan, &$documents, $dbForDatabases, $database, $collection, $data, $queries) {
return $dbForDatabases->updateDocuments(
$modified = $dbForProject->withPreserveDates(function () use ($plan, &$documents, $dbForProject, $database, $collection, $data, $queries) {
return $dbForProject->updateDocuments(
'database_' . $database->getSequence() . '_collection_' . $collection->getSequence(),
new Document($data),
$queries,
@@ -218,12 +216,12 @@ class Update extends Action
foreach ($documents as $document) {
$document->setAttribute('$databaseId', $database->getId());
$document->setAttribute('$'.$this->getCollectionsEventsContext().'Id', $collection->getId());
$document->setAttribute('$' . $this->getCollectionsEventsContext() . 'Id', $collection->getId());
}
$queueForStatsUsage
->addMetric($this->getDatabasesOperationWriteMetric(), \max(1, $modified))
->addMetric(str_replace('{databaseInternalId}', $database->getSequence(), $this->getDatabasesIdOperationWriteMetric()), \max(1, $modified));
$usage
->addMetric(METRIC_DATABASES_OPERATIONS_WRITES, \max(1, $modified))
->addMetric(str_replace('{databaseInternalId}', $database->getSequence(), METRIC_DATABASE_ID_OPERATIONS_WRITES), \max(1, $modified));
$response->dynamic(new Document([
'total' => $modified,
@@ -3,7 +3,6 @@
namespace Appwrite\Platform\Modules\Databases\Http\Databases\Collections\Documents\Bulk;
use Appwrite\Event\Event;
use Appwrite\Event\StatsUsage;
use Appwrite\Extend\Exception;
use Appwrite\Functions\EventProcessor;
use Appwrite\Platform\Modules\Databases\Http\Databases\Collections\Documents\Action;
@@ -12,6 +11,7 @@ use Appwrite\SDK\ContentType;
use Appwrite\SDK\Deprecated;
use Appwrite\SDK\Method;
use Appwrite\SDK\Response as SDKResponse;
use Appwrite\Usage\Context;
use Appwrite\Utopia\Response as UtopiaResponse;
use Utopia\Database\Database;
use Utopia\Database\Document;
@@ -58,7 +58,7 @@ class Upsert extends Action
group: $this->getSDKGroup(),
name: self::getName(),
description: '/docs/references/databases/upsert-documents.md',
auth: [AuthType::ADMIN, AuthType::SESSION, AuthType::KEY, AuthType::JWT],
auth: [AuthType::ADMIN, AuthType::KEY],
responses: [
new SDKResponse(
code: SwooleResponse::STATUS_CODE_CREATED,
@@ -78,8 +78,7 @@ class Upsert extends Action
->param('transactionId', null, fn (Database $dbForProject) => new Nullable(new UID($dbForProject->getAdapter()->getMaxUIDLength())), 'Transaction ID for staging the operation.', true, ['dbForProject'])
->inject('response')
->inject('dbForProject')
->inject('getDatabasesDB')
->inject('queueForStatsUsage')
->inject('usage')
->inject('queueForEvents')
->inject('queueForRealtime')
->inject('queueForFunctions')
@@ -89,7 +88,7 @@ class Upsert extends Action
->callback($this->action(...));
}
public function action(string $databaseId, string $collectionId, array $documents, ?string $transactionId, UtopiaResponse $response, Database $dbForProject, callable $getDatabasesDB, StatsUsage $queueForStatsUsage, Event $queueForEvents, Event $queueForRealtime, Event $queueForFunctions, Event $queueForWebhooks, array $plan, EventProcessor $eventProcessor): void
public function action(string $databaseId, string $collectionId, array $documents, ?string $transactionId, UtopiaResponse $response, Database $dbForProject, Context $usage, Event $queueForEvents, Event $queueForRealtime, Event $queueForFunctions, Event $queueForWebhooks, array $plan, EventProcessor $eventProcessor): void
{
$database = $dbForProject->getDocument('databases', $databaseId);
if ($database->isEmpty()) {
@@ -107,7 +106,7 @@ class Upsert extends Action
);
if ($hasRelationships) {
throw new Exception(Exception::GENERAL_BAD_REQUEST, 'Bulk upsert is not supported for ' . $this->getSDKNamespace() . ' with relationship attributes');
throw new Exception(Exception::GENERAL_BAD_REQUEST, 'Bulk upsert is not supported for ' . $this->getSDKNamespace() . ' with relationship attributes');
}
foreach ($documents as $key => $document) {
@@ -166,12 +165,11 @@ class Upsert extends Action
return;
}
$dbForDatabases = $getDatabasesDB($database);
$upserted = [];
try {
$modified = $dbForDatabases->withPreserveDates(function () use ($dbForDatabases, $database, $collection, $documents, $plan, &$upserted) {
return $dbForDatabases->upsertDocuments(
$modified = $dbForProject->withPreserveDates(function () use ($dbForProject, $database, $collection, $documents, $plan, &$upserted) {
return $dbForProject->upsertDocuments(
'database_' . $database->getSequence() . '_collection_' . $collection->getSequence(),
$documents,
onNext: function (Document $document) use ($plan, &$upserted) {
@@ -193,12 +191,12 @@ class Upsert extends Action
foreach ($upserted as $document) {
$document->setAttribute('$databaseId', $database->getId());
$document->setAttribute('$'.$this->getCollectionsEventsContext().'Id', $collection->getId());
$document->setAttribute('$' . $this->getCollectionsEventsContext() . 'Id', $collection->getId());
}
$queueForStatsUsage
->addMetric($this->getDatabasesOperationWriteMetric(), \max(1, $modified))
->addMetric(str_replace('{databaseInternalId}', $database->getSequence(), $this->getDatabasesIdOperationWriteMetric()), \max(1, $modified));
$usage
->addMetric(METRIC_DATABASES_OPERATIONS_WRITES, \max(1, $modified))
->addMetric(str_replace('{databaseInternalId}', $database->getSequence(), METRIC_DATABASE_ID_OPERATIONS_WRITES), \max(1, $modified));
$response->dynamic(new Document([
'total' => $modified,
@@ -3,7 +3,6 @@
namespace Appwrite\Platform\Modules\Databases\Http\Databases\Collections\Documents;
use Appwrite\Event\Event;
use Appwrite\Event\StatsUsage;
use Appwrite\Extend\Exception;
use Appwrite\Functions\EventProcessor;
use Appwrite\SDK\AuthType;
@@ -12,6 +11,7 @@ use Appwrite\SDK\Deprecated;
use Appwrite\SDK\Method;
use Appwrite\SDK\Parameter;
use Appwrite\SDK\Response as SDKResponse;
use Appwrite\Usage\Context;
use Appwrite\Utopia\Database\Documents\User;
use Appwrite\Utopia\Database\Validator\CustomId;
use Appwrite\Utopia\Response as UtopiaResponse;
@@ -85,7 +85,7 @@ class Create extends Action
new Parameter('documentId', optional: false),
new Parameter('data', optional: false),
new Parameter('permissions', optional: true),
new Parameter('transactionId', optional: true)
new Parameter('transactionId', optional: true),
],
deprecated: new Deprecated(
since: '1.8.0',
@@ -110,7 +110,7 @@ class Create extends Action
new Parameter('databaseId', optional: false),
new Parameter('collectionId', optional: false),
new Parameter('documents', optional: false),
new Parameter('transactionId', optional: true)
new Parameter('transactionId', optional: true),
],
deprecated: new Deprecated(
since: '1.8.0',
@@ -127,10 +127,9 @@ class Create extends Action
->param('transactionId', null, fn (Database $dbForProject) => new Nullable(new UID($dbForProject->getAdapter()->getMaxUIDLength())), 'Transaction ID for staging the operation.', true, ['dbForProject'])
->inject('response')
->inject('dbForProject')
->inject('getDatabasesDB')
->inject('user')
->inject('queueForEvents')
->inject('queueForStatsUsage')
->inject('usage')
->inject('queueForRealtime')
->inject('queueForFunctions')
->inject('queueForWebhooks')
@@ -139,7 +138,7 @@ class Create extends Action
->inject('eventProcessor')
->callback($this->action(...));
}
public function action(string $databaseId, string $documentId, string $collectionId, string|array $data, ?array $permissions, ?array $documents, ?string $transactionId, UtopiaResponse $response, Database $dbForProject, callable $getDatabasesDB, Document $user, Event $queueForEvents, StatsUsage $queueForStatsUsage, Event $queueForRealtime, Event $queueForFunctions, Event $queueForWebhooks, array $plan, Authorization $authorization, EventProcessor $eventProcessor): void
public function action(string $databaseId, string $documentId, string $collectionId, string|array $data, ?array $permissions, ?array $documents, ?string $transactionId, UtopiaResponse $response, Database $dbForProject, Document $user, Event $queueForEvents, Context $usage, Event $queueForRealtime, Event $queueForFunctions, Event $queueForWebhooks, array $plan, Authorization $authorization, EventProcessor $eventProcessor): void
{
$data = \is_string($data)
? \json_decode($data, true)
@@ -206,7 +205,7 @@ class Create extends Action
);
if ($isBulk && $hasRelationships) {
throw new Exception(Exception::GENERAL_BAD_REQUEST, 'Bulk create is not supported for ' . $this->getSDKNamespace() .' with relationship ' . $this->getStructureContext());
throw new Exception(Exception::GENERAL_BAD_REQUEST, 'Bulk create is not supported for ' . $this->getSDKNamespace() . ' with relationship ' . $this->getStructureContext());
}
$setPermissions = function (Document $document, ?array $permissions) use ($user, $isAPIKey, $isPrivilegedUser, $isBulk, $dbForProject, $authorization) {
@@ -448,12 +447,11 @@ class Create extends Action
return;
}
$dbForDatabases = $getDatabasesDB($database);
try {
$created = [];
$dbForDatabases->withPreserveDates(
function () use (&$created, $dbForDatabases, $database, $collection, $documents) {
$dbForDatabases->createDocuments(
$dbForProject->withPreserveDates(
function () use (&$created, $dbForProject, $database, $collection, $documents) {
$dbForProject->createDocuments(
'database_' . $database->getSequence() . '_collection_' . $collection->getSequence(),
$documents,
onNext: function ($doc) use (&$created) {
@@ -491,16 +489,16 @@ class Create extends Action
);
}
$queueForStatsUsage
->addMetric($this->getDatabasesOperationWriteMetric(), \max(1, $operations))
->addMetric(str_replace('{databaseInternalId}', $database->getSequence(), $this->getDatabasesIdOperationWriteMetric()), \max(1, $operations)); // per collection
$usage
->addMetric(METRIC_DATABASES_OPERATIONS_WRITES, \max(1, $operations))
->addMetric(str_replace('{databaseInternalId}', $database->getSequence(), METRIC_DATABASE_ID_OPERATIONS_WRITES), \max(1, $operations)); // per collection
$response->setStatusCode(SwooleResponse::STATUS_CODE_CREATED);
if ($isBulk) {
$response->dynamic(new Document([
'total' => count($created),
$this->getSDKGroup() => $created
$this->getSdkGroup() => $created
]), $this->getBulkResponseModel());
$this->triggerBulk(
@@ -4,13 +4,13 @@ namespace Appwrite\Platform\Modules\Databases\Http\Databases\Collections\Documen
use Appwrite\Databases\TransactionState;
use Appwrite\Event\Event;
use Appwrite\Event\StatsUsage;
use Appwrite\Extend\Exception;
use Appwrite\SDK\AuthType;
use Appwrite\SDK\ContentType;
use Appwrite\SDK\Deprecated;
use Appwrite\SDK\Method;
use Appwrite\SDK\Response as SDKResponse;
use Appwrite\Usage\Context;
use Appwrite\Utopia\Database\Documents\User;
use Appwrite\Utopia\Response as UtopiaResponse;
use Utopia\Database\Database;
@@ -79,9 +79,8 @@ class Delete extends Action
->inject('requestTimestamp')
->inject('response')
->inject('dbForProject')
->inject('getDatabasesDB')
->inject('queueForEvents')
->inject('queueForStatsUsage')
->inject('usage')
->inject('transactionState')
->inject('plan')
->inject('authorization')
@@ -96,9 +95,8 @@ class Delete extends Action
?\DateTime $requestTimestamp,
UtopiaResponse $response,
Database $dbForProject,
callable $getDatabasesDB,
Event $queueForEvents,
StatsUsage $queueForStatsUsage,
Context $usage,
TransactionState $transactionState,
array $plan,
Authorization $authorization
@@ -118,15 +116,14 @@ class Delete extends Action
throw new Exception($this->getParentNotFoundException(), params: [$collectionId]);
}
$dbForDatabases = $getDatabasesDB($database);
// Read permission should not be required for delete
$collectionTableId = 'database_' . $database->getSequence() . '_collection_' . $collection->getSequence();
if ($transactionId !== null) {
// Use transaction-aware document retrieval to see changes from same transaction
$document = $transactionState->getDocument($database, $collectionTableId, $documentId, $transactionId);
$document = $transactionState->getDocument($collectionTableId, $documentId, $transactionId);
} else {
$document = $authorization->skip(fn () => $dbForDatabases->getDocument($collectionTableId, $documentId));
$document = $authorization->skip(fn () => $dbForProject->getDocument($collectionTableId, $documentId));
}
if ($document->isEmpty()) {
@@ -190,8 +187,8 @@ class Delete extends Action
}
try {
$dbForDatabases->withRequestTimestamp($requestTimestamp, function () use ($dbForDatabases, $database, $collection, $documentId) {
$dbForDatabases->deleteDocument(
$dbForProject->withRequestTimestamp($requestTimestamp, function () use ($dbForProject, $database, $collection, $documentId) {
$dbForProject->deleteDocument(
'database_' . $database->getSequence() . '_collection_' . $collection->getSequence(),
$documentId
);
@@ -213,9 +210,9 @@ class Delete extends Action
authorization: $authorization
);
$queueForStatsUsage
->addMetric($this->getDatabasesOperationWriteMetric(), 1)
->addMetric(str_replace('{databaseInternalId}', $database->getSequence(), $this->getDatabasesIdOperationWriteMetric()), 1); // per collection
$usage
->addMetric(METRIC_DATABASES_OPERATIONS_WRITES, 1)
->addMetric(str_replace('{databaseInternalId}', $database->getSequence(), METRIC_DATABASE_ID_OPERATIONS_WRITES), 1); // per collection
$response->addHeader('X-Debug-Operations', 1);
@@ -3,13 +3,13 @@
namespace Appwrite\Platform\Modules\Databases\Http\Databases\Collections\Documents;
use Appwrite\Databases\TransactionState;
use Appwrite\Event\StatsUsage;
use Appwrite\Extend\Exception;
use Appwrite\SDK\AuthType;
use Appwrite\SDK\ContentType;
use Appwrite\SDK\Deprecated;
use Appwrite\SDK\Method;
use Appwrite\SDK\Response as SDKResponse;
use Appwrite\Usage\Context;
use Appwrite\Utopia\Database\Documents\User;
use Appwrite\Utopia\Response as UtopiaResponse;
use Utopia\Database\Database;
@@ -68,14 +68,13 @@ class Get extends Action
->param('transactionId', null, fn (Database $dbForProject) => new Nullable(new UID($dbForProject->getAdapter()->getMaxUIDLength())), 'Transaction ID to read uncommitted changes within the transaction.', true, ['dbForProject'])
->inject('response')
->inject('dbForProject')
->inject('getDatabasesDB')
->inject('queueForStatsUsage')
->inject('usage')
->inject('transactionState')
->inject('authorization')
->callback($this->action(...));
}
public function action(string $databaseId, string $collectionId, string $documentId, array $queries, ?string $transactionId, UtopiaResponse $response, Database $dbForProject, callable $getDatabasesDB, StatsUsage $queueForStatsUsage, TransactionState $transactionState, Authorization $authorization): void
public function action(string $databaseId, string $collectionId, string $documentId, array $queries, ?string $transactionId, UtopiaResponse $response, Database $dbForProject, Context $usage, TransactionState $transactionState, Authorization $authorization): void
{
$isAPIKey = User::isApp($authorization->getRoles());
$isPrivilegedUser = User::isPrivileged($authorization->getRoles());
@@ -87,7 +86,6 @@ class Get extends Action
$collection = $authorization->skip(fn () => $dbForProject->getDocument('database_' . $database->getSequence(), $collectionId));
$dbForDatabases = $getDatabasesDB($database);
if ($collection->isEmpty() || (!$collection->getAttribute('enabled', false) && !$isAPIKey && !$isPrivilegedUser)) {
throw new Exception($this->getParentNotFoundException(), params: [$collectionId]);
}
@@ -101,17 +99,16 @@ class Get extends Action
try {
$selects = Query::groupByType($queries)['selections'] ?? [];
$collectionTableId = 'database_' . $database->getSequence() . '_collection_' . $collection->getSequence();
$collectionTableId = 'database_' . $database->getSequence() . '_collection_' . $collection->getSequence();
// Use transaction-aware document retrieval if transactionId is provided
if ($transactionId !== null) {
$document = $transactionState->getDocument($database, $collectionTableId, $documentId, $transactionId, $queries);
$document = $transactionState->getDocument($collectionTableId, $documentId, $transactionId, $queries);
} elseif (! empty($selects)) {
// has selects, allow relationship on documents!
$document = $dbForDatabases->getDocument($collectionTableId, $documentId, $queries);
$document = $dbForProject->getDocument($collectionTableId, $documentId, $queries);
} else {
// has no selects, disable relationship looping on documents!
$document = $dbForDatabases->skipRelationships(fn () => $dbForDatabases->getDocument($collectionTableId, $documentId, $queries));
$document = $dbForProject->skipRelationships(fn () => $dbForProject->getDocument($collectionTableId, $documentId, $queries));
}
} catch (QueryException $e) {
throw new Exception(Exception::GENERAL_QUERY_INVALID, $e->getMessage());
@@ -133,9 +130,9 @@ class Get extends Action
operations: $operations
);
$queueForStatsUsage
->addMetric($this->getDatabasesOperationReadMetric(), max($operations, 1))
->addMetric(str_replace('{databaseInternalId}', $database->getSequence(), $this->getDatabasesIdOperationReadMetric()), $operations);
$usage
->addMetric(METRIC_DATABASES_OPERATIONS_READS, max($operations, 1))
->addMetric(str_replace('{databaseInternalId}', $database->getSequence(), METRIC_DATABASE_ID_OPERATIONS_READS), $operations);
$response->addHeader('X-Debug-Operations', $operations);

Some files were not shown because too many files have changed in this diff Show More