Merge branch '1.9.x' into feat/sites-build-spec-default

This commit is contained in:
Torsten Dittmann authored and GitHub committed 2026-06-12 16:45:18 +04:00
commit 0f6b36e508
424 files changed
+12957 -15954

No files matched your search

+74
View File
@@ -0,0 +1,74 @@
---
name: appwrite-testing
description: Choose, write, and review tests in the Appwrite server repository using a pragmatic test pyramid. Use when adding or changing PHPUnit tests, deciding between unit and e2e coverage, moving duplicated high-level coverage lower, testing API endpoints, workers, queues, permissions, auth/scopes, persistence, serialization, retry logic, validators, security regressions, or Swoole-sensitive behavior.
---
# Appwrite Testing
## Core Rule
Keep the test suite pyramid-shaped: many fast unit tests, fewer service/API e2e tests, and very few broad workflow tests. Push each assertion to the lowest level that still proves the behavior.
Prefer fast feedback over formal labels. A narrow integration-style unit test with deterministic fakes is better than a slow e2e test when it proves the same behavior.
## Choose The Test Level
Use `tests/unit/...` for:
- Pure validators, helpers, mappers, parsers, filters, retry classifiers, event builders, URL/security checks, and branching logic.
- Worker building blocks that can be called with fake adapters, in-memory collaborators, or scripted test doubles.
- Regression cases where inputs and observable outputs can be asserted without HTTP, Docker services, queues, Swoole coroutines, or real network calls.
- Boundary serialization/deserialization when the external side can be represented with a deterministic fake response or document.
Use `tests/e2e/Services/{Service}` for:
- HTTP route behavior, status codes, headers, cookies, response shape, SDK-visible contracts, and request validation through the real API path.
- Auth, scope, permission, project mode, platform, and side-specific behavior that depends on Appwrite's request lifecycle.
- Persistence, queue-visible behavior, worker integration, database adapter behavior, and DI/wiring that unit tests cannot prove.
- A small number of high-value service workflows that represent user-visible behavior across multiple Appwrite subsystems.
Avoid broad e2e coverage for every edge case. Cover edge cases in unit tests and keep e2e focused on the integration contract that lower layers cannot exercise.
## Appwrite Patterns
Structure tests as Arrange, Act, Assert. Test observable behavior, not the order of private calls or internal implementation details.
Write unit tests with `PHPUnit\Framework\TestCase` under the matching namespace in `tests/unit`. Use data providers for matrices. Prefer named fake classes over anonymous mocks when PHPStan clarity matters.
Write service e2e tests with `Tests\E2E\Client` and the existing scope traits such as `Scope`, `ProjectCustom`, `SideClient`, `SideServer`, or `ProjectConsole`. Reuse local service base traits when they exist.
Use deterministic test doubles for network, queue, mail, storage, database, and third-party boundaries whenever the point of the test is local behavior. Never call production third-party services from automated tests.
Generate unique IDs, emails, and names for e2e data to survive parallel runs. Cache setup data only when the test does not require precise counts or fresh state.
Keep assertions precise: status code, body fields, error type/message, permission outcome, retry count, adapter call count, or persisted value. Avoid asserting incidental full documents when a sparse behavior assertion is enough.
## Duplication Policy
If a higher-level test finds a bug and no lower-level test fails, add a lower-level regression test where practical.
Do not repeat all conditional branches at e2e level once unit tests cover them. The e2e test should prove routing, auth, serialization, persistence, or wiring only.
Delete or avoid high-level tests that no longer add confidence beyond lower-level coverage. Keep tests readable even if that means some local duplication in setup.
## Swoole And Workers
Be careful with Swoole coroutines in unit tests. Do not run coroutine integrations inside the shared unit process unless existing nearby tests prove the pattern is safe.
For workers, unit-test pure pieces such as batching, classification, cursor construction, retry/backoff decisions, and adapter interaction with fakes. Cover coroutine scheduling, queues, and full worker lifecycle through e2e or existing integration patterns.
Avoid sleeps and timing-sensitive assertions. If timing is unavoidable, isolate it, make it generous, and prefer polling helpers already used in the suite.
## Commands
Run focused checks first:
```bash
docker compose exec appwrite test tests/unit/
docker compose exec appwrite test tests/e2e/Services/[Service]
docker compose exec appwrite test tests/e2e/Services/[Service] --filter=[Method]
composer lint <file>
composer analyze
```
Use the narrowest command that validates the change, then broaden only when the touched behavior crosses module, API, worker, or shared-helper boundaries.
@@ -0,0 +1,4 @@
interface:
display_name: "Appwrite Testing"
short_description: "Choose and write Appwrite tests using a pragmatic test pyramid."
default_prompt: "Use the Appwrite testing skill to choose the right test level and add focused PHPUnit coverage for this change."
+3
View File
@@ -150,3 +150,6 @@ _APP_POOL_ADAPTER=stack
_APP_WORKER_SCREENSHOTS_ROUTER=http://appwrite
_TESTS_OAUTH2_GITHUB_CLIENT_ID=
_TESTS_OAUTH2_GITHUB_CLIENT_SECRET=
_APP_WORKERS_NUM=1
_APP_WORKER_MAX_COROUTINES=1
+91 -80
View File
@@ -139,6 +139,26 @@ jobs:
- name: Run Linter
run: composer lint
refactor:
name: Checks / Refactor
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Setup PHP
uses: shivammathur/setup-php@accd6127cb78bee3e8082180cb391013d204ef9f # 2.37.0
with:
php-version: '8.3'
tools: composer:v2
coverage: none
- name: Install dependencies
run: composer install --prefer-dist --no-progress --ignore-platform-reqs
- name: Run Rector
run: composer refactor:check
analyze:
name: Checks / Analyze
runs-on: ubuntu-latest
@@ -209,44 +229,74 @@ jobs:
outputs:
databases: ${{ steps.generate.outputs.databases }}
modes: ${{ steps.generate.outputs.modes }}
services: ${{ steps.generate.outputs.services }}
steps:
- name: Generate matrix
id: generate
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
with:
script: |
const allDatabases = ['MariaDB', 'PostgreSQL', 'MongoDB'];
const allModes = ['dedicated', 'shared'];
const runner4 = `runs-on=${context.runId}/runner=4cpu-linux-x64/volume=120g/spot=false/extras=otel`;
const runner8 = `runs-on=${context.runId}/runner=8cpu-linux-x64/volume=120g/spot=false/extras=otel`;
const defaultDatabases = ['MongoDB'];
const defaultModes = ['dedicated'];
const services = [
{ name: 'Account', runner: runner4, functional: true, processes: null, timeout: 20 },
{ name: 'Avatars', runner: runner4, functional: true, processes: null, timeout: 20 },
{ name: 'Console', runner: runner4, functional: true, processes: null, timeout: 20 },
{ name: 'Databases', runner: runner8, functional: false, processes: 3, timeout: 30 },
{ name: 'TablesDB', runner: runner8, functional: false, processes: 3, timeout: 30 },
{ name: 'Functions', runner: runner4, functional: false, processes: null, timeout: 20 },
{ name: 'FunctionsSchedule', runner: runner4, functional: true, processes: null, timeout: 20 },
{ name: 'GraphQL', runner: runner4, functional: false, processes: null, timeout: 20 },
{ name: 'Health', runner: runner4, functional: true, processes: null, timeout: 20 },
{ name: 'Advisor', runner: runner4, functional: true, processes: null, timeout: 20 },
{ name: 'Locale', runner: runner4, functional: true, processes: null, timeout: 20 },
{ name: 'Projects', runner: runner4, functional: true, processes: null, timeout: 20 },
{ name: 'Realtime', runner: runner4, functional: false, processes: null, timeout: 20 },
{ name: 'Sites', runner: runner4, functional: true, processes: null, timeout: 20 },
{ name: 'Proxy', runner: runner4, functional: true, processes: null, timeout: 20 },
{ name: 'Storage', runner: runner4, functional: true, processes: null, timeout: 20 },
{ name: 'Tokens', runner: runner4, functional: true, processes: null, timeout: 20 },
{ name: 'Teams', runner: runner4, functional: true, processes: null, timeout: 20 },
{ name: 'Users', runner: runner4, functional: true, processes: null, timeout: 20 },
{ name: 'ProjectWebhooks', runner: runner4, functional: false, processes: null, timeout: 20 },
{ name: 'Webhooks', runner: runner4, functional: true, processes: null, timeout: 20 },
{ name: 'VCS', runner: runner4, functional: true, processes: null, timeout: 20 },
{ name: 'Messaging', runner: runner4, functional: true, processes: null, timeout: 20 },
{ name: 'Migrations', runner: runner4, functional: true, processes: 1, timeout: 20 },
{ name: 'Project', runner: runner4, functional: true, processes: null, timeout: 20 },
{ name: 'Presences', runner: runner4, functional: true, processes: null, timeout: 20 },
];
core.setOutput('services', JSON.stringify(services));
const pr = context.payload.pull_request;
if (!pr) {
core.setOutput('databases', JSON.stringify(allDatabases));
core.setOutput('modes', JSON.stringify(allModes));
const isDatabaseVersionChanged = async () => {
const getContent = (ref) => github.rest.repos.getContent({
owner: context.repo.owner,
repo: context.repo.repo,
path: 'composer.lock',
ref,
});
const getDbVersion = (lock) => lock.packages?.find(p => p.name === 'utopia-php/database')?.version;
const [{ data: base }, { data: head }] = await Promise.all([
getContent(pr.base.sha),
getContent(pr.head.sha),
]);
const decode = (content) => JSON.parse(Buffer.from(content, 'base64').toString());
return getDbVersion(decode(base.content)) !== getDbVersion(decode(head.content));
};
if (!pr || await isDatabaseVersionChanged()) {
core.setOutput('databases', JSON.stringify(['MariaDB', 'PostgreSQL', 'MongoDB']));
core.setOutput('modes', JSON.stringify(['dedicated', 'shared']));
return;
}
const getContent = (ref) => github.rest.repos.getContent({
owner: context.repo.owner,
repo: context.repo.repo,
path: 'composer.lock',
ref,
});
const getDbVersion = (lock) => lock.packages?.find(p => p.name === 'utopia-php/database')?.version;
const [{ data: base }, { data: head }] = await Promise.all([
getContent(pr.base.sha),
getContent(pr.head.sha),
]);
const decode = (content) => JSON.parse(Buffer.from(content, 'base64').toString());
const databaseChanged = getDbVersion(decode(base.content)) !== getDbVersion(decode(head.content));
core.setOutput('databases', JSON.stringify(databaseChanged ? allDatabases : defaultDatabases));
core.setOutput('modes', JSON.stringify(databaseChanged ? allModes : defaultModes));
core.setOutput('databases', JSON.stringify(['MongoDB']));
core.setOutput('modes', JSON.stringify(['dedicated']));
build:
name: Build
@@ -379,7 +429,7 @@ jobs:
- name: Wait for Open Runtimes
timeout-minutes: 3
run: |
while ! docker compose logs openruntimes-executor | grep -q "Executor is ready."; do
while ! docker logs exc1 | grep -q "Executor is ready."; do
echo "Waiting for Executor to come online"
sleep 1
done
@@ -405,8 +455,8 @@ jobs:
docker compose logs
e2e_service:
name: Tests / E2E / ${{ matrix.database }} (${{ matrix.mode }}) / ${{ matrix.service }}
runs-on: ${{ matrix.runner || format('runs-on={0}/runner=4cpu-linux-x64/volume=120g/spot=false', github.run_id) }}
name: Tests / E2E / ${{ matrix.database }} (${{ matrix.mode }}) / ${{ matrix.service.name }}
runs-on: ${{ matrix.service.runner }}
needs: [build, matrix]
permissions:
contents: read
@@ -417,45 +467,7 @@ jobs:
matrix:
database: ${{ fromJSON(needs.matrix.outputs.databases) }}
mode: ${{ fromJSON(needs.matrix.outputs.modes) }}
service: [
Account,
Avatars,
Console,
Databases,
TablesDB,
Functions,
FunctionsSchedule,
GraphQL,
Health,
Advisor,
Locale,
Projects,
Realtime,
Sites,
Proxy,
Storage,
Tokens,
Teams,
Users,
ProjectWebhooks,
Webhooks,
VCS,
Messaging,
Migrations,
Project,
Presences
]
include:
- service: Databases
runner: runs-on=${{ github.run_id }}/runner=8cpu-linux-x64/volume=120g/spot=false
paratest_processes: 3
timeout_minutes: 30
- service: TablesDB
runner: runs-on=${{ github.run_id }}/runner=8cpu-linux-x64/volume=120g/spot=false
paratest_processes: 3
timeout_minutes: 30
- service: Migrations
paratest_processes: 1
service: ${{ fromJSON(needs.matrix.outputs.services) }}
steps:
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
@@ -513,7 +525,7 @@ jobs:
- name: Wait for Open Runtimes
timeout-minutes: 3
run: |
while ! docker compose logs openruntimes-executor | grep -q "Executor is ready."; do
while ! docker logs exc1 | grep -q "Executor is ready."; do
echo "Waiting for Executor to come online"
sleep 1
done
@@ -523,20 +535,19 @@ jobs:
with:
max_attempts: 2
retry_wait_seconds: 60
timeout_minutes: ${{ matrix.timeout_minutes || 20 }}
timeout_minutes: ${{ matrix.service.timeout }}
job_id: ${{ job.check_run_id }}
github_token: ${{ secrets.GITHUB_TOKEN }}
test_dir: tests/e2e/Services/${{ matrix.service }}
test_dir: tests/e2e/Services/${{ matrix.service.name }}
command: |
SERVICE_PATH="/usr/src/code/tests/e2e/Services/${{ matrix.service }}"
SERVICE_PATH="/usr/src/code/tests/e2e/Services/${{ matrix.service.name }}"
# Services that rely on sequential test method execution (shared static state)
FUNCTIONAL_FLAG="--functional"
case "${{ matrix.service }}" in
Databases|TablesDB|Functions|Realtime|GraphQL|ProjectWebhooks) FUNCTIONAL_FLAG="" ;;
esac
FUNCTIONAL_FLAG=""
if [ "${{ matrix.service.functional }}" = "true" ]; then
FUNCTIONAL_FLAG="--functional"
fi
PARATEST_PROCESSES="${{ matrix.paratest_processes }}"
PARATEST_PROCESSES="${{ matrix.service.processes }}"
if [ -z "$PARATEST_PROCESSES" ]; then
PARATEST_PROCESSES="$(nproc)"
fi
@@ -545,7 +556,7 @@ jobs:
-e _APP_E2E_RESPONSE_FORMAT="${{ github.event.inputs.response_format }}" \
-e _TESTS_OAUTH2_GITHUB_CLIENT_ID="${{ secrets.TESTS_OAUTH2_GITHUB_CLIENT_ID }}" \
-e _TESTS_OAUTH2_GITHUB_CLIENT_SECRET="${{ secrets.TESTS_OAUTH2_GITHUB_CLIENT_SECRET }}" \
appwrite vendor/bin/paratest --processes "$PARATEST_PROCESSES" $FUNCTIONAL_FLAG "$SERVICE_PATH" --exclude-group abuseEnabled --exclude-group screenshots --log-junit tests/e2e/Services/${{ matrix.service }}/junit.xml
appwrite vendor/bin/paratest --processes "$PARATEST_PROCESSES" $FUNCTIONAL_FLAG "$SERVICE_PATH" --exclude-group abuseEnabled --exclude-group screenshots --log-junit tests/e2e/Services/${{ matrix.service.name }}/junit.xml
- name: Failure Logs
if: failure()
@@ -667,7 +678,7 @@ jobs:
- name: Wait for Open Runtimes
timeout-minutes: 3
run: |
while ! docker compose logs openruntimes-executor | grep -q "Executor is ready."; do
while ! docker logs exc1 | grep -q "Executor is ready."; do
echo "Waiting for Executor to come online"
sleep 1
done
+1 -1
View File
@@ -33,7 +33,7 @@ jobs:
id: meta
uses: docker/metadata-action@030e881283bb7a6894de51c315a6bfe6a94e05cf # v6.0.0
with:
images: appwrite/cloud
images: appwrite/ce
tags: |
type=ref,event=tag
+1 -1
View File
@@ -117,7 +117,7 @@ Common injections: `$response`, `$request`, `$dbForProject`, `$dbForPlatform`, `
## Tracing with Utopia Span
In handlers, only call `Span::add($key, $value)`. **Never** call `Span::init`, `Span::error`, or `Span::finish` -- lifecycle is owned by the entry-point harness (`app/http.php`, `app/worker.php`, `app/realtime.php`, `Bus::dispatch`). For selective export, filter in the sampler in `app/init/span.php`.
In handlers, only call `Span::add($key, $value)`. **Never** call `Span::init`, `setError`, or `Span::finish` -- lifecycle is owned by the entry-point harness (`app/http.php`, `app/worker.php`, `app/realtime.php`, `Bus::dispatch`). For selective export, filter in the sampler in `app/init/span.php`.
Keys are `snake_case` with dots only for child relationships: `project.id` (id of project), `storage.bucket.id`. No dot otherwise: `inbound_bytes`, not `inbound.bytes`. No camelCase, no bare top-level keys (`function.id`, not `functionId`).
+1 -1
View File
@@ -12,7 +12,7 @@ RUN composer install --ignore-platform-reqs --optimize-autoloader \
--no-plugins --no-scripts --prefer-dist \
`if [ "$TESTING" != "true" ]; then echo "--no-dev"; fi`
FROM appwrite/base:1.4.1 AS base
FROM appwrite/base:1.4.3 AS base
LABEL maintainer="team@appwrite.io"
Binary file not shown.
Binary file not shown.
+7
View File
@@ -603,6 +603,13 @@ return [
'lengths' => [Database::LENGTH_KEY],
'orders' => [Database::ORDER_ASC],
],
[
'$id' => ID::custom('_key_type_expire'),
'type' => Database::INDEX_KEY,
'attributes' => ['type', 'expire'],
'lengths' => [],
'orders' => [Database::ORDER_ASC, Database::ORDER_ASC],
],
],
],
+2
View File
@@ -39,6 +39,7 @@ $console = [
'membershipsMfa' => true,
'membershipsUserId' => true,
'membershipsUserPhone' => true,
'membershipsUserAccessedAt' => true,
'mockNumbers' => [],
'invites' => System::getEnv('_APP_CONSOLE_INVITES', 'enabled') === 'enabled',
'limit' => (System::getEnv('_APP_CONSOLE_WHITELIST_ROOT', 'enabled') === 'enabled') ? 1 : 0, // limit signup to 1 user
@@ -48,6 +49,7 @@ $console = [
'disposableEmails' => false,
'canonicalEmails' => false,
'freeEmails' => false,
'corporateEmails' => false,
'invalidateSessions' => true
],
'authWhitelistEmails' => (!empty(System::getEnv('_APP_CONSOLE_WHITELIST_EMAILS', null))) ? \explode(',', System::getEnv('_APP_CONSOLE_WHITELIST_EMAILS', null)) : [],
+5
View File
@@ -241,6 +241,11 @@ return [
'description' => 'This email address must already be in its canonical form. Please remove aliases, tags, or provider-specific variations and try again.',
'code' => 400,
],
Exception::USER_EMAIL_NOT_CORPORATE => [
'name' => Exception::USER_EMAIL_NOT_CORPORATE,
'description' => 'Only corporate email addresses are allowed. Please use a work email address and try again.',
'code' => 400,
],
Exception::USER_PASSWORD_MISMATCH => [
'name' => Exception::USER_PASSWORD_MISMATCH,
'description' => 'Passwords do not match. Please check the password and confirm password.',
+19
View File
@@ -156,6 +156,25 @@ return [
'gitBranch' => 'dev',
'changelog' => \realpath(__DIR__ . '/../../docs/sdks/react-native/CHANGELOG.md'),
],
[
'key' => 'unity',
'name' => 'Unity',
'version' => '0.1.0',
'url' => 'https://github.com/appwrite/sdk-for-unity',
'package' => 'https://github.com/appwrite/sdk-for-unity',
'enabled' => true,
'beta' => true,
'dev' => true,
'hidden' => false,
'family' => APP_SDK_PLATFORM_CLIENT,
'prism' => 'csharp',
'source' => \realpath(__DIR__ . '/../sdks/client-unity'),
'gitUrl' => 'git@github.com:appwrite/sdk-for-unity.git',
'gitRepoName' => 'sdk-for-unity',
'gitUserName' => 'appwrite',
'gitBranch' => 'dev',
'changelog' => \realpath(__DIR__ . '/../../docs/sdks/unity/CHANGELOG.md'),
],
[
'key' => 'graphql',
'name' => 'GraphQL',
+14
View File
@@ -281,6 +281,20 @@ return [
'icon' => '',
'platforms' => ['client', 'server', 'console'],
],
'organization' => [
'key' => 'organization',
'name' => 'Organization',
'subtitle' => 'The Organization service allows you to manage organization-level projects.',
'description' => '',
'controller' => '', // Uses modules
'sdk' => true,
'docs' => true,
'docsUrl' => '',
'tests' => false,
'optional' => false,
'icon' => '',
'platforms' => ['server', 'console'],
],
'migrations' => [
'key' => 'migrations',
'name' => 'Migrations',
+31 -4
View File
@@ -7,6 +7,7 @@ use Appwrite\Auth\Phrase;
use Appwrite\Auth\Validator\Password;
use Appwrite\Auth\Validator\PasswordDictionary;
use Appwrite\Auth\Validator\PasswordHistory;
use Appwrite\Auth\Validator\PasswordStrength;
use Appwrite\Auth\Validator\PersonalData;
use Appwrite\Auth\Validator\Phone;
use Appwrite\Bus\Events\SessionCreated;
@@ -28,6 +29,7 @@ use Appwrite\SDK\Deprecated;
use Appwrite\SDK\Method;
use Appwrite\SDK\MethodType;
use Appwrite\SDK\Response as SDKResponse;
use Appwrite\SDK\Specification\Validator\PasswordFormat;
use Appwrite\Template\Template;
use Appwrite\URL\URL as URLParser;
use Appwrite\Usage\Context;
@@ -71,6 +73,7 @@ use Utopia\Platform\Enum;
use Utopia\Storage\Validator\FileName;
use Utopia\System\System;
use Utopia\Validator;
use Utopia\Validator\AllOf;
use Utopia\Validator\ArrayList;
use Utopia\Validator\Assoc;
use Utopia\Validator\Boolean;
@@ -255,7 +258,7 @@ Http::post('/v1/account')
->label('abuse-limit', 10)
->param('userId', '', fn (Database $dbForProject) => new CustomId(false, $dbForProject->getAdapter()->getMaxUIDLength()), 'User ID. Choose a custom ID or generate a random ID with `ID.unique()`. Valid chars are a-z, A-Z, 0-9, period, hyphen, and underscore. Can\'t start with a special char. Max length is 36 chars.', false, ['dbForProject'])
->param('email', '', new EmailValidator(), 'User email.')
->param('password', '', fn ($project, $passwordsDictionary) => new PasswordDictionary($passwordsDictionary, $project->getAttribute('auths', [])['passwordDictionary'] ?? false), 'New user password. Must be between 8 and 256 chars.', false, ['project', 'passwordsDictionary'])
->param('password', '', fn ($project, $passwordsDictionary) => new PasswordFormat(new AllOf([new PasswordStrength($project->getAttribute('auths', [])['passwordStrength'] ?? []), new PasswordDictionary($passwordsDictionary, enabled: $project->getAttribute('auths', [])['passwordDictionary'] ?? false)], Validator::TYPE_STRING)), 'New user password. Must be between 8 and 256 chars.', false, ['project', 'passwordsDictionary'])
->param('name', '', new Text(128), 'User name. Max length: 128 chars.', true)
->inject('request')
->inject('response')
@@ -348,6 +351,10 @@ Http::post('/v1/account')
throw new Exception(Exception::USER_EMAIL_FREE);
}
if ((($project->getId() === 'console') || ($plan['supportsCorporateEmailValidation'] ?? false)) && ($project->getAttribute('auths', [])['corporateEmails'] ?? false) && !$emailMetadata['emailIsCorporate']) {
throw new Exception(Exception::USER_EMAIL_NOT_CORPORATE);
}
try {
$userId = $userId == 'unique()' ? ID::unique() : $userId;
$user->setAttributes([
@@ -1696,6 +1703,10 @@ Http::get('/v1/account/sessions/oauth2/:provider/redirect')
$failureRedirect(Exception::USER_EMAIL_FREE);
}
if ((($project->getId() === 'console') || ($plan['supportsCorporateEmailValidation'] ?? false)) && ($project->getAttribute('auths', [])['corporateEmails'] ?? false) && !$emailMetadata['emailIsCorporate']) {
$failureRedirect(Exception::USER_EMAIL_NOT_CORPORATE);
}
try {
$userId = ID::unique();
$user->setAttributes([
@@ -1837,6 +1848,10 @@ Http::get('/v1/account/sessions/oauth2/:provider/redirect')
$failureRedirect(Exception::USER_EMAIL_FREE);
}
if ((($project->getId() === 'console') || ($plan['supportsCorporateEmailValidation'] ?? false)) && ($project->getAttribute('auths', [])['corporateEmails'] ?? false) && !$emailMetadata['emailIsCorporate']) {
$failureRedirect(Exception::USER_EMAIL_NOT_CORPORATE);
}
$user->setAttribute('email', $email);
$user->setAttribute('emailCanonical', $emailMetadata['emailCanonical']);
$user->setAttribute('emailIsCanonical', $emailMetadata['emailIsCanonical']);
@@ -2195,6 +2210,10 @@ Http::post('/v1/account/tokens/magic-url')
throw new Exception(Exception::USER_EMAIL_FREE);
}
if ((($project->getId() === 'console') || ($plan['supportsCorporateEmailValidation'] ?? false)) && ($project->getAttribute('auths', [])['corporateEmails'] ?? false) && !$emailMetadata['emailIsCorporate']) {
throw new Exception(Exception::USER_EMAIL_NOT_CORPORATE);
}
$user->setAttributes([
'$id' => $userId,
'$permissions' => [
@@ -2515,6 +2534,10 @@ Http::post('/v1/account/tokens/email')
throw new Exception(Exception::USER_EMAIL_FREE);
}
if ((($project->getId() === 'console') || ($plan['supportsCorporateEmailValidation'] ?? false)) && ($project->getAttribute('auths', [])['corporateEmails'] ?? false) && !$emailMetadata['emailIsCorporate']) {
throw new Exception(Exception::USER_EMAIL_NOT_CORPORATE);
}
$user->setAttributes([
'$id' => $userId,
'$permissions' => [
@@ -3271,8 +3294,8 @@ Http::patch('/v1/account/password')
contentType: ContentType::JSON
))
->label('abuse-limit', 10)
->param('password', '', fn ($project, $passwordsDictionary) => new PasswordDictionary($passwordsDictionary, $project->getAttribute('auths', [])['passwordDictionary'] ?? false), 'New user password. Must be at least 8 chars.', false, ['project', 'passwordsDictionary'])
->param('oldPassword', '', new Password(), 'Current user password. Must be at least 8 chars.', true)
->param('password', '', fn ($project, $passwordsDictionary) => new PasswordFormat(new AllOf([new PasswordStrength($project->getAttribute('auths', [])['passwordStrength'] ?? []), new PasswordDictionary($passwordsDictionary, enabled: $project->getAttribute('auths', [])['passwordDictionary'] ?? false)], Validator::TYPE_STRING)), 'New user password. Must be at least 8 chars.', false, ['project', 'passwordsDictionary'])
->param('oldPassword', '', new PasswordFormat(new Text(256, 0)), 'Current user password. Max length: 256 chars.', true)
->inject('response')
->inject('user')
->inject('project')
@@ -3436,6 +3459,10 @@ Http::patch('/v1/account/email')
throw new Exception(Exception::USER_EMAIL_FREE);
}
if ((($project->getId() === 'console') || ($plan['supportsCorporateEmailValidation'] ?? false)) && ($project->getAttribute('auths', [])['corporateEmails'] ?? false) && !$emailMetadata['emailIsCorporate']) {
throw new Exception(Exception::USER_EMAIL_NOT_CORPORATE);
}
$user
->setAttribute('email', $email)
->setAttribute('emailVerification', false) // After this user needs to confirm mail again
@@ -3888,7 +3915,7 @@ Http::put('/v1/account/recovery')
->label('abuse-key', 'url:{url},userId:{param-userId}')
->param('userId', '', fn (Database $dbForProject) => new UID($dbForProject->getAdapter()->getMaxUIDLength()), 'User ID.', false, ['dbForProject'])
->param('secret', '', new Text(256), 'Valid reset token.')
->param('password', '', fn ($project, $passwordsDictionary) => new PasswordDictionary($passwordsDictionary, $project->getAttribute('auths', [])['passwordDictionary'] ?? false), 'New user password. Must be between 8 and 256 chars.', false, ['project', 'passwordsDictionary'])
->param('password', '', fn ($project, $passwordsDictionary) => new PasswordFormat(new AllOf([new PasswordStrength($project->getAttribute('auths', [])['passwordStrength'] ?? []), new PasswordDictionary($passwordsDictionary, enabled: $project->getAttribute('auths', [])['passwordDictionary'] ?? false)], Validator::TYPE_STRING)), 'New user password. Must be between 8 and 256 chars.', false, ['project', 'passwordsDictionary'])
->inject('response')
->inject('user')
->inject('dbForProject')
+4 -4
View File
@@ -122,11 +122,11 @@ Http::post('/v1/graphql/mutation')
->action(function (Request $request, Response $response, GQLSchema $schema, Adapter $promiseAdapter) {
$query = $request->getParams();
if ($request->getHeader('x-sdk-graphql') == 'true') {
if ($request->getHeaderLine('x-sdk-graphql') == 'true') {
$query = $query['query'];
}
$type = $request->getHeader('content-type');
$type = $request->getHeaderLine('content-type');
if (\str_starts_with($type, 'application/graphql')) {
$query = parseGraphql($request);
@@ -173,11 +173,11 @@ Http::post('/v1/graphql')
->action(function (Request $request, Response $response, GQLSchema $schema, Adapter $promiseAdapter) {
$query = $request->getParams();
if ($request->getHeader('x-sdk-graphql') == 'true') {
if ($request->getHeaderLine('x-sdk-graphql') == 'true') {
$query = $query['query'];
}
$type = $request->getHeader('content-type');
$type = $request->getHeaderLine('content-type');
if (\str_starts_with($type, 'application/graphql')) {
$query = parseGraphql($request);
+210 -2
View File
@@ -17,6 +17,7 @@ use Appwrite\SDK\ContentType;
use Appwrite\SDK\Deprecated;
use Appwrite\SDK\Method;
use Appwrite\SDK\Response as SDKResponse;
use Appwrite\SDK\Specification\Validator\PasswordFormat;
use Appwrite\Utopia\Database\Validator\CompoundUID;
use Appwrite\Utopia\Database\Validator\CustomId;
use Appwrite\Utopia\Database\Validator\Queries\Messages;
@@ -236,6 +237,100 @@ Http::post('/v1/messaging/providers/sendgrid')
->dynamic($provider, Response::MODEL_PROVIDER);
});
Http::post('/v1/messaging/providers/ses')
->desc('Create Amazon SES provider')
->groups(['api', 'messaging'])
->label('audits.event', 'provider.create')
->label('audits.resource', 'provider/{response.$id}')
->label('event', 'providers.[providerId].create')
->label('scope', 'providers.write')
->label('resourceType', RESOURCE_TYPE_PROVIDERS)
->label('sdk', new Method(
namespace: 'messaging',
group: 'providers',
name: 'createSesProvider',
description: '/docs/references/messaging/create-ses-provider.md',
auth: [AuthType::ADMIN, AuthType::KEY],
responses: [
new SDKResponse(
code: Response::STATUS_CODE_CREATED,
model: Response::MODEL_PROVIDER,
)
]
))
->param('providerId', '', fn (Database $dbForProject) => new CustomId(false, $dbForProject->getAdapter()->getMaxUIDLength()), 'Provider ID. Choose a custom ID or generate a random ID with `ID.unique()`. Valid chars are a-z, A-Z, 0-9, period, hyphen, and underscore. Can\'t start with a special char. Max length is 36 chars.', false, ['dbForProject'])
->param('name', '', new Text(128), 'Provider name.')
->param('accessKey', '', new Text(0), 'AWS access key ID.', true)
->param('secretKey', '', new Text(0), 'AWS secret access key.', true)
->param('region', '', new Text(128), 'AWS region, for example us-east-1.', true)
->param('fromName', '', new Text(128, 0), 'Sender Name.', true)
->param('fromEmail', '', new Email(), 'Sender email address.', true)
->param('replyToName', '', new Text(128, 0), 'Name set in the reply to field for the mail. Default value is sender name.', true)
->param('replyToEmail', '', new Email(), 'Email set in the reply to field for the mail. Default value is sender email.', true)
->param('enabled', null, new Nullable(new Boolean()), 'Set as enabled.', true)
->inject('queueForEvents')
->inject('dbForProject')
->inject('response')
->action(function (string $providerId, string $name, string $accessKey, string $secretKey, string $region, string $fromName, string $fromEmail, string $replyToName, string $replyToEmail, ?bool $enabled, Event $queueForEvents, Database $dbForProject, Response $response) {
$providerId = $providerId == 'unique()' ? ID::unique() : $providerId;
$credentials = [];
if (!empty($accessKey)) {
$credentials['accessKey'] = $accessKey;
}
if (!empty($secretKey)) {
$credentials['secretKey'] = $secretKey;
}
if (!empty($region)) {
$credentials['region'] = $region;
}
$options = [
'fromName' => $fromName,
'fromEmail' => $fromEmail,
'replyToName' => $replyToName,
'replyToEmail' => $replyToEmail,
];
if (
$enabled === true
&& !empty($fromEmail)
&& !empty($credentials['accessKey'])
&& !empty($credentials['secretKey'])
&& !empty($credentials['region'])
) {
$enabled = true;
} else {
$enabled = false;
}
$provider = new Document([
'$id' => $providerId,
'name' => $name,
'provider' => 'ses',
'type' => MESSAGE_TYPE_EMAIL,
'enabled' => $enabled,
'credentials' => $credentials,
'options' => $options,
]);
try {
$provider = $dbForProject->createDocument('providers', $provider);
} catch (DuplicateException) {
throw new Exception(Exception::PROVIDER_ALREADY_EXISTS);
}
$queueForEvents
->setParam('providerId', $provider->getId());
$response
->setStatusCode(Response::STATUS_CODE_CREATED)
->dynamic($provider, Response::MODEL_PROVIDER);
});
Http::post('/v1/messaging/providers/resend')
->desc('Create Resend provider')
->groups(['api', 'messaging'])
@@ -364,7 +459,7 @@ Http::post('/v1/messaging/providers/smtp')
->param('host', '', new Text(0), 'SMTP hosts. Either a single hostname or multiple semicolon-delimited hostnames. You can also specify a different port for each host such as `smtp1.example.com:25;smtp2.example.com`. You can also specify encryption type, for example: `tls://smtp1.example.com:587;ssl://smtp2.example.com:465"`. Hosts will be tried in order.')
->param('port', 587, new Range(1, 65535), 'The default SMTP server port.', true)
->param('username', '', new Text(0), 'Authentication username.', true)
->param('password', '', new Text(0), 'Authentication password.', true)
->param('password', '', new PasswordFormat(new Text(0)), 'Authentication password.', true)
->param('encryption', '', new WhiteList(['none', 'ssl', 'tls']), 'Encryption type. Can be omitted, \'ssl\', or \'tls\'', true, enum: new Enum(name: 'SmtpEncryption'))
->param('autoTLS', true, new Boolean(), 'Enable SMTP AutoTLS feature.', true)
->param('mailer', '', new Text(0), 'The value to use for the X-Mailer header.', true)
@@ -1458,6 +1553,119 @@ Http::patch('/v1/messaging/providers/sendgrid/:providerId')
->dynamic($provider, Response::MODEL_PROVIDER);
});
Http::patch('/v1/messaging/providers/ses/:providerId')
->desc('Update Amazon SES provider')
->groups(['api', 'messaging'])
->label('audits.event', 'provider.update')
->label('audits.resource', 'provider/{response.$id}')
->label('event', 'providers.[providerId].update')
->label('scope', 'providers.write')
->label('resourceType', RESOURCE_TYPE_PROVIDERS)
->label('sdk', new Method(
namespace: 'messaging',
group: 'providers',
name: 'updateSesProvider',
description: '/docs/references/messaging/update-ses-provider.md',
auth: [AuthType::ADMIN, AuthType::KEY],
responses: [
new SDKResponse(
code: Response::STATUS_CODE_OK,
model: Response::MODEL_PROVIDER,
)
]
))
->param('providerId', '', fn (Database $dbForProject) => new UID($dbForProject->getAdapter()->getMaxUIDLength()), 'Provider ID.', false, ['dbForProject'])
->param('name', '', new Text(128), 'Provider name.', true)
->param('enabled', null, new Nullable(new Boolean()), 'Set as enabled.', true)
->param('accessKey', '', new Text(0), 'AWS access key ID.', true)
->param('secretKey', '', new Text(0), 'AWS secret access key.', true)
->param('region', '', new Text(128), 'AWS region, for example us-east-1.', true)
->param('fromName', '', new Text(128), 'Sender Name.', true)
->param('fromEmail', '', new Email(), 'Sender email address.', true)
->param('replyToName', '', new Text(128), 'Name set in the Reply To field for the mail. Default value is Sender Name.', true)
->param('replyToEmail', '', new Text(128), 'Email set in the Reply To field for the mail. Default value is Sender Email.', true)
->inject('queueForEvents')
->inject('dbForProject')
->inject('response')
->action(function (string $providerId, string $name, ?bool $enabled, string $accessKey, string $secretKey, string $region, string $fromName, string $fromEmail, string $replyToName, string $replyToEmail, Event $queueForEvents, Database $dbForProject, Response $response) {
$provider = $dbForProject->getDocument('providers', $providerId);
if ($provider->isEmpty()) {
throw new Exception(Exception::PROVIDER_NOT_FOUND);
}
$providerAttr = $provider->getAttribute('provider');
if ($providerAttr !== 'ses') {
throw new Exception(Exception::PROVIDER_INCORRECT_TYPE);
}
if (!empty($name)) {
$provider->setAttribute('name', $name);
}
$options = $provider->getAttribute('options');
if (!empty($fromName)) {
$options['fromName'] = $fromName;
}
if (!empty($fromEmail)) {
$options['fromEmail'] = $fromEmail;
}
if (!empty($replyToName)) {
$options['replyToName'] = $replyToName;
}
if (!empty($replyToEmail)) {
$options['replyToEmail'] = $replyToEmail;
}
$provider->setAttribute('options', $options);
$credentials = $provider->getAttribute('credentials');
if (!empty($accessKey)) {
$credentials['accessKey'] = $accessKey;
}
if (!empty($secretKey)) {
$credentials['secretKey'] = $secretKey;
}
if (!empty($region)) {
$credentials['region'] = $region;
}
$provider->setAttribute('credentials', $credentials);
if (!\is_null($enabled)) {
if ($enabled) {
if (
!empty($credentials['accessKey']) &&
!empty($credentials['secretKey']) &&
!empty($credentials['region']) &&
!empty($options['fromEmail'])
) {
$provider->setAttribute('enabled', true);
} else {
throw new Exception(Exception::PROVIDER_MISSING_CREDENTIALS);
}
} else {
$provider->setAttribute('enabled', false);
}
}
$provider = $dbForProject->updateDocument('providers', $provider->getId(), $provider);
$queueForEvents
->setParam('providerId', $provider->getId());
$response
->dynamic($provider, Response::MODEL_PROVIDER);
});
Http::patch('/v1/messaging/providers/resend/:providerId')
->desc('Update Resend provider')
->groups(['api', 'messaging'])
@@ -1602,7 +1810,7 @@ Http::patch('/v1/messaging/providers/smtp/:providerId')
->param('host', '', new Text(0), 'SMTP hosts. Either a single hostname or multiple semicolon-delimited hostnames. You can also specify a different port for each host such as `smtp1.example.com:25;smtp2.example.com`. You can also specify encryption type, for example: `tls://smtp1.example.com:587;ssl://smtp2.example.com:465"`. Hosts will be tried in order.', true)
->param('port', null, new Nullable(new Range(1, 65535)), 'SMTP port.', true)
->param('username', '', new Text(0), 'Authentication username.', true)
->param('password', '', new Text(0), 'Authentication password.', true)
->param('password', '', new PasswordFormat(new Text(0)), 'Authentication password.', true)
->param('encryption', '', new WhiteList(['none', 'ssl', 'tls']), 'Encryption type. Can be \'ssl\' or \'tls\'', true, enum: new Enum(name: 'SmtpEncryption'))
->param('autoTLS', null, new Nullable(new Boolean()), 'Enable SMTP AutoTLS feature.', true)
->param('mailer', '', new Text(0), 'The value to use for the X-Mailer header.', true)
+14 -2
View File
@@ -6,6 +6,7 @@ use Appwrite\Auth\MFA\Type\TOTP;
use Appwrite\Auth\Validator\Password;
use Appwrite\Auth\Validator\PasswordDictionary;
use Appwrite\Auth\Validator\PasswordHistory;
use Appwrite\Auth\Validator\PasswordStrength;
use Appwrite\Auth\Validator\PersonalData;
use Appwrite\Auth\Validator\Phone;
use Appwrite\Deletes\Identities as DeleteIdentities;
@@ -21,6 +22,7 @@ use Appwrite\SDK\ContentType;
use Appwrite\SDK\Deprecated;
use Appwrite\SDK\Method;
use Appwrite\SDK\Response as SDKResponse;
use Appwrite\SDK\Specification\Validator\PasswordFormat;
use Appwrite\Utopia\Database\Validator\CustomId;
use Appwrite\Utopia\Database\Validator\Queries\Identities;
use Appwrite\Utopia\Database\Validator\Queries\Memberships;
@@ -65,6 +67,8 @@ use Utopia\Http\Http;
use Utopia\Locale\Locale;
use Utopia\Platform\Enum;
use Utopia\System\System;
use Utopia\Validator;
use Utopia\Validator\AllOf;
use Utopia\Validator\ArrayList;
use Utopia\Validator\Assoc;
use Utopia\Validator\Boolean;
@@ -145,6 +149,10 @@ function createUser(Hash $hash, string $userId, ?string $email, ?string $passwor
throw new Exception(Exception::USER_EMAIL_FREE);
}
if ((($project->getId() === 'console') || ($plan['supportsCorporateEmailValidation'] ?? false)) && ($project->getAttribute('auths', [])['corporateEmails'] ?? false) && !($emailMetadata['emailIsCorporate'] ?? true)) {
throw new Exception(Exception::USER_EMAIL_NOT_CORPORATE);
}
$hashedPassword = null;
$isHashed = !$hash instanceof Plaintext;
@@ -280,7 +288,7 @@ Http::post('/v1/users')
->param('userId', '', fn (Database $dbForProject) => new CustomId(false, $dbForProject->getAdapter()->getMaxUIDLength()), 'User ID. Choose a custom ID or generate a random ID with `ID.unique()`. Valid chars are a-z, A-Z, 0-9, period, hyphen, and underscore. Can\'t start with a special char. Max length is 36 chars.', false, ['dbForProject'])
->param('email', null, new Nullable(new EmailValidator()), 'User email.', true)
->param('phone', null, new Nullable(new Phone()), 'Phone number. Format this number with a leading \'+\' and a country code, e.g., +16175551212.', true)
->param('password', '', fn ($project, $passwordsDictionary) => new PasswordDictionary($passwordsDictionary, $project->getAttribute('auths', [])['passwordDictionary'] ?? false), 'Plain text user password. Must be at least 8 chars.', true, ['project', 'passwordsDictionary'])
->param('password', '', fn ($project, $passwordsDictionary) => new PasswordFormat(new AllOf([new PasswordStrength($project->getAttribute('auths', [])['passwordStrength'] ?? []), new PasswordDictionary($passwordsDictionary, enabled: $project->getAttribute('auths', [])['passwordDictionary'] ?? false)], Validator::TYPE_STRING)), 'Plain text user password. Must be at least 8 chars.', true, ['project', 'passwordsDictionary'])
->param('name', '', new Text(128), 'User name. Max length: 128 chars.', true)
->inject('response')
->inject('project')
@@ -1431,7 +1439,7 @@ Http::patch('/v1/users/:userId/password')
]
))
->param('userId', '', fn (Database $dbForProject) => new UID($dbForProject->getAdapter()->getMaxUIDLength()), 'User ID.', false, ['dbForProject'])
->param('password', '', fn ($project, $passwordsDictionary) => new PasswordDictionary($passwordsDictionary, enabled: $project->getAttribute('auths', [])['passwordDictionary'] ?? false, allowEmpty: true), 'New user password. Must be at least 8 chars.', false, ['project', 'passwordsDictionary'])
->param('password', '', fn ($project, $passwordsDictionary) => new PasswordFormat(new AllOf([new PasswordStrength($project->getAttribute('auths', [])['passwordStrength'] ?? [], allowEmpty: true), new PasswordDictionary($passwordsDictionary, enabled: $project->getAttribute('auths', [])['passwordDictionary'] ?? false, allowEmpty: true)], Validator::TYPE_STRING)), 'New user password. Must be at least 8 chars.', false, ['project', 'passwordsDictionary'])
->inject('response')
->inject('project')
->inject('dbForProject')
@@ -1609,6 +1617,10 @@ Http::patch('/v1/users/:userId/email')
throw new Exception(Exception::USER_EMAIL_FREE);
}
if ((($project->getId() === 'console') || ($plan['supportsCorporateEmailValidation'] ?? false)) && ($project->getAttribute('auths', [])['corporateEmails'] ?? false) && !($emailMetadata['emailIsCorporate'] ?? true)) {
throw new Exception(Exception::USER_EMAIL_NOT_CORPORATE);
}
$user
->setAttribute('email', $email)
->setAttribute('emailVerification', false)
+13 -8
View File
@@ -293,7 +293,12 @@ function router(Http $utopia, Database $dbForPlatform, callable $getProjectDB, S
$body = $swooleRequest->getContent() ?: '';
$method = $swooleRequest->server['request_method'];
$requestHeaders = $request->getHeaders();
// getHeaders() follows PSR-7 and returns each header as a list of values;
// collapse to single-line strings for the function/site header map below.
$requestHeaders = \array_map(
static fn (array $values) => \implode(', ', $values),
$request->getHeaders()
);
if ($resource->isEmpty() || !$resource->getAttribute('enabled')) {
if ($type === 'function') {
@@ -803,7 +808,7 @@ Http::init()
->inject('request')
->action(function (Document $project, Request $request) {
if ($project->getId() === 'console') {
$message = empty($request->getHeader('x-appwrite-project')) ?
$message = empty($request->getHeaderLine('x-appwrite-project')) ?
'No Appwrite project was specified. Please specify your project ID when initializing your Appwrite SDK.' :
'This endpoint is not available for the console project. The Appwrite Console is a reserved project ID and cannot be used with the Appwrite SDKs and APIs. Please check if your project ID is correct.';
throw new AppwriteException(AppwriteException::GENERAL_ACCESS_FORBIDDEN, $message);
@@ -861,7 +866,7 @@ Http::init()
return;
}
$requestFormat = $request->getHeader('x-appwrite-response-format', System::getEnv('_APP_SYSTEM_RESPONSE_FORMAT', ''));
$requestFormat = $request->getHeaderLine('x-appwrite-response-format', System::getEnv('_APP_SYSTEM_RESPONSE_FORMAT', ''));
if ($requestFormat) {
if (version_compare($requestFormat, '1.4.0', '<')) {
$request->addFilter(new RequestV16());
@@ -899,7 +904,7 @@ Http::init()
}
}
$localeParam = (string) $request->getParam('locale', $request->getHeader('x-appwrite-locale', ''));
$localeParam = (string) $request->getParam('locale', $request->getHeaderLine('x-appwrite-locale', ''));
if (\in_array($localeParam, $localeCodes)) {
$locale->setDefault($localeParam);
}
@@ -919,7 +924,7 @@ Http::init()
/*
* Response format
*/
$responseFormat = $request->getHeader('x-appwrite-response-format', System::getEnv('_APP_SYSTEM_RESPONSE_FORMAT', ''));
$responseFormat = $request->getHeaderLine('x-appwrite-response-format', System::getEnv('_APP_SYSTEM_RESPONSE_FORMAT', ''));
if ($responseFormat) {
if (version_compare($responseFormat, '1.9.5', '<')) {
$response->addFilter(new ResponseV26());
@@ -1006,7 +1011,7 @@ Http::init()
// Application level CSRF protection
$origin = $request->getOrigin();
if (empty($origin) || !$devKey->isEmpty() || !empty($request->getHeader('x-appwrite-key'))) {
if (empty($origin) || !$devKey->isEmpty() || !empty($request->getHeaderLine('x-appwrite-key'))) {
return;
}
$route = $request->getRoute();
@@ -1198,7 +1203,7 @@ Http::error()
$line = $error->getLine();
$trace = $error->getTrace();
Span::error($error);
Span::current()?->setError($error);
switch ($class) {
case Utopia\Http\Exception::class:
@@ -1315,7 +1320,7 @@ Http::error()
}
$log->addTag('hostname', $request->getHostname());
$log->addTag('locale', (string)$request->getParam('locale', $request->getHeader('x-appwrite-locale', '')));
$log->addTag('locale', (string)$request->getParam('locale', $request->getHeaderLine('x-appwrite-locale', '')));
$log->addExtra('file', $error->getFile());
$log->addExtra('line', $error->getLine());
+1 -1
View File
@@ -46,7 +46,7 @@ Http::get('/v1/mock/tests/locale')
->inject('request')
->inject('response')
->action(function (Locale $locale, array $localeCodes, Request $request, Response $response) {
$localeParam = (string) $request->getParam('locale', $request->getHeader('x-appwrite-locale', ''));
$localeParam = (string) $request->getParam('locale', $request->getHeaderLine('x-appwrite-locale', ''));
if (\in_array($localeParam, $localeCodes)) {
$locale->setDefault($localeParam);
}
+4 -4
View File
@@ -201,19 +201,19 @@ Http::init()
if (! empty($apiKey->getProjectId())) {
$dbKey = $project->find(
key: 'secret',
find: $request->getHeader('x-appwrite-key', ''),
find: $request->getHeaderLine('x-appwrite-key', ''),
subject: 'keys'
);
} elseif (! empty($apiKey->getUserId())) {
$dbKey = $user->find(
key: 'secret',
find: $request->getHeader('x-appwrite-key', ''),
find: $request->getHeaderLine('x-appwrite-key', ''),
subject: 'keys'
);
} elseif (! empty($apiKey->getTeamId())) {
$dbKey = $team->find(
key: 'secret',
find: $request->getHeader('x-appwrite-key', ''),
find: $request->getHeaderLine('x-appwrite-key', ''),
subject: 'keys'
);
}
@@ -231,7 +231,7 @@ Http::init()
}
$sdkValidator = new WhiteList($servers, true);
$sdk = $request->getHeader('x-sdk-name', 'UNKNOWN');
$sdk = $request->getHeaderLine('x-sdk-name', 'UNKNOWN');
if ($sdk !== 'UNKNOWN' && $sdkValidator->isValid($sdk)) {
$sdks = $dbKey->getAttribute('sdks', []);
+4 -3
View File
@@ -529,6 +529,7 @@ $swoole->onRequest(function ($utopiaRequest, $utopiaResponse) use ($files, $swoo
$app->setCompression(System::getEnv('_APP_COMPRESSION_ENABLED', 'enabled') === 'enabled');
$app->setCompressionMinSize(intval(System::getEnv('_APP_COMPRESSION_MIN_SIZE_BYTES', '1024'))); // 1KB
$error = null;
try {
$authorization = $app->context()->get('authorization');
@@ -542,7 +543,7 @@ $swoole->onRequest(function ($utopiaRequest, $utopiaResponse) use ($files, $swoo
$route = $app->match($request)?->route;
Span::add('http.path', $route?->getPath() ?? 'unknown');
} catch (\Throwable $th) {
Span::error($th);
$error = $th;
$version = System::getEnv('_APP_VERSION', 'UNKNOWN');
@@ -577,7 +578,7 @@ $swoole->onRequest(function ($utopiaRequest, $utopiaResponse) use ($files, $swoo
$log->addTag('code', $th->getCode());
// $log->addTag('projectId', $project->getId()); // TODO: Figure out how to get ProjectID, if it becomes relevant
$log->addTag('hostname', $request->getHostname());
$log->addTag('locale', (string)$request->getParam('locale', $request->getHeader('x-appwrite-locale', '')));
$log->addTag('locale', (string)$request->getParam('locale', $request->getHeaderLine('x-appwrite-locale', '')));
$log->addExtra('file', $th->getFile());
$log->addExtra('line', $th->getLine());
@@ -631,7 +632,7 @@ $swoole->onRequest(function ($utopiaRequest, $utopiaResponse) use ($files, $swoo
$swooleResponse->end(\json_encode($output));
} finally {
Span::add('http.response.code', $response->getStatusCode());
Span::current()?->finish();
Span::current()?->finish(error: $error);
}
});
+6
View File
@@ -243,6 +243,12 @@ const RULE_STATUS_VERIFIED = 'verified';
// Message types
const MESSAGE_SEND_TYPE_INTERNAL = 'internal';
const MESSAGE_SEND_TYPE_EXTERNAL = 'external';
// External message fan-out tuning
const MESSAGE_RECIPIENTS_PAGE_SIZE = 5000; // Recipients resolved from the database per page while streaming a send; aligned with APP_LIMIT_COUNT to minimise round-trips on large topics
const MESSAGE_SEND_CONCURRENCY = 10; // Maximum adapter send requests running concurrently within a single send job
const MESSAGE_DELIVERY_ERRORS_LIMIT = 100; // Maximum number of per-recipient delivery errors retained on a message
const MESSAGE_SEND_MAX_RETRIES = 5; // Maximum number of attempts to deliver a batch when the provider throttles or fails transiently
const MESSAGE_SEND_RETRY_DELAY = 1.0; // Base seconds for exponential backoff between batch send retries; overridable in tests for an instant suite
// Mail Types
const MAIL_TYPE_VERIFICATION = 'verification';
const MAIL_TYPE_MAGIC_SESSION = 'magicSession';
+1
View File
@@ -231,6 +231,7 @@ Database::addFilter(
return $database->getAuthorization()->skip(fn () => $database
->find('authenticators', [
Query::equal('userInternalId', [$document->getSequence()]),
Query::orderDesc('$createdAt'),
Query::limit(APP_LIMIT_SUBQUERY),
]));
}
+2
View File
@@ -168,6 +168,7 @@ use Appwrite\Utopia\Response\Model\PolicyMembershipPrivacy;
use Appwrite\Utopia\Response\Model\PolicyPasswordDictionary;
use Appwrite\Utopia\Response\Model\PolicyPasswordHistory;
use Appwrite\Utopia\Response\Model\PolicyPasswordPersonalData;
use Appwrite\Utopia\Response\Model\PolicyPasswordStrength;
use Appwrite\Utopia\Response\Model\PolicySessionAlert;
use Appwrite\Utopia\Response\Model\PolicySessionDuration;
use Appwrite\Utopia\Response\Model\PolicySessionInvalidation;
@@ -460,6 +461,7 @@ Response::setModel(new OAuth2Microsoft());
Response::setModel(new OAuth2ProviderList());
Response::setModel(new PolicyPasswordDictionary());
Response::setModel(new PolicyPasswordHistory());
Response::setModel(new PolicyPasswordStrength());
Response::setModel(new PolicyPasswordPersonalData());
Response::setModel(new PolicySessionAlert());
Response::setModel(new PolicySessionDuration());
+12 -12
View File
@@ -24,7 +24,7 @@ use Utopia\Validator\WhiteList;
*/
return function (Container $container): void {
$getProjectId = static function (Request $request): string {
$projectId = $request->getHeader('x-appwrite-project', '');
$projectId = $request->getHeaderLine('x-appwrite-project', '');
if (!empty($projectId)) {
return $projectId;
@@ -36,7 +36,7 @@ return function (Container $container): void {
};
$getMode = static function (Request $request, Document $project) use ($getProjectId): string {
$mode = $request->getParam('mode', $request->getHeader('x-appwrite-mode', APP_MODE_DEFAULT));
$mode = $request->getParam('mode', $request->getHeaderLine('x-appwrite-mode', APP_MODE_DEFAULT));
$projectId = $getProjectId($request);
if (!empty($projectId) && $project->getId() !== $projectId) {
@@ -113,7 +113,7 @@ return function (Container $container): void {
};
$findDevKey = static function (Request $request, Document $project, array $servers, Authorization $authorization) use ($getDbForPlatform): Document {
$devKey = $request->getHeader('x-appwrite-dev-key', $request->getParam('devKey', ''));
$devKey = $request->getHeaderLine('x-appwrite-dev-key', $request->getParam('devKey', ''));
$key = $project->find('secret', $devKey, 'devKeys');
if (!$key) {
@@ -137,7 +137,7 @@ return function (Container $container): void {
}
$sdkValidator = new WhiteList($servers, true);
$sdk = \strtolower($request->getHeader('x-sdk-name', 'UNKNOWN'));
$sdk = \strtolower($request->getHeaderLine('x-sdk-name', 'UNKNOWN'));
if ($sdk !== 'unknown' && $sdkValidator->isValid($sdk)) {
$sdks = $key->getAttribute('sdks', []);
@@ -233,7 +233,7 @@ return function (Container $container): void {
);
if (empty($store->getProperty('id', '')) && empty($store->getProperty('secret', ''))) {
$sessionHeader = $request->getHeader('x-appwrite-session', '');
$sessionHeader = $request->getHeaderLine('x-appwrite-session', '');
if (!empty($sessionHeader)) {
$store->decode($sessionHeader);
@@ -241,7 +241,7 @@ return function (Container $container): void {
}
if (empty($store->getProperty('id', '')) && empty($store->getProperty('secret', ''))) {
$fallback = \json_decode($request->getHeader('x-fallback-cookies', ''), true);
$fallback = \json_decode($request->getHeaderLine('x-fallback-cookies', ''), true);
$store->decode((\is_array($fallback) && isset($fallback[$store->getKey()])) ? $fallback[$store->getKey()] : '');
}
@@ -271,7 +271,7 @@ return function (Container $container): void {
$user = new User([]);
}
$authJWT = $request->getHeader('x-appwrite-jwt', '');
$authJWT = $request->getHeaderLine('x-appwrite-jwt', '');
if (!empty($authJWT) && !$project->isEmpty()) {
if (!$user->isEmpty()) {
throw new Exception(Exception::USER_JWT_AND_COOKIE_SET);
@@ -300,8 +300,8 @@ return function (Container $container): void {
}
}
$accountKey = $request->getHeader('x-appwrite-key', '');
$accountKeyUserId = $request->getHeader('x-appwrite-user', '');
$accountKey = $request->getHeaderLine('x-appwrite-key', '');
$accountKeyUserId = $request->getHeaderLine('x-appwrite-user', '');
if (!empty($accountKeyUserId) && !empty($accountKey)) {
if (!$user->isEmpty()) {
@@ -329,9 +329,9 @@ return function (Container $container): void {
// Query params mirror the header fallback pattern used by ?project= and ?devKey=,
// allowing Console to embed impersonation in direct file/image URLs where headers cannot be set.
$impersonateUserId = $request->getHeader('x-appwrite-impersonate-user-id', (string)$request->getParam('impersonateUserId', ''));
$impersonateEmail = $request->getHeader('x-appwrite-impersonate-user-email', (string)$request->getParam('impersonateEmail', ''));
$impersonatePhone = $request->getHeader('x-appwrite-impersonate-user-phone', (string)$request->getParam('impersonatePhone', ''));
$impersonateUserId = $request->getHeaderLine('x-appwrite-impersonate-user-id', (string)($request->getParam('impersonateuserid', '') ?: $request->getParam('impersonateUserId', '')));
$impersonateEmail = $request->getHeaderLine('x-appwrite-impersonate-user-email', (string)($request->getParam('impersonateemail', '') ?: $request->getParam('impersonateEmail', '')));
$impersonatePhone = $request->getHeaderLine('x-appwrite-impersonate-user-phone', (string)($request->getParam('impersonatephone', '') ?: $request->getParam('impersonatePhone', '')));
if (!$user->isEmpty() && $user->getAttribute('impersonator', false)) {
$userDb = ($mode === APP_MODE_ADMIN || $project->getId() === 'console') ? $dbForPlatform : $dbForProject;
+14 -12
View File
@@ -222,12 +222,6 @@ $register->set('pools', function () {
'multiple' => false,
'schemes' => ['redis'],
],
'consumer' => [
'type' => 'consumer',
'dsns' => $fallbackForRedis,
'multiple' => false,
'schemes' => ['redis'],
],
'pubsub' => [
'type' => 'pubsub',
'dsns' => $fallbackForRedis,
@@ -254,6 +248,10 @@ $register->set('pools', function () {
$workerCount = intval(System::getEnv('_APP_CPU_NUM', swoole_cpu_num())) * intval(System::getEnv('_APP_WORKER_PER_CORE', 6));
$poolSize = max(1, (int)($instanceConnections / $workerCount));
// Queue workers consume jobs concurrently with coroutines; each in-flight
// job may hold a connection, so pools must cover the coroutine count.
$poolSize = max($poolSize, (int) System::getEnv('_APP_WORKER_MAX_COROUTINES', 1));
foreach ($connections as $key => $connection) {
$type = $connection['type'];
$multiple = $connection['multiple'];
@@ -340,9 +338,10 @@ $register->set('pools', function () {
// PubSub workers hold one long-lived subscribed connection and also need
// spare capacity for publishes from the same process.
$connectionPoolSize = $type === 'pubsub'
? max(2, $poolSize)
: $poolSize;
$connectionPoolSize = match ($type) {
'pubsub' => max(2, $poolSize),
default => $poolSize,
};
$pool = new Pool($poolAdapter, $name, $connectionPoolSize, function () use ($type, $resource, $dsn) {
// Get Adapter
@@ -364,9 +363,12 @@ $register->set('pools', function () {
default => null
};
case 'publisher':
case 'consumer':
// Publishers never block on receive, so one connection backs both broker slots.
return match ($dsn->getScheme()) {
'redis' => new Queue\Broker\Redis(new Queue\Connection\Redis($dsn->getHost(), $dsn->getPort())),
'redis' => (function () use ($dsn) {
$connection = new Queue\Connection\Redis($dsn->getHost(), $dsn->getPort());
return new Queue\Broker\Redis($connection, $connection);
})(),
default => null
};
case 'cache':
@@ -451,7 +453,7 @@ $register->set('smtp', function () {
);
});
$register->set('geodb', function () {
return new Reader(__DIR__ . '/../assets/dbip/dbip-country-lite-2025-12.mmdb');
return new Reader(__DIR__ . '/../assets/dbip/dbip-country-lite-2026-06.mmdb');
});
$register->set('passwordsDictionary', function () {
$content = \file_get_contents(__DIR__ . '/../assets/security/10k-common-passwords');
+22 -22
View File
@@ -387,7 +387,7 @@ return function (Container $context): void {
// Get session from header for SSR clients
if (empty($store->getProperty('id', '')) && empty($store->getProperty('secret', ''))) {
$sessionHeader = $request->getHeader('x-appwrite-session', '');
$sessionHeader = $request->getHeaderLine('x-appwrite-session', '');
if (! empty($sessionHeader)) {
$store->decode($sessionHeader);
@@ -399,7 +399,7 @@ return function (Container $context): void {
if (empty($store->getProperty('id', '')) && empty($store->getProperty('secret', ''))) {
$response->addHeader('X-Debug-Fallback', 'true');
$fallback = $request->getHeader('x-fallback-cookies', '');
$fallback = $request->getHeaderLine('x-fallback-cookies', '');
$fallback = \json_decode($fallback, true);
$store->decode(((is_array($fallback) && isset($fallback[$store->getKey()])) ? $fallback[$store->getKey()] : ''));
}
@@ -432,7 +432,7 @@ return function (Container $context): void {
$user = new User([]);
}
$authJWT = $request->getHeader('x-appwrite-jwt', '');
$authJWT = $request->getHeaderLine('x-appwrite-jwt', '');
if (! empty($authJWT) && ! $project->isEmpty()) { // JWT authentication
if (! $user->isEmpty()) {
throw new Exception(Exception::USER_JWT_AND_COOKIE_SET);
@@ -462,8 +462,8 @@ return function (Container $context): void {
}
// Account based on account API key
$accountKey = $request->getHeader('x-appwrite-key', '');
$accountKeyUserId = $request->getHeader('x-appwrite-user', '');
$accountKey = $request->getHeaderLine('x-appwrite-key', '');
$accountKeyUserId = $request->getHeaderLine('x-appwrite-user', '');
if (! empty($accountKeyUserId) && ! empty($accountKey)) {
if (! $user->isEmpty()) {
throw new Exception(Exception::USER_API_KEY_AND_SESSION_SET);
@@ -491,9 +491,9 @@ return function (Container $context): void {
// Impersonation: if current user has impersonator capability and headers/params are set, act as another user
// Query params mirror the header fallback pattern used by ?project= and ?devKey=,
// allowing Console to embed impersonation in direct file/image URLs where headers cannot be set.
$impersonateUserId = $request->getHeader('x-appwrite-impersonate-user-id', (string)$request->getParam('impersonateUserId', ''));
$impersonateEmail = $request->getHeader('x-appwrite-impersonate-user-email', (string)$request->getParam('impersonateEmail', ''));
$impersonatePhone = $request->getHeader('x-appwrite-impersonate-user-phone', (string)$request->getParam('impersonatePhone', ''));
$impersonateUserId = $request->getHeaderLine('x-appwrite-impersonate-user-id', (string)($request->getParam('impersonateuserid', '') ?: $request->getParam('impersonateUserId', '')));
$impersonateEmail = $request->getHeaderLine('x-appwrite-impersonate-user-email', (string)($request->getParam('impersonateemail', '') ?: $request->getParam('impersonateEmail', '')));
$impersonatePhone = $request->getHeaderLine('x-appwrite-impersonate-user-phone', (string)($request->getParam('impersonatephone', '') ?: $request->getParam('impersonatePhone', '')));
if (!$user->isEmpty() && $user->getAttribute('impersonator', false)) {
$userDb = (APP_MODE_ADMIN === $mode || $project->getId() === 'console') ? $dbForPlatform : $dbForProject;
$targetUser = null;
@@ -525,10 +525,10 @@ return function (Container $context): void {
/** @var Appwrite\Utopia\Request $request */
/** @var Utopia\Database\Database $dbForPlatform */
/** @var Utopia\Database\Document $console */
$projectId = $request->getParam('project', $request->getHeader('x-appwrite-project', ''));
$projectId = $request->getParam('project', $request->getHeaderLine('x-appwrite-project', ''));
// Realtime channel "project" can send project=Query array
if (! \is_string($projectId)) {
$projectId = $request->getHeader('x-appwrite-project', '');
$projectId = $request->getHeaderLine('x-appwrite-project', '');
}
// Backwards compatibility for new services, originally project resources
@@ -920,9 +920,9 @@ return function (Container $context): void {
* - 'default' => Requests for Client and Server Side
* - 'admin' => Request from the Console on non-console projects
*/
$mode = $request->getParam('mode', $request->getHeader('x-appwrite-mode', APP_MODE_DEFAULT));
$mode = $request->getParam('mode', $request->getHeaderLine('x-appwrite-mode', APP_MODE_DEFAULT));
$projectId = $request->getParam('project', $request->getHeader('x-appwrite-project', ''));
$projectId = $request->getParam('project', $request->getHeaderLine('x-appwrite-project', ''));
if (!empty($projectId) && $project->getId() !== $projectId) {
$mode = APP_MODE_ADMIN;
}
@@ -932,7 +932,7 @@ return function (Container $context): void {
$context->set('requestTimestamp', function ($request) {
// TODO: Move this to the Request class itself
$timestampHeader = $request->getHeader('x-appwrite-timestamp');
$timestampHeader = $request->getHeaderLine('x-appwrite-timestamp');
$requestTimestamp = null;
if (! empty($timestampHeader)) {
try {
@@ -946,7 +946,7 @@ return function (Container $context): void {
}, ['request']);
$context->set('devKey', function (Request $request, Document $project, array $servers, Database $dbForPlatform, Authorization $authorization) {
$devKey = $request->getHeader('x-appwrite-dev-key', $request->getParam('devKey', ''));
$devKey = $request->getHeaderLine('x-appwrite-dev-key', $request->getParam('devKey', ''));
// Check if given key match project's development keys
$key = $project->find('secret', $devKey, 'devKeys');
@@ -972,7 +972,7 @@ return function (Container $context): void {
// add sdk to key
$sdkValidator = new WhiteList($servers, true);
$sdk = \strtolower($request->getHeader('x-sdk-name', 'UNKNOWN'));
$sdk = \strtolower($request->getHeaderLine('x-sdk-name', 'UNKNOWN'));
if ($sdk !== 'unknown' && $sdkValidator->isValid($sdk)) {
$sdks = $key->getAttribute('sdks', []);
@@ -1001,7 +1001,7 @@ return function (Container $context): void {
} else {
$route = $utopia->match($request)?->route;
$path = ! empty($route) ? $route->getPath() : $request->getURI();
$orgHeader = $request->getHeader('x-appwrite-organization', '');
$orgHeader = $request->getHeaderLine('x-appwrite-organization', '');
if (str_starts_with($path, '/v1/projects/:projectId')) {
$uri = $request->getURI();
$pid = explode('/', $uri)[3];
@@ -1047,7 +1047,7 @@ return function (Container $context): void {
}
if ($allowed) {
$host = $request->getQuery('appwrite-hostname', $request->getHeader('x-appwrite-hostname', '')) ?? '';
$host = $request->getQuery('appwrite-hostname', $request->getHeaderLine('x-appwrite-hostname', '')) ?? '';
if (! empty($host)) {
return $host;
}
@@ -1057,7 +1057,7 @@ return function (Container $context): void {
}, ['request', 'apiKey']);
$context->set('apiKey', function (Request $request, Document $project, Document $team, Document $user): ?Key {
$key = $request->getHeader('x-appwrite-key');
$key = $request->getHeaderLine('x-appwrite-key');
if (empty($key)) {
return null;
@@ -1065,9 +1065,9 @@ return function (Container $context): void {
$key = Key::decode($project, $team, $user, $key);
$userHeader = $request->getHeader('x-appwrite-user');
$organizationHeader = $request->getHeader('x-appwrite-organization');
$projectHeader = $request->getHeader('x-appwrite-project');
$userHeader = $request->getHeaderLine('x-appwrite-user');
$organizationHeader = $request->getHeaderLine('x-appwrite-organization');
$projectHeader = $request->getHeaderLine('x-appwrite-project');
if (! empty($key->getProjectId())) {
if (empty($projectHeader) || $projectHeader !== $key->getProjectId()) {
@@ -1170,7 +1170,7 @@ return function (Container $context): void {
['host' => \gethostname(), 'project' => $project->getId()]
);
$timeout = \intval($request->getHeader('x-appwrite-timeout'));
$timeout = \intval($request->getHeaderLine('x-appwrite-timeout'));
if (!empty($timeout) && Http::isDevelopment()) {
$database->setTimeout($timeout);
}
+2 -2
View File
@@ -12,7 +12,7 @@ $traceProjectId = System::getEnv('_APP_TRACE_PROJECT_ID', '');
$traceFunctionId = System::getEnv('_APP_TRACE_FUNCTION_ID', '');
$traceEnabled = $traceProjectId !== '' || $traceFunctionId !== '';
Span::addExporter(new Exporter\Pretty(), function (Span $span) use ($traceEnabled, $traceProjectId, $traceFunctionId): bool {
Span::setExporters(new Exporter\Pretty(sampler: function (Span $span) use ($traceEnabled, $traceProjectId, $traceFunctionId): bool {
if (\str_starts_with($span->getAction(), 'listener.')) {
return $span->getError() !== null;
}
@@ -29,4 +29,4 @@ Span::addExporter(new Exporter\Pretty(), function (Span $span) use ($traceEnable
}
return true;
});
}));
+51 -24
View File
@@ -699,8 +699,15 @@ $server->onWorkerStart(function (int $workerId) use ($server, $register, $stats,
);
}
// Restore authorization after subscribe
if ($authorization !== null) {
// meta can be empty as well as the channels are not required query param to connect
// channels and queries can be sent via message later on
// so if meta is empty we are not subscribing above to the projectId
if (!isset($realtime->connections[$connection])) {
$realtime->subscribe($projectId, $connection, '', $roles, [], [], $userId);
}
if ($authorization !== null && isset($realtime->connections[$connection])) {
$realtime->connections[$connection]['authorization'] = $authorization;
}
@@ -845,6 +852,7 @@ $server->onOpen(function (int $connection, SwooleRequest $request) use ($server,
Span::add('realtime.origin', $request->getOrigin());
}
$error = null;
try {
/** @var Document $project */
$project = $connectionContainer->get('project');
@@ -914,7 +922,7 @@ $server->onOpen(function (int $connection, SwooleRequest $request) use ($server,
$channels = Realtime::convertChannels($request->getQuery('channels', []), $user->getId());
$channelCount = \count($channels);
$updateStats = static function (string $projectId, ?string $teamId, string $payloadJson) use ($register, $stats): void {
$updateStats = static function (string $projectId, ?string $teamId) use ($register, $stats): void {
$register->get('telemetry.connectionCounter')->add(1);
$register->get('telemetry.workerClientCounter')->add(1, $register->get('telemetry.workerAttributes'));
$register->get('telemetry.connectionCreatedCounter')->add(1);
@@ -928,7 +936,6 @@ $server->onOpen(function (int $connection, SwooleRequest $request) use ($server,
triggerStats([
METRIC_REALTIME_CONNECTIONS => 1,
METRIC_REALTIME_OUTBOUND => \strlen($payloadJson),
], $projectId);
};
@@ -949,9 +956,12 @@ $server->onOpen(function (int $connection, SwooleRequest $request) use ($server,
$realtime->subscribe($project->getId(), $connection, '', $roles, [], [], $user->getId());
$realtime->connections[$connection]['authorization'] = $authorization;
$updateStats($project->getId(), $project->getAttribute('teamId'));
$server->send([$connection], $connectedPayloadJson);
$outboundBytes += \strlen($connectedPayloadJson);
$updateStats($project->getId(), $project->getAttribute('teamId'), $connectedPayloadJson);
triggerStats([
METRIC_REALTIME_OUTBOUND => \strlen($connectedPayloadJson),
], $project->getId());
$subscriptionMode = 'message';
$success = true;
return;
@@ -969,6 +979,8 @@ $server->onOpen(function (int $connection, SwooleRequest $request) use ($server,
throw new Exception(Exception::REALTIME_POLICY_VIOLATION, $e->getMessage());
}
$sanitizedUser = empty($user->getId()) ? null : $response->output($user, Response::MODEL_ACCOUNT);
$mapping = [];
foreach ($subscriptions as $index => $subscription) {
$subscriptionId = ID::unique();
@@ -985,31 +997,33 @@ $server->onOpen(function (int $connection, SwooleRequest $request) use ($server,
$mapping[$index] = $subscriptionId;
}
$realtime->connections[$connection]['authorization'] = $authorization;
$updateStats($project->getId(), $project->getAttribute('teamId'));
$subscriptionCount = \count($subscriptions);
if (!empty($subscriptions)) {
$register->get('telemetry.workerSubscriptionCounter')->add(\count($subscriptions), $register->get('telemetry.workerAttributes'));
}
$realtime->connections[$connection]['authorization'] = $authorization;
$user = empty($user->getId()) ? null : $response->output($user, Response::MODEL_ACCOUNT);
$connectedPayloadJson = json_encode([
'type' => 'connected',
'data' => [
'channels' => $names,
'subscriptions' => $mapping,
'user' => $user
'user' => $sanitizedUser
]
]);
$server->send([$connection], $connectedPayloadJson);
$outboundBytes += \strlen($connectedPayloadJson);
$updateStats($project->getId(), $project->getAttribute('teamId'), $connectedPayloadJson);
triggerStats([
METRIC_REALTIME_OUTBOUND => \strlen($connectedPayloadJson),
], $project->getId());
$success = true;
} catch (Throwable $th) {
Span::error($th);
$error = $th;
// Convert known Utopia DB exceptions to AppwriteException so isPublishable()
// suppresses expected client errors (permission denied, query timeout) from Sentry.
@@ -1067,7 +1081,7 @@ $server->onOpen(function (int $connection, SwooleRequest $request) use ($server,
if (!empty($logUser?->getId())) {
Span::add('user.id', $logUser->getId());
}
Span::current()?->finish();
Span::current()?->finish(error: $error);
}
});
@@ -1086,6 +1100,7 @@ $server->onMessage(function (int $connection, string $message) use ($container,
Span::add('realtime.inbound_bytes', $rawSize);
Span::add('realtime.container.id', $containerId);
$error = null;
try {
$response = new Response(new SwooleResponse());
@@ -1207,13 +1222,16 @@ $server->onMessage(function (int $connection, string $message) use ($container,
$outboundBytes += $bytes;
if ($project !== null && !$project->isEmpty()) {
triggerStats([METRIC_REALTIME_OUTBOUND => $bytes], $project->getId());
triggerStats([
METRIC_REALTIME_OUTBOUND => $bytes,
METRIC_REALTIME_CONNECTIONS_MESSAGES_SENT => 1,
], $project->getId());
}
}
$success = true;
} catch (Throwable $th) {
Span::error($th);
$error = $th;
// Convert known Utopia DB exceptions to AppwriteException so isPublishable()
// suppresses expected client errors (permission denied, query timeout) from Sentry.
@@ -1259,7 +1277,7 @@ $server->onMessage(function (int $connection, string $message) use ($container,
Span::add('project.id', $project?->getId() ?? $projectId);
Span::add('user.id', $realtime->connections[$connection]['userId'] ?? null);
Span::add('realtime.message_type', $messageType);
Span::current()?->finish();
Span::current()?->finish(error: $error);
}
});
@@ -1279,20 +1297,27 @@ $server->onClose(function (int $connection) use ($realtime, $stats, $register, $
try {
if (array_key_exists($connection, $realtime->connections)) {
$stats->decr($realtime->connections[$connection]['projectId'], 'connectionsTotal');
// These decrements are symmetric with the increments in updateStats (onOpen),
// which run when the connection enters $realtime->connections — the same gate as the
// array_key_exists check above. The connectionsTotal stat is keyed by project, so it's
// additionally guarded: a connection can exist without a stored projectId (e.g. only an
// authorization entry left after an orphaning re-subscribe).
$register->get('telemetry.connectionCounter')->add(-1);
$register->get('telemetry.workerClientCounter')->add(-1, $register->get('telemetry.workerAttributes'));
if (!empty($projectId)) {
$stats->decr($projectId, 'connectionsTotal');
}
$subscriptionsBeforeClose = \count($realtime->getSubscriptionMetadata($connection));
if ($subscriptionsBeforeClose > 0) {
$register->get('telemetry.workerSubscriptionCounter')->add(-$subscriptionsBeforeClose, $register->get('telemetry.workerAttributes'));
}
$projectId = $realtime->connections[$connection]['projectId'];
/** @var array<string, Document> $presencesById */
$presencesById = $realtime->connections[$connection]['presences'] ?? [];
if (
!empty($presencesById)
&& !empty($projectId)
&& $projectId !== 'console'
) {
go(function () use ($presencesById, $projectId, $userId, $container, $presenceState): void {
@@ -1346,7 +1371,7 @@ $server->onClose(function (int $connection) use ($realtime, $stats, $register, $
);
$presenceState->triggerUsage($publisherForUsage, $project, -$deletionCount);
} catch (Throwable $th) {
Span::error($th);
Span::current()?->setError($th);
logError($th, 'realtimeOnClosePresenceDeletion', tags: [
'projectId' => $projectId,
'presences' => \count($presences)
@@ -1373,7 +1398,7 @@ $server->onClose(function (int $connection) use ($realtime, $stats, $register, $
}
}
} catch (Throwable $th) {
Span::error($th);
Span::current()?->setError($th);
logError($th, 'realtimeOnClosePresenceCleanup', tags: [
'projectId' => $projectId,
]);
@@ -1383,22 +1408,24 @@ $server->onClose(function (int $connection) use ($realtime, $stats, $register, $
});
}
triggerStats([
METRIC_REALTIME_CONNECTIONS => -1,
], $projectId);
if (!empty($projectId)) {
triggerStats([
METRIC_REALTIME_CONNECTIONS => -1,
], $projectId);
}
}
$success = true;
} catch (\Throwable $th) {
// Log only; do not rethrow. If we let this bubble, Swoole dumps full coroutine
// backtraces and unsubscribe() below would never run (connection cleanup would fail).
Console::error('Realtime onClose error: ' . $th->getMessage());
Span::error($th);
Span::current()?->setError($th);
} finally {
try {
$realtime->unsubscribe($connection);
} catch (\Throwable $th) {
Console::error('Realtime onClose unsubscribe error: ' . $th->getMessage());
Span::error($th);
Span::current()?->setError($th);
}
Span::add('realtime.success', $success);
+5 -2
View File
@@ -983,12 +983,12 @@ $hostPath = rtrim($this->getParam('hostPath', ''), '/');
- appwrite
openruntimes-executor:
container_name: openruntimes-executor
container_name: exc1
hostname: exc1
<<: *x-logging
restart: unless-stopped
stop_signal: SIGINT
image: openruntimes/executor:0.11.4
image: openruntimes/executor:0.24.1
networks:
- appwrite
- runtimes
@@ -1001,6 +1001,8 @@ $hostPath = rtrim($this->getParam('hostPath', ''), '/');
# It's not possible to share mount file between 2 containers without host mount (copying is too slow)
- /tmp:/tmp:rw
environment:
- OPR_EXECUTOR_CONNECTION_STORAGE=local://localhost
- OPR_EXECUTOR_CONNECTION_BUILD_CACHE_STORAGE=
- OPR_EXECUTOR_INACTIVE_TRESHOLD=$_APP_COMPUTE_INACTIVE_THRESHOLD
- OPR_EXECUTOR_MAINTENANCE_INTERVAL=$_APP_COMPUTE_MAINTENANCE_INTERVAL
- OPR_EXECUTOR_NETWORK=$_APP_COMPUTE_RUNTIMES_NETWORK
@@ -1209,3 +1211,4 @@ volumes:
appwrite-sites:
appwrite-builds:
appwrite-config:
openruntimes-node-modules-cache:
+18 -26
View File
@@ -12,9 +12,10 @@ use Utopia\Database\Validator\Authorization;
use Utopia\Logger\Log;
use Utopia\Logger\Logger;
use Utopia\Platform\Service;
use Utopia\Pools\Group;
use Utopia\Queue\Adapter\Swoole;
use Utopia\Queue\Broker\Pool as BrokerPool;
use Utopia\Queue\Broker\Redis as BrokerRedis;
use Utopia\Queue\Connection\Locking;
use Utopia\Queue\Connection\Redis as RedisConnection;
use Utopia\Queue\Server;
use Utopia\Span\Span;
use Utopia\System\System;
@@ -38,22 +39,6 @@ $container->set('project', fn () => new Document([]), []);
$container->set('log', fn () => new Log(), []);
$container->set('consumer', function (Group $pools) {
return new BrokerPool(consumer: $pools->get('consumer'));
}, ['pools']);
$container->set('consumerDatabases', function (BrokerPool $consumer) {
return $consumer;
}, ['consumer']);
$container->set('consumerMigrations', function (BrokerPool $consumer) {
return $consumer;
}, ['consumer']);
$container->set('consumerStatsUsage', function (BrokerPool $consumer) {
return $consumer;
}, ['consumer']);
$container->set('certificates', function () {
$email = System::getEnv('_APP_EMAIL_CERTIFICATES', System::getEnv('_APP_SYSTEM_SECURITY_EMAIL_ADDRESS'));
if (empty($email)) {
@@ -80,20 +65,27 @@ if (\str_starts_with($workerName, 'databases')) {
$queueName = System::getEnv('_APP_QUEUE_NAME', 'v1-' . strtolower($workerName));
}
/** @var \Utopia\Pools\Group $pools */
$pools = $container->get('pools');
$redisHost = System::getEnv('_APP_REDIS_HOST', 'redis');
$redisPort = (int) System::getEnv('_APP_REDIS_PORT', 6379);
// A dedicated connection drives the blocking receive loop; a lock-guarded one
// serializes acks/publishes from the per-message coroutines.
$adapter = new Swoole(
$pools->get('consumer')->pop()->getResource(),
new BrokerRedis(
receive: new RedisConnection($redisHost, $redisPort),
commands: new Locking(new RedisConnection($redisHost, $redisPort)),
),
System::getEnv('_APP_WORKERS_NUM', 1),
$queueName
$queueName,
maxCoroutines: (int) System::getEnv('_APP_WORKER_MAX_COROUTINES', 1),
resources: $container,
);
$worker = new Server($adapter, $container);
$worker = new Server($adapter);
try {
$worker->init()->action(function () use ($worker, $registerWorkerMessageResources, $queueName) {
$registerWorkerMessageResources($worker->getContainer());
$registerWorkerMessageResources($worker->context());
Span::init("worker.{$queueName}");
});
@@ -103,7 +95,7 @@ try {
$container->set('bus', function ($register) use ($worker) {
return $register->get('bus')->setResolver(
fn (string $name) => $worker->getContainer()->get($name)
fn (string $name) => $worker->context()->get($name)
);
}, ['register']);
@@ -126,7 +118,7 @@ $worker
->action(function (Throwable $error, ?Logger $logger, Log $log, Document $project, Authorization $authorization) use ($queueName) {
$version = System::getEnv('_APP_VERSION', 'UNKNOWN');
Span::error($error);
Span::current()?->setError($error);
if ($logger) {
$log->setNamespace('appwrite-worker');
+11 -8
View File
@@ -16,6 +16,8 @@
"analyze": "./vendor/bin/phpstan analyse -c phpstan.neon --memory-limit=1G",
"bench": "vendor/bin/phpbench run --report=benchmark",
"check": "./vendor/bin/phpstan analyse -c phpstan.neon --memory-limit=1G",
"refactor": "vendor/bin/rector process",
"refactor:check": "vendor/bin/rector process --dry-run",
"installer:clean": "php src/Appwrite/Platform/Installer/Server.php --clean",
"installer:dev": "docker compose build && composer installer:clean && php src/Appwrite/Platform/Installer/Server.php --docker"
},
@@ -57,7 +59,7 @@
"utopia-php/audit": "^2.4",
"utopia-php/auth": "0.5.*",
"utopia-php/cache": "^3.0",
"utopia-php/cli": "0.23.*",
"utopia-php/cli": "^0.24",
"utopia-php/compression": "0.1.*",
"utopia-php/config": "1.*",
"utopia-php/console": "0.1.*",
@@ -65,7 +67,7 @@
"utopia-php/detector": "0.2.*",
"utopia-php/domains": "^2.2",
"utopia-php/emails": "0.7.*",
"utopia-php/dns": "1.7.*",
"utopia-php/dns": "^1.8",
"utopia-php/dsn": "0.2.1",
"utopia-php/http": "^2.0@RC",
"utopia-php/fetch": "^1.1",
@@ -74,13 +76,13 @@
"utopia-php/locale": "0.8.*",
"utopia-php/lock": "0.2.*",
"utopia-php/logger": "0.8.*",
"utopia-php/messaging": "0.22.*",
"utopia-php/messaging": "1.0.*",
"utopia-php/migration": "1.*",
"utopia-php/platform": "1.0.0-rc3",
"utopia-php/platform": "^1.0@RC",
"utopia-php/pools": "1.*",
"utopia-php/span": "1.1.*",
"utopia-php/span": "3.0.*",
"utopia-php/preloader": "0.2.*",
"utopia-php/queue": "0.18.*",
"utopia-php/queue": "^0.21",
"utopia-php/servers": "0.4.*",
"utopia-php/registry": "0.5.*",
"utopia-php/storage": "2.*",
@@ -100,14 +102,15 @@
},
"require-dev": {
"ext-fileinfo": "*",
"appwrite/sdk-generator": "*",
"appwrite/sdk-generator": "^2.0",
"brianium/paratest": "7.*",
"phpunit/phpunit": "12.*",
"swoole/ide-helper": "6.*",
"phpstan/phpstan": "^2.0",
"textalk/websocket": "1.5.*",
"czproject/git-php": "4.*",
"laravel/pint": "1.*"
"laravel/pint": "1.*",
"rector/rector": "^2.4"
},
"provide": {
"ext-phpiredis": "*"
Generated
+380 -569
View File
File diff suppressed because it is too large. Load diff
+38 -4
View File
@@ -349,6 +349,8 @@ services:
- appwrite-embedding
environment:
- _APP_ENV
- _APP_WORKERS_NUM=1
- _APP_WORKER_MAX_COROUTINES=1
- _APP_WORKER_PER_CORE
- _APP_POOL_ADAPTER
- _APP_OPENSSL_KEY_V1
@@ -383,6 +385,8 @@ services:
- appwrite-embedding
environment:
- _APP_ENV
- _APP_WORKERS_NUM=1
- _APP_WORKER_MAX_COROUTINES=8
- _APP_WORKER_PER_CORE
- _APP_POOL_ADAPTER
- _APP_OPENSSL_KEY_V1
@@ -424,6 +428,8 @@ services:
environment:
- _APP_ENV
- _APP_WORKERS_NUM=1
- _APP_WORKER_MAX_COROUTINES=8
- _APP_WORKER_PER_CORE
- _APP_POOL_ADAPTER
- _APP_OPENSSL_KEY_V1
@@ -484,6 +490,8 @@ services:
- appwrite-embedding
environment:
- _APP_ENV
- _APP_WORKERS_NUM=1
- _APP_WORKER_MAX_COROUTINES=1
- _APP_WORKER_PER_CORE
- _APP_POOL_ADAPTER
- _APP_OPENSSL_KEY_V1
@@ -504,7 +512,6 @@ services:
- _APP_DB_USER_VECTORSDB
- _APP_DB_PASS_VECTORSDB
- _APP_LOGGING_CONFIG
- _APP_WORKERS_NUM
- _APP_QUEUE_NAME
- _APP_DATABASE_SHARED_TABLES
@@ -527,6 +534,8 @@ services:
- appwrite-embedding
environment:
- _APP_ENV
- _APP_WORKERS_NUM=1
- _APP_WORKER_MAX_COROUTINES=8
- _APP_WORKER_PER_CORE
- _APP_POOL_ADAPTER
- _APP_OPENSSL_KEY_V1
@@ -605,6 +614,8 @@ services:
- _APP_OPTIONS_FORCE_HTTPS
# Basic
- _APP_ENV
- _APP_WORKERS_NUM=1
- _APP_WORKER_MAX_COROUTINES=8
- _APP_WORKER_PER_CORE
- _APP_POOL_ADAPTER
- _APP_LOGGING_CONFIG
@@ -665,6 +676,8 @@ services:
- ./src:/usr/src/code/src
environment:
- _APP_ENV
- _APP_WORKERS_NUM=1
- _APP_WORKER_MAX_COROUTINES=8
- _APP_WORKER_PER_CORE
- _APP_POOL_ADAPTER
- _APP_OPENSSL_KEY_V1
@@ -705,6 +718,8 @@ services:
- ${_APP_DB_HOST:-mongodb}
environment:
- _APP_ENV
- _APP_WORKERS_NUM=1
- _APP_WORKER_MAX_COROUTINES=8
- _APP_WORKER_PER_CORE
- _APP_POOL_ADAPTER
- _APP_OPENSSL_KEY_V1
@@ -736,6 +751,8 @@ services:
- redis
environment:
- _APP_ENV
- _APP_WORKERS_NUM=1
- _APP_WORKER_MAX_COROUTINES=8
- _APP_WORKER_PER_CORE
- _APP_POOL_ADAPTER
- _APP_OPENSSL_KEY_V1
@@ -782,6 +799,8 @@ services:
# - smtp
environment:
- _APP_ENV
- _APP_WORKERS_NUM=1
- _APP_WORKER_MAX_COROUTINES=1
- _APP_WORKER_PER_CORE
- _APP_POOL_ADAPTER
- _APP_OPENSSL_KEY_V1
@@ -823,6 +842,8 @@ services:
- redis
environment:
- _APP_ENV
- _APP_WORKERS_NUM=1
- _APP_WORKER_MAX_COROUTINES=1
- _APP_WORKER_PER_CORE
- _APP_POOL_ADAPTER
- _APP_OPENSSL_KEY_V1
@@ -883,6 +904,8 @@ services:
- ${_APP_DB_HOST:-mongodb}
environment:
- _APP_ENV
- _APP_WORKERS_NUM=1
- _APP_WORKER_MAX_COROUTINES=1
- _APP_WORKER_PER_CORE
- _APP_POOL_ADAPTER
- _APP_OPENSSL_KEY_V1
@@ -1044,6 +1067,8 @@ services:
- ${_APP_DB_HOST:-mongodb}
environment:
- _APP_ENV
- _APP_WORKERS_NUM=1
- _APP_WORKER_MAX_COROUTINES=1
- _APP_WORKER_PER_CORE
- _APP_POOL_ADAPTER
- _APP_OPENSSL_KEY_V1
@@ -1078,6 +1103,8 @@ services:
- appwrite-embedding
environment:
- _APP_ENV
- _APP_WORKERS_NUM=1
- _APP_WORKER_MAX_COROUTINES=1
- _APP_WORKER_PER_CORE
- _APP_POOL_ADAPTER
- _APP_OPENSSL_KEY_V1
@@ -1218,11 +1245,11 @@ services:
- appwrite
openruntimes-executor:
container_name: openruntimes-executor
container_name: exc1
hostname: exc1
<<: *x-logging
stop_signal: SIGINT
image: openruntimes/executor:0.11.4
image: openruntimes/executor:0.24.1
restart: unless-stopped
networks:
- appwrite
@@ -1236,6 +1263,8 @@ services:
# It's not possible to share mount file between 2 containers without host mount (copying is too slow)
- /tmp:/tmp:rw
environment:
- OPR_EXECUTOR_CONNECTION_STORAGE=local://localhost
- OPR_EXECUTOR_CONNECTION_BUILD_CACHE_STORAGE=
- OPR_EXECUTOR_IMAGE_PULL=enabled
- OPR_EXECUTOR_INACTIVE_TRESHOLD=$_APP_COMPUTE_INACTIVE_THRESHOLD
- OPR_EXECUTOR_MAINTENANCE_INTERVAL=$_APP_COMPUTE_MAINTENANCE_INTERVAL
@@ -1356,7 +1385,11 @@ services:
timeout: 5s
retries: 10
start_period: 10s
command: "postgres"
command: >
postgres
-c fsync=off
-c synchronous_commit=off
-c full_page_writes=off
appwrite-embedding:
image: appwrite/embedding:0.1.0
@@ -1494,3 +1527,4 @@ volumes:
appwrite-builds:
appwrite-config:
appwrite-models:
openruntimes-node-modules-cache:
@@ -0,0 +1 @@
Create a new Amazon SES provider.
@@ -0,0 +1 @@
Update an Amazon SES provider by its unique ID.
+5
View File
@@ -0,0 +1,5 @@
# Change Log
## 0.1.0
* Initial release
+19
View File
@@ -0,0 +1,19 @@
<?php
declare(strict_types=1);
use Rector\Config\RectorConfig;
use Rector\PHPUnit\Set\PHPUnitSetList;
return RectorConfig::configure()
->withBootstrapFiles([
__DIR__ . '/app/init/constants.php',
])
->withPaths([
__DIR__ . '/tests',
])
->withSkipPath(__DIR__ . '/vendor')
->withSkipPath(__DIR__ . '/tests/resources')
->withSets([
PHPUnitSetList::PHPUNIT_CODE_QUALITY,
]);
@@ -0,0 +1,89 @@
<?php
namespace Appwrite\Auth\Validator;
/**
* PasswordStrength.
*
* Validates password complexity rules.
*/
class PasswordStrength extends Password
{
protected int $min;
protected bool $uppercase;
protected bool $lowercase;
protected bool $number;
protected bool $symbols;
public function __construct(array $policy = [], bool $allowEmpty = false)
{
parent::__construct($allowEmpty);
$this->min = $policy['min'] ?? 8;
$this->uppercase = $policy['uppercase'] ?? false;
$this->lowercase = $policy['lowercase'] ?? false;
$this->number = $policy['number'] ?? false;
$this->symbols = $policy['symbols'] ?? false;
}
public function getDescription(): string
{
$requirements = [
'between ' . $this->min . ' and 256 characters long',
];
if ($this->uppercase) {
$requirements[] = 'include an uppercase letter';
}
if ($this->lowercase) {
$requirements[] = 'include a lowercase letter';
}
if ($this->number) {
$requirements[] = 'include a number';
}
if ($this->symbols) {
$requirements[] = 'include a symbol';
}
return 'Password must be ' . \implode(', ', $requirements) . '.';
}
/**
* @param mixed $value
*/
public function isValid($value): bool
{
if (!parent::isValid($value)) {
return false;
}
if ($this->allowEmpty && \strlen($value) === 0) {
return true;
}
if (\strlen($value) < $this->min) {
return false;
}
if ($this->uppercase && !\preg_match('/[A-Z]/', $value)) {
return false;
}
if ($this->lowercase && !\preg_match('/[a-z]/', $value)) {
return false;
}
if ($this->number && !\preg_match('/\d/', $value)) {
return false;
}
if ($this->symbols && !\preg_match('/[^\p{L}\p{N}\s]/u', $value)) {
return false;
}
return true;
}
}
+1
View File
@@ -84,6 +84,7 @@ class Exception extends \Exception
public const string USER_EMAIL_ALREADY_EXISTS = 'user_email_already_exists';
public const string USER_EMAIL_DISPOSABLE = 'user_email_disposable';
public const string USER_EMAIL_FREE = 'user_email_free';
public const string USER_EMAIL_NOT_CORPORATE = 'user_email_not_corporate';
public const string USER_EMAIL_NOT_CANONICAL = 'user_email_not_canonical';
public const string USER_PASSWORD_MISMATCH = 'user_password_mismatch';
public const string USER_SESSION_NOT_FOUND = 'user_session_not_found';
+3 -2
View File
@@ -40,8 +40,9 @@ class Resolvers
);
}
// Response headers follow PSR-7: keys are lowercased and values are lists.
$headers = $from->getHeaders();
$fallbackCookies = $headers['X-Fallback-Cookies'] ?? null;
$fallbackCookies = $headers['x-fallback-cookies'] ?? null;
if ($fallbackCookies === null) {
return;
}
@@ -347,7 +348,7 @@ class Resolvers
$request->addHeader('x-appwrite-source', 'graphql');
// Drop json content type so post args are used directly.
if (\str_starts_with($request->getHeader('content-type'), 'application/json')) {
if (\str_starts_with($request->getHeaderLine('content-type'), 'application/json')) {
$request->removeHeader('content-type');
}
+9
View File
@@ -213,6 +213,15 @@ class V24 extends Migration
$this->dbForProject->purgeCachedCollection($id);
break;
case 'tokens':
try {
$this->createIndexFromCollection($this->dbForProject, $id, '_key_type_expire');
} catch (Throwable $th) {
Console::warning("Failed to create index \"_key_type_expire\" from {$id}: {$th->getMessage()}");
}
$this->dbForProject->purgeCachedCollection($id);
break;
case 'databases':
if ($collectionType === 'projects') {
try {
@@ -73,7 +73,7 @@ class Install extends Action
Config $config,
array $paths
): void {
$acceptHeader = $request->getHeader('accept');
$acceptHeader = $request->getHeaderLine('accept');
$wantsStream = stripos($acceptHeader, 'text/event-stream') !== false;
if ($wantsStream) {
@@ -38,7 +38,7 @@ class Validate extends Action
public static function validateCsrf(Request $request): bool
{
$cookie = $request->getCookie(Server::CSRF_COOKIE);
$header = $request->getHeader('x-appwrite-installer-csrf');
$header = $request->getHeaderLine('x-appwrite-installer-csrf');
return $cookie !== '' && $header !== '' && hash_equals($cookie, $header);
}
@@ -42,6 +42,7 @@ class Get extends Action
description: '/docs/references/avatars/get-browser.md',
auth: [AuthType::ADMIN, AuthType::SESSION, AuthType::KEY, AuthType::JWT],
type: MethodType::LOCATION,
locationAuth: ['Project', 'ImpersonateUserId'],
responses: [
new SDKResponse(
code: Response::STATUS_CODE_OK,
@@ -42,6 +42,7 @@ class Get extends Action
description: '/docs/references/avatars/get-credit-card.md',
auth: [AuthType::ADMIN, AuthType::SESSION, AuthType::KEY, AuthType::JWT],
type: MethodType::LOCATION,
locationAuth: ['Project', 'ImpersonateUserId'],
responses: [
new SDKResponse(
code: Response::STATUS_CODE_OK,
@@ -54,6 +54,7 @@ class Get extends Action
description: '/docs/references/avatars/get-favicon.md',
auth: [AuthType::ADMIN, AuthType::SESSION, AuthType::KEY, AuthType::JWT],
type: MethodType::LOCATION,
locationAuth: ['Project', 'ImpersonateUserId'],
responses: [
new SDKResponse(
code: Response::STATUS_CODE_OK,
@@ -42,6 +42,7 @@ class Get extends Action
description: '/docs/references/avatars/get-flag.md',
auth: [AuthType::ADMIN, AuthType::SESSION, AuthType::KEY, AuthType::JWT],
type: MethodType::LOCATION,
locationAuth: ['Project', 'ImpersonateUserId'],
responses: [
new SDKResponse(
code: Response::STATUS_CODE_OK,
@@ -45,6 +45,7 @@ class Get extends Action
description: '/docs/references/avatars/get-image.md',
auth: [AuthType::ADMIN, AuthType::SESSION, AuthType::KEY, AuthType::JWT],
type: MethodType::LOCATION,
locationAuth: ['Project', 'ImpersonateUserId'],
responses: [
new SDKResponse(
code: Response::STATUS_CODE_OK,
@@ -44,6 +44,7 @@ class Get extends Action
description: '/docs/references/avatars/get-initials.md',
auth: [AuthType::ADMIN, AuthType::SESSION, AuthType::KEY, AuthType::JWT],
type: MethodType::LOCATION,
locationAuth: ['Project', 'ImpersonateUserId'],
responses: [
new SDKResponse(
code: Response::STATUS_CODE_OK,
@@ -42,6 +42,7 @@ class Get extends Action
description: '/docs/references/avatars/get-qr.md',
auth: [AuthType::ADMIN, AuthType::SESSION, AuthType::KEY, AuthType::JWT],
type: MethodType::LOCATION,
locationAuth: ['Project', 'ImpersonateUserId'],
responses: [
new SDKResponse(
code: Response::STATUS_CODE_OK,
@@ -57,6 +57,7 @@ class Get extends Action
description: '/docs/references/avatars/get-screenshot.md',
auth: [AuthType::ADMIN, AuthType::SESSION, AuthType::KEY, AuthType::JWT],
type: MethodType::LOCATION,
locationAuth: ['Project', 'ImpersonateUserId'],
responses: [
new SDKResponse(
code: Response::STATUS_CODE_OK,
@@ -144,18 +144,6 @@ class Base extends Action
'activate' => $activate,
]));
$function = $function
->setAttribute('latestDeploymentId', $deployment->getId())
->setAttribute('latestDeploymentInternalId', $deployment->getSequence())
->setAttribute('latestDeploymentCreatedAt', $deployment->getCreatedAt())
->setAttribute('latestDeploymentStatus', $deployment->getAttribute('status', ''));
$dbForProject->updateDocument('functions', $function->getId(), new Document([
'latestDeploymentId' => $deployment->getId(),
'latestDeploymentInternalId' => $deployment->getSequence(),
'latestDeploymentCreatedAt' => $deployment->getCreatedAt(),
'latestDeploymentStatus' => $deployment->getAttribute('status', ''),
]));
$publisherForBuilds->enqueue(new BuildMessage(
project: $project,
resource: $function,
@@ -258,18 +246,6 @@ class Base extends Action
'activate' => $activate,
]));
$site = $site
->setAttribute('latestDeploymentId', $deployment->getId())
->setAttribute('latestDeploymentInternalId', $deployment->getSequence())
->setAttribute('latestDeploymentCreatedAt', $deployment->getCreatedAt())
->setAttribute('latestDeploymentStatus', $deployment->getAttribute('status', ''));
$dbForProject->updateDocument('sites', $site->getId(), new Document([
'latestDeploymentId' => $deployment->getId(),
'latestDeploymentInternalId' => $deployment->getSequence(),
'latestDeploymentCreatedAt' => $deployment->getCreatedAt(),
'latestDeploymentStatus' => $deployment->getAttribute('status', ''),
]));
$sitesDomain = $platform['sitesDomain'];
$domain = ID::unique() . "." . $sitesDomain;
@@ -103,6 +103,15 @@ class Create extends Action
// Map aggregate permissions into the multiple permissions they represent.
$permissions = Permission::aggregate($permissions) ?? [];
// Resolve the data-plane database BEFORE writing the metadata row. For a
// dedicated product database this throws while provisioning (no DSN yet);
// writing the row first would leave an orphaned collection with no backing
// collection, which lists fine but fails every document operation.
/**
* @var Database $dbForDatabases
*/
$dbForDatabases = $getDatabasesDB($database);
try {
$collection = $dbForProject->createDocument('database_' . $database->getSequence(), new Document([
'$id' => $collectionId,
@@ -122,11 +131,6 @@ class Create extends Action
throw new Exception(Exception::DATABASE_NOT_FOUND, params: [$databaseId]);
}
/**
* @var Database $dbForDatabases
*/
$dbForDatabases = $getDatabasesDB($database);
$collectionKey = 'database_' . $database->getSequence() . '_collection_' . $collection->getSequence();
$databaseKey = 'database_' . $database->getSequence();
@@ -199,6 +199,24 @@ class Create extends Action
$database = $dbForProject->getDocument('databases', $databaseId);
$this->createMetadataCollection($dbForProject, $database);
$queueForEvents->setParam('databaseId', $database->getId());
$response
->setStatusCode(SwooleResponse::STATUS_CODE_CREATED)
->dynamic($database, UtopiaResponse::MODEL_DATABASE);
}
/**
* Create the per-database metadata collection (`database_{internalId}`) that holds
* the database's collection definitions. Extracted so product overrides that
* provision a backend asynchronously (e.g. documentsdb, vectorsdb) can initialise
* the metadata exactly like a standard database create instead of leaving it
* uninitialised, which makes the first collection create fail.
*/
protected function createMetadataCollection(Database $dbForProject, Document $database): void
{
$collections = $this->getDatabaseCollection();
if (empty($collections)) {
throw new Exception(Exception::GENERAL_SERVER_ERROR, 'The "collections" collection is not configured.');
@@ -217,19 +235,11 @@ class Create extends Action
try {
$dbForProject->createCollection('database_' . $database->getSequence(), $attributes, $indexes);
} catch (DuplicateException) {
throw new Exception(Exception::DATABASE_ALREADY_EXISTS, params: [$databaseId]);
throw new Exception(Exception::DATABASE_ALREADY_EXISTS, params: [$database->getId()]);
} catch (IndexException $e) {
throw new Exception(Exception::INDEX_INVALID);
} catch (LimitException) {
// TODO: @Jake, how do we handle this collection/table?
// there's no context awareness at this level on what the api is.
throw new Exception(Exception::COLLECTION_LIMIT_EXCEEDED, params: [$databaseId]);
throw new Exception(Exception::COLLECTION_LIMIT_EXCEEDED, params: [$database->getId()]);
}
$queueForEvents->setParam('databaseId', $database->getId());
$response
->setStatusCode(SwooleResponse::STATUS_CODE_CREATED)
->dynamic($database, UtopiaResponse::MODEL_DATABASE);
}
}
@@ -25,7 +25,7 @@ class Timeout extends Action
->inject('request')
->inject('dbForProject')
->callback(function (Request $request, Database $dbForProject) {
$timeout = \intval($request->getHeader('x-appwrite-timeout'));
$timeout = \intval($request->getHeaderLine('x-appwrite-timeout'));
if (!empty($timeout) && Http::isDevelopment()) {
$dbForProject->setTimeout($timeout);
@@ -33,7 +33,7 @@ class Update extends DatetimeUpdate
$this
->setHttpMethod(self::HTTP_REQUEST_METHOD_PATCH)
->setHttpPath('/v1/tablesdb/:databaseId/tables/:tableId/columns/datetime/:key')
->desc('Update dateTime column')
->desc('Update datetime column')
->groups(['api', 'database', 'schema'])
->label('scope', ['tables.write', 'collections.write', 'columns.write', 'attributes.write'])
->label('resourceType', RESOURCE_TYPE_DATABASES)
@@ -94,6 +94,13 @@ class Create extends CollectionAction
// Map aggregate permissions into the multiple permissions they represent.
$permissions = Permission::aggregate($permissions) ?? [];
// Resolve the data-plane database BEFORE writing the metadata row. For a
// dedicated product database this throws while provisioning (no DSN yet);
// writing the row first would leave an orphaned collection with no backing
// collection, which lists fine but fails every document operation.
/** @var Database $dbForDatabases */
$dbForDatabases = $getDatabasesDB($database);
try {
$collection = $dbForProject->createDocument('database_' . $database->getSequence(), new Document([
'$id' => $collectionId,
@@ -114,8 +121,6 @@ class Create extends CollectionAction
} catch (NotFoundException) {
throw new Exception(Exception::DATABASE_NOT_FOUND);
}
/** @var Database $dbForDatabases */
$dbForDatabases = $getDatabasesDB($database);
$attributes = [];
$indexes = [];
@@ -44,7 +44,7 @@ class Create extends CreateDocumentAction
$this
->setHttpMethod(self::HTTP_REQUEST_METHOD_POST)
->setHttpPath('/v1/vectorsdb/embeddings/text')
->desc('Create Text Embeddings')
->desc('Create text embeddings')
->groups(['api', 'database'])
->label('scope', 'documents.write')
->label('resourceType', RESOURCE_TYPE_EMBEDDINGS_TEXT)
@@ -166,7 +166,7 @@ class Create extends Action
throw new Exception(Exception::STORAGE_FILE_TYPE_UNSUPPORTED);
}
$contentRange = $request->getHeader('content-range');
$contentRange = $request->getHeaderLine('content-range');
$deploymentId = ID::unique();
$chunk = 1;
$chunks = 1;
@@ -175,7 +175,7 @@ class Create extends Action
$start = $request->getContentRangeStart();
$end = $request->getContentRangeEnd();
$fileSize = $request->getContentRangeSize();
$deploymentId = $request->getHeader('x-appwrite-id', $deploymentId);
$deploymentId = $request->getHeaderLine('x-appwrite-id', $deploymentId);
// TODO make `end >= $fileSize` in next breaking version
if (is_null($start) || is_null($end) || is_null($fileSize) || $end > $fileSize) {
throw new Exception(Exception::STORAGE_INVALID_CONTENT_RANGE);
@@ -202,8 +202,27 @@ class Create extends Action
$metadata = ['content_type' => $deviceForLocal->getFileMimeType($fileTmpName)];
$completed = false;
$mergeUploadMetadata = function (array $stored, array $current): array {
$merged = \array_merge($stored, $current);
if (isset($stored['parts']) || isset($current['parts'])) {
$parts = $stored['parts'] ?? [];
foreach (($current['parts'] ?? []) as $part => $value) {
$parts[(int) $part] = $value;
}
\ksort($parts);
$merged['parts'] = $parts;
$merged['chunks'] = \count($parts);
}
return $merged;
};
$type = $request->getHeaderLine('x-sdk-language') === 'cli' ? 'cli' : 'manual';
try {
$locks($lockKey, 600, function () use (&$chunks, $dbForProject, $deploymentId, &$metadata, &$completed, $response): void {
$locks($lockKey, 600, function () use ($activate, &$chunks, $commands, $contentRange, $dbForProject, $deploymentId, $deviceForFunctions, $entrypoint, $fileSize, &$function, &$metadata, $path, $type, &$completed, $response): void {
$deployment = $dbForProject->getDocument('deployments', $deploymentId);
if (!$deployment->isEmpty()) {
@@ -220,6 +239,35 @@ class Create extends Action
return;
}
}
if ($deployment->isEmpty()) {
$deviceForFunctions->prepareUpload($path, $metadata['content_type'] ?? '', $chunks, $metadata);
if (!empty($contentRange)) {
$deployment = $dbForProject->createDocument('deployments', new Document([
'$id' => $deploymentId,
'$permissions' => [
Permission::read(Role::any()),
Permission::update(Role::any()),
Permission::delete(Role::any()),
],
'resourceInternalId' => $function->getSequence(),
'resourceId' => $function->getId(),
'resourceType' => 'functions',
'entrypoint' => $entrypoint,
'buildCommands' => $commands,
'startCommand' => $function->getAttribute('startCommand', ''),
'sourcePath' => $path,
'sourceSize' => $fileSize,
'totalSize' => $fileSize,
'sourceChunksTotal' => $chunks,
'sourceChunksUploaded' => 0,
'activate' => $activate,
'sourceMetadata' => $metadata,
'type' => $type
]));
}
}
}, timeout: 120.0);
} catch (LockContention) {
$response->addHeader('Retry-After', '5');
@@ -231,23 +279,21 @@ class Create extends Action
return;
}
$chunksUploaded = $deviceForFunctions->upload($fileTmpName, $path, $chunk, $chunks, $metadata);
$chunksUploaded = $deviceForFunctions->uploadChunk($fileTmpName, $path, $chunk, $chunks, $metadata);
if (empty($chunksUploaded)) {
throw new Exception(Exception::GENERAL_SERVER_ERROR, 'Failed moving file');
}
$type = $request->getHeader('x-sdk-language') === 'cli' ? 'cli' : 'manual';
try {
$locks($lockKey, 600, function () use ($activate, &$chunks, $chunksUploaded, $commands, $dbForProject, $deploymentId, $deviceForFunctions, $entrypoint, $fileSize, &$function, $functionId, $path, &$metadata, $platform, $project, $publisherForBuilds, $queueForEvents, $response, $type): void {
$locks($lockKey, 600, function () use ($activate, &$chunks, $chunksUploaded, $commands, $dbForProject, $deploymentId, $deviceForFunctions, $entrypoint, $fileSize, &$function, $functionId, $path, &$metadata, $mergeUploadMetadata, $platform, $project, $publisherForBuilds, $queueForEvents, $response, $type): void {
$deployment = $dbForProject->getDocument('deployments', $deploymentId);
$uploaded = 0;
if (!$deployment->isEmpty()) {
$chunks = $deployment->getAttribute('sourceChunksTotal', 1);
$uploaded = $deployment->getAttribute('sourceChunksUploaded', 0);
$metadata = \array_merge($deployment->getAttribute('sourceMetadata', []), $metadata);
$metadata = $mergeUploadMetadata($deployment->getAttribute('sourceMetadata', []), $metadata);
if ($uploaded === $chunks) {
$queueForEvents->reset();
@@ -259,9 +305,11 @@ class Create extends Action
}
}
$chunksUploaded = max($uploaded, $chunksUploaded);
$chunksUploaded = max($uploaded, $chunksUploaded, (int) ($metadata['chunks'] ?? 0));
if ($chunksUploaded === $chunks && $uploaded < $chunks) {
$deviceForFunctions->finalizeUpload($path, $chunks, $metadata);
if ($activate) {
// Remove deploy for all other deployments.
$activeDeployments = $dbForProject->find('deployments', [
@@ -303,12 +351,6 @@ class Create extends Action
'type' => $type
]));
$function = $dbForProject->updateDocument('functions', $function->getId(), new Document([
'latestDeploymentId' => $deployment->getId(),
'latestDeploymentInternalId' => $deployment->getSequence(),
'latestDeploymentCreatedAt' => $deployment->getCreatedAt(),
'latestDeploymentStatus' => $deployment->getAttribute('status', ''),
]));
} else {
$deployment = $dbForProject->updateDocument('deployments', $deploymentId, new Document([
'sourceSize' => $fileSize,
@@ -326,42 +368,10 @@ class Create extends Action
platform: $platform,
));
} else {
if ($deployment->isEmpty()) {
$deployment = $dbForProject->createDocument('deployments', new Document([
'$id' => $deploymentId,
'$permissions' => [
Permission::read(Role::any()),
Permission::update(Role::any()),
Permission::delete(Role::any()),
],
'resourceInternalId' => $function->getSequence(),
'resourceId' => $function->getId(),
'resourceType' => 'functions',
'entrypoint' => $entrypoint,
'buildCommands' => $commands,
'startCommand' => $function->getAttribute('startCommand', ''),
'sourcePath' => $path,
'sourceSize' => $fileSize,
'totalSize' => $fileSize,
'sourceChunksTotal' => $chunks,
'sourceChunksUploaded' => $chunksUploaded,
'activate' => $activate,
'sourceMetadata' => $metadata,
'type' => $type
]));
$function = $dbForProject->updateDocument('functions', $function->getId(), new Document([
'latestDeploymentId' => $deployment->getId(),
'latestDeploymentInternalId' => $deployment->getSequence(),
'latestDeploymentCreatedAt' => $deployment->getCreatedAt(),
'latestDeploymentStatus' => $deployment->getAttribute('status', ''),
]));
} else {
$deployment = $dbForProject->updateDocument('deployments', $deploymentId, new Document([
'sourceChunksUploaded' => $chunksUploaded,
'sourceMetadata' => $metadata,
]));
}
$deployment = $dbForProject->updateDocument('deployments', $deploymentId, new Document([
'sourceChunksUploaded' => $chunksUploaded,
'sourceMetadata' => $metadata,
]));
}
$metadata = null;
@@ -13,6 +13,7 @@ use Appwrite\SDK\Response as SDKResponse;
use Appwrite\Utopia\Response;
use Utopia\Database\Database;
use Utopia\Database\Document;
use Utopia\Database\Exception\Transaction as TransactionException;
use Utopia\Database\Query;
use Utopia\Database\Validator\UID;
use Utopia\Platform\Action;
@@ -89,8 +90,16 @@ class Delete extends Action
throw new Exception(Exception::DEPLOYMENT_NOT_FOUND);
}
if (!$dbForProject->deleteDocument('deployments', $deployment->getId())) {
throw new Exception(Exception::GENERAL_SERVER_ERROR, 'Failed to remove deployment from DB');
try {
if (!$dbForProject->deleteDocument('deployments', $deployment->getId())) {
throw new Exception(Exception::GENERAL_SERVER_ERROR, 'Failed to remove deployment from DB');
}
} catch (TransactionException) {
$deploymentExists = !$dbForProject->getDocument('deployments', $deployment->getId())->isEmpty();
if ($deploymentExists && !$dbForProject->deleteDocument('deployments', $deployment->getId())) {
throw new Exception(Exception::GENERAL_SERVER_ERROR, 'Failed to remove deployment from DB');
}
}
if (!empty($deployment->getAttribute('sourcePath', ''))) {
@@ -52,7 +52,8 @@ class Get extends Action
)
],
contentType: ContentType::ANY,
type: MethodType::LOCATION
type: MethodType::LOCATION,
locationAuth: ['Project', 'ImpersonateUserId'],
))
->param('functionId', '', fn (Database $dbForProject) => new UID($dbForProject->getAdapter()->getMaxUIDLength()), 'Function ID.', false, ['dbForProject'])
->param('deploymentId', '', fn (Database $dbForProject) => new UID($dbForProject->getAdapter()->getMaxUIDLength()), 'Deployment ID.', false, ['dbForProject'])
@@ -106,7 +107,7 @@ class Get extends Action
->addHeader('Content-Disposition', 'attachment; filename="' . $deploymentId . '-' . $type . '.tar.gz"');
$size = $device->getFileSize($path);
$rangeHeader = $request->getHeader('range');
$rangeHeader = $request->getHeaderLine('range');
if (!empty($rangeHeader)) {
$start = $request->getRangeStart();
@@ -120,18 +120,6 @@ class Create extends Action
'buildLogs' => '',
]));
$function = $function
->setAttribute('latestDeploymentId', $deployment->getId())
->setAttribute('latestDeploymentInternalId', $deployment->getSequence())
->setAttribute('latestDeploymentCreatedAt', $deployment->getCreatedAt())
->setAttribute('latestDeploymentStatus', $deployment->getAttribute('status', ''));
$dbForProject->updateDocument('functions', $function->getId(), new Document([
'latestDeploymentId' => $function->getAttribute('latestDeploymentId'),
'latestDeploymentInternalId' => $function->getAttribute('latestDeploymentInternalId'),
'latestDeploymentCreatedAt' => $function->getAttribute('latestDeploymentCreatedAt'),
'latestDeploymentStatus' => $function->getAttribute('latestDeploymentStatus'),
]));
$publisherForBuilds->enqueue(new BuildMessage(
project: $project,
resource: $function,
@@ -12,6 +12,7 @@ use Executor\Executor;
use Utopia\Database\Database;
use Utopia\Database\DateTime;
use Utopia\Database\Document;
use Utopia\Database\Exception\Transaction as TransactionException;
use Utopia\Database\Validator\UID;
use Utopia\Platform\Action;
use Utopia\Platform\Scope\HTTP;
@@ -91,17 +92,30 @@ class Update extends Action
$endTime = new \DateTime('now');
$duration = $endTime->getTimestamp() - $startTime->getTimestamp();
$deployment = $dbForProject->updateDocument('deployments', $deployment->getId(), new Document([
'buildEndedAt' => DateTime::now(),
'buildDuration' => $duration,
'status' => 'canceled'
]));
if ($deployment->getSequence() === $function->getAttribute('latestDeploymentInternalId', '')) {
$function = $function->setAttribute('latestDeploymentStatus', $deployment->getAttribute('status', ''));
$dbForProject->updateDocument('functions', $function->getId(), new Document([
'latestDeploymentStatus' => $function->getAttribute('latestDeploymentStatus'),
try {
$deployment = $dbForProject->updateDocument('deployments', $deployment->getId(), new Document([
'buildEndedAt' => DateTime::now(),
'buildDuration' => $duration,
'status' => 'canceled'
]));
} catch (TransactionException) {
$deployment = $dbForProject->getDocument('deployments', $deployment->getId());
if ($deployment->isEmpty()) {
throw new Exception(Exception::DEPLOYMENT_NOT_FOUND);
}
if (\in_array($deployment->getAttribute('status'), ['ready', 'failed'])) {
throw new Exception(Exception::BUILD_ALREADY_COMPLETED);
}
if ($deployment->getAttribute('status') !== 'canceled') {
$deployment = $dbForProject->updateDocument('deployments', $deployment->getId(), new Document([
'buildEndedAt' => DateTime::now(),
'buildDuration' => $duration,
'status' => 'canceled'
]));
}
}
try {
@@ -174,19 +174,6 @@ class Create extends Base
'activate' => $activate,
]));
$function = $function
->setAttribute('latestDeploymentId', $deployment->getId())
->setAttribute('latestDeploymentInternalId', $deployment->getSequence())
->setAttribute('latestDeploymentCreatedAt', $deployment->getCreatedAt())
->setAttribute('latestDeploymentStatus', $deployment->getAttribute('status', ''));
$dbForProject->updateDocument('functions', $function->getId(), new Document([
'latestDeploymentId' => $function->getAttribute('latestDeploymentId'),
'latestDeploymentInternalId' => $function->getAttribute('latestDeploymentInternalId'),
'latestDeploymentCreatedAt' => $function->getAttribute('latestDeploymentCreatedAt'),
'latestDeploymentStatus' => $function->getAttribute('latestDeploymentStatus'),
]));
$this->updateEmptyManualRule($project, $function, $deployment, $dbForPlatform, $authorization);
$publisherForBuilds->enqueue(new BuildMessage(
@@ -13,7 +13,6 @@ use Appwrite\Extend\Exception as AppwriteException;
use Appwrite\Functions\Validator\Headers;
use Appwrite\Platform\Modules\Compute\Base;
use Appwrite\SDK\AuthType;
use Appwrite\SDK\ContentType;
use Appwrite\SDK\Method;
use Appwrite\SDK\Response as SDKResponse;
use Appwrite\Usage\Context;
@@ -81,7 +80,6 @@ class Create extends Base
model: Response::MODEL_EXECUTION,
)
],
contentType: ContentType::MULTIPART,
))
->param('functionId', '', fn (Database $dbForProject) => new UID($dbForProject->getAdapter()->getMaxUIDLength()), 'Function ID.', false, ['dbForProject'])
->param('body', '', new Text(10485760, 0), 'HTTP body of execution. Default value is empty string.', true)
@@ -519,7 +517,7 @@ class Create extends Base
$execution->setAttribute('responseBody', $executionResponse['body'] ?? '');
$execution->setAttribute('responseHeaders', $headers);
$acceptTypes = \explode(', ', $request->getHeader('accept'));
$acceptTypes = \explode(', ', $request->getHeaderLine('accept'));
foreach ($acceptTypes as $acceptType) {
if (\str_starts_with($acceptType, 'application/json') || \str_starts_with($acceptType, 'application/*')) {
$response->setContentType(Response::CONTENT_TYPE_JSON);
@@ -308,7 +308,7 @@ class Create extends Base
]));
// Backwards compatibility with 1.6 behaviour
$requestFormat = $request->getHeader('x-appwrite-response-format', System::getEnv('_APP_SYSTEM_RESPONSE_FORMAT', ''));
$requestFormat = $request->getHeaderLine('x-appwrite-response-format', System::getEnv('_APP_SYSTEM_RESPONSE_FORMAT', ''));
if ($requestFormat && version_compare($requestFormat, '1.7.0', '<')) {
// build from template
$template = new Document([]);
@@ -344,12 +344,6 @@ class Create extends Base
referenceType: 'branch'
);
$function = $dbForProject->updateDocument('functions', $function->getId(), new Document([
'latestDeploymentId' => $deployment->getId(),
'latestDeploymentInternalId' => $deployment->getSequence(),
'latestDeploymentCreatedAt' => $deployment->getCreatedAt(),
'latestDeploymentStatus' => $deployment->getAttribute('status', ''),
]));
} elseif (!$template->isEmpty()) {
// Deploy non-VCS from template
$deploymentId = ID::unique();
@@ -370,13 +364,6 @@ class Create extends Base
'activate' => true,
]));
$function = $dbForProject->updateDocument('functions', $function->getId(), new Document([
'latestDeploymentId' => $deployment->getId(),
'latestDeploymentInternalId' => $deployment->getSequence(),
'latestDeploymentCreatedAt' => $deployment->getCreatedAt(),
'latestDeploymentStatus' => $deployment->getAttribute('status', ''),
]));
$publisherForBuilds->enqueue(new BuildMessage(
project: $project,
resource: $function,
@@ -182,6 +182,17 @@ class Update extends Base
$isConnected = !empty($function->getAttribute('providerRepositoryId', ''));
// Omitted providerRepositoryId (null) on a connected function — preserve existing VCS values
if ($isConnected && $providerRepositoryId === null) {
$providerRepositoryId = $function->getAttribute('providerRepositoryId', '');
$installationId = $function->getAttribute('installationId', '');
$providerBranch = $providerBranch ?: $function->getAttribute('providerBranch', '');
$providerRootDirectory = $providerRootDirectory ?: $function->getAttribute('providerRootDirectory', '');
$repositoryId = $function->getAttribute('repositoryId', '');
$repositoryInternalId = $function->getAttribute('repositoryInternalId', '');
$installation = $dbForPlatform->getDocument('installations', $installationId);
}
// Git disconnect logic. Disconnecting only when providerRepositoryId is empty, allowing for continue updates without disconnecting git
if ($isConnected && ($providerRepositoryId !== null && empty($providerRepositoryId))) {
$repositories = $dbForPlatform->find('repositories', [
@@ -30,6 +30,7 @@ use Utopia\Database\Exception\Conflict;
use Utopia\Database\Exception\Duplicate;
use Utopia\Database\Exception\Restricted;
use Utopia\Database\Exception\Structure;
use Utopia\Database\Exception\Transaction as TransactionException;
use Utopia\Database\Query;
use Utopia\Detector\Detection\Rendering\SSR;
use Utopia\Detector\Detection\Rendering\XStatic;
@@ -247,24 +248,32 @@ class Builds extends Action
->setParam('deploymentId', $deployment->getId());
if ($deployment->getAttribute('status') === 'canceled') {
$this->cancelDeployment($deployment->getId(), $dbForProject, $queueForRealtime);
$resource = $this->updateLatestDeployment($dbForProject, $resource);
$this->finalizeCanceledDeployment($deployment->getId(), $dbForProject, $queueForRealtime);
return;
}
$deploymentId = $deployment->getId();
$deployment->setAttribute('buildStartedAt', $startTime);
$deployment->setAttribute('status', 'processing');
$deployment = $dbForProject->updateDocument('deployments', $deployment->getId(), new Document([
$updated = $dbForProject->updateDocuments('deployments', new Document([
'buildStartedAt' => $startTime,
'status' => 'processing',
]));
]), [
Query::equal('$id', [$deploymentId]),
Query::notEqual('status', 'canceled'),
]);
if ($deployment->getSequence() === $resource->getAttribute('latestDeploymentInternalId', '')) {
$resource = $dbForProject->updateDocument($resource->getCollection(), $resource->getId(), new Document(['latestDeploymentStatus' => $deployment->getAttribute('status', '')]));
if ($updated === 0) {
$resource = $this->updateLatestDeployment($dbForProject, $resource);
$this->finalizeCanceledDeployment($deploymentId, $dbForProject, $queueForRealtime);
return;
}
$deployment = $dbForProject->getDocument('deployments', $deploymentId);
$resource = $this->updateLatestDeployment($dbForProject, $resource);
Span::add('deployment.status', 'processing');
$queueForRealtime
@@ -389,7 +398,7 @@ class Builds extends Action
Console::execute('mkdir -p ' . \escapeshellarg('/tmp/builds/' . $deploymentId), '', $stdout, $stderr);
if ($dbForProject->getDocument('deployments', $deploymentId)->getAttribute('status') === 'canceled') {
$this->cancelDeployment($deployment->getId(), $dbForProject, $queueForRealtime);
$this->finalizeCanceledDeployment($deployment->getId(), $dbForProject, $queueForRealtime);
return;
}
@@ -531,15 +540,22 @@ class Builds extends Action
}
/** Request the executor to build the code... */
$deployment->setAttribute('status', 'building');
$deployment = $dbForProject->updateDocument('deployments', $deployment->getId(), new Document([
$updated = $dbForProject->updateDocuments('deployments', new Document([
'status' => 'building',
]));
]), [
Query::equal('$id', [$deploymentId]),
Query::notEqual('status', 'canceled'),
]);
if ($updated === 0) {
$this->finalizeCanceledDeployment($deploymentId, $dbForProject, $queueForRealtime);
return;
}
$deployment = $dbForProject->getDocument('deployments', $deploymentId);
Span::add('deployment.status', 'building');
if ($deployment->getSequence() === $resource->getAttribute('latestDeploymentInternalId', '')) {
$resource = $dbForProject->updateDocument($resource->getCollection(), $resource->getId(), new Document(['latestDeploymentStatus' => $deployment->getAttribute('status', '')]));
}
$resource = $this->updateLatestDeployment($dbForProject, $resource);
$queueForRealtime
->setPayload($deployment->getArrayCopy())
@@ -670,11 +686,16 @@ class Builds extends Action
deployment: $deployment
);
$cacheKey = $this->getNodeModulesCacheKey($project, $resource, $runtime, $version, $command);
Span::add('build.node_modules_cache.enabled', $cacheKey !== '');
Span::add('build.node_modules_cache.key', $cacheKey);
$response = null;
$err = null;
if ($dbForProject->getDocument('deployments', $deploymentId)->getAttribute('status') === 'canceled') {
$this->cancelDeployment($deployment->getId(), $dbForProject, $queueForRealtime);
$this->finalizeCanceledDeployment($deployment->getId(), $dbForProject, $queueForRealtime);
return;
}
@@ -683,32 +704,14 @@ class Builds extends Action
$span = Span::current();
Co::join([
Co\go(function () use ($executor, &$response, $project, $deployment, $source, $resource, $runtime, $vars, $command, $cpus, $memory, $timeout, &$err, $version, $span) {
Co\go(function () use ($executor, &$response, $project, $deployment, $source, $resource, $runtime, $vars, $command, $cacheKey, $cpus, $memory, $timeout, &$err, $version, $span) {
try {
if ($version === 'v2') {
$command = 'tar -zxf /tmp/code.tar.gz -C /usr/code && cd /usr/local/src/ && ./build.sh';
} else {
$outputDirectory = $deployment->getAttribute('buildOutput') ?? $resource->getAttribute('outputDirectory');
if ($resource->getCollection() === 'sites') {
$listFilesCommand = '';
// Start separation, enter build folder
$listFilesCommand .= 'echo "{APPWRITE_DETECTION_SEPARATOR_START}" && cd /usr/local/build';
// Enter output directory, if set
if (! empty($outputDirectory)) {
$listFilesCommand .= ' && cd ' . \escapeshellarg($outputDirectory);
}
// Print files, and end separation
$listFilesCommand .= ' && find . -name \'node_modules\' -prune -o -type f -print && echo "{APPWRITE_DETECTION_SEPARATOR_END}"';
// Use SSR file listing
if (empty($command)) {
$command = $listFilesCommand;
} else {
$command .= ' && ' . $listFilesCommand;
}
$command = $this->prepareSiteBuildCommand($command, $outputDirectory ?? '');
}
$command = 'tar -zxf /tmp/code.tar.gz -C /mnt/code && helpers/build.sh ' . \trim(\escapeshellarg($command));
@@ -728,7 +731,8 @@ class Builds extends Action
destination: APP_STORAGE_BUILDS . "/app-{$project->getId()}",
variables: $vars,
command: $command,
outputDirectory: $outputDirectory ?? ''
outputDirectory: $outputDirectory ?? '',
cacheKey: $cacheKey
);
} catch (ExecutorTimeout $error) {
@@ -809,6 +813,10 @@ class Builds extends Action
$streamLog = \str_replace('{APPWRITE_LINEBREAK_PLACEHOLDER}', "\n", $streamLog);
$streamParts = \explode(' ', $streamLog, 2);
if (! isset($streamParts[1])) {
continue;
}
// TODO: use part[0] as timestamp when switching to dbForLogs for build logs
$currentLogs .= $streamParts[1];
@@ -843,7 +851,7 @@ class Builds extends Action
$latestDeployment = $dbForProject->getDocument('deployments', $deploymentId);
if ($latestDeployment->getAttribute('status') === 'canceled') {
$this->cancelDeployment($deployment->getId(), $dbForProject, $queueForRealtime);
$this->finalizeCanceledDeployment($deployment->getId(), $dbForProject, $queueForRealtime);
return;
}
@@ -871,31 +879,13 @@ class Builds extends Action
$logs .= $log['content'];
}
// Separate logs for SSR detection
$detectionLogs = '';
if (\str_contains($logs, '{APPWRITE_DETECTION_SEPARATOR_START}')) {
[$logsBefore, $detectionLogsStart] = \explode('{APPWRITE_DETECTION_SEPARATOR_START}', $logs, 2);
[$detectionLogs, $logsAfter] = \explode('{APPWRITE_DETECTION_SEPARATOR_END}', $detectionLogsStart, 2);
$logs = $logsBefore . $logsAfter;
}
['logs' => $logs, 'detectionLogs' => $detectionLogs] = $this->splitSiteDetectionLogs($logs);
$deployment->setAttribute('buildLogs', $logs);
$adapter = null;
if ($resource->getCollection() === 'sites' && ! empty($detectionLogs)) {
$files = \explode("\n", $detectionLogs); // Parse output
$files = \array_filter($files); // Remove empty
$files = \array_map(fn ($file) => \trim($file), $files); // Remove whitepsaces
$files = \array_map(fn ($file) => \str_starts_with($file, './') ? \substr($file, 2) : $file, $files); // Remove beginning ./
$detector = new Rendering($resource->getAttribute('framework', ''));
foreach ($files as $file) {
$detector->addInput($file);
}
$detector
->addOption(new SSR())
->addOption(new XStatic());
$detection = $detector->detect();
$detection = $this->detectSiteRendering($resource->getAttribute('framework', ''), $detectionLogs);
$adapter = $resource->getAttribute('adapter', '');
if (empty($adapter)) {
@@ -930,16 +920,21 @@ class Builds extends Action
$deployment->setAttribute('buildLogs', $logs);
/** Update the status */
$endTime = DateTime::now();
$durationEnd = \microtime(true);
$deployment->setAttribute('buildEndedAt', $endTime);
$deployment->setAttribute('buildDuration', \intval(\ceil($durationEnd - $durationStart)));
$deployment->setAttribute('status', 'ready');
$deployment = $dbForProject->updateDocument('deployments', $deploymentId, new Document([
'buildEndedAt' => $deployment->getAttribute('buildEndedAt'),
'buildDuration' => $deployment->getAttribute('buildDuration'),
'buildLogs' => $deployment->getAttribute('buildLogs'),
'status' => 'ready',
]));
Span::add('deployment.status', 'ready');
Span::add('build.duration', $deployment->getAttribute('buildDuration'));
if ($deployment->getSequence() === $resource->getAttribute('latestDeploymentInternalId', '')) {
$resource = $dbForProject->updateDocument($resource->getCollection(), $resource->getId(), new Document(['latestDeploymentStatus' => $deployment->getAttribute('status', '')]));
}
$resource = $this->updateLatestDeployment($dbForProject, $resource);
if ($isVcsEnabled) {
$this->runGitAction('ready', $github, $providerCommitHash, $owner, $repositoryName, $project, $resource, $deployment->getId(), $dbForProject, $dbForPlatform, $queueForRealtime, $platform);
@@ -1026,6 +1021,8 @@ class Builds extends Action
Span::add('build.activated', true);
}
$resource = $this->updateLatestDeployment($dbForProject, $resource);
$this->afterDeploymentSuccess(
$project,
$deployment,
@@ -1095,19 +1092,12 @@ class Builds extends Action
}
}
$endTime = DateTime::now();
$durationEnd = \microtime(true);
$deployment = $dbForProject->updateDocument('deployments', $deployment->getId(), new Document([
'buildEndedAt' => $endTime,
'buildDuration' => \intval(\ceil($durationEnd - $durationStart)),
]));
Span::add('build.duration', $deployment->getAttribute('buildDuration'));
$queueForRealtime
->setPayload($deployment->getArrayCopy())
->trigger();
if ($dbForProject->getDocument('deployments', $deploymentId)->getAttribute('status') === 'canceled') {
$this->cancelDeployment($deployment->getId(), $dbForProject, $queueForRealtime);
$this->finalizeCanceledDeployment($deployment->getId(), $dbForProject, $queueForRealtime);
return;
}
@@ -1139,7 +1129,7 @@ class Builds extends Action
}
} catch (\Throwable $th) {
if ($dbForProject->getDocument('deployments', $deploymentId)->getAttribute('status') === 'canceled') {
$this->cancelDeployment($deployment->getId(), $dbForProject, $queueForRealtime);
$this->finalizeCanceledDeployment($deployment->getId(), $dbForProject, $queueForRealtime);
return;
}
@@ -1182,9 +1172,7 @@ class Builds extends Action
'buildLogs' => $message,
]));
if ($deployment->getSequence() === $resource->getAttribute('latestDeploymentInternalId', '')) {
$resource = $dbForProject->updateDocument($resource->getCollection(), $resource->getId(), new Document(['latestDeploymentStatus' => $deployment->getAttribute('status', '')]));
}
$resource = $this->updateLatestDeployment($dbForProject, $resource);
$queueForRealtime
->setPayload($deployment->getArrayCopy())
@@ -1333,6 +1321,79 @@ class Builds extends Action
return '';
}
protected function getNodeModulesCacheKey(Document $project, Document $resource, array $runtime, string $version, string $command): string
{
if ($version !== 'v5' || $command === '' || $command === '0') {
return '';
}
$hashContext = [
'version' => 'v1',
'projectId' => $project->getId(),
'resourceType' => $resource->getCollection(),
'resourceId' => $resource->getId(),
'runtime' => $runtime['image'] ?? '',
];
return \substr(\hash('sha256', \json_encode($hashContext, JSON_THROW_ON_ERROR)), 0, 48);
}
protected function prepareSiteBuildCommand(string $command, string $outputDirectory): string
{
$listFilesCommand = 'echo "{APPWRITE_DETECTION_SEPARATOR_START}" && cd /usr/local/build';
if (! empty($outputDirectory)) {
$listFilesCommand .= ' && cd ' . \escapeshellarg($outputDirectory);
}
$listFilesCommand .= ' && find . -name \'node_modules\' -prune -o -type f -print && echo "{APPWRITE_DETECTION_SEPARATOR_END}"';
if (empty($command)) {
return $listFilesCommand;
}
return "{$command} && {$listFilesCommand}";
}
/**
* @return array{logs: string, detectionLogs: string}
*/
protected function splitSiteDetectionLogs(string $logs): array
{
if (! \str_contains($logs, '{APPWRITE_DETECTION_SEPARATOR_START}')) {
return [
'logs' => $logs,
'detectionLogs' => '',
];
}
[$logsBefore, $detectionLogsStart] = \explode('{APPWRITE_DETECTION_SEPARATOR_START}', $logs, 2);
[$detectionLogs, $logsAfter] = \explode('{APPWRITE_DETECTION_SEPARATOR_END}', $detectionLogsStart, 2);
return [
'logs' => "{$logsBefore}{$logsAfter}",
'detectionLogs' => $detectionLogs,
];
}
protected function detectSiteRendering(string $framework, string $detectionLogs): object
{
$files = \explode("\n", $detectionLogs);
$files = \array_filter($files);
$files = \array_map(\trim(...), $files);
$files = \array_map(fn ($file) => \str_starts_with($file, './') ? \substr($file, 2) : $file, $files);
$detector = new Rendering($framework);
foreach ($files as $file) {
$detector->addInput($file);
}
return $detector
->addOption(new SSR())
->addOption(new XStatic())
->detect();
}
/**
* @throws Structure
* @throws \Utopia\Database\Exception
@@ -1472,20 +1533,61 @@ class Builds extends Action
}
}
private function cancelDeployment(string $deploymentId, Database $dbForProject, Realtime $queueForRealtime)
private function updateLatestDeployment(Database $dbForProject, Document $resource): Document
{
$latestDeployment = $dbForProject->findOne('deployments', [
Query::equal('resourceType', [$resource->getCollection()]),
Query::equal('resourceInternalId', [$resource->getSequence()]),
Query::orderDesc('$createdAt'),
]);
$updates = $latestDeployment->isEmpty()
? [
'latestDeploymentCreatedAt' => '',
'latestDeploymentInternalId' => '',
'latestDeploymentId' => '',
'latestDeploymentStatus' => '',
]
: [
'latestDeploymentCreatedAt' => $latestDeployment->getCreatedAt(),
'latestDeploymentInternalId' => $latestDeployment->getSequence(),
'latestDeploymentId' => $latestDeployment->getId(),
'latestDeploymentStatus' => $latestDeployment->getAttribute('status', ''),
];
return $dbForProject->updateDocument(
$resource->getCollection(),
$resource->getId(),
new Document($updates)
);
}
private function finalizeCanceledDeployment(string $deploymentId, Database $dbForProject, Realtime $queueForRealtime)
{
Span::add('deployment.status', 'canceled');
$deployment = $dbForProject->getDocument('deployments', $deploymentId);
$attempts = 0;
$logs = $deployment->getAttribute('buildLogs', '');
$date = \date('H:i:s');
$logs .= "\033[90m[$date] \033[90m[\033[0mappwrite\033[90m]\033[33m Build has been canceled. \033[0m\n";
while (true) {
try {
$deployment = $dbForProject->getDocument('deployments', $deploymentId);
$deployment->setAttribute('buildLogs', $logs);
$deployment = $dbForProject->updateDocument('deployments', $deployment->getId(), new Document([
'buildLogs' => $deployment->getAttribute('buildLogs'),
]));
$logs = $deployment->getAttribute('buildLogs', '');
$date = \date('H:i:s');
$logs .= "\033[90m[$date] \033[90m[\033[0mappwrite\033[90m]\033[33m Build has been canceled. \033[0m\n";
$deployment->setAttribute('buildLogs', $logs);
$deployment = $dbForProject->updateDocument('deployments', $deployment->getId(), new Document([
'buildLogs' => $deployment->getAttribute('buildLogs'),
]));
break;
} catch (TransactionException $exception) {
if (++$attempts >= 5) {
throw $exception;
}
}
}
$queueForRealtime
->setPayload($deployment->getArrayCopy())
@@ -10,7 +10,6 @@ use Appwrite\Role;
use Exception;
use Utopia\Compression\Compression;
use Utopia\Config\Config;
use Utopia\Console;
use Utopia\Database\Database;
use Utopia\Database\Document;
use Utopia\Database\Helpers\ID;
@@ -18,6 +17,7 @@ use Utopia\Database\Query;
use Utopia\Fetch\Client as FetchClient;
use Utopia\Platform\Action;
use Utopia\Queue\Message;
use Utopia\Span\Span;
use Utopia\Storage\Device;
use Utopia\System\System;
use Utopia\Telemetry\Adapter as Telemetry;
@@ -59,7 +59,7 @@ class Screenshots extends Action
Device $deviceForFiles,
Telemetry $telemetry
): void {
Console::log('Screenshot action started');
Span::add('project.id', $project->getId());
$payload = $message->getPayload();
@@ -70,9 +70,8 @@ class Screenshots extends Action
$screenshotMessage = Screenshot::fromArray($payload);
$counter = $telemetry->createCounter('worker.screenshots.capture');
Console::log('Site screenshot started');
$deploymentId = $screenshotMessage->deploymentId;
Span::add('deployment.id', $deploymentId);
$deployment = $dbForProject->getDocument('deployments', $deploymentId);
if ($deployment->isEmpty()) {
@@ -80,12 +79,15 @@ class Screenshots extends Action
}
$siteId = $deployment->getAttribute('resourceId');
Span::add('site.id', $siteId);
$site = $dbForProject->getDocument('sites', $siteId);
if ($site->isEmpty()) {
throw new \Exception('Site not found');
}
Span::add('site.framework', $site->getAttribute('framework', ''));
// Realtime preparation
$event = "sites.[siteId].deployments.[deploymentId].update";
$queueForRealtime
@@ -205,6 +207,8 @@ class Screenshots extends Action
throw new \Exception($screenshotError);
}
Span::add('screenshot.count', \count($screenshots));
$mimeType = "image/png";
$updates = new Document([]);
@@ -268,10 +272,6 @@ class Screenshots extends Action
'deploymentScreenshotLight' => $deployment->getAttribute('screenshotLight', ''),
]));
} catch (\Throwable $th) {
Console::warning("Screenshot failed to generate:");
Console::warning($th->getMessage());
Console::warning($th->getTraceAsString());
$date = \date('H:i:s');
$this->appendToLogs($dbForProject, $deployment->getId(), $queueForRealtime, "[$date] [appwrite] Screenshot capturing failed. Deployment will continue. \n");
@@ -9,6 +9,7 @@ use Appwrite\Platform\Action;
use Appwrite\SDK\AuthType;
use Appwrite\SDK\Method;
use Appwrite\SDK\Response as SDKResponse;
use Appwrite\SDK\Specification\Validator\PasswordFormat;
use Appwrite\Utopia\Response;
use Utopia\Database\Database;
use Utopia\Database\Document;
@@ -60,7 +61,7 @@ class Create extends Action
->param('adminSecret', '', new Text(512), 'Source\'s Admin Secret')
->param('database', '', new Text(512), 'Source\'s Database Name')
->param('username', '', new Text(512), 'Source\'s Database Username')
->param('password', '', new Text(512), 'Source\'s Database Password')
->param('password', '', new PasswordFormat(new Text(512)), 'Source\'s Database Password')
->param('port', 5432, new Integer(true), 'Source\'s Database Port', true)
->inject('response')
->inject('dbForProject')
@@ -7,6 +7,7 @@ use Appwrite\Platform\Action;
use Appwrite\SDK\AuthType;
use Appwrite\SDK\Method;
use Appwrite\SDK\Response as SDKResponse;
use Appwrite\SDK\Specification\Validator\PasswordFormat;
use Appwrite\Utopia\Response;
use Utopia\Database\Document;
use Utopia\Migration\Sources\NHost;
@@ -53,7 +54,7 @@ class Get extends Action
->param('adminSecret', '', new Text(512), 'Source\'s Admin Secret.')
->param('database', '', new Text(512), 'Source\'s Database Name.')
->param('username', '', new Text(512), 'Source\'s Database Username.')
->param('password', '', new Text(512), 'Source\'s Database Password.')
->param('password', '', new PasswordFormat(new Text(512)), 'Source\'s Database Password.')
->param('port', 5432, new Integer(true), 'Source\'s Database Port.', true)
->inject('response')
->callback($this->action(...));
@@ -9,6 +9,7 @@ use Appwrite\Platform\Action;
use Appwrite\SDK\AuthType;
use Appwrite\SDK\Method;
use Appwrite\SDK\Response as SDKResponse;
use Appwrite\SDK\Specification\Validator\PasswordFormat;
use Appwrite\Utopia\Response;
use Utopia\Database\Database;
use Utopia\Database\Document;
@@ -60,7 +61,7 @@ class Create extends Action
->param('apiKey', '', new Text(512), 'Source\'s API Key')
->param('databaseHost', '', new Text(512), 'Source\'s Database Host')
->param('username', '', new Text(512), 'Source\'s Database Username')
->param('password', '', new Text(512), 'Source\'s Database Password')
->param('password', '', new PasswordFormat(new Text(512)), 'Source\'s Database Password')
->param('port', 5432, new Integer(true), 'Source\'s Database Port', true)
->inject('response')
->inject('dbForProject')
@@ -7,6 +7,7 @@ use Appwrite\Platform\Action;
use Appwrite\SDK\AuthType;
use Appwrite\SDK\Method;
use Appwrite\SDK\Response as SDKResponse;
use Appwrite\SDK\Specification\Validator\PasswordFormat;
use Appwrite\Utopia\Response;
use Utopia\Database\Document;
use Utopia\Migration\Sources\Supabase;
@@ -53,7 +54,7 @@ class Get extends Action
->param('apiKey', '', new Text(512), 'Source\'s API Key.')
->param('databaseHost', '', new Text(512), 'Source\'s Database Host.')
->param('username', '', new Text(512), 'Source\'s Database Username.')
->param('password', '', new Text(512), 'Source\'s Database Password.')
->param('password', '', new PasswordFormat(new Text(512)), 'Source\'s Database Password.')
->param('port', 5432, new Integer(true), 'Source\'s Database Port.', true)
->inject('response')
->callback($this->action(...));
@@ -94,6 +94,7 @@ class Create extends Action
'disposableEmails' => false,
'canonicalEmails' => false,
'freeEmails' => false,
'corporateEmails' => false,
'mockNumbers' => [],
'sessionAlerts' => false,
'membershipsUserName' => false,
@@ -156,7 +156,7 @@ class Upsert extends PlatformAction
if (empty($userInternalId)) {
throw new Exception(Exception::GENERAL_SERVER_ERROR, 'Failed to resolve valid user internal ID.');
}
$isGraphQL = $request->getHeader('x-appwrite-source') === 'graphql';
$isGraphQL = $request->getHeaderLine('x-appwrite-source') === 'graphql';
$presenceData = [
'userInternalId' => $userInternalId,
@@ -77,6 +77,7 @@ class Get extends Action
return [
'password-dictionary' => Response::MODEL_POLICY_PASSWORD_DICTIONARY,
'password-history' => Response::MODEL_POLICY_PASSWORD_HISTORY,
'password-strength' => Response::MODEL_POLICY_PASSWORD_STRENGTH,
'password-personal-data' => Response::MODEL_POLICY_PASSWORD_PERSONAL_DATA,
'session-alert' => Response::MODEL_POLICY_SESSION_ALERT,
'session-duration' => Response::MODEL_POLICY_SESSION_DURATION,
@@ -109,6 +110,18 @@ class Get extends Action
]),
Response::MODEL_POLICY_PASSWORD_HISTORY,
],
'password-strength' => [
new Document(\array_merge([
'min' => 8,
'uppercase' => false,
'lowercase' => false,
'number' => false,
'symbols' => false,
], $auths['passwordStrength'] ?? [], [
'$id' => 'password-strength',
])),
Response::MODEL_POLICY_PASSWORD_STRENGTH,
],
'password-personal-data' => [
new Document([
'$id' => 'password-personal-data',
@@ -159,6 +172,7 @@ class Get extends Action
'userPhone' => $auths['membershipsUserPhone'] ?? false,
'userName' => $auths['membershipsUserName'] ?? false,
'userMFA' => $auths['membershipsMfa'] ?? false,
'userAccessedAt' => $auths['membershipsUserAccessedAt'] ?? false,
]),
Response::MODEL_POLICY_MEMBERSHIP_PRIVACY,
],
@@ -55,6 +55,7 @@ class Update extends Action
->param('userPhone', null, new Boolean(), 'Set to true if you want make user phone number visible to all team members, or false to hide it.', optional: true)
->param('userName', null, new Boolean(), 'Set to true if you want make user name visible to all team members, or false to hide it.', optional: true)
->param('userMFA', null, new Boolean(), 'Set to true if you want make user MFA status visible to all team members, or false to hide it.', optional: true)
->param('userAccessedAt', null, new Boolean(), 'Set to true if you want make user last access time visible to all team members, or false to hide it.', optional: true)
->inject('response')
->inject('dbForPlatform')
->inject('project')
@@ -69,6 +70,7 @@ class Update extends Action
?bool $userPhone,
?bool $userName,
?bool $userMFA,
?bool $userAccessedAt,
Response $response,
Database $dbForPlatform,
Document $project,
@@ -92,6 +94,9 @@ class Update extends Action
if ($userMFA !== null) {
$auths['membershipsMfa'] = $userMFA;
}
if ($userAccessedAt !== null) {
$auths['membershipsUserAccessedAt'] = $userAccessedAt;
}
$updates = new Document([
'auths' => $auths,
@@ -0,0 +1,115 @@
<?php
namespace Appwrite\Platform\Modules\Project\Http\Project\Policies\PasswordStrength;
use Appwrite\Event\Event;
use Appwrite\Platform\Action;
use Appwrite\SDK\AuthType;
use Appwrite\SDK\Method;
use Appwrite\SDK\Response as SDKResponse;
use Appwrite\Utopia\Response;
use Utopia\Database\Database;
use Utopia\Database\Document;
use Utopia\Database\Validator\Authorization;
use Utopia\Platform\Scope\HTTP;
use Utopia\Validator\Boolean;
use Utopia\Validator\Range;
class Update extends Action
{
use HTTP;
public static function getName()
{
return 'updateProjectPasswordStrengthPolicy';
}
public function __construct()
{
$this
->setHttpMethod(Action::HTTP_REQUEST_METHOD_PATCH)
->setHttpPath('/v1/project/policies/password-strength')
->desc('Update password strength policy')
->groups(['api', 'project'])
->label('scope', ['policies.write', 'project.policies.write'])
->label('event', 'projects.[projectId].policies.[policy].update')
->label('audits.event', 'projects.[projectId].policies.[policy].update')
->label('audits.resource', 'project/{response.$id}')
->label('sdk', new Method(
namespace: 'project',
group: 'policies',
name: 'updatePasswordStrengthPolicy',
description: <<<'EOT'
Update the password strength requirements for users in the project.
EOT,
auth: [AuthType::ADMIN, AuthType::KEY],
responses: [
new SDKResponse(
code: Response::STATUS_CODE_OK,
model: Response::MODEL_POLICY_PASSWORD_STRENGTH,
),
],
))
->param('min', null, new Range(8, 256), 'Minimum password length. Value must be between 8 and 256. Default is 8.', optional: true)
->param('uppercase', null, new Boolean(), 'Whether passwords must include at least one uppercase letter.', optional: true)
->param('lowercase', null, new Boolean(), 'Whether passwords must include at least one lowercase letter.', optional: true)
->param('number', null, new Boolean(), 'Whether passwords must include at least one number.', optional: true)
->param('symbols', null, new Boolean(), 'Whether passwords must include at least one symbol.', optional: true)
->inject('response')
->inject('dbForPlatform')
->inject('project')
->inject('authorization')
->inject('queueForEvents')
->callback($this->action(...));
}
public function action(
?int $min,
?bool $uppercase,
?bool $lowercase,
?bool $number,
?bool $symbols,
Response $response,
Database $dbForPlatform,
Document $project,
Authorization $authorization,
Event $queueForEvents,
): void {
$auths = $project->getAttribute('auths', []);
$auths['passwordStrength'] = \array_merge([
'min' => 8,
'uppercase' => false,
'lowercase' => false,
'number' => false,
'symbols' => false,
], $auths['passwordStrength'] ?? []);
if ($min !== null) {
$auths['passwordStrength']['min'] = $min;
}
if ($uppercase !== null) {
$auths['passwordStrength']['uppercase'] = $uppercase;
}
if ($lowercase !== null) {
$auths['passwordStrength']['lowercase'] = $lowercase;
}
if ($number !== null) {
$auths['passwordStrength']['number'] = $number;
}
if ($symbols !== null) {
$auths['passwordStrength']['symbols'] = $symbols;
}
$project = $authorization->skip(fn () => $dbForPlatform->updateDocument('projects', $project->getId(), new Document([
'auths' => $auths,
])));
$queueForEvents
->setParam('projectId', $project->getId())
->setParam('policy', 'password-strength');
$response->dynamic(new Document(\array_merge($auths['passwordStrength'], [
'$id' => 'password-strength',
])), Response::MODEL_POLICY_PASSWORD_STRENGTH);
}
}
@@ -103,6 +103,15 @@ class XList extends Action
'$id' => 'password-history',
'total' => $auths['passwordHistory'] ?? 0,
]),
new Document(\array_merge([
'min' => 8,
'uppercase' => false,
'lowercase' => false,
'number' => false,
'symbols' => false,
], $auths['passwordStrength'] ?? [], [
'$id' => 'password-strength',
])),
new Document([
'$id' => 'password-personal-data',
'enabled' => $auths['personalDataCheck'] ?? false,
@@ -10,6 +10,7 @@ use Appwrite\SDK\AuthType;
use Appwrite\SDK\ContentType;
use Appwrite\SDK\Method;
use Appwrite\SDK\Response as SDKResponse;
use Appwrite\SDK\Specification\Validator\PasswordFormat;
use Appwrite\Template\Template;
use Appwrite\Utopia\Response;
use Appwrite\Utopia\Response as UtopiaResponse;
@@ -64,7 +65,7 @@ class Create extends Action
->param('host', '', new Hostname(), 'SMTP server host name', optional: true, deprecated: true) // Backwards compatibility
->param('port', null, new Integer(), 'SMTP server port', optional: true, deprecated: true) // Backwards compatibility
->param('username', '', new Text(256), 'SMTP server username', optional: true, deprecated: true) // Backwards compatibility
->param('password', '', new Text(256), 'SMTP server password', optional: true, deprecated: true) // Backwards compatibility
->param('password', '', new PasswordFormat(new Text(256)), 'SMTP server password', optional: true, deprecated: true) // Backwards compatibility
->param('secure', '', new WhiteList(['tls', 'ssl'], true), 'Does SMTP server use secure connection', optional: true, deprecated: true) // Backwards compatibility
->inject('response')
->inject('project')
@@ -7,6 +7,7 @@ use Appwrite\Platform\Action;
use Appwrite\SDK\AuthType;
use Appwrite\SDK\Method;
use Appwrite\SDK\Response as SDKResponse;
use Appwrite\SDK\Specification\Validator\PasswordFormat;
use Appwrite\Utopia\Response;
use PHPMailer\PHPMailer\PHPMailer;
use Throwable;
@@ -62,7 +63,7 @@ class Update extends Action
->param('host', null, new Nullable(new Hostname()), 'SMTP server hostname (domain)', optional: true)
->param('port', null, new Nullable(new Integer()), 'SMTP server port', optional: true)
->param('username', null, new Nullable(new Text(256, 0)), 'SMTP server username. Pass an empty string to clear a previously set value.', optional: true)
->param('password', null, new Nullable(new Text(256, 0)), 'SMTP server password. Pass an empty string to clear a previously set value. This property is stored securely and cannot be read in future (write-only).', optional: true)
->param('password', null, new Nullable(new PasswordFormat(new Text(256, 0))), 'SMTP server password. Pass an empty string to clear a previously set value. This property is stored securely and cannot be read in future (write-only).', optional: true)
->param('senderEmail', null, new Nullable(new Email(allowEmpty: true)), 'Email address shown in inbox as the sender of the email. Pass an empty string to clear a previously set value.', optional: true)
->param('senderName', null, new Nullable(new Text(256, 0)), 'Name shown in inbox as the sender of the email. Pass an empty string to clear a previously set value.', optional: true)
->param('replyToEmail', null, new Nullable(new Email(allowEmpty: true)), 'Email used when user replies to the email. Pass an empty string to clear a previously set value.', optional: true)
@@ -80,6 +80,7 @@ use Appwrite\Platform\Modules\Project\Http\Project\Policies\MembershipPrivacy\Up
use Appwrite\Platform\Modules\Project\Http\Project\Policies\PasswordDictionary\Update as UpdatePasswordDictionaryPolicy;
use Appwrite\Platform\Modules\Project\Http\Project\Policies\PasswordHistory\Update as UpdatePasswordHistoryPolicy;
use Appwrite\Platform\Modules\Project\Http\Project\Policies\PasswordPersonalData\Update as UpdatePasswordPersonalDataPolicy;
use Appwrite\Platform\Modules\Project\Http\Project\Policies\PasswordStrength\Update as UpdatePasswordStrengthPolicy;
use Appwrite\Platform\Modules\Project\Http\Project\Policies\SessionAlert\Update as UpdateSessionAlertPolicy;
use Appwrite\Platform\Modules\Project\Http\Project\Policies\SessionDuration\Update as UpdateSessionDurationPolicy;
use Appwrite\Platform\Modules\Project\Http\Project\Policies\SessionInvalidation\Update as UpdateSessionInvalidationPolicy;
@@ -168,6 +169,7 @@ class Http extends Service
$this->addAction(UpdateMembershipPrivacyPolicy::getName(), new UpdateMembershipPrivacyPolicy());
$this->addAction(UpdatePasswordDictionaryPolicy::getName(), new UpdatePasswordDictionaryPolicy());
$this->addAction(UpdatePasswordHistoryPolicy::getName(), new UpdatePasswordHistoryPolicy());
$this->addAction(UpdatePasswordStrengthPolicy::getName(), new UpdatePasswordStrengthPolicy());
$this->addAction(UpdatePasswordPersonalDataPolicy::getName(), new UpdatePasswordPersonalDataPolicy());
$this->addAction(UpdateSessionAlertPolicy::getName(), new UpdateSessionAlertPolicy());
$this->addAction(UpdateSessionDurationPolicy::getName(), new UpdateSessionDurationPolicy());
@@ -83,6 +83,13 @@ class Create extends Action
$auths = [
'limit' => 0,
'maxSessions' => 0,
'passwordStrength' => [
'min' => 8,
'uppercase' => false,
'lowercase' => false,
'number' => false,
'symbols' => false,
],
'passwordHistory' => 0,
'passwordDictionary' => false,
'duration' => TOKEN_EXPIRATION_LOGIN_LONG,
@@ -90,6 +97,7 @@ class Create extends Action
'disposableEmails' => false,
'canonicalEmails' => false,
'freeEmails' => false,
'corporateEmails' => false,
'mockNumbers' => [],
'sessionAlerts' => false,
'membershipsUserName' => false,
@@ -54,10 +54,11 @@ class Action extends PlatformAction
return $dbForPlatform->getDocument('rules', $rule->getId());
});
if (
$existingRule->isEmpty() ||
$existingRule->getAttribute('domain', '') !== $rule->getAttribute('domain', '')
) {
if ($existingRule->isEmpty()) {
return true;
}
if ($existingRule->getAttribute('domain', '') !== $rule->getAttribute('domain', '')) {
return false;
}
@@ -40,7 +40,7 @@ class Create extends Action
->setHttpMethod(Action::HTTP_REQUEST_METHOD_POST)
->setHttpPath('/v1/proxy/rules/redirect')
->groups(['api', 'proxy'])
->desc('Create Redirect rule')
->desc('Create redirect rule')
->label('scope', 'rules.write')
->label('event', 'rules.[ruleId].create')
->label('audits.event', 'rule.create')
@@ -166,7 +166,7 @@ class Create extends Action
throw new Exception(Exception::STORAGE_FILE_TYPE_UNSUPPORTED);
}
$contentRange = $request->getHeader('content-range');
$contentRange = $request->getHeaderLine('content-range');
$deploymentId = ID::unique();
$chunk = 1;
$chunks = 1;
@@ -175,7 +175,7 @@ class Create extends Action
$start = $request->getContentRangeStart();
$end = $request->getContentRangeEnd();
$fileSize = $request->getContentRangeSize();
$deploymentId = $request->getHeader('x-appwrite-id', $deploymentId);
$deploymentId = $request->getHeaderLine('x-appwrite-id', $deploymentId);
// TODO make `end >= $fileSize` in next breaking version
if (is_null($start) || is_null($end) || is_null($fileSize) || $end > $fileSize) {
throw new Exception(Exception::STORAGE_INVALID_CONTENT_RANGE);
@@ -202,8 +202,35 @@ class Create extends Action
$metadata = ['content_type' => $deviceForLocal->getFileMimeType($fileTmpName)];
$completed = false;
$mergeUploadMetadata = function (array $stored, array $current): array {
$merged = \array_merge($stored, $current);
if (isset($stored['parts']) || isset($current['parts'])) {
$parts = $stored['parts'] ?? [];
foreach (($current['parts'] ?? []) as $part => $value) {
$parts[(int) $part] = $value;
}
\ksort($parts);
$merged['parts'] = $parts;
$merged['chunks'] = \count($parts);
}
return $merged;
};
$type = $request->getHeaderLine('x-sdk-language') === 'cli' ? 'cli' : 'manual';
$commands = [];
if (!empty($installCommand)) {
$commands[] = $installCommand;
}
if (!empty($buildCommand)) {
$commands[] = $buildCommand;
}
try {
$locks($lockKey, 600, function () use (&$chunks, $dbForProject, $deploymentId, &$metadata, &$completed, $response): void {
$locks($lockKey, 600, function () use ($activate, $authorization, &$chunks, $commands, $contentRange, $dbForPlatform, $dbForProject, $deploymentId, $deviceForSites, $fileSize, &$metadata, $outputDirectory, $path, $platform, $project, &$site, $type, &$completed, $response): void {
$deployment = $dbForProject->getDocument('deployments', $deploymentId);
if (!$deployment->isEmpty()) {
@@ -220,6 +247,65 @@ class Create extends Action
return;
}
}
if ($deployment->isEmpty()) {
$deviceForSites->prepareUpload($path, $metadata['content_type'] ?? '', $chunks, $metadata);
if (!empty($contentRange)) {
$deployment = $dbForProject->createDocument('deployments', new Document([
'$id' => $deploymentId,
'$permissions' => [
Permission::read(Role::any()),
Permission::update(Role::any()),
Permission::delete(Role::any()),
],
'resourceInternalId' => $site->getSequence(),
'resourceId' => $site->getId(),
'resourceType' => 'sites',
'buildCommands' => \implode(' && ', $commands),
'startCommand' => $site->getAttribute('startCommand', ''),
'buildOutput' => $outputDirectory,
'adapter' => $site->getAttribute('adapter', ''),
'fallbackFile' => $site->getAttribute('fallbackFile', ''),
'sourcePath' => $path,
'sourceSize' => $fileSize,
'totalSize' => $fileSize,
'sourceChunksTotal' => $chunks,
'sourceChunksUploaded' => 0,
'activate' => $activate,
'sourceMetadata' => $metadata,
'type' => $type,
]));
$sitesDomain = $platform['sitesDomain'];
$domain = ID::unique() . "." . $sitesDomain;
// TODO: (@Meldiron) Remove after 1.7.x migration
$isMd5 = System::getEnv('_APP_RULES_FORMAT') === 'md5';
$ruleId = $isMd5 ? md5($domain) : ID::unique();
$authorization->skip(
fn () => $dbForPlatform->createDocument('rules', new Document([
'$id' => $ruleId,
'projectId' => $project->getId(),
'projectInternalId' => $project->getSequence(),
'domain' => $domain,
'type' => 'deployment',
'trigger' => 'deployment',
'deploymentId' => $deployment->isEmpty() ? '' : $deployment->getId(),
'deploymentInternalId' => $deployment->isEmpty() ? '' : $deployment->getSequence(),
'deploymentResourceType' => 'site',
'deploymentResourceId' => $site->getId(),
'deploymentResourceInternalId' => $site->getSequence(),
'status' => 'verified',
'certificateId' => '',
'search' => implode(' ', [$ruleId, $domain]),
'owner' => 'Appwrite',
'region' => $project->getAttribute('region')
]))
);
}
}
}, timeout: 120.0);
} catch (LockContention) {
$response->addHeader('Retry-After', '5');
@@ -231,31 +317,21 @@ class Create extends Action
return;
}
$chunksUploaded = $deviceForSites->upload($fileTmpName, $path, $chunk, $chunks, $metadata);
$chunksUploaded = $deviceForSites->uploadChunk($fileTmpName, $path, $chunk, $chunks, $metadata);
if (empty($chunksUploaded)) {
throw new Exception(Exception::GENERAL_SERVER_ERROR, 'Failed moving file');
}
$type = $request->getHeader('x-sdk-language') === 'cli' ? 'cli' : 'manual';
$commands = [];
if (!empty($installCommand)) {
$commands[] = $installCommand;
}
if (!empty($buildCommand)) {
$commands[] = $buildCommand;
}
try {
$locks($lockKey, 600, function () use ($activate, $authorization, $commands, &$chunks, $chunksUploaded, $dbForPlatform, $dbForProject, $deploymentId, $deviceForSites, $fileSize, &$metadata, $outputDirectory, $path, $platform, $project, $publisherForBuilds, $queueForEvents, $response, &$site, $siteId, $type): void {
$locks($lockKey, 600, function () use ($activate, $authorization, $commands, &$chunks, $chunksUploaded, $dbForPlatform, $dbForProject, $deploymentId, $deviceForSites, $fileSize, &$metadata, $mergeUploadMetadata, $outputDirectory, $path, $platform, $project, $publisherForBuilds, $queueForEvents, $response, &$site, $siteId, $type): void {
$deployment = $dbForProject->getDocument('deployments', $deploymentId);
$uploaded = 0;
if (!$deployment->isEmpty()) {
$chunks = $deployment->getAttribute('sourceChunksTotal', 1);
$uploaded = $deployment->getAttribute('sourceChunksUploaded', 0);
$metadata = \array_merge($deployment->getAttribute('sourceMetadata', []), $metadata);
$metadata = $mergeUploadMetadata($deployment->getAttribute('sourceMetadata', []), $metadata);
if ($uploaded === $chunks) {
$queueForEvents->reset();
@@ -267,9 +343,11 @@ class Create extends Action
}
}
$chunksUploaded = max($uploaded, $chunksUploaded);
$chunksUploaded = max($uploaded, $chunksUploaded, (int) ($metadata['chunks'] ?? 0));
if ($chunksUploaded === $chunks && $uploaded < $chunks) {
$deviceForSites->finalizeUpload($path, $chunks, $metadata);
if ($activate) {
// Remove deploy for all other deployments.
$activeDeployments = $dbForProject->find('deployments', [
@@ -311,18 +389,6 @@ class Create extends Action
'type' => $type,
]));
$site = $site
->setAttribute('latestDeploymentId', $deployment->getId())
->setAttribute('latestDeploymentInternalId', $deployment->getSequence())
->setAttribute('latestDeploymentCreatedAt', $deployment->getCreatedAt())
->setAttribute('latestDeploymentStatus', $deployment->getAttribute('status', ''));
$dbForProject->updateDocument('sites', $site->getId(), new Document([
'latestDeploymentId' => $deployment->getId(),
'latestDeploymentInternalId' => $deployment->getSequence(),
'latestDeploymentCreatedAt' => $deployment->getCreatedAt(),
'latestDeploymentStatus' => $deployment->getAttribute('status', ''),
]));
$sitesDomain = $platform['sitesDomain'];
$domain = ID::unique() . "." . $sitesDomain;
@@ -367,73 +433,10 @@ class Create extends Action
platform: $platform,
));
} else {
if ($deployment->isEmpty()) {
$deployment = $dbForProject->createDocument('deployments', new Document([
'$id' => $deploymentId,
'$permissions' => [
Permission::read(Role::any()),
Permission::update(Role::any()),
Permission::delete(Role::any()),
],
'resourceInternalId' => $site->getSequence(),
'resourceId' => $site->getId(),
'resourceType' => 'sites',
'buildCommands' => \implode(' && ', $commands),
'startCommand' => $site->getAttribute('startCommand', ''),
'buildOutput' => $outputDirectory,
'adapter' => $site->getAttribute('adapter', ''),
'fallbackFile' => $site->getAttribute('fallbackFile', ''),
'sourcePath' => $path,
'sourceSize' => $fileSize,
'totalSize' => $fileSize,
'sourceChunksTotal' => $chunks,
'sourceChunksUploaded' => $chunksUploaded,
'activate' => $activate,
'sourceMetadata' => $metadata,
'type' => $type,
]));
$site = $site
->setAttribute('latestDeploymentId', $deployment->getId())
->setAttribute('latestDeploymentInternalId', $deployment->getSequence())
->setAttribute('latestDeploymentCreatedAt', $deployment->getCreatedAt())
->setAttribute('latestDeploymentStatus', $deployment->getAttribute('status', ''));
$dbForProject->updateDocument('sites', $site->getId(), new Document([
'latestDeploymentId' => $site->getAttribute('latestDeploymentId'),
'latestDeploymentInternalId' => $site->getAttribute('latestDeploymentInternalId'),
'latestDeploymentCreatedAt' => $site->getAttribute('latestDeploymentCreatedAt'),
'latestDeploymentStatus' => $site->getAttribute('latestDeploymentStatus'),
]));
$sitesDomain = $platform['sitesDomain'];
$domain = ID::unique() . "." . $sitesDomain;
$ruleId = md5($domain);
$authorization->skip(
fn () => $dbForPlatform->createDocument('rules', new Document([
'$id' => $ruleId,
'projectId' => $project->getId(),
'projectInternalId' => $project->getSequence(),
'domain' => $domain,
'type' => 'deployment',
'trigger' => 'deployment',
'deploymentId' => $deployment->isEmpty() ? '' : $deployment->getId(),
'deploymentInternalId' => $deployment->isEmpty() ? '' : $deployment->getSequence(),
'deploymentResourceType' => 'site',
'deploymentResourceId' => $site->getId(),
'deploymentResourceInternalId' => $site->getSequence(),
'status' => 'verified',
'certificateId' => '',
'search' => implode(' ', [$ruleId, $domain]),
'owner' => 'Appwrite',
'region' => $project->getAttribute('region')
]))
);
} else {
$deployment = $dbForProject->updateDocument('deployments', $deploymentId, new Document([
'sourceChunksUploaded' => $chunksUploaded,
'sourceMetadata' => $metadata,
]));
}
$deployment = $dbForProject->updateDocument('deployments', $deploymentId, new Document([
'sourceChunksUploaded' => $chunksUploaded,
'sourceMetadata' => $metadata,
]));
}
$metadata = null;
@@ -13,6 +13,7 @@ use Appwrite\SDK\Response as SDKResponse;
use Appwrite\Utopia\Response;
use Utopia\Database\Database;
use Utopia\Database\Document;
use Utopia\Database\Exception\Transaction as TransactionException;
use Utopia\Database\Query;
use Utopia\Database\Validator\UID;
use Utopia\Platform\Action;
@@ -89,8 +90,16 @@ class Delete extends Action
throw new Exception(Exception::DEPLOYMENT_NOT_FOUND);
}
if (!$dbForProject->deleteDocument('deployments', $deployment->getId())) {
throw new Exception(Exception::GENERAL_SERVER_ERROR, 'Failed to remove deployment from DB');
try {
if (!$dbForProject->deleteDocument('deployments', $deployment->getId())) {
throw new Exception(Exception::GENERAL_SERVER_ERROR, 'Failed to remove deployment from DB');
}
} catch (TransactionException) {
$deploymentExists = !$dbForProject->getDocument('deployments', $deployment->getId())->isEmpty();
if ($deploymentExists && !$dbForProject->deleteDocument('deployments', $deployment->getId())) {
throw new Exception(Exception::GENERAL_SERVER_ERROR, 'Failed to remove deployment from DB');
}
}
if (!empty($deployment->getAttribute('sourcePath', ''))) {
@@ -51,6 +51,7 @@ class Get extends Action
)
],
type: MethodType::LOCATION,
locationAuth: ['Project', 'ImpersonateUserId'],
contentType: ContentType::ANY,
))
->param('siteId', '', fn (Database $dbForProject) => new UID($dbForProject->getAdapter()->getMaxUIDLength()), 'Site ID.', false, ['dbForProject'])
@@ -99,7 +100,7 @@ class Get extends Action
}
$size = $device->getFileSize($path);
$rangeHeader = $request->getHeader('range');
$rangeHeader = $request->getHeaderLine('range');
$response
->setContentType('application/gzip')
@@ -135,19 +135,7 @@ class Create extends Action
'status' => 'waiting',
'buildPath' => '',
'buildLogs' => '',
'type' => $request->getHeader('x-sdk-language') === 'cli' ? 'cli' : 'manual'
]));
$site = $site
->setAttribute('latestDeploymentId', $deployment->getId())
->setAttribute('latestDeploymentInternalId', $deployment->getSequence())
->setAttribute('latestDeploymentCreatedAt', $deployment->getCreatedAt())
->setAttribute('latestDeploymentStatus', $deployment->getAttribute('status', ''));
$dbForProject->updateDocument('sites', $site->getId(), new Document([
'latestDeploymentId' => $site->getAttribute('latestDeploymentId'),
'latestDeploymentInternalId' => $site->getAttribute('latestDeploymentInternalId'),
'latestDeploymentCreatedAt' => $site->getAttribute('latestDeploymentCreatedAt'),
'latestDeploymentStatus' => $site->getAttribute('latestDeploymentStatus'),
'type' => $request->getHeaderLine('x-sdk-language') === 'cli' ? 'cli' : 'manual'
]));
// Preview deployments for sites
@@ -12,6 +12,7 @@ use Executor\Executor;
use Utopia\Database\Database;
use Utopia\Database\DateTime;
use Utopia\Database\Document;
use Utopia\Database\Exception\Transaction as TransactionException;
use Utopia\Database\Validator\UID;
use Utopia\Platform\Action;
use Utopia\Platform\Scope\HTTP;
@@ -89,17 +90,30 @@ class Update extends Action
$endTime = new \DateTime('now');
$duration = $endTime->getTimestamp() - $startTime->getTimestamp();
$deployment = $dbForProject->updateDocument('deployments', $deployment->getId(), new Document([
'buildEndedAt' => DateTime::now(),
'buildDuration' => $duration,
'status' => 'canceled'
]));
if ($deployment->getSequence() === $site->getAttribute('latestDeploymentInternalId', '')) {
$site = $site->setAttribute('latestDeploymentStatus', $deployment->getAttribute('status', ''));
$dbForProject->updateDocument('sites', $site->getId(), new Document([
'latestDeploymentStatus' => $site->getAttribute('latestDeploymentStatus'),
try {
$deployment = $dbForProject->updateDocument('deployments', $deployment->getId(), new Document([
'buildEndedAt' => DateTime::now(),
'buildDuration' => $duration,
'status' => 'canceled'
]));
} catch (TransactionException) {
$deployment = $dbForProject->getDocument('deployments', $deployment->getId());
if ($deployment->isEmpty()) {
throw new Exception(Exception::DEPLOYMENT_NOT_FOUND);
}
if (\in_array($deployment->getAttribute('status'), ['ready', 'failed'])) {
throw new Exception(Exception::BUILD_ALREADY_COMPLETED);
}
if ($deployment->getAttribute('status') !== 'canceled') {
$deployment = $dbForProject->updateDocument('deployments', $deployment->getId(), new Document([
'buildEndedAt' => DateTime::now(),
'buildDuration' => $duration,
'status' => 'canceled'
]));
}
}
try {
@@ -184,18 +184,6 @@ class Create extends Base
'activate' => $activate,
]));
$site = $site
->setAttribute('latestDeploymentId', $deployment->getId())
->setAttribute('latestDeploymentInternalId', $deployment->getSequence())
->setAttribute('latestDeploymentCreatedAt', $deployment->getCreatedAt())
->setAttribute('latestDeploymentStatus', $deployment->getAttribute('status', ''));
$dbForProject->updateDocument('sites', $site->getId(), new Document([
'latestDeploymentId' => $site->getAttribute('latestDeploymentId'),
'latestDeploymentInternalId' => $site->getAttribute('latestDeploymentInternalId'),
'latestDeploymentCreatedAt' => $site->getAttribute('latestDeploymentCreatedAt'),
'latestDeploymentStatus' => $site->getAttribute('latestDeploymentStatus'),
]));
$sitesDomain = $platform['sitesDomain'];
$domain = ID::unique() . "." . $sitesDomain;
@@ -183,6 +183,17 @@ class Update extends Base
$isConnected = !empty($site->getAttribute('providerRepositoryId', ''));
// Omitted providerRepositoryId (null) on a connected site — preserve existing VCS values
if ($isConnected && $providerRepositoryId === null) {
$providerRepositoryId = $site->getAttribute('providerRepositoryId', '');
$installationId = $site->getAttribute('installationId', '');
$providerBranch = $providerBranch ?: $site->getAttribute('providerBranch', '');
$providerRootDirectory = $providerRootDirectory ?: $site->getAttribute('providerRootDirectory', '');
$repositoryId = $site->getAttribute('repositoryId', '');
$repositoryInternalId = $site->getAttribute('repositoryInternalId', '');
$installation = $dbForPlatform->getDocument('installations', $installationId);
}
// Git disconnect logic. Disconnecting only when providerRepositoryId is empty, allowing for continue updates without disconnecting git
if ($isConnected && ($providerRepositoryId !== null && empty($providerRepositoryId))) {
$repositories = $dbForPlatform->find('repositories', [
@@ -187,7 +187,7 @@ class Create extends Action
$fileTmpName = (\is_array($file['tmp_name']) && isset($file['tmp_name'][0])) ? $file['tmp_name'][0] : $file['tmp_name'];
$fileSize = (\is_array($file['size']) && isset($file['size'][0])) ? $file['size'][0] : $file['size'];
$contentRange = $request->getHeader('content-range');
$contentRange = $request->getHeaderLine('content-range');
$fileId = $fileId === 'unique()' ? ID::unique() : $fileId;
$chunk = 1;
$chunks = 1;
@@ -196,7 +196,7 @@ class Create extends Action
$start = $request->getContentRangeStart();
$end = $request->getContentRangeEnd();
$fileSize = $request->getContentRangeSize();
$fileId = $request->getHeader('x-appwrite-id', $fileId);
$fileId = $request->getHeaderLine('x-appwrite-id', $fileId);
// TODO make `end >= $fileSize` in next breaking version
if (is_null($start) || is_null($end) || is_null($fileSize) || $end > $fileSize) {
throw new Exception(Exception::STORAGE_INVALID_CONTENT_RANGE);
@@ -260,8 +260,8 @@ class Create extends Action
};
try {
$locks($lockKey, 600, function () use ($bucket, &$chunks, $contentRange, $dbForProject, $deviceForFiles, $fileId, $fileName, $fileSize, &$metadata, $path, $permissions, $response, &$completed): void {
$file = $dbForProject->getDocument('bucket_' . $bucket->getSequence(), $fileId);
$locks($lockKey, 600, function () use ($authorization, $bucket, &$chunks, $contentRange, $dbForProject, $deviceForFiles, $fileId, $fileName, $fileSize, &$metadata, $path, $permissions, $response, &$completed): void {
$file = $authorization->skip(fn () => $dbForProject->getDocument('bucket_' . $bucket->getSequence(), $fileId));
if (!$file->isEmpty()) {
$chunks = $file->getAttribute('chunksTotal', 1);
$uploaded = $file->getAttribute('chunksUploaded', 0);
@@ -325,7 +325,7 @@ class Create extends Action
}
$finalizeUpload = function (int $chunksUploaded) use ($authorization, $bucket, &$chunks, $contentRange, $dbForProject, $deviceForFiles, $fileId, $fileName, $fileSize, &$metadata, $mergeUploadMetadata, $path, $permissions, $queueForEvents, $response): void {
$file = $dbForProject->getDocument('bucket_' . $bucket->getSequence(), $fileId);
$file = $authorization->skip(fn () => $dbForProject->getDocument('bucket_' . $bucket->getSequence(), $fileId));
$uploaded = 0;
if (!$file->isEmpty()) {
@@ -57,7 +57,8 @@ class Get extends Action
)
],
contentType: ContentType::ANY,
type: MethodType::LOCATION
type: MethodType::LOCATION,
locationAuth: ['Project', 'ImpersonateUserId'],
))
->param('bucketId', '', new UID(), 'Storage bucket ID. You can create a new storage bucket using the Storage service [server integration](https://appwrite.io/docs/server/storage#createBucket).')
->param('fileId', '', new UID(), 'File ID.')
@@ -127,7 +128,7 @@ class Get extends Action
$size = $file->getAttribute('sizeOriginal', 0);
$rangeHeader = $request->getHeader('range');
$rangeHeader = $request->getHeaderLine('range');
if (!empty($rangeHeader)) {
$start = $request->getRangeStart();
$end = $request->getRangeEnd();
@@ -71,6 +71,7 @@ class Get extends Action
)
],
type: MethodType::LOCATION,
locationAuth: ['Project', 'ImpersonateUserId'],
contentType: ContentType::IMAGE
))
->param('bucketId', '', new UID(), 'Storage bucket unique ID. You can create a new storage bucket using the Storage service [server integration](https://appwrite.io/docs/server/storage#createBucket).')
@@ -119,7 +119,7 @@ class Get extends Action
$size = $file->getAttribute('sizeOriginal', 0);
$rangeHeader = $request->getHeader('range');
$rangeHeader = $request->getHeaderLine('range');
if (!empty($rangeHeader)) {
$start = $request->getRangeStart();
$end = $request->getRangeEnd();
@@ -58,7 +58,8 @@ class Get extends Action
)
],
contentType: ContentType::ANY,
type: MethodType::LOCATION
type: MethodType::LOCATION,
locationAuth: ['Project', 'ImpersonateUserId'],
))
->param('bucketId', '', new UID(), 'Storage bucket unique ID. You can create a new storage bucket using the Storage service [server integration](https://appwrite.io/docs/server/storage#createBucket).')
->param('fileId', '', new UID(), 'File ID.')
@@ -136,7 +137,7 @@ class Get extends Action
$size = $file->getAttribute('sizeOriginal', 0);
$rangeHeader = $request->getHeader('range');
$rangeHeader = $request->getHeaderLine('range');
if (!empty($rangeHeader)) {
$start = $request->getRangeStart();
$end = $request->getRangeEnd();
@@ -204,6 +204,10 @@ class Create extends Action
throw new Exception(Exception::USER_EMAIL_FREE);
}
if ((($project->getId() === 'console') || ($plan['supportsCorporateEmailValidation'] ?? false)) && ($project->getAttribute('auths', [])['corporateEmails'] ?? false) && !($emailMetadata['emailIsCorporate'] ?? true)) {
throw new Exception(Exception::USER_EMAIL_NOT_CORPORATE);
}
$hash = $proofForPassword->hash($proofForPassword->generate());
$userId = ID::unique();
Loaded 100 of 424 files, more files were not shown because too many files have changed in this diff. Show more