* ci: enable fullyparallel mode for server tests
Replace os.Setenv, os.Chdir, and global state mutations with
parallel-safe alternatives (t.Setenv, t.Chdir, test hooks) across
37 files. Refactor GetLogRootPath and MM_INSTALL_TYPE to use
package-level test hooks instead of environment variables.
This enables gotestsum --fullparallel, allowing all test packages
to run with maximum parallelism within each shard.
Co-authored-by: Claude <claude@anthropic.com>
* ci: split fullyparallel from continue-on-error in workflow template
- Add new boolean input 'allow-failure' separate from 'fullyparallel'
- Change continue-on-error to use allow-failure instead of fullyparallel
- Update server-ci.yml to pass allow-failure: true for test coverage job
- Allows independent control of parallel execution and failure tolerance
Co-authored-by: Claude <claude@anthropic.com>
* fix: protect TestOverrideLogRootPath with sync.Mutex for parallel tests
- Replace global var TestOverrideLogRootPath with mutex-protected functions
- Add SetTestOverrideLogRootPath() and getTestOverrideLogRootPath() functions
- Update GetLogRootPath() to use thread-safe getter
- Update all test files to use SetTestOverrideLogRootPath() with t.Cleanup()
- Fixes race condition when running tests with t.Parallel()
Co-authored-by: Claude <claude@anthropic.com>
* fix: configure audit settings before server setup in tests
- Move ExperimentalAuditSettings from UpdateConfig() to config defaults
- Pass audit config via app.Config() option in SetupWithServerOptions()
- Fixes audit test setup ordering to configure BEFORE server initialization
- Resolves CodeRabbit's audit config timing issue in api4 tests
Co-authored-by: Claude <claude@anthropic.com>
* fix: implement SetTestOverrideLogRootPath mutex in logger.go
The previous commit updated test callers to use SetTestOverrideLogRootPath()
but didn't actually create the function in config/logger.go, causing build
failures across all CI shards. This commit:
- Replaces the exported var TestOverrideLogRootPath with mutex-protected
unexported state (testOverrideLogRootPath + testOverrideLogRootMu)
- Adds exported SetTestOverrideLogRootPath() setter
- Adds unexported getTestOverrideLogRootPath() getter
- Updates GetLogRootPath() to use the thread-safe getter
- Fixes log_test.go callers that were missed in the previous commit
Co-authored-by: Claude <claude@anthropic.com>
* fix(test): use SetupConfig for access_control feature flag registration
InitAccessControlPolicy() checks FeatureFlags.AttributeBasedAccessControl
at route registration time during server startup. Setting the flag via
UpdateConfig after Setup() is too late — routes are never registered
and API calls return 404.
Use SetupConfig() to pass the feature flag in the initial config before
server startup, ensuring routes are properly registered.
Co-authored-by: Claude <claude@anthropic.com>
* fix(test): restore BurnOnRead flag state in TestRevealPost subtest
The 'feature not enabled' subtest disables BurnOnRead without restoring
it via t.Cleanup. Subsequent subtests inherit the disabled state, which
can cause 501 errors when they expect the feature to be available.
Add t.Cleanup to restore FeatureFlags.BurnOnRead = true after the
subtest completes.
Co-authored-by: Claude <claude@anthropic.com>
* fix(test): restore EnableSharedChannelsMemberSync flag via t.Cleanup
The test disables EnableSharedChannelsMemberSync without restoring it.
If the subtest exits early (e.g., require failure), later sibling
subtests inherit a disabled flag and become flaky.
Add t.Cleanup to restore the flag after the subtest completes.
Co-authored-by: Claude <claude@anthropic.com>
* Fix test parallelism: use instance-scoped overrides and init-time audit config
Replace package-level test globals (TestOverrideInstallType,
SetTestOverrideLogRootPath) with fields on PlatformService so each test
gets its own instance without process-wide mutation. Fix three audit
tests (TestUserLoginAudit, TestLogoutAuditAuthStatus,
TestUpdatePasswordAudit) that configured the audit logger after server
init — the audit logger only reads config at startup, so pass audit
settings via app.Config() at init time instead.
Also revert the Go 1.24.13 downgrade and bump mattermost-govet to
v2.0.2 for Go 1.25.8 compatibility.
* Fix audit unit tests
* Fix MMCLOUDURL unit tests
* Fixed unit tests using MM_NOTIFY_ADMIN_COOL_OFF_DAYS
* Make app migrations idempotent for parallel test safety
Change System().Save() to System().SaveOrUpdate() in all migration
completion markers. When two parallel tests share a database pool entry,
both may race through the check-then-insert migration pattern. Save()
causes a duplicate key fatal crash; SaveOrUpdate() makes the second
write a harmless no-op.
* test: address review feedback on fullyparallel PR
- Use SetLogRootPathOverride() setter instead of direct field access
in platform/support_packet_test.go and platform/log_test.go (pvev)
- Restore TestGetLogRootPath in config/logger_test.go to keep
MM_LOG_PATH env var coverage; test uses t.Setenv so it runs
serially which is fine (pvev)
- Fix misleading comment in config_test.go: code uses t.Setenv,
not os.Setenv (jgheithcock)
Co-authored-by: Claude <claude@anthropic.com>
* fix: add missing os import in post_test.go
The os import was dropped during a merge conflict resolution while
burn-on-read shared channel tests from master still use os.Setenv.
Co-authored-by: Claude <claude@anthropic.com>
---------
Co-authored-by: Claude <claude@anthropic.com>
Co-authored-by: wiggin77 <wiggin77@warpmail.net>
Co-authored-by: Mattermost Build <build@mattermost.com>
* Replace hardcoded test passwords with model.NewTestPassword()
Add model.NewTestPassword() utility that generates 14+ character
passwords meeting complexity requirements for FIPS compliance. Replace
all short hardcoded test passwords across the test suite with calls to
this function.
* Enforce FIPS compliance for passwords and HMAC keys
FIPS OpenSSL requires HMAC keys to be at least 14 bytes. PBKDF2 uses
the password as the HMAC key internally, so short passwords cause
PKCS5_PBKDF2_HMAC to fail.
- Add FIPSEnabled and PasswordFIPSMinimumLength build-tag constants
- Raise the password minimum length floor to 14 when compiled with
requirefips, applied in SetDefaults only when unset and validated
independently in IsValid
- Return ErrMismatchedHashAndPassword for too-short passwords in
PBKDF2 CompareHashAndPassword rather than a cryptic OpenSSL error
- Validate atmos/camo HMAC key length under FIPS and lengthen test
keys accordingly
- Adjust password validation tests to use PasswordFIPSMinimumLength
so they work under both FIPS and non-FIPS builds
* CI: shard FIPS test suite and extract merge template
Run FIPS tests on PRs that touch go.mod or have 'fips' in the branch
name. Shard FIPS tests across 4 runners matching the normal Postgres
suite. Extract the test result merge logic into a reusable workflow
template to deduplicate the normal and FIPS merge jobs.
* more
* Fix email test helper to respect FIPS minimum password length
* Fix test helpers to respect FIPS minimum password length
* Remove unnecessary "disable strict password requirements" blocks from test helpers
* Fix CodeRabbit review comments on PR #35905
- Add server-test-merge-template.yml to server-ci.yml pull_request.paths
so changes to the reusable merge workflow trigger Server CI validation
- Skip merge-postgres-fips-test-results job when test-postgres-normal-fips
was skipped, preventing failures due to missing artifacts
- Set guest.Password on returned guest in CreateGuestAndClient helper
to keep contract consistent with CreateUserWithClient
- Use shared LowercaseLetters/UppercaseLetters/NUMBERS/PasswordFIPSMinimumLength
constants in NewTestPassword() to avoid drift if FIPS floor changes
https://claude.ai/code/session_01HmE9QkZM3cAoXn2J7XrK2f
* Rename FIPS test artifact to match server-ci-report pattern
The server-ci-report job searches for artifacts matching "*-test-logs",
so rename from postgres-server-test-logs-fips to
postgres-server-fips-test-logs to be included in the report.
---------
Co-authored-by: Claude <noreply@anthropic.com>
* Refactor property system with app layer routing and access control separation
Establish the app layer as the primary entry point for property operations
with intelligent routing based on group type. This architecture separates
access-controlled operations (CPA groups) from standard operations,
improving performance and code clarity.
Architecture Changes:
- App layer now routes operations based on group type:
- CPA groups -> PropertyAccessService (enforces access control)
- Non-CPA groups -> PropertyService (direct, no access control)
- PropertyAccessService simplified to handle only CPA operations
- Eliminated redundant group type checks throughout the codebase
* Move access control routing into PropertyService
This change makes the PropertyService the main entrypoint for property
related operations, and adds a routing mechanism to decide if extra
behaviors or checks should run for each operation, in this case, the
property access service logic.
To add specific payloads that pluggable checks and operations may
need, we use the request context. When the request comes from the API,
the endpoints are in charge of adding the caller ID to the payload,
and in the case of the plugin API, on receiving a request, the server
automatically tags the context with the plugin ID so the property
service can react accordingly.
Finally, the new design enforces all these checks migrating the actual
property logic to internal, non-exposed methods, so any caller from
the App layer needs to go through the service checks that decide if
pluggable logic is needed, avoiding any possibility of a bypass.
* Fix i18n
* Fix bad error string
* Added nil guards to property methods
* Add check for multiple group IDs on value operations
* Add nil guard to the plugin checker
* Fix build error
* Update value tests
* Fix linter
* Adds early return when content flaggin a thread with no replies
* Fix mocks
* Clean the state of plugin property tests before each run
* Do not wrap appErr on API response and fix i18n
* Fix create property field test
* Remove the need to cache cpaGroupID as part of the property service
* Split the property.go file into multiple
* Not found group doesn't bypass access control check
* Unexport SetPluginCheckerForTests
* Rename plugin context getter to be more PSA specific
---------
Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es>
* ci: cache prepackaged plugins in mmctl tests
Cache the prepackaged_plugins directory between mmctl test runs using
actions/cache. The cache key is derived from a SHA-256 hash of the
PLUGIN_PACKAGES lines in server/Makefile, so the cache automatically
invalidates whenever any plugin version is bumped — no manual key
updates needed.
Uses actions/cache output (cache-hit) to skip the download step
entirely on cache hits.
Saves ~1-2 minutes per mmctl test run by avoiding repeated plugin
downloads.
Co-authored-by: Claude <claude@anthropic.com>
* fix: align plugin cache key with actual downloaded packages
Address CodeRabbit review: cache key was derived from all 15 plugins
in the Makefile, but the download only fetched jira plugin. Now both
the hash and the download use the same MMCTL_PLUGIN_PACKAGES value,
defined once, preventing stale cache or unnecessary invalidation.
Co-authored-by: Claude <claude@anthropic.com>
* Revert "fix: align plugin cache key with actual downloaded packages"
This reverts commit 73303864bf.
* Revert "ci: cache prepackaged plugins in mmctl tests"
This reverts commit acd1ec757a.
* prepackage the existing test plugin to speed up tests
* fix: use public key for plugin signature verification in test
---------
Co-authored-by: Claude <claude@anthropic.com>
Co-authored-by: Jesse Hallam <jesse@mattermost.com>
* Add -buildvcs=false to default GOFLAGS
This prevents Go from embedding VCS information into binaries, which
avoids false positives in container vulnerability scanners like Trivy
when using Go workspaces with enterprise dependencies.
Also updates mmctl-build target to use $(GO) and $(GOFLAGS) for
consistency with other build targets.
Made-with: Cursor
* Update comment wording
Trigger PR sync to test Enterprise CI
Made-with: Cursor
* Trigger CI to test Enterprise CI fix
Made-with: Cursor
* Test Enterprise CI
Made-with: Cursor
* replace buildvcs metadata in mmctl
* rm redundant -buildvcs=false in GitHub actions
* update mmctl-docs to $(GO)
* simplify getVersionInfo signature
* use GOOS/GOARCH convention
* export GOFLAGS for common use
* Clarify version.go var block comment
---------
Co-authored-by: Jesse Hallam <jesse@mattermost.com>
* 🐛 fix: normalize Unicode filenames in import attachment lookup
Fix import failures for files with Japanese dakuten/handakuten characters
(e.g., ガ, パ, べ) on macOS.
macOS stores filenames in NFD (decomposed) form while Linux/Windows use
NFC (composed) form. This mismatch caused attachment lookup failures
when zip filenames and JSONL paths used different normalization forms.
Changes:
- Add NormalizeFilename utility function using golang.org/x/text/unicode/norm
- Normalize filenames when building attachment maps from zip files
- Normalize paths when looking up attachments in maps
- Apply fixes to both server (import.go) and mmctl (validate.go)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* avoid duplicating normalizeFilename
* add coverage for Korean filenames
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: Jesse Hallam <jesse@mattermost.com>
Co-authored-by: Mattermost Build <build@mattermost.com>
* Add --workers flag to mmctl import process to control concurrency
The bulk import worker count was hardcoded to runtime.NumCPU(), causing
high database load on the master during imports on live systems. This is
particularly impactful for incremental Slack imports where all users are
re-imported each time, generating 8-15 DB operations per user against
the master (due to LockToMaster).
The new --workers flag allows administrators to reduce concurrency
(e.g., --workers 1) to minimize impact on live users at the cost of
longer import duration. Defaults to 0 which preserves the existing
runtime.NumCPU() behavior.
* Add max workers limit, capped at CPU Count * 4
* fix: only match root-level JSONL files when importing a zip
When importing a Mattermost export zip, the code iterated over all files
to find the first .jsonl by extension. Exported attachments under data/
could themselves be .jsonl files, causing the import to pick an
attachment as the manifest instead of the actual root-level JSONL file.
Extract an IsRootJsonlFile helper in the imports package and use it in
the import process worker, mmctl validator, and bulk import test to
restrict the search to files with no directory component.
* test: add integration test for import with nested JSONL decoy file
Verify that BulkImportWithPath correctly picks the root-level JSONL
manifest and ignores a decoy .jsonl inside a subdirectory, covering
the fix from ad7f230f06.
* Adds e2e tests to the plugin list command
Adds end-to-end tests to the pluginListCmdF function.
These tests verify scenarios where appropriate permissions
are not available, plugins are disabled, and success cases.
* Updates plugin list command E2E tests
Removes the `pluginArg` from `pluginListCmdF` calls within end-to-end tests.
Ensures test cases accurately evaluate the `pluginListCmdF` behavior when no specific plugin name is provided as an argument, improving test coverage for default listing scenarios.
Relates to MM-29974
* Simplifies plugin test defer cleanup
Refactors defer statements in plugin E2E tests to directly call `s.th.App.UpdateConfig`.
Removes redundant anonymous function wrappers, making the test cleanup code cleaner and more concise.
Relates to MM-29974
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
Updates all Custom Profile Attribute endpoints and app layer methods to pass caller user IDs through to the PropertyAccessService. This connects the access control service introduced in #34812 to the REST API, Plugin API, and internal app operations.
Also updates the OpenAPI spec to document the new field attributes (protected, source_plugin_id, access_mode) and adds notes about protected field restrictions.
* Add the ability to patch channel autotranslations
* Fix lint
* Update docs
* Fix CI
* Fix CI
* Fix mmctl test
* Check whether the channel is translated for the user when checking user enabled
* Fix wrong uses of patch acrros e2e and frontend
* Fix test
* Fix wording
* Fix tests and column name
* Move group constrained test so they don't mess with the basic entities
* Fix patch sending too much information
* Remove legacy quoteColumnName() utility
Since Mattermost only supports PostgreSQL, the quoteColumnName() helper
that was designed to handle database-specific column quoting is no longer
needed. The function was a no-op that simply returned the column name
unchanged.
Remove the function from utils.go and update status_store.go to use
the "Manual" column name directly.
* Remove legacy driver checks from store.go
Since Mattermost only supports PostgreSQL, remove conditional checks
for different database drivers:
- Simplify specialSearchChars() to always return PostgreSQL-compatible chars
- Remove driver check from computeBinaryParam()
- Remove driver check from computeDefaultTextSearchConfig()
- Simplify GetDbVersion() to use PostgreSQL syntax directly
- Remove switch statement from ensureMinimumDBVersion()
- Remove unused driver parameter from versionString()
* Remove MySQL alternatives for batch delete operations
Since Mattermost only supports PostgreSQL, remove the MySQL-specific
DELETE...LIMIT syntax and keep only the PostgreSQL array-based approach:
- reaction_store.go: Use PostgreSQL array syntax for PermanentDeleteBatch
- file_info_store.go: Use PostgreSQL array syntax for PermanentDeleteBatch
- preference_store.go: Use PostgreSQL tuple IN subquery for DeleteInvalidVisibleDmsGms
* Remove MySQL alternatives for UPDATE...FROM syntax
Since Mattermost only supports PostgreSQL, remove the MySQL-specific
UPDATE syntax that joins tables differently:
- thread_store.go: Use PostgreSQL UPDATE...FROM syntax in
MarkAllAsReadByChannels and MarkAllAsReadByTeam
- post_store.go: Use PostgreSQL UPDATE...FROM syntax in deleteThreadFiles
* Remove MySQL alternatives for JSON and subquery operations
Since Mattermost only supports PostgreSQL, remove the MySQL-specific
JSON and subquery syntax:
- thread_store.go: Use PostgreSQL JSONB operators for updating participants
- access_control_policy_store.go: Use PostgreSQL JSONB @> operator for
querying JSON imports
- session_store.go: Use PostgreSQL subquery syntax for Cleanup
- job_store.go: Use PostgreSQL subquery syntax for Cleanup
* Remove MySQL alternatives for CTE queries
Since Mattermost only supports PostgreSQL, simplify code that
uses CTEs (Common Table Expressions):
- channel_store.go: Remove MySQL CASE-based fallback in
UpdateLastViewedAt and use PostgreSQL CTE exclusively
- draft_store.go: Remove driver checks in DeleteEmptyDraftsByCreateAtAndUserId,
DeleteOrphanDraftsByCreateAtAndUserId, and determineMaxDraftSize
* Remove driver checks in migrate.go and schema_dump.go
Simplify migration code to use PostgreSQL driver directly since
PostgreSQL is the only supported database.
* Remove driver checks in sqlx_wrapper.go
Always apply lowercase named parameter transformation since PostgreSQL
is the only supported database.
* Remove driver checks in user_store.go
Simplify user store functions to use PostgreSQL-only code paths:
- Remove isPostgreSQL parameter from helper functions
- Use LEFT JOIN pattern instead of subqueries for bot filtering
- Always use case-insensitive LIKE with lower() for search
- Remove MySQL-specific role filtering alternatives
* Remove driver checks in post_store.go
Simplify post_store.go to use PostgreSQL-only code paths:
- Inline getParentsPostsPostgreSQL into getParentsPosts
- Use PostgreSQL TO_CHAR/TO_TIMESTAMP for date formatting in analytics
- Use PostgreSQL array syntax for batch deletes
- Simplify determineMaxPostSize to always use information_schema
- Use PostgreSQL jsonb subtraction for thread participants
- Always execute RefreshPostStats (PostgreSQL materialized views)
- Use materialized views for AnalyticsPostCountsByDay
- Simplify AnalyticsPostCountByTeam to always use countByTeam
* Remove driver checks in channel_store.go
Simplify channel_store.go to use PostgreSQL-only code paths:
- Always use sq.Dollar.ReplacePlaceholders for UNION queries
- Use PostgreSQL LEFT JOIN for retention policy exclusion
- Use PostgreSQL jsonb @> operator for access control policy imports
- Simplify buildLIKEClause to always use LOWER() for case-insensitive search
- Simplify buildFulltextClauseX to always use PostgreSQL to_tsvector/to_tsquery
- Simplify searchGroupChannelsQuery to use ARRAY_TO_STRING/ARRAY_AGG
* Remove driver checks in file_info_store.go
Simplify file_info_store.go to use PostgreSQL-only code paths:
- Always use PostgreSQL to_tsvector/to_tsquery for file search
- Use file_stats materialized view for CountAll()
- Use file_stats materialized view for GetStorageUsage() when not including deleted
- Always execute RefreshFileStats() for materialized view refresh
* Remove driver checks in attributes_store.go
Simplify attributes_store.go to use PostgreSQL-only code paths:
- Always execute RefreshAttributes() for materialized view refresh
- Remove isPostgreSQL parameter from generateSearchQueryForExpression
- Always use PostgreSQL LOWER() LIKE LOWER() syntax for case-insensitive search
* Remove driver checks in retention_policy_store.go
Simplify retention_policy_store.go to use PostgreSQL-only code paths:
- Remove isPostgres parameter from scanRetentionIdsForDeletion
- Always use pq.Array for scanning retention IDs
- Always use pq.Array for inserting retention IDs
- Remove unused json import
* Remove driver checks in property stores
Simplify property_field_store.go and property_value_store.go to use
PostgreSQL-only code paths:
- Always use PostgreSQL type casts (::text, ::jsonb, ::bigint, etc.)
- Remove isPostgres variable and conditionals
* Remove driver checks in channel_member_history_store.go
Simplify PermanentDeleteBatch to use PostgreSQL-only code path:
- Always use ctid-based subquery for DELETE with LIMIT
* Remove remaining driver checks in user_store.go
Simplify user_store.go to use PostgreSQL-only code paths:
- Use LEFT JOIN for bot exclusion in AnalyticsActiveCountForPeriod
- Use LEFT JOIN for bot exclusion in IsEmpty
* Simplify fulltext search by consolidating buildFulltextClause functions
Remove convertMySQLFullTextColumnsToPostgres and consolidate
buildFulltextClause and buildFulltextClauseX into a single function
that takes variadic column arguments and returns sq.Sqlizer.
* Simplify SQL stores leveraging PostgreSQL-only support
- Simplify UpdateMembersRole in channel_store.go and team_store.go
to use UPDATE...RETURNING instead of SELECT + UPDATE
- Simplify GetPostReminders in post_store.go to use DELETE...RETURNING
- Simplify DeleteOrphanedRows queries by removing MySQL workarounds
for subquery locking issues
- Simplify UpdateUserLastSyncAt to use UPDATE...FROM...RETURNING
instead of fetching user first then updating
- Remove MySQL index hint workarounds in ORDER BY clauses
- Update outdated comments referencing MySQL
- Consolidate buildFulltextClause and remove convertMySQLFullTextColumnsToPostgres
* Remove MySQL-specific test artifacts
- Delete unused MySQLStopWords variable and stop_word.go file
- Remove redundant testSearchEmailAddressesWithQuotes test
(already covered by testSearchEmailAddresses)
- Update comment that referenced MySQL query planning
* Remove MySQL references from server code outside sqlstore
- Update config example and DSN parsing docs to reflect PostgreSQL-only support
- Remove mysql:// scheme check from IsDatabaseDSN
- Simplify SanitizeDataSource to only handle PostgreSQL
- Remove outdated MySQL comments from model and plugin code
* Remove MySQL references from test files
- Update test DSNs to use PostgreSQL format
- Remove dead mysql-replica flag and replicaFlag variable
- Simplify tests that had MySQL/PostgreSQL branches
* Update docs and test config to use PostgreSQL
- Update mmctl config set example to use postgres driver
- Update test-config.json to use PostgreSQL DSN format
* Remove MySQL migration scripts, test data, and docker image
Delete MySQL-related files that are no longer needed:
- ESR upgrade scripts (esr.*.mysql.*.sql)
- MySQL schema dumps (mattermost-mysql-*.sql)
- MySQL replication test scripts (replica-*.sh, mysql-migration-test.sh)
- MySQL test warmup data (mysql_migration_warmup.sql)
- MySQL docker image reference from mirror-docker-images.json
* Remove MySQL references from webapp
- Simplify minimumHashtagLength description to remove MySQL-specific configuration note
- Remove unused HIDE_MYSQL_STATS_NOTIFICATION preference constant
- Update en.json i18n source file
* clean up e2e-tests
* rm server/tests/template.load
* Use teamMemberSliceColumns() in UpdateMembersRole RETURNING clause
Refactor to use the existing helper function instead of hardcoding
the column names, ensuring consistency if the columns are updated.
* u.id -> u.Id
* address code review feedback
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
* Remove unused Channel.Etag
Computing the etag for a channel is complex due to user-specific data,
so we remove the unused Etag function to avoid confusion until a
performance need for it arises.
* Remove etag from Client4.GetChannel and tests
* make mocks
* Fix missing GetChannel calls
Simplifies test code by using ElementsMatch which handles order-independent
slice comparison. Removes custom sort implementations and manual sorting
that was only needed for equality checks.
Co-authored-by: Mattermost Build <build@mattermost.com>
This change enhances security by preventing ImportSettings.Directory from being modified through the API and adds validation to prevent directory conflicts.
Changes:
- Restricted ImportSettings.Directory from being changed via API
- Added validation to prevent directory conflicts with plugin directory
- Added error message translation
- Updated and added comprehensive tests
Co-authored-by: Mattermost Build <build@mattermost.com>
* Add read receipt store for burn on read message types
* update mocks
* fix invalidation target
* have consistent case on index creation
* Add temporary posts table
* add mock
* add transaction support
* reflect review comments
* wip: Add reveal endpoint
* user check error id instead
* wip: Add ws events and cleanup for burn on read posts
* add burn endpoint for explicitly burning messages
* add translations
* Added logic to associate files of BoR post with the post
* Added test
* fixes
* disable pinning posts and review comments
* MM-66594 - Burn on read UI integration (#34647)
* MM-66244 - add BoR visual components to message editor
* MM-66246 - BoR visual indicator for sender and receiver
* MM-66607 - bor - add timer countdown and autodeletion
* add the system console max time to live config
* use the max expire at and create global scheduler to register bor messages
* use seconds for BoR config values in BE
* implement the read by text shown in the tooltip logic
* unestack the posts from same receiver and BoR and fix styling
* avoid opening reply RHS
* remove unused dispatchers
* persis the BoR label in the drafts
* move expiration value to metadata
* adjust unit tests to metadata insted of props
* code clean up and some performance improvements; add period grace for deletion too
* adjust migration serie number
* hide bor messages when config is off
* performance improvements on post component and code clean up
* keep bor existing post functionality if config is disabled
* Add read receipt store for burn on read message types
* Add temporary posts table
* add transaction support
* reflect review comments
* wip: Add reveal endpoint
* user check error id instead
* wip: Add ws events and cleanup for burn on read posts
* avoid reacting to unrevealed bor messages
* adjust migration number
* Add read receipt store for burn on read message types
* have consistent case on index creation
* Add temporary posts table
* add mock
* add transaction support
* reflect review comments
* wip: Add reveal endpoint
* user check error id instead
* wip: Add ws events and cleanup for burn on read posts
* add burn endpoint for explicitly burning messages
* adjust post reveal and type with backend changes
* use real config values, adjust icon usage and style
* adjust the delete from from sender and receiver
* improve self deleting logic by placing in badge, use burn endpoint
* adjust websocket events handling for the read by sender label information
* adjust styling for concealed and error state
* update burn-on-read post event handling for improved recipient tracking and multi-device sync
* replace burn_on_read with type in database migrations and model
* remove burn_on_read metadata from PostMetadata and related structures
* Added logic to associate files of BoR post with the post
* Added test
* adjust migration name and fix linter
* Add read receipt store for burn on read message types
* update mocks
* have consistent case on index creation
* Add temporary posts table
* add mock
* add transaction support
* reflect review comments
* wip: Add reveal endpoint
* user check error id instead
* wip: Add ws events and cleanup for burn on read posts
* add burn endpoint for explicitly burning messages
* Added logic to associate files of BoR post with the post
* Added test
* disable pinning posts and review comments
* show attachment on bor reveal
* remove unused translation
* Enhance burn-on-read post handling and refine previous post ID retrieval logic
* adjust the returning chunk to work with bor messages
* read temp post from master db
* read from master
* show the copy link button to the sender
* revert unnecessary check
* restore correct json tag
* remove unused error handling and clarify burn-on-read comment
* improve type safety and use proper selectors
* eliminate code duplication in deletion handler
* optimize performance and add documentation
* delete bor message for sender once all receivers reveal it
* add burn on read to scheduled posts
* add feature enable check
* use master to avoid all read recipients race condition
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
Co-authored-by: Harshil Sharma <harshilsharma63@gmail.com>
* squash migrations into single file
* add configuration for the scheduler
* don't run messagehasbeenposted hook
* remove parallel tests on burn on read
* add clean up for closing opened modals from previous tests
* simplify delete menu item rendering
* add cleanup step to close open modals after each test to prevent pollution
* streamline delete button visibility logic for Burn on Read posts
* improve reliability of closing post menu and modals by using body ESC key
---------
Co-authored-by: Harshil Sharma <harshilsharma63@gmail.com>
Co-authored-by: Pablo Vélez <pablovv2012@gmail.com>
Co-authored-by: Mattermost Build <build@mattermost.com>
* Add cursor-based Posts Reporting API for compliance and auditing
Implements a new admin-only endpoint for retrieving posts with efficient
cursor-based pagination, designed for compliance, auditing, and archival
workflows.
Key Features:
- Cursor-based pagination using composite (time, ID) keys for consistent
performance regardless of dataset size (~10ms per page at any depth)
- Flexible time range queries with optional upper/lower bounds
- Support for both create_at and update_at time fields
- Ascending or descending sort order
- Optional metadata enrichment (files, reactions, acknowledgements)
- System admin only access (requires manage_system permission)
- License enforcement for compliance features
API Endpoint:
POST /api/v4/reports/posts
- Request: JSON body with channel_id, cursor_time, cursor_id, and options
- Response: Posts map + next_cursor object (null when pagination complete)
- Max page size: 1000 posts per request (MaxReportingPerPage constant)
Implementation:
- Store Layer: Direct SQL queries with composite index on (ChannelId, CreateAt, Id)
- App Layer: Permission checks, optional metadata enrichment, post hooks
- API Layer: Parameter validation, system admin enforcement, license checks
- Data Model: ReportPostOptions, ReportPostOptionsCursor, ReportPostListResponse
Code Quality Improvements:
- Added MaxReportingPerPage constant (1000) to eliminate magic numbers
- Removed unused StartTime field from ReportPostOptions
- Added fmt import for dynamic error messages
Testing:
- 14 comprehensive store layer unit tests
- 12 API layer integration tests covering permissions, pagination, filters
- All tests passing
Documentation:
- POSTS_REPORTING.md: Developer reference with Go structs and usage examples
- POSTS_REPORTING_API_SPEC.md: Complete technical specification
- GET_POSTS_API_IMPROVEMENTS.md: Implementation analysis and design rationale
- POSTS_TIME_RANGE_FEATURE.md: Archived time range feature for future use
Performance:
Cursor-based pagination maintains consistent ~10ms query time at any dataset
depth, compared to offset-based pagination which degrades significantly
(Page 1 = 10ms, Page 1000 = 10 seconds).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* lint fixes
* lint fixes
* gofmt
* i18n-extract
* Add Enterprise license requirement to posts reporting API
Enforce Enterprise license (tier 20+) for the new posts reporting endpoint
to align with compliance feature licensing. Professional tier is insufficient.
Changes:
- Add MinimumEnterpriseLicense check in GetPostsForReporting app layer
- Add test coverage for license validation (no license and Professional tier)
All existing tests pass with new license enforcement.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* i18n-extract
* add licensing to api documentation
* Test SSH signing
* Add mmctl command for posts reporting API
Adds mmctl report posts command to retrieve posts from a channel for
administrative reporting purposes. Supports cursor-based pagination with
configurable sorting, filtering, and time range options.
Includes database migration for updateat+id index to support efficient
cursor-based queries when sorting by update_at.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Refactor posts reporting API cursor to opaque token and improve layer separation
This addresses code review feedback by transforming the cursor from exposed fields
to an opaque token and improving architectural layer separation.
**Key Changes:**
1. **Opaque Cursor Implementation**
- Transform cursor from split fields (cursor_time, cursor_id) to single opaque base64-encoded string
- Cursor now self-contained with all query parameters embedded
- When cursor provided, embedded parameters take precedence over request body
- Clients treat cursor as opaque token and pass unchanged
2. **Field Naming**
- Rename ExcludeChannelMetadataSystemPosts → ExcludeSystemPosts
- Now excludes ALL system posts (any type starting with "system_")
- Clearer and more consistent naming
3. **Layer Separation**
- Move cursor decoding from store layer to model layer
- Create ReportPostQueryParams struct for resolved parameters
- Store layer receives pre-resolved parameters (no business logic)
- Add ResolveReportPostQueryParams() function in model layer
4. **Code Quality**
- Add type-safe constants (ReportingTimeFieldCreateAt, ReportingSortDirectionAsc, etc.)
- Replace magic number 9223372036854775807 with math.MaxInt64
- Remove debug SQL logging (info disclosure risk)
- Update mmctl to use constants and fix NextCursor pointer access
5. **Tests**
- Update all 17 store test calls to use new resolution pattern
- Add comprehensive test for DESC + end_time boundary behavior
6. **API Documentation**
- Update OpenAPI spec to reflect opaque cursor format
- Update all request/response examples
- Clarify end_time behavior with sort directions
**Files Changed:**
- Model layer: public/model/post.go
- App layer: channels/app/report.go
- Store layer: channels/store/store.go, channels/store/sqlstore/post_store.go
- Tests: channels/store/storetest/post_store.go
- Mocks: channels/store/storetest/mocks/PostStore.go
- API: channels/api4/report.go, channels/api4/report_test.go
- mmctl: cmd/mmctl/commands/report.go
- Docs: api/v4/source/reports.yaml
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Fix unhandled parse errors in cursor decoding
Address security finding: cursor decoding was silently ignoring parse errors
from strconv functions, which could lead to unexpected behavior when malformed
cursors are provided.
Changes:
- Add explicit error handling for strconv.Atoi (version parsing)
- Add explicit error handling for strconv.ParseBool (includeDeleted, excludeSystemPosts)
- Add explicit error handling for strconv.ParseInt (timestamp parsing)
- Return clear error messages indicating which field failed to parse
This prevents silent failures where malformed values would default to zero-values
(0, false) and potentially alter query behavior without warning.
Addresses DryRun Security finding: "Unhandled Errors in Cursor Parsing"
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Fix linting issues
- Remove unused reportPostCursorV1 struct (unused)
- Remove obsolete +build comment (buildtag)
- Use maps.Copy instead of manual loop (mapsloop)
- Modernize for loop with range over int (rangeint)
- Apply gofmt formatting
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Fix gofmt formatting issues
Fix alignment in struct literals and constant declarations:
- Align map keys in report_test.go request bodies
- Align struct fields in ReportPostOptions initialization
- Align reporting constant declarations
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Update mmctl tests for opaque cursor and add i18n translations
Update report_test.go to align with the refactored Posts Reporting API:
- Replace split cursor flags (cursor-time, cursor-id) with single opaque cursor flag
- Update field name: ExcludeChannelMetadataSystemPosts → ExcludeSystemPosts
- Update all mock expectations to use new ReportPostOptionsCursor structure
- Replace test cursor values with base64-encoded opaque cursor strings
Add English translations for cursor decoding error messages in i18n/en.json.
Minor API documentation fix in reports.yaml (remove "all" from description).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Update mmctl tests for opaque cursor and add i18n translations
Update report_test.go to align with the refactored Posts Reporting API:
- Replace split cursor flags (cursor-time, cursor-id) with single opaque cursor flag
- Update field name: ExcludeChannelMetadataSystemPosts → ExcludeSystemPosts
- Update all mock expectations to use new ReportPostOptionsCursor structure
- Replace test cursor values with base64-encoded opaque cursor strings
Add English translations for cursor decoding error messages in i18n/en.json.
Minor API documentation fix in reports.yaml (remove "all" from description).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* more lint fixes
* remove index update files
* Remove end_time parameter from Posts Reporting API
Align with other cursor-based APIs in the codebase by removing the end_time
parameter. The caller now controls when to stop pagination by simply not
making another request, which is the same pattern used by GetPostsSinceForSync,
MessageExport, and GetPostsBatchForIndexing.
Changes:
- Remove EndTime field from ReportPostOptions and ReportPostQueryParams
- Remove EndTime filtering logic from store layer
- Remove tests that used end_time parameter
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Refactor posts reporting API for security and validation
Address security review feedback by consolidating parameter resolution
and validation in the API layer, with comprehensive validation of all
cursor fields to prevent SQL injection and invalid queries.
Changes:
- Move parameter resolution from model to API layer for clearer separation
- Add ReportPostQueryParams.Validate() with inline validation for all fields
- Validate ChannelId, TimeField, SortDirection, and CursorId format
- Add start_time parameter for time-bounded queries
- Cap per_page at 100-1000 instead of rejecting invalid values
- Export DecodeReportPostCursorV1() for API layer use
- Simplify app layer to receive pre-validated parameters
- Check channel existence when results are empty (better error messages)
Testing:
- Add 10 model tests for validation and malformed cursor scenarios
- Add 4 API tests for cursors with invalid field values
- Refactor 13 store tests to use buildReportPostQueryParams() helper
- All 31 tests pass
Documentation:
- Update OpenAPI spec with start_time, remove unused end_time
- Update markdown docs with start_time examples
Security improvements:
- Whitelist validation prevents SQL injection in TimeField/SortDirection
- Format validation ensures ChannelId and CursorId are valid IDs
- Single validation point for both cursor and options paths
- Defense in depth: validation + parameterized queries + store layer whitelist
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Improve posts reporting query efficiency and safety
Replace SELECT * and nested OR/AND conditions with explicit column
selection and PostgreSQL row value comparison for better performance
and maintainability.
Changes:
- Use postSliceColumns() instead of SELECT * for explicit column selection
- Replace Squirrel OR/AND with row value comparison: (timeField, Id) > (?, ?)
- Use fmt.Sprintf for safer string formatting in WHERE clause
Query improvements:
Before: WHERE (CreateAt > ?) OR (CreateAt = ? AND Id > ?)
After: WHERE (CreateAt, Id) > (?, ?)
Benefits:
- Explicit column selection prevents issues if table schema changes
- Row value comparison is more concise and better optimized by PostgreSQL
- Follows existing patterns in post_store.go (postSliceColumns)
- Standard SQL:2003 syntax
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Change posts reporting response from map to ordered array
Replace the Posts map with an ordered array to preserve query sort order
and provide a more natural API response for sequential processing.
Changes:
- ReportPostListResponse.Posts: map[string]*Post → []*Post
- Store layer returns posts array directly (already sorted by query)
- App layer iterates by index for metadata enrichment
- Remove applyPostsWillBeConsumedHook call (not applicable to reporting)
- Update API tests to iterate arrays instead of map lookups
- Update store tests to convert array to map for deduplication checks
- Remove unused "maps" import
Benefits:
- Preserves query sort order (ASC/DESC, create_at/update_at)
- More natural for sequential processing/export workflows
- Simpler response structure for reporting/compliance use cases
- Aligns with message export/compliance patterns (no plugin hooks)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Fix linting issues in posts reporting tests
Replace inefficient loops with append(...) for better performance.
Changes:
- Use append(postSlice, result.Posts...) instead of loop
- Simplifies code and follows staticcheck recommendations
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Fix store test AppError nil checking
Use require.Nil instead of require.NoError for *AppError returns
to avoid Go interface nil pointer issues.
When DecodeReportPostCursorV1 returns nil *AppError and it's assigned
to error interface, the interface becomes non-nil even though the
pointer is nil. This causes require.NoError to fail incorrectly.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Mattermost Build <build@mattermost.com>
* Adds default values to the attrs of CPA fields and refactors the app layer
* Fix mmctl tests
* Fix types and linter
* Fix model test
---------
Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es>
Co-authored-by: Mattermost Build <build@mattermost.com>
* [MM-65830] Fix mmctl system status exit code for health check failures
Make mmctl system status return non-zero exit codes when health checks fail.
This addresses the customer blocker issue where AWS ECS health checks were
broken after the distroless Docker image change, as they rely on exit codes
to determine service health.
Changes:
- Return error (non-zero exit) when server status != "OK"
- Return error when database_status != "OK"
- Return error when filestore_status != "OK"
- Return success (exit code 0) only when all components are healthy
- Add comprehensive test coverage for all health check scenarios
- Maintain backward compatibility for missing status fields
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* [MM-65830] Refactor mmctl system status to exit silently on health failures
Update mmctl system status to exit with code 1 when components are unhealthy
without outputting error messages. This addresses the customer blocker where
AWS ECS health checks were broken after the distroless Docker image change,
as they rely on exit codes to determine service health.
Changes:
- Add withClientAndExitCode wrapper for commands returning (bool, error)
- Modify systemStatusCmdF to return (bool, error) instead of error
- Return (true, nil) when any component is unhealthy (causes exit code 1)
- Return (false, nil) when all components are healthy (causes exit code 0)
- Return (false, error) for actual API/network failures
- Always print status information before checking health
- Update comprehensive test coverage for all scenarios
The refactored approach uses a clean adapter pattern that converts (bool, error)
signatures to standard error returns while delegating to existing withClient
infrastructure to avoid code duplication.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* [MM-65830] Ensure printer output is flushed before exit
Add printer.Flush() call before os.Exit(1) in withClientAndExitCode wrapper
to ensure all buffered output is properly written before the program exits.
This ensures status information is always displayed to users even when
exiting with non-zero exit codes.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Add godoc comment
* Output content then error out
---------
Co-authored-by: Claude <noreply@anthropic.com>
* Updates buildFieldAttrs to preseve existing attrs when editing a field
* Fix preserve option issue for select/multiselect type fields
* Fix linter
---------
Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es>
* Improves `mmctl cpa` subcommands' output to show human readable values instead of IDs
* Adds mmctl docs updates
* Fixed linter
---------
Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es>
* Adds Custom Profile Attribute field commands to mmctl
* Fix linter
* Refactor buildFieldAttrs
* Reverse test to match implementation
* Adds a confirmation prompt
* Refactor the tests
---------
Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es>
Co-authored-by: Mattermost Build <build@mattermost.com>
- Removed UpdateUserEmailCmd and UpdateUsernameCmd commands
- Deleted updateUserEmailCmdF and updateUsernameCmdF functions
- Commands were deprecated in favor of 'mmctl user edit email/username'
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-authored-by: Claude <noreply@anthropic.com>
* MM-64807: Deprecate format parameter in client config endpoint
- Remove requirement for format=old query parameter in /api/v4/config/client
- Endpoint now returns client configuration by default without parameters
- Maintain backward compatibility - format parameter is accepted but ignored
- Replace GetOldClientConfig with GetClientConfig across all clients
- Update API documentation to reflect simplified endpoint
- Update webapp client to remove format parameter usage
The endpoint previously returned HTTP 501 without format=old parameter.
Now it returns the client configuration directly, making the API more
intuitive while preserving compatibility with existing clients.
* Update i18n strings after format parameter deprecation
* Update E2E tests to use getClientConfig instead of getClientConfigOld
- Replace getClientConfigOld calls in playwright test library
- Aligns with format parameter deprecation in MM-64807
* Keep format=old parameter in webapp getClientConfig for pre-v11 compatibility
* MM-63726: Adjust default MaxOpenConns/MaxIdleConns settings
- Change MaxOpenConns from 300 to 100
- Change MaxIdleConns from 20 to 50
- Establish a healthier 2:1 ratio instead of the previous 15:1
- Remove hardcoded values from config files to use new defaults
* Fix mmctl config test for updated MaxIdleConns default
Update test expectation from 20 to 50 to match the new default
value for SqlSettings.MaxIdleConns that was changed in the previous
commit.
* MM-6449 - manage channel access rules permissions backend part
* add the system console changes to show the new permission
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
* Add mmctl user edit command
* Add e2e tests
* Use right client method to update authdata
* implement local API endpoint
* Don't allow users to clear the authdata
* make mmctl-docs
* Fix casing
* Fix example
* make mmctl-docs
* Simplify error message
* Fix test
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
Domenico RizzoandGitHub
Felipe MartinandGitHub
Harrison HealeyandGitHub
David KrauserandGitHub
Daniel Espino GarcíaandGitHub
Ibrahim Serdar AcikgozandGitHub
Alejandro García MontoroandGitHub
Nicolas Le CamandGitHub
Agniva De SarkerandGitHub