* [INS-241] New detector (datadogapikey) for datadog apikeys
* Analyzer updated to cater endpoint
* Added new tests for anlyzers
* Removed print statement
* resolved comments and fixed integration tests.
* resolved comments
* changed cli prompt
* Fixed the comments and added app key validation in analyzer
* renamed regex variable
* Added found verified endpoint to ExtraData
* Clean up Analyze function by removing comments
Removed commented-out code for appKey and endpoint.
* [INS-286] Added support to analyze just the apikey in datadog's analyzer
* fixed linter issue
* fixed comment and introduced snake case to make analyzer code cosistent and also fixed flaky tests
* resolved bugbot comment
* updated protos
* resolve conflicts
* updated protobuffs and resolved bugbot comments
* made regex idiomatic
* fixed ssrf vulnerability
* fixed string formatting
* added detector for artifactory reference tokens
* add artifactory reference token detector to the no cloud endpoints list
* address mustansir feedback; remove the invalid host deletion
* use detectors.DetectorHttpClientWithNoLocalAddresses instead of common.SaneHttpClient() just like sibling artifactory detector
* enabled symlinks with maximum depth support
* resolved concurrency bugs and added maxDepthOption to cli
* Removed visited path map and tracked symlink depth by maintaining a counter variable
* separated symlink scanning from scanDir
* resolved bugbot comments
* introduced hash as a separator to avoid collisions
* add google gemini api key detector
* change detector name to google cloud api key, mark as verified if 403 is returned
* add build tags for integration test
* changes in defaults.go
* Revert "changes in defaults.go"
This reverts commit 12e7b6f4aa.
* Revert "change detector name to google cloud api key, mark as verified if 403 is returned"
This reverts commit e46bb29b40.
* revert google cloud api changes, change keyword to gemini, add extra field active_google_key
* use aizasy as keyword instead of gemini
* close response body after draining
* remove \b from regex to support keys that end with -
* add \b to the beginning
This adds a new input source to TruffleHog, accessible via `trufflehog json-enumerator`.
This input source requires a list of filenames, each of which is an NDJSON-formatted sequence of objects that take one of two forms:
Form 1: `{"data": "utf-8 string", "metadata": <non-null JSON value>}`
Form 2: `{"data_b64": "base64-encoded bytestring", "metadata": <non-null JSON value>}`
The `data` / `data_b64` field specifies the content to be scanned. The `metadata` field is arbitrary, and is simply propagated downstream with scan results from the corresponding content.
Note that although `trufflehog json-enumerator` requires a list of filenames to be given, the NDJSON data that you wish to scan may not need to be first written to disk. On Linux and macOS, at least, you can use shell process substitution to set up a named pipe from a producer process, like `trufflehog json-enumerator <(some-program-that-emits-ndjson)`.
This adds a generic detector and verifier for generic JWTs.
This uses regular expressions for detection. Only public key cryptography algorithms are supported. Additionally, OIDC Discovery is attempted against the issuer to fetch the public key for signature verification.
Bonus Cleanup:
* Fix mixed indentation in alchemy test
* Use `strings.ReplaceAll(...)` instead of `strings.Replace(..., -1)`
* Use integer range loop instead of explicit counting loop
* Revise and expand detectors.go comments
---------
Co-authored-by: Kashif Khan <70996046+kashifkhan0771@users.noreply.github.com>
* Removed redundant IncludeRepos mentions.
* removed proto for IncludeRepos and remade protos
* removed another instance of includeRepos
* reverted proto removal and
* actually deprecated the field
* ran make protos
This commit prevents TruffleHog from executing arbitrary commands located in archived malicious git repositories. Thanks to Adam Reiser at Cisco Talos for pointing this out!
This approach uses Git's recommended best practice for sanitizing untrusted git configs: git clone all local file:// git repos prior to scanning. Executing git clone does not execute any of the potentially malicious git configs in the untrusted repo directory, and the output only includes "safe" default configs, similar to what we see when cloning from remote.
We explored a few other approaches (allowlist, denylist, etc), but those carried lots of complications.
A few notes about how this works:
This only applies to local repositories scanned using the git subcommand.
Remote git targets are not impacted.
Local git targets are now cloned to temp by default prior to scanning. Users can specify a --clone-path argument if they don't want to use the default temp dir. Users can specify --trust-local-git-config if they want to trust the repo as is and bypass cloning.
Local --bare repos are handled appropriately.
This approach knocks out (most...all?) of this class of malicious git config vulnerabilities.
Testing:
There's coverage for most of the new code, including: test cases for the specific issue reported, local bare clones, local repos with staged commits, etc. All are passing.
* change in protos to support scanning confluence comments
* rename comment -> commentId
* use snakecase for comment_id
---------
Co-authored-by: Shahzad Haider <76992801+shahzadhaider1@users.noreply.github.com>
* fix legacy json flag for gitlab private repos
* some code
* incorporated code from main
* remove cloned repositories after the scan is complete
* enhanced the code
* comment addressed
* changed the approach for persisting the repositories for legacy json printing
* addressed comments; updated the variable names
* Added support for additional validation rules in custom detector
* refactored the approach to support validations for each regex in config
* resolved comments
Added a field in the proto definition for Bitbucket sources to explicitly allow writing to the user's secrets manager. Initially this will be used to keep new and unused Bitbucket refresh tokens inside of the secret that we pull the config from. This process needed to happen in the first place because Bitbucket as an OAuth authorization server currently revokes any previously issued refresh tokens whenever the newest refresh token is issued with an access token, effectively making refresh tokens in the config file single-use.
Update the proto definitions found for Bitbucket sources to have OAuth option in the config file. Also regenerated the Protobuf files with the changes.
* gitlab groups init
* added list group projects api
* list group projects updated
* added duplicate repo scan check
* comments addressed
* added error when repo and group id flags are provided at the same time
* added test case for gitlab group projects
* add bitbucket app password scanner
* clean up regex and username pattern logic
* feat: re-intro bitbucket in engine.
* feat: add BitbucketAppPassword detector type to proto files
* Update pkg/detectors/bitbucketapppassword/bitbucketapppassword.go
Co-authored-by: Amaan Ullah <amaanuj.dev@gmail.com>
* refactor(bitbucket): tests (+patterns) ; code cleanup
- add switch{} block + credentialPattern[] for readability
- + tidy nested loops
- reflect codebase conventions in verification scope
- hard drain io.Discard, body.Close on res call in verify func
- intro the patterns test as it was not introoed
- standardize the integration test based off others living in repo
* Update pkg/detectors/bitbucketapppassword/bitbucketapppassword.go
Co-authored-by: Amaan Ullah <amaanuj.dev@gmail.com>
---------
Co-authored-by: Brandon Yan <yanjbrandon@gmail.com>
Co-authored-by: x-stp <x-stp@users.noreply.github.com>
Co-authored-by: Amaan Ullah <aman.ullah.jalal@trufflesec.com>
Co-authored-by: Kashif Khan <70996046+kashifkhan0771@users.noreply.github.com>
Co-authored-by: Amaan Ullah <amaanuj.dev@gmail.com>
Description:
Add support for excluding paths in Docker source scanning:
Add ExcludePaths field to Docker protobuf
Implement path exclusion logic in docker.go
Add comprehensive test coverage for exact and wildcard path matching
Update engine to pass exclude paths configuration
Add CLI support for --exclude-paths flag
The implementation supports:
Exact path matching (e.g., /var/log/test)
Wildcard path matching (e.g., /var/log/test/*)
Multiple exclude paths
Tests ensure proper handling of:
Exact path exclusions
Wildcard exclusions
Edge cases and similar paths
References:
https://github.com/trufflesecurity/trufflehog/issues/2216?utm_source=chatgpt.com
Mustansir Muzaffar
Amaan Ullah
Nabeel Alam
Muneeb Ullah Khan
Shahzad Haider
meredith
Brad Larsen
Jordan Tunstill
joeleonjr
Casey Tran
Dustin Decker
Charlie Gunyon
Tanner Jones