When TruffleHog times out in pre-receive hooks, it fails to output
diagnostic logs.
Changes:
- Ensure log output is flushed before process termination in all exit paths:
* Defer log sync in run() function for normal exits
* Sync logs in signal handler before os.Exit(0)
* Sync logs before os.Exit(183) when results are found
* Sync logs in logFatalFunc before os.Exit() calls
- Use exec.CommandContext instead of exec.Command for git log/diff
to ensure processes are killed when context is cancelled
- Add WaitDelay to git commands to provide grace period for cleanup
This ensures diagnostic output is captured when git operations block in
pre-receive hook environments and logs are visible when process is killed.
Co-authored-by: Kashif Khan <70996046+kashifkhan0771@users.noreply.github.com>
Fixes#3338
TruffleHog fails to parse git commit dates when the system locale is non-English (e.g., German outputs Sa. instead of Sat) preventing users from scanning git repos(all git-based sources like local git repo, github, gitlab, etc). This is because the format that is being used --date=format:%a %b %d %H:%M:%S %Y %z is dependent on strftime(3) C library function which is locale dependent(uses POSIX LC_ALL, LC_TIME, LANG). Setting LC_ALL=C or any other locale on the user's side like LC_ALL=de_DE.UTF-8 trufflehog ... doesn't help because the code overwrites the subprocess environment.
This means git only receives GIT_DIR and no locale settings. Rather than fixing env inheritance or adding LC_TIME=C, this PR switches to --date=iso-strict which outputs locale-independent ISO 8601 timestamps (2024-09-28T07:59:21+00:00). I think this is a pretty good solution: iso format has been stable since Git 2.2 (2014), requires no environment manipulation, and uses Go's native time.RFC3339 parser.
Error message when parsing:
2026-01-10T00:28:06-05:00 error trufflehog failed to parse commit date {"source_manager_worker_id": "rc4SK", "unit_kind": "dir", "unit": "/var/folders/hw/r5j4bcyd3472ccwjz0klh5840000gn/T/trufflehog-25309-2688385567", "repo": "file:///Users/xxx/xxx/sample-repo", "commit": "5f506baa305831998a2e15aa07cd381a69fde48f", "latestState": "AuthorDateLine", "error": "parsing time \"Sa. Jan. 10 00:11:44 2026 -0500\" as \"Mon Jan 2 15:04:05 2006 -0700\": cannot parse \"Sa. Jan. 10 00:11:44 2026 -0500\" as \"Mon\""}
Tested on homebrew git on mac(apple git seems to completely ignoring locales) and linux (debian)
* Expanded test coverage for binary git diffs
* reverted old changes and added binary test in chunker
* revert more
* added filesystem binary file scan test
* Better Test Output For Git Diff Parsing
This test file previously compared parsed diffs using a method `Equal` on the Diff
struct. This would fail (and stop) at the first mismatch, and left reporting the
failure to the caller. The existing test output made it hard to understand _what_
the difference was when the difference was in the diff content itself because the
_content_ of the contentWriter field wasn't actually processed and printed.
This change adds an `assertDiffEqualToExpected` function that leverages much of the
existing test structure but centralizes the assertion _and_ reporting of the differences
into a single reusable method.
I'm experimenting with ways to manipulate the git log that's being used to generate diffs (specifically, limiting its depth in some cases). Tweaking RepoPath for this locally isn't a huge deal but making the log command extensible seems like it could generally be useful, so I figured I'd do it.
* ensures that cmd.Wait() is always called, even if there's a panic in the FromReader function or if stdOut.Close() returns an error
* close stdout and ensure wait is called when handling binaries
* process cleanup improvements
* lint
This is a follow-up to #2713 that fixes the strange test error.
As suspected, the failure was caused by additional diffs not being included in the test's expected data.
This fixes#2683. It scans the commit author, committer (which is typically GitHub <noreply@github.com> for GitHub, but can be different), and message.
It also scans Git notes.
* correctly use the buffered file writer
* use value from source
* reorder fields
* use only the DetectorKey as a map field
* correctly use the buffered file writer
* use value from source
* reorder fields
* add tests and update
* Fix issue with buffer slices growing
* fix test
* fix
* add singleton
* use shared pool
* optimize
* rename and cleanup
* use correct calculation to grow buffer
* only grow if needed
* address comments
* remove unused
* remove
* rip out Grow
* address coment
* use 2k default buffer
* update comment allow large buffers to be garbage collected
Waiting for the sub-command will block until all of `stdout` has been
read. In some cases, we return early due to failed chunking without
reading all of the data, and thus, get stuck waiting for the command to
finish. Closing the pipe will ensure `Wait` does not block on that I/O.
* correctly use the buffered file writer
* use value from source
* reorder fields
* use only the DetectorKey as a map field
* address comments and use factory function
* fix optional params
* remove commented out code
* Write large diffs to tmp files
* address comments
* Move bufferedfilewriter to own pkg
* update test
* swallow write err
* use buffer pool
* use size vs len
* use interface
* fix test
* update comments
* fix test
* remove unused
* remove
* remove unused
* move parser and commit struct closer to where they are used
* linter change
* add more kvp pairs to error
* fix test
* update
* address comments
* remove bufferedfile writer
* address comments
* adjust interface
* fix finalize
* address comments
* lint
* remove guard
* fix
* add TODO
* feat: initial support for bare repositories
* feat: use concatenation instead of formatting and os.Getenv instead of os.Environ
Signed-off-by: Savely Krasovsky <savely@krasovs.ky>
* fix: go-git update with pre-receive hooks fix
Signed-off-by: Savely Krasovsky <savely@krasovs.ky>
* fix: remove info about pre-receive hook from README.md for now
Signed-off-by: Savely Krasovsky <savely@krasovs.ky>
* fix: don't scan staged while using --bare option, fixes to make it work with the latest master
Signed-off-by: Savely Krasovsky <savely@krasovs.ky>
* fix: small refactor according to #1518
Signed-off-by: Savely Krasovsky <savely@krasovs.ky>
---------
Signed-off-by: Savely Krasovsky <savely@krasovs.ky>
* gitparse: Use an object for currentDiff instead of a pointer
* gitparse: Use an object for currentCommit instead of a pointer
* Revert "gitparse: Use an object for currentCommit instead of a pointer"
This reverts commit c5f0708b4a.
Amaan UllahandGitHub
Jordan TunstillandGitHub
Nabeel AlamandGitHub
Martin LocklearandGitHub
Cody RoseandGitHub
ahravandGitHub
Dustin DeckerandGitHub
Richard GomezandGitHub
MiccahandGitHub
Cody RoseandGitHub
Bill RichandGitHub
Savely KrasovskyandGitHub
Zachary RiceandGitHub