Automating secret detection in Git with GitLeaks and TruffleHog

Secret detection: where keys leak into the repository

A secret rarely ends up in a repository because of malicious intent — usually it is a side effect of haste.

A key is temporarily written directly into the code to test an integration faster, an environment file is committed “along the way” with working changes, and a build token is left in the build configuration.

A day later, such a repository has already been cloned to a dozen machines, and the value lives in the commit history regardless of the fact that the line was deleted long ago.

History is the most underrated leak point. Deleting a line in a new commit does not erase it from previous ones: git log -p or a regular clone retrieves the old value in seconds, including forgotten branches and forks.

There is only one reliable answer — consider such a key compromised and revoke it, rather than hoping to rewrite history.

A separate category is environment files and deployment configurations. They are added to the repository before ignore rules, and then the values spread through images, caches, and build logs.

Checking at this layer — GitLeaks and TruffleHog — catches keys by format and entropy before they reach the working environment.

Therefore, secret detection is not a one-time cleanup but ongoing control: checking at the commit stage, scanning in the build pipeline, and inventorying what has already accumulated in history. We explain how this is structured and where to place checks.

What secret scanning delivers: measurable effect and classes of failures

Manual review finds a secret only if the reviewer deliberately reads lines with keys in file changes — in practice, only isolated cases are caught this way.

Automated secret scanning shifts the detection point left: the check runs on every commit and blocks the merge before the key reaches the main branch. The share of findings closed before merge grows from a few percent to 85–95% — the leak simply never reaches the shared branch.

The second effect is that an entire class of failures disappears. A key that has entered history is not removed by a commit: it remains in clones, forks, and build logs, and the response comes down to rotation and damage assessment.

The scanner walks the commit history and all branches, so it also finds what lies in old branches that no one looks at.

The third is incident response time. A report is generated on every merge, not once a quarter, so the interval between a key appearing and its rotation shrinks from weeks to minutes.

Findings are prioritized not by file alphabet: the rule separates keys by access type and environment, and TruffleHog additionally checks whether it is live — a test placeholder key and a production token stop looking the same and no longer compete for the team’s attention.

Secret detection: comparing check modes before the build

Check modes do not replace each other — each covers its own moment in the code lifecycle. Pre-commit checking stops a secret before it enters history, but it depends on discipline: the hook is easy to bypass with a skip flag, and on a new machine it is not there at all.

Checking in the pipeline works without local setup and sees what actually went to the server, but with a delay of a few minutes. Open-source scanners integrate into any of these points with the same rule configuration; only the launch location changes.

Server-side control when accepting changes cannot be bypassed by the developer, but a false positive blocks the whole team’s work, so a triage procedure is needed.

Periodic archive scanning is the only mode that finds what has been lying in history for months and years: it blocks nothing, but requires rotating found keys after the fact.

In practice, two or three modes are implemented at once: a fast local one, a mandatory server-side one, and a scheduled scan. The order of addition does not matter — their rules are shared, so the second mode connects to the first without rewriting the configuration.

Implementation order: from repository inventory to handover

Implementation begins not with installing a scanner but with inventorying: there are usually many times more repositories than are visible from the hosting interface — archived ones, branches, mirrors, contractor projects.

Until there is a complete list, any setup will cover half the surface, and a false sense of control costs more than no checking at all.

1. Repository inventory. We collect a list from the hosting, build pipeline, and local mirrors, mark active, archived, and external ones, and record the owner and access level for each.
2. Prioritization. Based on history composition and commit frequency, we determine where full history scanning is needed and where checking new changes is sufficient — this is the difference of hours of work.
3. Control scheme. We choose check points: pre-commit checking on the developer’s machine, a build step, a periodic scheduled run. In practice, a fast check on commit and a full one at night are combined.
4. Sensitivity threshold. We configure rules so as to filter out obvious noise (key examples in documentation, test environments) and not miss real leaks. The threshold is set by finding types, not by a single global value — otherwise the team gets used to ignoring the report.
5. Rule exceptions. We formalize an explicit list with reason and lifetime: permanent exceptions quickly turn into a blind spot, from which a key then leaks.
6. In-flow checking. We run the combination of GitLeaks and TruffleHog on real commits, measure the false positive rate and check time, and tune the threshold to an acceptable level.
7. Handover to the team. We provide the incident response procedure for a finding, the procedure for revoking a compromised key, and rotation of responsible people.

Each stage ends with an artifact — a list, a rule, or a procedure, so the result does not depend on a specific engineer.

Handover artifacts: rules, policies, rotation procedure

Secret scanning automation work ends not with a running scanner but with a package of artifacts that the team can use to continue on its own.

A scanner without rules and procedures quickly becomes a source of noise, and noise becomes a habit of closing findings without looking, so we hand over not only a configured GitLeaks but also everything that makes its checks meaningful.

• Rule set — templates for our stack, sensitivity thresholds, and key format descriptions, organized by category: updating one group does not break the others.
• Exception policy — a registry of paths and files that are not checked (test data, configuration examples), with justification, owner, and review date for each entry.
• Control scheme — three lines of defense: checking on the developer’s machine before commit, a blocking check on branch merge, and a nightly scan of the entire repository history.
• Rotation procedure — the sequence of actions when triggered: who revokes the key, in what order, how it is confirmed that the old access is closed, and where the incident is recorded.
• Team instructions — a one-page memo: how to run the check locally, how to dismiss a false positive, and whom to go to for a disputed finding.
• Responsibility matrix — who owns the rules, who triages the queue of findings, and who reviews exceptions once a quarter.

We put everything next to the code and in the team’s knowledge base, plus walk through a leak scenario with on-call engineers. This way the rules are updated rather than becoming outdated after we leave.

Case study: how secret scanning closed leaks in an old repository

The client came with a situation familiar to almost any long-lived repository: dozens of people have access, the history goes back years, and it contains keys that no one has revoked.

A separate risk area is old branches: they are not deleted “just in case,” and that is exactly where the scanner finds the most.

What a full history scan showed

We ran the check not only on the current state but also across the entire history: all branches and tags, including long-merged ones. We found 143 hits, of which 19 were active keys to the payment infrastructure, the internal image registry, and external services.

About two-thirds of the findings were sitting in branches outside the mainline — when checking only new changes, they would never have surfaced. We used a combination of detectors: rules for our key formats plus TruffleHog for entropy-based search.

What changed in the process

The check was divided into three levels: on the developer’s machine before pushing changes, a mandatory one on the build server, and a regular full-history scan with a report.

We keep the exception list for test placeholders explicit — without that, the stream of false positives kills trust in the check; in our case, their share fell from about 70% to 8%.

What was found was not simply deleted: every key was revoked and rotated, because a secret that has entered history is considered compromised.

Metrics before and after: response time to a finding — two or three days versus three minutes now; the signal comes with the file, branch, and commit indicated, and the responsible person receives it automatically.

Repeat leaks over six months of observation — zero versus six over a comparable period before implementation. There are no more branches with findings left in history.

How does Git history scanning differ from checking new commits?

Checking new commits works at the entrance: interception before changes are committed and control over what goes to the shared repository on push. The scanner looks only at added and modified lines, so the pass takes seconds and does not interfere with work.

The ceiling of this mode is that it sees the present and knows nothing about what already lies in history.

A one-time pass solves a different task. It walks all repository objects: every commit, all branches including abandoned ones, tags, and files deleted from the working tree but remaining in Git history.

On a repository with a long history, this is thousands of objects and minutes of work instead of seconds — but it finds what has been invisible in the current state of the code for years.

Depth of checking is the key difference between the modes. Flow control analyzes the difference between states; a one-time pass examines every tree snapshot, so it catches a secret added and deleted a dozen commits ago.

Rewritten history and branches that have not been visited for a long time are accounted for separately: findings there survive any edits to the working copy.

The main consequence: a one-time pass fixes nothing by itself. Deleting a file and cleaning history does not revoke the key — copies have already spread through clones, builds, and forks.

Therefore, its result is not a list of lines but a key rotation plan: the compromised one is revoked, a new one is issued, and only after that does it make sense to clean history.

In practice, the modes complement each other: ongoing control prevents new leaks from appearing, and a one-time pass closes accumulated debt. The order of actions is counterintuitive — first key rotation, then work with history.

False positives, bypassing checks, and old branches

Noise in reports is the main reason a secret scanner gets disabled after a month. Test data, key examples in documentation, configuration templates, and demo environments produce false positives and blur priorities.

We separate findings into levels: known placeholders go into a low-priority set, real keys stay at the top — the engineer triages a few signals instead of a hundred.

The second line is bypassing the check. As long as control relies on conscientiousness, someone will commit with a hook skip flag or move a change directly into a protected branch.

That is why the check sits on the server side, before merge: it cannot be disabled locally, and the change history remains complete.

How do you distinguish test data from real data?

By context: file path, variable name, environment, neighboring lines, value format. An example from documentation does not raise an alarm, while a production token in code does, so triage is by priority, not as one continuous list.

What should be done with abandoned branches?

Do not delete blindly. First an inventory: which branches are alive, which are forgotten, and what lies in them. Abandoned ones go to the archive with permissions disabled; live ones go under protection with a mandatory check before merge.

Let’s discuss the task: what we need from you to start

To come back with an estimate and implementation plan, rather than with clarifying questions a week later, we need a minimal set of inputs. Nothing beyond this is required at the start — we gather the rest as we go, once we see how your secret detection pipeline is structured.

• List of repositories and groups: where the code lives, what has been archived, which projects we deliberately leave out of scope — the volume of the initial run depends on this.
• Read-only access: a service account in the Git platform or a personal token with a limited lifetime and issuance log.
• Integration point: at which stage we connect the check — before merge, in the nightly build, or as a separate scheduled job.
• Security owner: the person who triages hits and owns the escalation matrix.
• Response procedure: what we consider critical, who receives the notification, and how much time is allotted for revoking and reissuing a key.
• Access to the secret store: so that a found value can be rotated rather than merely marked as a finding.
• Commit history policy: which branches may be rewritten and which are forbidden to touch — otherwise the baseline will show thousands of already irrelevant lines.
• Working hours and windows: when heavy runs can be launched and who remains available.

Once this is gathered, in one short conversation we fix the boundaries of the first stage and come back with a timeline estimate and plan: repository order, scanner integration method, acceptance criteria. Describe the task — we will come back with an estimate and a plan.