OWASP ZAP in the pipeline: why the scanner drowns in false positives
OWASP ZAP is connected to the pipeline with a single line, and the first run usually pleases: the report is full of findings, security checks during build are finally automated.
A month passes — the exclusion file already has dozens of rules, and the build is green regardless of what is in the code.
The reason lies in the nature of dynamic analysis.
DAST in the pipeline walks through the application as an external user and does not know the context: a missing security header on the login page and on the error page is the same to it, a form without a token on an internal staging environment looks like a hole, test data does not differ from real data.
For one real defect there are dozens of alerts that the developer closes with the wording "not our area."
Then the avalanche starts. An exclusion by rule ID suppresses a whole class of checks, an exclusion by URL mask suppresses an entire section of the application. The report grows, people stop reading it, the build failure threshold is raised so as not to block release.
False positives cease to be a report problem and become a process problem: the security check turns into a formality, and a new significant alert drowns among the old ones.
The result of such setup — a vulnerability goes to production with the release, while the report looks clean.
We break down how this is closed: separation of scanning modes, managed rules instead of disabled ones, marking of confirmed false positives, and a threshold that stops release only on something significant.
Dynamic analysis in the pipeline: measurable effect
Dynamic analysis checks not the source code but the running application: the scanner goes through URLs and scenarios just as an external attacker would, and records what is visible from the outside. Static checks do not replace such a scope — they work with a different layer.
The key metric here is time to defect detection. When a check is run once a quarter before release, the found problem lives in the main branch for weeks, and the context of the fix is already lost by the time of the report.
Running on every merge to the main branch reduces this interval to tens of minutes — the developer reviews the report while the task is still in progress.
The coverage of checks grows not from the number of rules, but from scenario-based runs.
Authenticated traversal under each role, passing through forms and file uploads, a separate perimeter for APIs — this is how paths are covered that are not in the source code and appear only in the built environment.
Whole classes of failures gradually disappear from reports: unclosed access between roles, data leaks in responses, lack of limits on repeated requests.
The OWASP ZAP scanner is installed as a separate pipeline step, and filtering rules cut off known noise — otherwise the team stops reading the report.
The trigger threshold is configured by risk level: the build fails on confirmed issues and continues on findings that require manual review.
Dynamic scanner run modes: when what fits
Dynamic checks can be embedded in different ways, and the mode is chosen not by the tool's capabilities, but by how much time is available for response and how predictable the test environment is.
A quick check of changes on every commit catches regressions in security headers, cookies, and data leaks, but does not perform active attacks.
A full run finds injections and authorization bypasses, but takes tens of minutes and requires a running staging environment with controlled data.
In practice, modes are combined: a cheap check on every change, an expensive one on a schedule or before deployment. Below is an honest comparison, including cases where the mode does not pay off.
| Mode | When it fits | Limitations |
|---|---|---|
| Change check on every commit | Dozens of builds per day, fast feedback needed for the developer | Works passively: does not find injections and authorization bypasses; sees only pages the traversal reached |
| Full scheduled run | Nightly builds, stable test environment | Takes tens of minutes, depends on data and staging state, some findings require manual review |
| Full run before deployment | Release stage, a stop valve is needed | Findings appear late, when changes are already in the branch; lengthens the deployment cycle |
| API check by interface description | Services without a UI, service-to-service communication | Coverage equals completeness of the description: an outdated description creates blind spots |
| Run with an authenticated session | Restricted sections, personal account | Requires an account and its maintenance, token lifetime is limited; the share of false positives grows |
The scanner's common limitation is that it does not understand business logic: it will not see price tampering, limit bypasses, or races with parallel requests.
Therefore, OWASP ZAP in the pipeline is installed together with finding filtering and build stop rules — otherwise noise devalues the checks. We break down how to set up false positive filtering and blocking thresholds.
Setting up checks in the pipeline: from audit to handover
A pipeline audit begins with inventory: which branches are built, where artifacts go, which environments are brought up automatically, and who owns the service.
Without this map, scanning is placed at the wrong stage — it either slows down the build or does not reach the deployed application. Next, we design the scheme and place OWASP ZAP at the identified points.
- Pipeline audit. We look at build and deployment stages, the time of each and the points where the test staging environment is brought up; we look for a place to scan without increasing run time.
- Scanning scope. We record URLs and roles in the perimeter, what we exclude (static files, third-party services), which accounts the scanner receives.
- Scanner rule setup. We select the rule set and thresholds for the application type, enable context and login under an account — restricted sections are checked, not only the login form.
- False positive filtering. We mark confirmed noise and maintain an exclusion file with justification; we return a rule if conditions have changed.
- Run modes. A quick set — on every branch change, full scanning — on a schedule; build fails only on new high-severity findings.
- Result verification. We run the scheme on a staging environment with a deliberately introduced vulnerability: the known issue is caught, the excluded noise does not surface.
- Handover to the team. We hand over the configuration, the rule update procedure, and the review order: who looks at the report, within what time frame they close findings, how they change thresholds.
False positive filtering rules and basic configuration
The OWASP ZAP scanner without configured exclusions quickly turns into a noise generator: the same finding about a missing security header surfaces in every run, the report grows, and real defects are lost in it.
Therefore, together with the scanner, the customer receives a set of artifacts — from configuration files to a procedure for reviewing reports.
- Scanner configuration: rule levels, significance threshold, set of active and passive checks, separate profiles for a quick run on every commit and a full one on a schedule.
- Exclusion policies and false positive filtering rules: binding to a path, parameter, or response pattern; a suppressed finding does not stop release but remains in the run log.
- Authentication context matrix: under which roles we scan, how the session is maintained so that restricted sections are not checked as public.
- Staging environment scheme: where the scanner is deployed, on which environments it runs, how it is isolated from production data.
- Report review procedure: who reviews new findings and within what time frame, how an exclusion is documented and when it is revisited.
- Integration scheme into the delivery pipeline: at which step the check sits and under which conditions it stops release.
- Acceptance checklist: a control run on a staging environment with pre-introduced vulnerabilities — shows that filters do not suppress more than needed.
Exclusions do not remain static: rules are reviewed together with the team so that suppressed findings do not accumulate and the filter does not turn into a way to ignore problems.
Case study: how noise was removed from scanner reports
Before implementation, OWASP ZAP was run manually once every two weeks, and the report was put in a shared folder. A typical run produced around 400 findings; after manual review, 25–30 real ones remained — the share of false positives stayed at 85%.
Engineers spent two days on review, the release slipped, and people got used to not looking at the report: "the same list again."
What was configured
Scanning was moved to every merge request — a quick pass over changed routes, and a full application traversal at night.
Filtering relies on three rules: cutting off by response code and signature (the same response to all input data is not a vulnerability), an exclusion list for endpoints that intentionally return test data, and deduplication of findings by vulnerability class and parameter.
Thresholds were set separately: the pipeline stops only on confirmed high-severity findings, the rest goes to the report. False positives that were nevertheless confirmed manually go into a suppression rule with a comment — the next build no longer raises them.
Result
The share of false positives is about 10%, with 25–35 findings in the report instead of 400. Review takes an hour and a half to two hours instead of two days, and scanning on merge fits within minutes and does not slow down the pipeline.
Confirmed vulnerabilities are closed before release: over the quarter, not a single high-severity finding reached production.
How OWASP ZAP differs from source code analysis
These are two different objects of verification, and they should not be confused. Source code analysis reads what is written: data flows from entry point to dangerous call, parameter handling, cryptography work.
OWASP ZAP looks at the running application from the outside and checks actual behavior, not the developer's intent.
Static analysis sees the data path as a whole, including code that has never been executed. But it does not know the real environment configuration: how the reverse proxy is configured, which headers the load balancer adds, whether internal addresses are closed. A dynamic scanner answers exactly this.
There is a downside too: ZAP does not point to the line with the error, while static analysis produces noise on paths that are not assembled in a real request. Therefore, comparing approaches is not about "which is better," but about the class of defects that needs to be closed.
In practice, the methods complement each other. Code review is placed on every commit — it is fast and points to the exact location.
ZAP is run on the pre-production staging environment after deployment: scanning goes through real routes, with real authentication and sessions, and alerts are filtered so that the report does not drown in noise.
The choice of verification method is dictated by the stage: code checking — at the entrance to the build pipeline, application checking — before release.
If an answer is needed to the question "is the hole exploitable," OWASP ZAP provides it; if "why did it appear" — source code review.
Typical technical doubts before implementing checks
Before work starts, engineers have the same questions: will the check slow down the build, where should it run, how to pass authentication, and what to do if a scanner is already deployed.
The configuration depends on the answers — which rules are enabled, at which stage the traversal starts, and by which threshold the build is considered unsuccessful. Below we briefly address four doubts that most often surface before the first run.
It is important to divide checks into two modes. Passive analysis examines already generated traffic and has almost no effect on build time; active traversal sends its own requests and requires a separate time budget. Planning is built around this division.
Will checks noticeably slow down the build?
We keep passive mode on every change — it does not add noticeable delay. We move active traversal outside the fast build: to a schedule or nightly run, where we limit traversal depth and the number of concurrent requests. For the incremental run, we assign only changed routes, and we launch the full profile less often.
Do we need a separate staging environment for checks?
Active checks write data and can leave garbage, so we direct them to an isolated environment with anonymized content, not to the production one. The environment is brought up from the same configurations as production — otherwise some alerts will be false.
How to check sections behind authorization?
We pass the scanner a saved session and context with a token for the required role, and exclude the login form from traversal so that it does not waste time brute-forcing it.
Separately, we configure session refresh on expiration and a run for each role — some vulnerabilities are visible only with the permissions of a specific user.
We already have a scanner deployed — what will change?
We move settings into configuration next to the code: rules, exclusions, and thresholds become reproducible, and the result becomes comparable between runs. We add false positive marking so that the report does not drown in noise and the build fails only on confirmed findings.
Let's discuss the task: what to send for assessment
Assessing the implementation of security checks in the build pipeline depends not on the size of the project, but on how clear the scope is: how many services, where staging environments are brought up, who reviews findings.
The more input data you send during the first conversation, the more accurate the work estimate will be and the fewer clarifying iterations there will be.
To discuss the task substantively, please send: a pipeline diagram and list of stages, a list of services and their types (web interface, API), which environments are brought up automatically and how they are filled with data, whether a scanner is already deployed and its current configuration, and who on the team is responsible for reviewing findings.
This is enough to propose an integration scheme, run modes, and a procedure for working with false positive filtering. The scope of work and timelines are determined individually after familiarization with the scope.