CodeQL setup doesn't stick: false positives and noise in reports
Formally, the check is in the pipeline, the branch is protected, the report opens — and yet no one looks at it.
Configuring CodeQL most often comes down to enabling a ready-made “everything” query suite without specifying build steps: the analyzer parses the code partially, the data path breaks at module boundaries, and dozens of findings unrelated to vulnerabilities surface in the report.
The cause of false positives is predictable. The tool doesn't know about custom input wrappers, internal sanitizers, and the corporate data access layer, so it considers any input that reaches a query unsafe.
The same pattern repeats across hundreds of files — the database fills with similar entries, and the real unsafe concatenation drowns among them.
The main loss from such noise is not the time spent triaging, but trust in the tool. Developers start closing findings without looking, add exclusions for directories, disable the check step during an urgent release — and static analysis quietly falls out of the process.
Ignoring findings becomes the norm before the team has time to assess the real effect of the rollout.
The right configuration answers three questions: which branches and directories to scan, which build steps to pass to the analyzer, and which queries actually make sense for this codebase.
Then comes work with the report itself — dividing findings by criticality, establishing a baseline so that new code is checked strictly, while old debt doesn't block merges.
We break down how this is addressed: from building a correct database and setting thresholds to rules under which the check becomes mandatory but doesn't hinder releases.
What changes after implementation: check time and code coverage
The check stops being a merge bottleneck
The first run of CodeQL builds a database over the entire project history and takes minutes; afterward analysis is incremental — only changed files and related modules are recomputed. The verdict on a merge request arrives in minutes, in parallel with tests. Fixes go into the same branch where they were found.
Analysis coverage is visible in numbers
We configure the scope: new commits, release branches, project languages.
The metric is the share of changed lines that passed the check; at the start it's usually below half, after enabling mandatory merge checking it moves toward 100% for new code, and we pull in history in portions. Uncovered areas are visible, not “somewhere, probably.”
Classes of defects that don't reach the merge
Injection into queries, unsafe deserialization, path traversal, secrets leaking in configurations. Analysis follows the data flow from source to dangerous call, so it catches composite chains too, not just pattern matches. Findings are closed in the branch — code without known exploitation paths goes to release.
Defects become a metric, not a one-off action
We count the number of issues per release, the share of false positives, and the time from finding to fix. The merge threshold prevents accumulations from growing, and comparison across releases shows whether the rules are working. After a few cycles, it's clear which classes of errors the team has stopped introducing.
Check run modes: comparison of options and limitations
CodeQL doesn't parse a file in isolation from the rest of the code: before checking, a project database is built, and which data transfer chains the engine sees depends on its composition.
The run mode is a choice between feedback speed and coverage completeness, not just a trigger setting. Below is a comparison of scanning options that teams realistically choose between at the start.
| Run option | When it fits | Analysis limitations |
|---|---|---|
| Analysis of changed files on push | Task branches, fast feedback to the author of the change | Cross-file chains aren't assembled: a leak through two or three modules will go unnoticed |
| Check on every commit in a pull request | Mandatory check before merging into the main branch | The database is rebuilt on every update; with frequent changes, builds queue up |
| Full scheduled run | Nightly control of the entire repository, including long-untouched modules | The finding arrives with a delay, and finding the culprit among parallel changes takes longer |
| Analysis of release branches and tags | Before building the artifact that goes to production | Late detection: the fix is already layered on top and requires review |
| Manual run | Incident investigation or a check before an external audit | No regularity: no one will notice a missed run |
How the modes combine
In practice, they aren't opposed: a quick filter over changed paths gives the author of a change feedback in minutes, a check on every commit in a pull request covers the main branch, and a full scheduled run picks up what didn't make the slice — generated code, configurations, outdated modules.
Discrepancies between modes are resolved through exclusions in the query configuration, so the same finding doesn't resurface at release.
CodeQL setup step by step: from repository audit to handover
CodeQL setup begins not with editing the configuration but with an inventory: until you can see which repositories contain what and how often they change, any rule set is a shot in the dark.
Below is the order we follow on each project; each step ends with an artifact that we hand over to the team.
- Inventory of repositories. We compile a list of services, languages, and release branches, separately marking archived projects — we don't run analysis on them.
- Building the code database. We deploy the database for each language: for interpreted ones — pure analysis, for build-based ones — a pass with compilation; dependencies and code generation are recorded in the task description.
- Query rule set. We take the standard set as a basis, add rules for our stack, and disable noisy ones — the result is versioned as a configuration file, not as manual edits in the interface.
- Pipeline integration. On changes — a quick pass, in the nightly run — a full one; reports don't block merges if the analyzer itself breaks.
- Cutting false positives. We review each batch of findings with developers: some are closed with a code fix, some with an exclusion with a reason and owner.
- Testing on real changes. We run it on actual branches: does the analysis catch a new vulnerability and does it break the build where it shouldn't.
- Handover. We deliver configurations, a list of exclusions with justifications, instructions for reading reports, and the escalation procedure for others' findings.
What we hand over at the end: configurations, rules, and policies
Everything that appears during CodeQL setup stays with the team in the repository and knowledge base — not in chat and not in someone's head.
The configuration lives next to the code and goes through review like any normal change, so checks don't fall apart when a contractor changes or an engineer leaves the project.
- Analysis configuration file — run conditions (branch change, merge request, scheduled nightly run), list of languages and branches, analysis depth.
- Rule sets — divided into mandatory and recommended, tied to module criticality: payment and authorization paths are checked more strictly than the rest.
- Exclusion scheme — what we don't check and why, with an owner and review deadline, so the list doesn't turn into a dumping ground.
- Report access policy — who sees findings, who can close them, how results are divided by roles and areas of responsibility.
- Finding response policy — review deadlines by criticality level, who does initial triage, at what point a task is created.
- Integration with the issue tracker — automatic creation of defects by template, with criticality labels and a module owner.
- Check documentation — how to run the analysis locally, how to read the report, how to add or weaken a rule.
- Development plan — which languages and checks are connected next, what gradually leaves the exclusions.
With such a kit, a new engineer becomes self-sufficient in one day, without calls and retelling context.
Case study: reduced noise and check time in the change flow
A repository with a dozen services, a shared build pipeline, static analysis runs — but no one reads the report. Over a quarter, about four hundred findings accumulated, more than half false: generated code, test directories, routine type casts. Developers learned to bypass the blocking check, and it stopped catching anything.
We began setting up CodeQL by splitting analysis into two profiles. A quick one runs over changed files on every merge request, a full one runs on schedule for the entire branch.
Generated code and fixtures went into exclusions, and the trigger threshold was tied to criticality: the check fails only if an exploitation path is found, not a potentially dangerous call. Additionally, we described rules for our data access layer — where the standard set was silent.
The share of false positives dropped from 62% to 14%. The time for the check to pass on a merge request decreased from 18–25 minutes to 4–6 — the result arrives before the author switches to the next task.
The share of findings closed before merge grew from 30% to 87%: a fix at the moment code is written takes minutes, while analyzing consequences in the working environment takes hours.
Separately, we established a policy: bypassing the mandatory check is possible only by explicit decision of the responsible person and remains in history. After two months, such cases are few, and the necessity of the check itself has ceased to be a subject of dispute. We review the rules as the codebase grows, otherwise the set will start making noise again.
What to do if checks are already deployed but reports are ignored?
Deployed checks whose reports no one opens are not security but background load on the build.
The cause is usually not CodeQL itself, but how its results are integrated: the first weeks bring hundreds of findings in inherited code, and trust in the report falls faster than time is found for triage.
First, the merge blocking threshold is revisited. The check is attached only to new changes — rules apply to lines actually being edited in the merge request, while old code goes into the baseline.
The build starts failing on a few findings instead of hundreds, and the team has a reason to review them rather than disable the check entirely.
The second step is revising the rule set: queries with a high false positive rate are disabled, and test directories, generated code, and external dependencies are excluded from analysis. Rules remain where a finding means an exploitable risk — such a list is not learned to be ignored.
The third step is responsibility. Each directory gets an owner assigned in the CODEOWNERS file, and finding triage is assigned to an on-duty engineer with a one-day response window.
A finding without an owner lives until it goes stale; a finding with an owner is closed or marked as accepted risk with justification.
The result is tracked by metrics: share of triaged findings, time to first triage, number of suppressions with justification. When these numbers move in the right direction, analysis stops being a formality and starts influencing release decisions.
Technical doubts: build, closed repositories, and load
Before starting, the same doubts come up — they concern not the analysis capabilities but how it will fit into the current development process.
Below are short answers for each point we encounter most often: non-standard builds, closed repositories, pipeline load, multilingual projects, and custom rules.
The build is non-standard — will analysis even work?
For compiled languages, parsing happens based on the build, so the key condition is reproducibility of the build step in a clean environment.
If the build pulls data from outside or requires manual actions, we switch to analysis without compilation: some checks at data flow boundaries are lost, but the baseline set of defects remains. On a pilot branch, this is visible in one run, before rolling out to all repositories.
Repositories are closed, code must not leave the perimeter
Analysis runs on executors inside your environment, and source code stays in your infrastructure. If necessary, we provide only aggregated results without code fragments externally — or provide nothing at all if reports are collected locally.
How much will this slow down the pipeline?
A full run with the main rule set takes minutes and is scheduled at night. Incremental mode works on merge requests — only changed files and what depends on them are checked, so developer wait time stays within a normal check step.
We have several languages in one project
Each part is analyzed with its own rule set, and results are consolidated into one report. We mark up monorepositories by paths so service owners receive only their own issues and don't mute others'.
Can we add our own check rules?
Yes, we describe custom rules for your internal conventions: prohibition of unsafe calls, mandatory access control checks, requirements for personal data processing.
Such rules are versioned together with the configuration so changes go through review rather than appearing silently.
Let's discuss the task: we'll select the set of checks for your project
Tell us which repositories and languages need to be covered by analysis, how the pipeline is currently organized, and what prevents reports from being read.
We'll conduct an audit on a pilot branch, show how many findings are actually significant, and propose a configuration for your change flow. The scope of work and timelines are calculated individually after reviewing the task.