Container scanning: where image control breaks
A base image in a project is usually not reviewed: an engineer takes a public build, adds application dependencies, and pushes it to the registry.
An update of a "foreign" base build goes unnoticed — it is not in the project tree, no one looks at its layers, and the tag remains the same, only the content under it changes.
Weeks later, container scanning shows a critical vulnerability in a system package that is not in the application source code. It came from the base layer — and dozens of other services run on the same build.
Patching one service does not close anything: the next release of a neighbor will be built on top of the same hole.
They try to control images before deployment manually: look at the tag, run through the package list, compare with the previous release.
On a few dozen services and two or three environments, this is not reproducible — the result depends on who is on duty, and the layer history is not stored anywhere.
Vulnerable images in production live not because they were not searched for, but because they were searched for once and in the wrong place.
The cost: an emergency rollback in the midst of an incident, an unscheduled release, and a manual walk through the registry to assess the scale.
Some services remain on the old build for months — it needs to be rebuilt, rechecked, and rolled out, and this is manual work for each service individually.
We break down how this is closed: where a mandatory gate appears in the build, what it checks before deployment, and how a vulnerability that came from a foreign base build is caught.
What early image scanning in the pipeline gives
Early image scanning shifts the control point left: scanning happens immediately after the build, before publication to the registry.
The scanner parses layers and the manifest, compares system packages and application dependencies against the vulnerability database, and returns findings with severity levels. At this stage, a failure costs a rebuild, not a release rollback.
The build policy cuts off high and critical findings in the layers that reach the runtime environment; notes on development-level dependencies remain a warning. This removes a whole class of failures — a vulnerable base image, a forgotten package, a library without updates.
We configure blocking thresholds according to the container environment risk classification described in OWASP: this provides a common language with the security team instead of manual agreements.
Deployment time does not grow: the check runs in parallel with tests, only changed layers are scanned, and the unchanged base image is taken from cache. The result arrives before the build finishes — there is no separate waiting stage in the pipeline.
A release that previously went into operation and was rolled back now simply does not proceed further.
Incident response time is determined not by the scanner's speed but by the availability of an inventory.
Each image has a list of components with versions attached, so when information about a new vulnerability appears, we find all affected images with a single query rather than manually iterating through the registry. Then — a rebuild on the fixed base layer, and rollback to the previous tag remains available.
The result: control is moved to where the fix is cheaper, and data about the image composition remains at hand at the moment of an incident.
Container scanning methods: comparing modes
Modes differ in at which stage of the image lifecycle the check is performed and which layer it sees.
A build-time check catches vulnerabilities before publication, scanning in the image registry insures tags that came from outside, and periodic inventory finds what is already running on testbeds. Therefore, container scanning is built as a sequence of modes rather than choosing one.
| Mode | When it fits | Limitations |
|---|---|---|
| At image build | Your code, control at the entrance: the pipeline stops on critical findings | Slows down the build; does not see images built outside the pipeline |
| At publication to the registry | Multiple teams and a shared registry: a unified policy for all tags | Reacts after the build fact; requires integration with the registry |
| Before deployment | Clusters with strict requirements: launch is rejected until policy is passed | Fragile: breaks deployment when rules change, an exception list is needed |
| Periodic inventory of running | Long-lived testbeds: shows actual state rather than intent | Lags: the image is already running while the check is in progress |
A common limitation of checks is that the vulnerability database is updated, and an image that was clean yesterday gets critical findings today. In addition, the check does not distinguish a library that is actually called at runtime from dead code.
Therefore, the policy is built on priorities: block only what already has a fixed version, the rest goes to a queue with a fixed remediation deadline.
How we embed checks into the pipeline: stages of work
The order of stages depends on how your build pipeline is arranged: where the image is built, how it gets to the registry, who is authorized to publish it. We embed container scanning so that the check step does not stretch the build and does not stop releases on false positives.
- Audit of the build pipeline. We analyze the stages where images are formed and what access policies to the registry are already in effect. We record the build time before changes — this is the reference point for estimating overhead.
- Image check scheme. We determine what and at which step we check: base layer, dependencies, credentials in layers, launch configuration. Fast checks — on every commit, full analysis — on the release branch.
- Setting up admission rules. Critical vulnerabilities block publication, medium level goes to a queue for review. We formalize exceptions in writing, with a reason and a review date, otherwise the rules quickly become a formality.
- Testbed check. We run the scenario on real project images: measure the added time, look at the share of false positives, and check rollback to the previous build.
- Handover of regulations. Description of the scheme, the procedure for reading reports, rules for updating the vulnerability database, and a responsibility matrix: who handles the notes and how to change thresholds without modifying the pipeline itself.
Admission policies, reports, and schemes: what remains with the team
Scanning ceases to be a one-time check when it relies on artifacts that live in the repository and internal knowledge base. Below is the minimum with which the on-duty shift and release engineers work without our participation.
- Check configurations in the repository: build stages, severity thresholds, exception rules for false positives and external dependencies.
- Image admission policies — which vulnerability levels block a release, what passes by exception, and who signs it off for a term.
- Separate rules for service and application images so that a release does not stall because of findings that do not apply to the container's role.
- Pipeline schemes: the place of the check in the chain and the reaction to a trigger — stop, warning, or release with a mark.
- Vulnerability report templates: composition of findings, priority, responsible person, remediation deadline, sign of recurrence.
- Instructions for the on-duty shift: how to read the report, what to do when blocked before a release, and by what sign to escalate.
- Regulations for synchronizing vulnerability databases and reviewing exceptions — so that a temporary relaxation does not become permanent.
The artifacts lie next to the code and are linked to the pipeline scheme: when editing a policy, it is immediately clear which checks are affected. The decision to admit an image becomes reproducible — it is made by a described rule, not by a specific engineer.
Case study: how vulnerable images were removed from deployment
What was
Images were built by the same jobs as now, but vulnerabilities surfaced already in production — they were found by a perimeter scanner or an incident.
Security service notes accumulated in a separate registry: there were several dozen critical ones, and the closure time reached three weeks because updating the base layer pulled a rebuild of dependent services.
What was changed
The check was placed between the build and sending the image to the registry; the report based on Trivy was split into two thresholds.
Critical vulnerabilities with an available fix block deployment, medium ones go as a task with a deadline, false positives are suppressed by exceptions in the policy file. Bypassing the threshold is possible only by editing the policy — this remains in the change history.
| Metric | Before | After |
|---|---|---|
| Share of rejected images | no blocking | 17–20% of builds |
| From detection to fix | 2 days | 3–5 hours |
| Closure of security notes | up to 3 weeks | 2–4 days |
| Critical vulnerabilities in production | dozens | units |
Scanning added about forty seconds to the build — the cost was accepted deliberately because deployment time was determined by waiting for checks, not by the build.
The share of rejected images did not stabilize immediately: at first almost a quarter of builds were rejected, then services switched to supported base images, and the metric holds at one or two releases per week.
How does base image scanning differ from layer scanning?
Base image scanning answers the question of whether vulnerable legacy came from the parent layer. The scanner goes through the manifest and compares distribution package versions: the verdict hardly depends on your code and changes when the base image itself is updated.
Layer scanning adds everything you put on top: compiled binaries, application dependencies, copied configs. Scanners like Trivy read the layer history, and secrets also surface here — a file deleted in the next layer remains in the previous one and is retrieved from the registry. Therefore, base image control does not cancel layer-by-layer scanning.
The third level is scanning a running instance. It sees the actually mounted filesystem, process permissions, and attached volumes, including packages installed at startup.
The strictness of rules grows with the level: on the base image, the build is failed on critical findings; on layers, they warn and require review; on a running instance, rollout is blocked only for reachable vulnerabilities.
Common technical doubts before implementing checks
Engineering objections before implementing checks are usually the same: analysis will slow down the build, raise noise with false positives, and force rebuilding already released images. Below is how each of them is addressed in practice, without stopping the release.
How to suppress false positives?
We do not argue with findings; instead, we configure the policy: exception rules indicating the package, path, and reason, with a mandatory review date and an owner.
Only critical and high severity vulnerabilities block the release; the rest goes to the report and is handled according to plan. This way the queue of requests to "disable the check" does not grow.
How much will the check slow down the build?
The base image is scanned once and reused from cache; on each build, only the changed layer is checked — that is, new code and updated dependencies. A full pass is launched on a schedule, not on every commit, so the increase in build task time is measured in seconds, not minutes.
What to do with already published images?
We enable observation mode: the registry is traversed entirely, findings are collected into a report, and the release is not blocked.
Then a separate replacement plan goes for critical findings, and the gate with blocking is enabled only for new builds — old tags live until a scheduled update, without an emergency rewrite of the release.
Can third-party vendor images be checked?
Yes, but you cannot trust someone else's report: we check the base layer and dependencies with our own rules, and we rebuild opaquely built base images ourselves from source.
We pin tags by the image checksum, otherwise the vendor will substitute the content under the same name, and the check will lose its meaning.
Will we have to change our scanning tool?
No. The existing scanner is connected as a separate step in the build chain, and reports are brought to a unified form — triggers from different sources end up in the same queue.
We make a replacement only if the tool does not support the required output format or does not cover the required vulnerability classes.
Let's discuss the task: where the implementation of checks begins
The implementation of checks begins not with configuring the scanner but with an inventory: what is already being built, from which base images, and where it goes upon release.
The first stage is access and analysis of the current pipeline, because policies written without understanding branching and build order block releases instead of protecting them.
The work plan for the pipeline is built in waves: first, a build-time check in observation mode, then blocking rules for critical services.
To return with an estimate and a plan, not with general words, we need a short starting set from your side:
- read access to repositories and branching settings — we see where the entry points are and who is authorized to change rules;
- access to the image registry with permission to read tag metadata and layer history;
- description of the current pipeline: stages, triggers, who and how releases to the production environment;
- the person responsible for rules — an engineer who makes decisions on severity thresholds and exceptions;
- a list of services that cannot be stopped, and a list of those where blocking mode is acceptable;
- information about base images: own or public, who is responsible for updating them;
- a contact for acceptance — the one who will confirm that the check does not break the release.
Describe the task in two or three paragraphs: how many services, what pipeline, where the boundary between build and release currently lies.
We will return with a timeline estimate, a sequence of stages, and what you will get at each step — from the first run in observation mode to blocking policies.