Encryption at rest: where protection breaks
Encryption at rest is usually considered a solved problem: the virtual machine disk is encrypted, so nothing can be read from the storage medium. Yet data protection on the medium breaks at the periphery — where copies appear: snapshots, exports, cache, logs. It is these places that produce most incidents, not the primary volume.
The most common channel is unencrypted snapshots and backups. A volume snapshot inherits the parent's encryption, while a database dump or directory copy sent to object storage goes out in plaintext: encryption is configured at the disk level, but not at the export level.
A backup leak means the entire database can be extracted from the archive — with its schema and relationships.
The second layer is application cache and log archives. Sessions and temporary exports land on the node's local disk, while logs contain tokens and personal data in plaintext. Formally this is not "data"; in practice, it is ready access to the service.
A separate risk is insider access: if the key lies next to the data, in the same project or on the same node, encryption protects against removal of the medium, but not against a compromised account.
Keys are moved to a separate vault with access control and an access log — for example, HashiCorp Vault — and encryption is built on two levels: block volume and key management service.
An encryption policy then answers three questions: what is encrypted by default, where keys live, and who rotates them. Below we examine how this is covered.
What changes after implementation: risk classes and response time
The first effect is that the failure class "medium outside the perimeter" disappears.
A disk from the server room, a decommissioned drive, a removed disk snapshot, or an archived copy without the key reads as a random set of blocks: volume encryption works at the device level and does not depend on application care.
The share of unprotected data is reduced not partially, but to zero — both backups and temporary volumes come under protection.
The second effect is access separation for keys. The key lives separately from the data, in HashiCorp Vault or a managed key service, and is issued to a specific role for a specific operation for a limited time.
This breaks the link "access to disk = access to contents": the storage administrator sees metadata but cannot decrypt the volume. Scheduled rotation does not require re-encrypting the entire array — only the wrapping key changes.
The third is that data access auditing becomes verifiable. Each key issuance leaves a record: who, when, to which resource, and on what basis received access. Incident investigation proceeds from the log, not from polling teams and reconciling application logs.
Hence the savings in response time. The scenario "a medium is missing" used to mean days of investigation: inventorying copies, estimating scope, notifications. Now it comes down to hours: record the fact, revoke the key, issue a new one.
A leak is closed by revoking access, not by rewriting all copies — this reduces both the risk window and the amount of manual work during an incident.
Comparison of approaches: encryption on disk, in storage, and for secrets
Approaches to encryption at rest do not replace each other: they cover different risk classes, and the choice depends on whom we trust with the key and what we protect — a medium, a separate object, or a secret.
Full disk and volume encryption protects against removal of the medium and snapshot leakage, but is powerless against a process that has already gained access to a mounted volume.
Object storage encryption works at the level of individual objects and versions and does not require rebuilding machines. Secrets are moved to a separate perimeter: a secrets manager grants access by role, while a key management service handles rotation and an access log.
| Option | When it fits | Limitations |
|---|---|---|
| Full disk, volume, and snapshot encryption | Virtual machines, databases on local volumes, backups | The key is needed at boot; inside a running system, data is read in plaintext |
| Object storage encryption | Files, exports, archives, versioned objects | Covers only the object; access rights and lifecycle are configured separately |
| Separate perimeter for secrets and keys | Passwords, tokens, signing keys, database access credentials | Discipline is required: secrets must not end up in environment variables and repositories |
| Hybrid scheme | Multiple data types and access audit requirements | Harder to maintain: you will have to describe which key belongs where and who owns it |
In practice, schemes are combined: the volume and snapshots are encrypted at the lower level, objects — in storage, and access to everything else lives in a separate perimeter with rotation and auditing.
Implementation stages: from data inventory to handover
Encryption at rest breaks not during configuration, but during operation: a key forgotten when decommissioning a node, or a backup that no one has tried to decrypt, nullifies all the work.
That is why implementation follows a chain where each step relies on the previous one and is verified before moving on.
- Data inventory. We compile a list of volumes, databases, object storages, copies, and exports — including those on engineers' machines. Without this list, any scheme covers only half the perimeter.
- Data classification. We label arrays by sensitivity: where a full volume is needed, where object encryption is sufficient, and where data should not be stored at all. The class determines both the protection method and the access mode.
- Key scheme. We build a hierarchy: root key in the secrets vault, data encryption keys — separately per perimeter and service, with separate rights. Key rotation is designed in from the start, together with the schedule and decommissioning procedure.
- Encryption configuration. We enable LUKS on block volumes, server-side encryption in object storage, encryption in the DBMS and in backups. We verify that keys do not leak into images, configs, and logs.
- Recovery verification. We decrypt a copy on a clean test environment strictly according to the instructions, not from an engineer's memory. If recovery is not reproducible on the first try from the document, the procedure is not ready.
- Audit and access procedure. We enable logging of key operations, review who has the right to decrypt, and reduce the circle to what is truly necessary.
- Handover and training. We hand over the scheme, rotation and recovery instructions, and walk through them with the customer's on-call shift.
The result of the work is not a report, but your team's ability to perform rotation and bring up data from a copy without us.
Handover artifacts: configurations, key schemes, procedures
The work ends not with encryption enabled, but with a set of artifacts that your team can use to operate and audit protection without our support.
We hand over working configurations, not a "how to configure" memo — they can be reapplied and compared with the current state of the infrastructure.
- Working configurations — LUKS profiles for disks and partitions, server-side encryption parameters for buckets and databases, binding to the key vault.
- Key hierarchy scheme — root key, data encryption keys, and wrapping keys: what protects what, where it resides, and how each level is rotated.
- Data access policies — which roles and service accounts are allowed to decrypt, under what conditions, and how the key access log is maintained.
- Rotation and revocation procedure — scheduled rotation calendar, criteria for unscheduled rotation (leak, employee departure), and service behavior when a key is revoked.
- Recovery instruction — steps: access to the key vault, mounting the volume, restoring from backup, integrity check; a separate scenario for losing one vault.
- Responsibility matrix — who owns the keys, who grants access, who approves rotation, and to whom escalation goes during an incident.
- Encryption documentation — a data map listing covered perimeters, rationale for the chosen scheme, and honest boundaries: what encryption does not cover.
Documents reside next to the configurations in the same repository: when the infrastructure changes, an outdated instruction breaks the check at the build stage rather than surfacing during an incident.
Case study: migration to encryption without service downtime
The client is a service with distributed storage: dozens of volumes, some created manually and left unencrypted because the procedure could not keep up with infrastructure growth.
Stopping the platform for re-encryption was not possible — the load is around the clock, and the maintenance window was not even enough to restart nodes. We examine what the audit showed and what changed afterward.
What we found before the start
The inventory revealed an unpleasant picture: 34% of volumes were not covered by encryption, and records were kept in a spreadsheet that diverged from reality by eight machines.
Key rotation took 14 manual operations and stretched over days; backup verification took about nine hours selectively, without confirmed recovery.
The bottleneck turned out to be in one segment: volumes there could only be re-encrypted with downtime, so they were postponed, and they were exactly the ones that accumulated the longest.
What we changed
We migrated layer by layer: first cold replicas, then production nodes — re-encryption ran in the background with live traffic, without connection breaks.
Keys were moved to Vault and linked to access policies, rotation became a single procedure with automatic reconciliation, and each copy began to be restored to a separate test environment.
Results by metrics: share of uncovered volumes — 34% → 0, key rotation — 14 manual steps → one run, backup verification — nine hours selectively → forty minutes for all copies. Downtime during migration — zero, failures during re-encryption — zero.
How does disk encryption differ from storage encryption?
The difference is determined not by the algorithm, but by where ciphertext turns into plaintext. Disk encryption works at the block device level: the volume is decrypted by the kernel after the key is entered, and after that all I/O proceeds in plaintext.
The decryption point is inside the same machine, so the trust boundary runs along the hypervisor and host: root, a crash dump, or an attached debugger see the data just like the application itself.
Storage encryption decrypts the object on the service side — at the network boundary or inside it. The provider receives the key and, if the scheme is not client-side, sees the plaintext along with metadata: object names, sizes, access times.
On the other hand, key management is centralized — access policies, rotation, revocation, key access log, and separate keys per bucket or prefix.
What remains uncovered: in the first case — a memory snapshot, a live process, a mounted volume; in the second — the storage service itself, its administrators, and anyone who has obtained the right to decrypt. Neither approach protects against someone who already has access to the decrypted representation.
That is why they are combined. Disk encryption covers the physical medium and removed disks; storage encryption provides granularity and managed keys; and application-level secrets move the trust boundary to the client — only the client holds the key.
Key management becomes a separate perimeter: where the root key lives, who can rotate it, what happens upon revocation, and who loses access to data at that moment.
Questions engineers ask before starting
Before starting encryption at rest, engineers ask roughly the same set of questions — about performance, keys, and compatibility with already running systems.
Below are short answers without vague wording; we work through details for a specific infrastructure during the design stage.
How much will encryption slow down operations?
On CPUs with hardware acceleration, block encryption overhead stays within a few percent of throughput and is almost invisible in application response times. For object storage, server-side encryption does not consume application resources at all.
Before implementation, we take measurements on the customer's workload profile: read, write, and rewrite operations — and show the difference before and after.
What happens if a key is lost?
The data will become unreadable — this is mathematics, not a support question. That is why the master key is stored in a secure vault with versioning and separation of duties: issuance requires more than one administrator.
A trusted copy of the key resides separately from the encrypted data, and rotation is performed according to procedure, not "when we remember."
Can encryption be enabled on a running system?
Yes, in stages: first we encrypt new volumes and new writes, then we cover existing data with background rewriting and resume points. The process can be paused and resumed — interrupting re-encryption does not destroy already processed blocks. Rollback at an intermediate stage is provided for.
What about access to old copies and how long should they be retained?
Backups remain encrypted with the keys that were in effect at the time they were taken, so key version history is retained no shorter than the copies themselves. If the key for an old backup is deleted before the copy's retention period, recovery from it becomes impossible. We synchronize copy and key retention periods in a single policy.
Let's discuss the task: what we will need from you at the start
To make the assessment substantive and the encryption implementation plan rely on facts rather than assumptions, at the start it is enough to list the inputs.
A ready-made technical specification is not needed — we need a snapshot of the infrastructure and the constraints you will have to work within.
- composition of systems and storage locations: virtual machine disks, server volumes, databases, object storage, backups;
- what we encrypt: only new data, or migration of existing volumes and archives is required;
- protection requirements: internal policy, industry regulations, audit expectations, whether key isolation from infrastructure administrators is needed;
- where keys are stored: a separate key management service, hardware module, or secrets vault, and who has been granted access;
- key owners: owner, rotation approval procedure, key decommissioning procedure;
- work windows: allowable interruptions, continuity requirements, readiness to rehearse switching on a test environment or replica;
- environment and access: own site or cloud, orchestrator, network segmentation, what rights will be granted to us.
From this set, a work map is assembled: where encryption can be enabled without downtime, where data migration is needed, and where it is enough to change the scheme on new volumes.
Describe the task — we will return with an assessment, an implementation plan, and a list of risks for each area.







