Dynamic secrets: why static keys don't protect
A static key in configuration lives longer than the task it was created for. It settles in the repository, in build pipeline variables, on developers' laptops, and in log exports — and remains valid even if the service was rewritten long ago. Changing a password in one place is not enough: there are already dozens of copies.
Credential leaks rarely look like a hack. More often it is a key left in repository history, a database dump, an application log with connection parameters, or a contractor whose access was never revoked.
It is hard to notice, and the result is the same: someone who was not granted access can connect to the database or storage.
The second problem is shared credentials. The same set of keys is used by the payment service, internal analytics, and a scheduled report.
From the logs it is impossible to understand whose request came in, and revoking access for one participant without breaking the others is impossible.
Revoking access turns into an emergency procedure: issue a new key, update the configuration in all environments, restart services. While this is being done, the old key continues to work — the vulnerability window is measured in days.
Dynamic secrets in HashiCorp Vault break this coupling: credentials are created for a specific request, have a short access lifetime, and are revoked automatically.
Below we break down how this works in practice — from connecting the vault to services to scheduled rotation and targeted access revocation.
Rotation without downtime and access revocation in minutes
Permanent keys live in configurations and environment variables for months. They are copied between environments, handed to contractors, forgotten to be revoked after an employee leaves — and one such key opens access to the entire environment.
When an incident has already occurred, revocation turns into manually going around dozens of services, and some of them continue to work with the old password.
Dynamic secrets change the model: credentials are issued for a specific workload and role, rather than stored in a file. The lifetime of short-lived credentials is minutes or hours, and when it expires, access disappears by itself, without human involvement.
Automatic access issuance relies on the service's own authentication, so the password does not need to be passed anywhere, stored, or synchronized between teams — in our practice this is HashiCorp Vault.
Secret rotation stops being a separate operation with a maintenance window. The service requests new data in advance, continues working on the current data, and the previous data is revoked after the switch — no service downtime occurs.
During implementation, we check that each role has a reasonable lifetime, that the application correctly handles rejection of the old password, and that it can re-authenticate.
Access revocation becomes a single action: we block the role — and all credentials issued on its basis are marked invalid. The log shows which service received what and when, so incident investigation relies on facts, not guesses.
We configure this together with the service runtime environment and configuration storage, so that issuance and disabling do not depend on manual steps.
Methods of issuing access: from static to short-term keys
The fork is simple: where credentials live and how long they live. Static keys in configs and environment variables speed up startup, but they have no lifetime — a leak through logs, memory dumps, or a repository gives indefinite access that no one will notice.
Moving secrets into a secure vault removes the question “who sees the file” but not the question of rotation: the values remain long-lived. Below are the criteria for choosing a mode for a specific environment.
| Option | When it fits | Limitations |
|---|---|---|
| Static keys in configuration | Local debugging, external systems without rotation support | No revocation or lifetime; changing the value is a manual operation on all nodes |
| Secrets in a secure key-value vault | You need to remove keys from code and repositories, the provider cannot issue temporary access | Values are long-lived, rotation and distribution remain with engineers |
| Short-term credentials on request | Databases, clouds, internal APIs with dynamic issuance support | The service needs its own identity and the ability to update access on the fly |
| Federation by party identity | Environments without long-lived keys, exchange between clusters and external providers | Requires preconfigured trust and synchronized time |
| Hybrid scheme | Production environments: static only for initial startup, the rest is dynamic | Debugging is more complex: you need to see which issuance path worked |
In practice, the hybrid scheme turns out to be the working answer.
A long-lived key remains exactly where startup is impossible without it: initial service authentication before the vault — Vault in this role itself issues a short-term token — and access to a system that does not support temporary keys.
Everything else is issued for a limited time, so access revocation stops being a separate operation and happens by itself when the term expires.
How we implement: audit, design, configuration, validation, handover
Secret management implementation begins not with deploying a vault, but with inventory: where passwords, keys, and tokens currently reside — in application configurations, environment variables, files on servers, repository history.
Until this map is assembled, automatic rotation gives a false sense of order and leaves blind spots. Below are the stages, each leaving a working intermediate result.
- Inventory and access audit. We record every secret, its owner, lifetime, and who reads it — a service, a person, an external system. We separately mark what cannot be changed automatically, so as not to stop exchange with contractors.
- Access issuance design. We determine which services receive short-lived credentials to the database, which receive certificates, and which receive static keys. Here we also choose application authentication methods and lease lifetime.
- Environment configuration. We deploy a fault-tolerant vault based on Vault, enable an audit log of every read, configure backups and automatic unsealing after restart. We connect the necessary issuance mechanisms — databases, certificates, encryption.
- Pilot on one service. We switch the application to obtaining a secret from the vault: we check lease renewal, correct revocation, and behavior when the vault is unavailable.
- Validation on staging. We run scenarios for revocation, application restart under load, and loss of connection to the vault. We check that failure of one issuance does not bring down the whole environment.
- Handover of procedures. We give your team the issuance design, developer instructions, rotation and on-call procedures, and audit log analysis — further support is done by your own team or with our assistance.
Configurations, access policies, and rotation procedures
After the work is completed, the infrastructure must remain manageable without us: secrets are issued on request, lifetime is set in advance, and issued access can be checked and revoked at any moment.
Therefore, we hand over not only a configured vault but also a set of artifacts by which the team reproduces the Vault configuration and changes it without parsing someone else's code.
- Access policies — read and write rules by path, tied to roles rather than people: the development environment is more open, production secrets are seen only by those who need them for the task.
- Roles for services — a separate role for each application with its own set of paths: compromise of one service does not open access to neighboring services' secrets.
- Rotation procedure — a table of “secret type / lifetime / reissue moment / responsible person,” plus the procedure for manual revocation and checking issuance history.
- Integration design — how applications, build pipelines, and cluster nodes obtain secrets, where data for initial authentication is stored, and what happens when the vault is unavailable.
- Secret documentation — a path map, issuance templates, environment variable names, and instructions for the on-call engineer: revoke access, see who received it, connect a new service.
- Set of checks — tests confirming that the policy issues a secret to the right role and denies all others; they run on every configuration change.
- Emergency access plan — sealed credentials and the procedure for using them if the vault is unavailable.
Case: reducing access lifetime from months to hours
Before implementation, access to databases and external services lived in configurations and was issued manually: credentials were changed once a quarter, and when an engineer left, dozens of services had to be traversed and one had to remember where each static password was set.
Access revocation took on average 3-5 days, and all that time the former employee formally remained in the system. It was impossible to check who currently had valid credentials — inventory took weeks and still remained incomplete.
What changed
We switched services to dynamic secrets in HashiCorp Vault: credentials are created for a specific task, live from several minutes to an hour, and are revoked automatically. Rotation does not require a person — issuance and revocation are recorded in the log with role and time.
Metrics before and after
- Credential lifetime: from several months to 15-60 minutes.
- Full access revocation time: from 3-5 days to 2-3 minutes — disabling the role immediately stops issuance of new secrets.
- Share of manual access operations: decreased from approximately 80% to isolated cases — only approval of a new role remained manual.
- Eliminated failure class: leakage of a static password from configuration or a repository stopped being an incident — by the time it is detected, such a secret is already invalid.
A side effect — there are almost no “eternal” passwords left in configurations, repositories, and environment variables. Auditors received a complete picture: who, when, and to which systems received access.
How do dynamic secrets differ from static keys?
A static key is created once and lives until manual rotation: a line in the application config, an environment variable in the build, an entry in a password vault.
Its lifetime is unlimited, and the access scope is usually wider than a specific service needs — the same database login is often reused by several environments.
Until the key is revoked, it remains valid for anyone who obtained it accidentally: from logs, a dump, or an old repository branch.
Dynamic secrets are issued on request. HashiCorp Vault creates credentials for a specific role at the moment of access: the service presents its role and receives a login-password pair that exists only for it.
The lifetime is set by a lease — minutes or hours; when it expires, access is revoked automatically, without an on-call engineer or manual rotation.
This leads to the difference in access scope and the cost of key compromise. A static key is shared: revocation affects everyone who uses it, so it is changed rarely and reluctantly, and a leaked key requires rotation at all points and searching for the leak source.
A dynamic key is issued for a specific role, its revocation does not affect neighboring services, and the attack window is limited by the lease term.
The trust model also changes: instead of “we trust everyone who has the file with the key” — “we trust the issuing center and its log,” where it is visible who and when received access.
Static keys do not disappear: for external systems without issuance support, they remain a working option, and the task is to minimize the number of such places.
Common concerns: compatibility, fault tolerance, rollback
Concerns usually surface not at the start, but when the pilot is already issuing secrets to a production service: what happens if the vault fails, will this work with an old application, how to roll back. Below are short answers to what is most often asked before enabling in production.
What happens if the vault becomes unavailable?
Each application receives secrets through a local agent that keeps a cache of the last issued values. As long as the secret lifetime has not expired, the service works normally on them, and vault unavailability does not cause a failure.
New issuances are queued and retried with increasing delay, and if the delay is prolonged, the backup source configured during implementation takes over.
We deploy the vault itself as a cluster of several nodes: if the active role fails, switching happens automatically, and state snapshots are stored outside the nodes.
Will this work with old applications?
Yes, even if the application cannot access the vault directly. The agent injects secrets into a configuration file or environment variables and rereads them on a signal — no code changes are needed.
For services with a strict key format, we keep an intermediate layer that returns the value in the expected form.
How is the rollback plan arranged?
We describe the reverse transition before enabling: the static path with keys remains working, and the switch is done with a single flag in the configuration.
After returning, all issued secrets are forcibly revoked so that no live copies remain in the system, and accounts for dynamic secrets are closed.
Let's discuss your access scheme and come back with a plan
So that the implementation assessment is based on facts rather than assumptions, we suggest discussing the task concretely: send the infrastructure environment diagram and the list of systems that receive secrets. Below is what exactly removes uncertainty in the work plan.
- Environment diagram: which environments exist (development, testing, production), where services live, and how they see the secret vault.
- Credential consumers: applications, build and deployment pipelines, operators, databases, external integrations — indicating exactly which access each needs.
- Current access model: where secrets currently reside — environment variables, configs, repositories — who issues them, and how they reach the runtime environment.
- Rotation requirements: credential lifetime, owner, what should happen in case of compromise and access revocation.
- Constraints: regulatory requirements, systems that cannot be stopped, change windows, and approval procedures.
- Failure history: what has already broken due to leaks or expired passwords, and which operations are still done manually.
The review takes place in one conversation: we go through the diagram, mark where secrets will remain static and where they can realistically be moved to dynamic, and name the migration stages.
As a result, you receive a work plan with the sequence of system migration, an implementation estimate by stage, and a list of source data that is missing to start.