Turnkey automatic TLS certificate renewal setup

Why an expired TLS certificate stops a production environment

An expired certificate is not gradual degradation but an instant complete failure: trust chain validation breaks on the client side, and the request does not reach the application.

The user sees an insecure connection warning, integrations get a validation error, and notifications from external systems simply are not delivered. The product is formally alive, but inaccessible from the outside.

This is most expensive where TLS secures more than just the external perimeter. Internal services, calls to a message broker, payment integrations, and mobile applications rely on the same chain of trust.

Downtime due to a certificate looks to the business like “the whole product is down,” even though the code, database, and load balancer are operating normally.

This is usually noticed at the moment of failure because renewal was manual.

The expiration date was entered in one person's calendar — and that person is on vacation; the certificate was issued, but they forgot to restart the service that holds the old file in memory; it was updated on the ingress node but not on the internal one.

Short-lived Let's Encrypt certificates reduced the interval between issuances to weeks, and the frequency of manual operations only increased.

We break down how this class of failure is closed automatically: monitoring the remaining validity period, issuance, delivery to services, and verification that the new certificate is the one that went into operation.

What automatic certificate renewal provides

Renewal no longer depends on one person

Validity checks, issuance of a new key pair, installation on load balancers, and service reload proceed on a schedule — well before expiration, with retries if the network fails or the certificate authority is unavailable.

The successful renewal rate reaches 99% and above; the remaining cases are those where configuration needs to be fixed, but even they are visible in advance.

Unplanned downtime window measured in minutes

An expired certificate takes down traffic entirely, including service-to-service calls: the user sees an error, and troubleshooting starts from scratch.

Automation warns about expiration weeks in advance, so instead of an incident there is a planned switchover — the unplanned downtime window shrinks from hours to minutes.

The “forgot to renew” failure class goes away

Manual issuance brings recurring errors: a missed subdomain, an incomplete intermediate certificate chain, different versions on the load balancer and on the application server.

Automation reconciles the entire set of names against the registry, so the discrepancy is found at the validation stage, not from a user complaint.

Manual operations are replaced by a registry and monitoring

Instead of a calendar reminder and a “who is responsible for this domain” spreadsheet, there remains a certificate inventory with expiration dates and owners, and notifications reach the on-call engineer in advance.

This removes several hours of manual work per month and, more importantly, maintains service continuity: name coverage is complete, and rollback to the previous certificate is possible at any time.

Issuance options: comparison of modes and limitations

The choice of renewal mode is determined by two things: where the TLS termination point is located and who holds the private key. This determines where issuance automation can even live — on the node itself, on the load balancer, or in a separate issuance environment.

Comparison of options

What determines the choice

The criterion here is not issuance convenience, but whether the service can pick up a new certificate without manual intervention.

If configuration reload is not automated, a scheduled task dutifully updates the file, while the application continues to serve the old certificate — and failure occurs at exactly the same moment as in manual mode.

Implementation procedure: from audit to handover to operations

Automatic certificate renewal breaks not at issuance, but at the junctions: some services are renewed on a schedule, some manually, some read the file from an image. We structure the work so that each step produces a verifiable result and leaves a rollback path.

1. Infrastructure audit. We collect the list of names and TLS termination points: reverse proxies, load balancers, cluster nodes, mail, remote access gateways. We record where each service reads the certificate from — a file, a secret, or an image.

2. Issuance scheme. We decide which names get a certificate from a public authority and which from an internal one, and where a single name with subdomain wildcarding is sufficient. We agree on the domain ownership validation method and the owner of the DNS zone.

3. Automation setup. We place the renewal client on a dedicated node or in a container; issuance proceeds on a schedule with a margin before expiration. Keys and chains are stored centrally, and services receive them by subscription rather than by copying files.

4. Renewal verification. We run the cycle in forced mode, without waiting for the date: whether the certificate was renewed, whether services picked it up without a restart, and whether chain validation passes on clients.

5. Controlled rollout. We roll out to the production environment in groups: first non-production nodes, then a share of traffic. We keep the previous certificate and fast configuration rollback, and monitor handshake errors.

6. Handover to operations. We hand over the placement scheme, renewal policy, remaining validity notifications, and instructions for the on-call shift: how to verify issuance and what to do if domain validation fails.

What we hand over: configurations and procedures

We hand over not a set of files but a working automatic TLS certificate renewal environment: renewal configurations, an issuance scheme with key storage, and procedures by which your engineer and on-call shift can operate it without our involvement.

Everything lives in the repository next to the infrastructure description — changes go through review and remain in version history, not in chat.

• Automatic renewal configurations: task schedules, binding to domains and ownership validation method, launch 30 days before expiration so there is a margin for manual investigation.
• Issuance and key storage scheme: where the private part is generated, how it reaches load balancers and web servers, and which services have no access to it by design.
• Access policies: roles for issuance, reissuance, and revocation, separation of rights between the on-call shift and your engineers, and a configuration change log.
• Operations procedure: what to check on the monitoring dashboard, which alerts count as an incident, and to whom and within what timeframes to escalate.
• Rollback instructions: how to return the previous key pair or configuration, how long it takes, and by what sign to confirm that rollback completed correctly.
• Environment documentation: component diagram, failure points, acceptance checklist, and analysis of typical renewal errors with causes.
• Alerting rules: triggers for approaching expiration, a failed issuance attempt, and a mismatch between the certificate and the expected one.

Case study: a node fleet moved to renewal without manual operations

Starting point

A fleet of 180 nodes — load balancers, API gateways, internal panels — was renewed manually: an engineer issued a certificate by ticket, distributed files across nodes, and restarted the service.

Over a quarter, about six incidents accumulated: an expired certificate on a forgotten node, an intermediate chain mismatch after partial replacement, and failures of old clients on a new cipher suite.

Each planned renewal produced a restart window of 15–25 minutes per node and 10–12 person-hours of on-call shift time per month.

What was changed

Issuance was moved to an ACME flow via Let's Encrypt: the client on the node requests the certificate itself 30 days before expiration, validates the chain before replacement, and reloads the configuration without breaking active connections.

Rollout proceeded in groups — 5% of the fleet, then a quarter, then the rest; at each stage we checked the successful handshake rate and response time; rollback meant returning the previous file from local storage.

Metrics before and after

• Manual renewals: 100% → 0, all 180 nodes renew on a schedule.
• Certificate-related failures: ~6 per quarter → none in three quarters of observation.
• Downtime during renewal: 15–25 minutes per node → zero, the service does not stop.
• On-call shift labor: 10–12 hours per month → about 20 minutes to review alerts.

Monitoring looks at expiration and chain validation result: the on-call engineer sees only those signals where automatic renewal actually failed.

What to do if traffic is already terminated at the perimeter?

If traffic is already terminated at the perimeter, the certificate lives on the shared proxy layer, not on the backends — that is where renewal automation is installed.

The issuance client runs next to the reverse proxy or as its module: it passes domain ownership validation, obtains the certificate, and reloads the configuration without breaking active connections. Internal services behind the proxy do not terminate TLS, so their scheme does not change.

A mixed scheme — when some services go through a shared proxy and some hold local keys on their own nodes. We separate by the criterion “who owns the domain and can confirm rights to it”: if the domain points to the perimeter, automation is at the perimeter.

If a service must present the certificate itself and the key must not leave the host, a separate agent is deployed next to it, and the proxy operates in TLS passthrough mode — routing by node name without decryption.

Implementation goes from inventory to enablement. First we record where TLS is terminated for each domain, who issued the certificates, and where the DNS zones are located.

Then we run a test issuance in the same environment: ownership validation is visible, production certificates are not affected. Only after that do we enable service reload on the update event.

Compatibility with the current scheme rests on the fact that renewal does not change routing: the same reverse proxy, the same rules, only the source of the certificate file changes.

If ownership validation fails, the current certificate works until the end of its term, so we launch renewal well before expiration and move validity control into monitoring.

Common technical questions before work starts

Before starting, people ask not about certificate issuance itself but about system behavior in non-standard scenarios: what happens if renewal fails, whether a service stop will be required, and what to do with nodes outside the environment. Below are answers to what most often comes up in technical meetings.

Will replacement happen without stopping the service?

Where the application reloads the certificate without a restart, replacement happens on the fly. Where it cannot, we put the restart into an agreed maintenance window and do it in stages, taking nodes out of balancing one by one. For external users this is invisible.

What should be done with nodes outside the main environment?

Before the start, we perform an inventory: the list includes everything that serves traffic over a secure channel, including isolated segments and partner sites.

For them, a separate update route is configured with its own delivery and control channels — otherwise such nodes quietly drop out of the overall cycle.

Who is responsible for operation after handover?

We hand over the system together with the procedure: where to check status, what to do if renewal fails, and who receives the notification. We support the first period, then by agreement. Responsibility for operations transfers together with documentation and on-call shift training.

How do you check expiration?

Monitoring counts the remaining days for each certificate and warns in advance, before entering the critical zone. Separately, we check what is actually served externally, not what is on disk — the discrepancy between them is the most common cause of sudden failures.

Let's discuss your task: where we start

Assessment and the work plan depend on the number of entry points and the structure of network segments: a dozen services in one environment and a distributed system with an internal certificate authority are different scopes.

We start with a conversation and collecting source data: from it we build a certificate map and determine exactly where configuration will have to be changed.

Then comes the plan: what we do, what remains on your side, what work windows are needed, and how we confirm that renewal proceeds without failures — a trial issuance in advance, practicing revocation, and notifications about failed attempts.

What we need from you to come back with an assessment and plan:

• List of services and entry points: load balancers, reverse proxies, gateways, mail, internal services that serve traffic externally.
• Segment diagram: where there is internet access, which DNS zones you control, and whether any closed environments without external access remain.
• Certificate owners for each domain — who renews them now and approves configuration changes.
• Work windows and change procedure: when the production environment can be touched, who approves, and how to roll back.
• How keys are issued and stored today: public authorities or corporate, and where private keys are located.
• Audit requirements: log retention period, access to keys, and whether separation of rights is required.

Describe the task — we will come back with an assessment and work plan. If some data is not yet known, we will start with certificate inventory: it removes the main uncertainty and immediately shows the areas where renewal breaks.