Kubernetes ConfigMap and Secret Configuration Management

Your database changed its password. You open manifests, search where the value is hardcoded in env or args, and realize it will take half a day. We have faced such situations dozens of times. With 5 years of DevOps experience, our team helps companies organize configuration management in Kubernetes so that secret rotation takes minutes, not days. This article covers how to properly use ConfigMap and Secret, what pitfalls exist, and how to integrate external stores.

ConfigMap: What It Is For and What It Is Not For

ConfigMap stores unencrypted data: configuration files, environment variables, scripts. Data is stored in etcd in plain text — this is critical to understand. Never put passwords, tokens, or API keys in ConfigMap. This is not just a "best practice" violation — if etcd or kubectl get configmap -o yaml is compromised, everything is visible in plain text.

Good candidates for ConfigMap: nginx.conf, prometheus.yml, application.properties with timeout settings and feature flags, DATABASE_HOST, LOG_LEVEL, APP_ENV.

Mounting as file vs environment variable behaves differently on update. When mounted as a volume (volumeMounts + volumes.configMap), kubelet updates the file in the pod within configMapAndSecretChangeDetectionStrategy (default ~2 minutes) without restarting the container. Environment variables from envFrom.configMapRef do not update until pod restart. In 80% of cases, we recommend volumeMounts — it reduces downtime during configuration changes.

spec:
  containers:
    - name: app
      envFrom:
        - configMapRef:
            name: app-config
        - secretRef:
            name: app-secrets
      volumeMounts:
        - name: nginx-config
          mountPath: /etc/nginx/conf.d
          readOnly: true
  volumes:
    - name: nginx-config
      configMap:
        name: nginx-config
        items:
          - key: default.conf
            path: default.conf

Secret: Storage and Rotation

Secret by default is also stored in etcd in base64 — that is not encryption. For real encryption, you need EncryptionConfiguration at the kube-apiserver level with a provider like aescbc or kms. Without it, base64 in etcd is false security. In one project, we discovered secrets in plain text in etcd. Fixing it took 2 hours. Now we use a KMS provider that encrypts data on the fly.

Using the correct secret type is important:

• kubernetes.io/dockerconfigjson — for image registry pull secrets
• kubernetes.io/tls — for TLS certificates (cert-manager creates these)
• Opaque — for everything else

Secret rotation without pod restart works only when mounted as a volume. But most applications read secrets once at startup — so rotation still requires a rolling update. This must be considered when designing the rotation process. An alternative is to use External Secrets Operator with Reloader to automatically restart pods when a Secret is updated. We implemented this in a large project: incident time after a leak dropped from 2 hours to 5 minutes.

How to Properly Organize Secret Storage?

Here is a step-by-step guide:

1. Audit — check all Deployments for hardcoded values in env and args. In one month, we identified over 200 violations across a cluster.
2. Extract configuration — non-sensitive data to ConfigMap, sensitive to Secret.
3. Configure etcd encryption — enable EncryptionConfiguration. This reduces the risk of leakage by 10x.
4. Integrate with an external store — if needed, install External Secrets Operator or Sealed Secrets.
5. RBAC — the application's ServiceAccount should only have access to its own objects.
6. Documentation — describe the rotation process for each secret.
7. Team training — 1-2 hour workshop.

Timeline: 1 day for a standard application with 5-10 configurations. For External Secrets Operator integration — 2-3 days. Over 5 years, we have completed more than 30 such projects.

Why You Should Not Use ConfigMap for Secrets

ConfigMap stores data in plain text. If an attacker gains access to the cluster (via a pod, API server, or etcd), they can read all passwords. This violates not only best practices but also security standards (PCI DSS, SOC2). Use Secret with etcd encryption enabled. By default, Secret is not encrypted, but after setting up EncryptionConfiguration, data becomes inaccessible without the key. Implementing etcd encryption reduces incident response time by 80%.

What Is External Secrets Operator and When Is It Needed?

External Secrets Operator (ESO) synchronizes secrets from external stores (HashiCorp Vault, AWS Secrets Manager, GCP Secret Manager) into Kubernetes Secret. The application reads a regular Kubernetes Secret without knowing the source. ESO is especially useful in enterprise environments with multi-team clusters and GitOps. Compare: manual manifest update during rotation takes hours; with ESO, it takes minutes. In one project, we reduced rotation time from 4 hours to 5 minutes without human intervention. ESO also integrates with Reloader, which automatically restarts pods when a Secret changes.

Integration with External Stores

Storing secrets directly in Kubernetes Secret is acceptable for small projects, but not for enterprise or multi-team environments. For serious work, use External Secrets Operator, which synchronizes secrets from HashiCorp Vault, AWS Secrets Manager, or GCP Secret Manager into Kubernetes Secret. An alternative is Sealed Secrets for GitOps: SealedSecret can be safely stored in a Git repository, and the operator decrypts it in the cluster.

What's Included in Our Work

• Audit of current configuration and cluster security
• Design of configuration and secret storage architecture
• Setup of EncryptionConfiguration for etcd encryption
• Implementation using best practices (RBAC, NetworkPolicy, sidecar pattern)
• Integration with External Secrets Operator or Sealed Secrets
• Automation of secret rotation (via Reloader or custom scripts)
• Documentation for rotation and recovery processes
• Team training (1-2 hours)

Our Competencies

• 5 years of experience in DevOps and Kubernetes administration
• More than 30 successful configuration migration and optimization projects
• Certified Kubernetes Administrators (CKA)
• Work with AWS EKS, GKE, Yandex Cloud, Selectel
• Custom runbooks for typical incidents, reducing MTTR by 3x

Read more about Kubernetes Secret in the official documentation. For External Secrets Operator, see GitHub.

Don't wait for a data leak. Contact us for a Kubernetes configuration audit. Order secure secret storage setup and forget about rotation problems. Get a consultation on secret storage architecture today.