Using Kubernetes Secrets in API Proxies (Apigee hybrid only)

omidt · 2025-11-29T12:27:59.777Z

In Apigee hybrid, “Kubernetes Secrets” typically refers to the internal credentials (like service account keys and TLS certificates) required to run the runtime components. However, Apigee hybrid also supports a powerful feature that allows read-only access to custom Kubernetes Secrets directly from within your API proxies.

This capability enables a clean separation of concerns and enhances security by ensuring sensitive data never leaves your cluster or your control.

Note: This feature is available exclusively for Apigee hybrid. It relies on direct access to the underlying Kubernetes cluster to create and manage secrets. It is not available in Apigee X, where the Kubernetes infrastructure is fully managed by Google and abstracted away from the customer.

Use Cases: Kubernetes Secrets vs. Key Value Maps (KVMs)

While both Kubernetes Secrets and Apigee KVMs allow you to store and retrieve data at runtime, they serve different purposes and security models.

Feature	Kubernetes Secrets (Proxy Access)	Apigee Key Value Maps (KVMs)
Primary Use Case	Highly sensitive credentials (private keys, certs, backend passwords) that must remain within the cluster boundary.	Configuration data, routing tables, or non-critical secrets managed via Apigee API.
Management	Managed via `kubectl` (or other k8s tooling) by Cluster Admins.	Managed via Apigee API, `apigeecli`, or other Apigee tooling.
Data Residency	High. Data is stored natively in Kubernetes (etcd) and cached in the Runtime. It is never replicated to the Apigee Management Plane (Google Cloud).	High. Encrypted KVMs are stored in the Apigee runtime’s primary datastore, Cassandra, where all data is encrypted at rest. With Cassandra backups, this encrypted data will be duplicated to destinations outside of the cluster. Most important to note, however, is that KVM entries can be retrieved via the Apigee API, for anyone with correct permissions.
Updates	Requires updating/replacing the secret.	Can be updated dynamically.

When to use Kubernetes Secrets:

You have strict compliance requirements that prevent sensitive keys (like a private key for a target system) from being persisted in Apigee’s Cassandra datastore.
You are adopting a GitOps workflow where secrets are managed as part of your infrastructure code.
You need to enforce a strict separation of duties between the team managing the platform and the developers building the APIs.

What if you’d like to manage a non-sensitive configuration parameter?
Consider using PropertySets: Key Value Maps vs Property Sets in Apigee: Understanding the Differences

Separation of Duties: A Persona-Based Approach

This feature natively supports a secure workflow involving two distinct personas:

1. The Systems Administrator / Operator

Role: Has kubectl access and is authorized to handle raw sensitive material (e.g., the production database password).
Action: Creates the Kubernetes Secret in the cluster.
Result: The secret exists safely in the infrastructure, but the Admin does not need to know how it is used in the API.

2. The API Proxy Developer

Role: Builds the API logic but is not authorized to or responsible for managing the sensitive information.
Action: References the secret using a predefined variable name in the proxy code.
Result: The developer successfully connects to the backend without ever viewing or handling the sensitive credential.

Implementation Guide

Step 1: The Operator Creates the Secret

The Systems Administrator creates the secret in the Kubernetes namespace where the Apigee runtime components are deployed (typically apigee).

Rules & Constraints:

Naming Convention: The secret must be named using the pattern <org>-<env>-policy-secret.
File Format: Supported files are *.properties, *.key, *.pem, *.crt.
Filename: Must be lowercase.

Example:
Imagine you need to store a backend credential file named credentials.properties for the production environment in the organization my-org.

Create the properties file:

# credentials.properties
username=admin
password=SuperSecretPassword123!

Create the Kubernetes Secret:

kubectl create secret generic my-org-dev-policy-secret \
  --namespace apigee \
  --from-file=./credentials.properties

Note: Changes to secrets (creation or deletion) typically take up to 90 seconds to propagate to the apigee-runtime pods.

Step 2: The Developer Accesses the Secret

The API Developer can now reference the data using flow variables. The Apigee runtime automatically detects the secret and populates variables in the private.secret namespace.

The variable syntax is: private.secret.{filename}.{key}

Example:
To retrieve the password defined above, the developer uses the variable:
private.secret.credentials.properties.password

Usage in a Policy:
The developer can inject this password into a backend request using an AssignMessage policy, without ever hardcoding it.

<AssignMessage name="AM-Set-Backend-Auth">
  <AssignVariable>
    <Name>request.header.Authorization</Name>
    <Template>Basic {private.secret.credentials.properties.password}</Template>
  </AssignVariable>
</AssignMessage>

Summary of Constraints

Environment Scope Only: Secrets are scoped to a specific environment. You cannot create org-scoped or proxy-scoped secrets with this method. This means all API Proxies deployed to an environment can access all secrets for that environment.
Immutability: Kubernetes Secrets cannot be updated in place for this feature. To update a value, the Operator must delete and recreate the secret.
Lowercase Only: Ensure all filenames inside the secret are lowercase (e.g., creds.properties, not Creds.properties).

Congratulations, you’ve unlocked a new approach to handling sensitive credentials inside the API Proxy!

Plan your next step with a Google Cloud Sales Specialist