Skip to content

topology.kubernetes.io labels should be removed from security labels #43723

@moscicky

Description

@moscicky

Is there an existing issue for this?

  • I have searched the existing issues

Version

not directly connected to cilium version. Depends on k8s version

What happened?

Starting from Kubernetes 1.35 topology.kubernetes.io labels were added to pods.

This results in CIDs being duplicated:

  1. From what I've observed those labels are not added atomically when POD is first created so it results in 2 CIDs being created: first without new labels and second with new labels
  2. For multi-zonal deployments pods from the same deployment can have different values of topology.kubernetes.io/zone and will have different CIDs.

CID duplication can lead to CID exhaustion in clusters with large CID churn (limit is 65k CIDs). We have hit this in Scalability Tests using k8s 1.35.

How can we reproduce the issue?

I've run k8s scalability tests (with clusterloader2) with k8s 1.34 and 1.35 and compared ciliumidentities and label sets.

Cilium Version

Kernel Version

Kubernetes Version

1.35

Regression

Worked fine with k8s 1.34

Sysdump

No response

Relevant log output

Anything else?

Proposed fix: exclude topology labels by default

Cilium Users Document

  • Are you a user of Cilium? Please add yourself to the Users doc

Code of Conduct

  • I agree to follow this project's Code of Conduct

Metadata

Metadata

Assignees

Labels

area/agentCilium agent related.kind/bugThis is a bug in the Cilium logic.kind/community-reportThis was reported by a user in the Cilium community, eg via Slack.release-blocker/1.18This issue will prevent the release of the next version of Cilium.sig/policyImpacts whether traffic is allowed or denied based on user-defined policies.upgrade-impactThis PR has potential upgrade or downgrade impact.

Type

Projects

Status

Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions