-
-
Notifications
You must be signed in to change notification settings - Fork 86
Cilium Ingress - Multiple Filter Chains with Overlapping Matching Rules #1824
Copy link
Copy link
Open
Description
Issue Summary
When multiple Kubernetes Ingress resources share the same hostname but have different path rules, Cilium Envoy pods report errors about "multiple filter chains with overlapping matching rules" and stop serving traffic for all Ingress objects on that shared IP.
Environment
- Cilium Version: 1.19.2
- Kubernetes Version: v1.31.13
- Ingress Controller: Cilium Ingress Controller
- LoadBalancer: MetallLB
Observed Behavior
Error Messages in Cilium Envoy Logs
[2026-03-30 08:44:39.823][7][warning][config] [external/envoy/source/extensions/config_subscription/grpc/grpc_subscription_impl.cc:138]
gRPC config for type.googleapis.com/envoy.config.listener.v3.Listener rejected:
Error adding/updating listener(s) kube-system/cilium-ingress/listener:
error adding listener '127.0.0.1:17489': multiple filter chains with overlapping matching rules are defined
Affected Ingress Resources
Four separate Ingress resources were configured with the same hostname apim.example.com:
| Ingress Name | Namespace | Path | Backend Service | Port |
|---|---|---|---|---|
| gravitee-api-management | stg-apim-stack | /management | gravitee-api-management | 8083 |
| gravitee-api-portal | stg-apim-stack | /portal | gravitee-api-portal | 8083 |
| gravitee-gateway | stg-apim-stack | /gateway | gravitee-gateway | 8082 |
| gravitee-portal | stg-apim-stack | /console | gravitee-portal | 8085 |
Impact
- Service Disruption: All Ingress objects on the shared IP stop serving traffic when any one of them has overlapping path rules
- Cascading Failure: The error affects the entire listener configuration, not just the problematic Ingress
- Manual Intervention Required: Requires merging all Ingress resources into a single object and restarting Cilium Envoy pods
Expected Behavior
According to Kubernetes Ingress specification and Cilium documentation:
- Multiple Ingress resources should be able to coexist with the same hostname as long as their path rules are distinct
- Path-based routing should allow different backends to serve requests based on URL paths
- A misconfiguration in one Ingress should not affect other valid Ingresses sharing the same IP
Root Cause Analysis
The issue appears to be related to how Cilium Envoy processes multiple Ingress resources:
- Envoy Filter Chain Configuration: When multiple Ingress objects share a hostname, Cilium creates filter chains for each path rule
- Path Matching Overlap: Even with distinct paths (
/management,/portal,/gateway,/console), the Envoy configuration may be generating overlapping match rules - Listener Rejection: The Envoy configuration validation rejects the entire listener when any overlap is detected
Workaround Applied
The issue was resolved by merging all four Ingress resources into a single Ingress object:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: gravitee-ingress
namespace: stg-apim-stack
spec:
ingressClassName: cilium
rules:
- host: apim.example.com
http:
paths:
- backend:
service:
name: gravitee-api-management
port:
number: 8083
path: /management
pathType: Prefix
- backend:
service:
name: gravitee-api-portal
port:
number: 8083
path: /portal
pathType: Prefix
- backend:
service:
name: gravitee-gateway
port:
number: 8082
path: /gateway
pathType: Prefix
- backend:
service:
name: gravitee-portal
port:
number: 8085
path: /console
pathType: Prefix
tls:
- hosts:
- apim.example.com
secretName: apim.example.comSteps to Reproduce
- Create multiple Ingress resources with the same hostname but different paths
- Apply all Ingress resources to a cluster using Cilium Ingress Controller
- Check Cilium Envoy pod logs for "multiple filter chains with overlapping matching rules" errors
- Verify that traffic is not being served for any of the Ingress objects
Severity
High - This issue causes complete service disruption for all Ingress objects sharing a LoadBalancer IP when path configuration issues occur.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels