feat: Direct connection support between provider clusters by Gabbe64 · Pull Request #3115 · liqotech/liqo

Gabbe64 · 2025-08-28T21:08:17Z

Description

This PR introduces the possibility to make pods deployed on provider clusters communicate directly by using an established network connection (liqoctl network connect).

This feature is opt-in: users can enable it by adding the annotation key use-direct-connections: true to the Service exposing the pods that should communicate directly. This way, the default behavior remains unchanged, while also giving users the possibility to choose whether to route traffic through the consumer cluster or directly.

Implementation

As discussed during the last community meetings, the modified components are the virtual-kubelet and the ShadowEndpointslice controller.

Transmission - virtual-kubelet

When reflecting Endpointslices and the corresponding Service has the right annotation, some data is collected from the informers (without direct the API server calls) then serialized as JSON and reflected in the annotation field of the ShadowEndpointslice.

This data is:

The clusterID where the endpoint is deployed,
The local IPs contained in the Endpointslice of the consumer,
The same IPs, remapped by the consumer cluster.

Reception

The ShadowEndpointslice controller (in the provider cluster) receives this data and uses it to remap the addresses so to use the direct connections towards other provider clusters (if available).

To achieve this:

Endpoints leading to pods on a different provider are identified using point 3.
They are replaced with direct addresses, derived from the remote cluster’s podCIDR (point 1) and the host part extracted from the local IP (point 2); this required the implementation of a "forced mapping".

…mmunication

…IP objects

…wEPS

adamjensenbot · 2025-08-28T21:08:28Z

Hi @Gabbe64. Thanks for your PR!

I am @adamjensenbot.
You can interact with me issuing a slash command in the first line of a comment.
Currently, I understand the following commands:

/rebase: Rebase this PR onto the master branch (You can add the option test=true to launch the tests
when the rebase operation is completed)
/merge: Merge this PR into the master branch
/build Build Liqo components
/test Launch the E2E and Unit tests
/hold, /unhold Add/remove the hold label to prevent merging with /merge

Make sure this PR appears in the liqo changelog, adding one of the following labels:

feat: 🚀 New Feature
fix: 🐛 Bug Fix
refactor: 🧹 Code Refactoring
docs: 📝 Documentation
style: 💄 Code Style
perf: 🐎 Performance Improvement
test: ✅ Tests
chore: 🚚 Dependencies Management
build: 📦 Builds Management
ci: 👷 CI/CD
revert: ⏪ Reverts Previous Changes

… files

cheina97 · 2025-09-02T10:56:20Z

/test

aleoli · 2025-09-10T13:01:55Z

/test

cheina97 · 2026-02-10T13:42:23Z

pkg/liqo-controller-manager/offloading/shadowendpointslice-controller/mapping.go

 )

 // MapEndpointsWithConfiguration maps the endpoints of the shadowendpointslice.
 func MapEndpointsWithConfiguration(ctx context.Context, cl client.Client,


This function is has a parametr called clusterID whic is overrided by the GetConnectionDataByIP function. The options are 2:

the parameter is useless

the GetConnectionDataByIP is shadowing the value

Yes, that was a mistake.

The name of the parameter was changed and it stores the clusterID of the local cluster.

cheina97 · 2026-02-10T13:45:33Z

pkg/liqo-controller-manager/offloading/shadowendpointslice-controller/mapping.go

+				if addrHasBeenRemapped {
+					break


The break statement exits the inner address loop (for j := range endpoints[i].Addresses) after the first address is remapped. If an endpoint has multiple addresses, only the first one gets processed

cheina97 · 2026-02-10T14:01:49Z

pkg/utils/directconnection/directconnection.go

+func (l *InfoList) GetConnectionDataByIP(ip string) (clusterID, originalIP string, found bool) {
+	for _, entry := range l.Items {
+		for i, remappedIP := range entry.RemappedIPs {
+			if ip == remappedIP {
+				return entry.ClusterID, entry.IPs[i], true
+			}
+		}
+	}
+	return "", "", false
+}


Assumes len(entry.IPs) == len(entry.RemappedIPs) without validation. If these slices have different lengths (which is possible via the Add() method), accessing entry.IPs[i] will cause a panic.

cheina97 · 2026-02-10T14:03:20Z

pkg/virtualKubelet/reflection/exposition/endpointslice.go

+				continue
+			}
+
+			objectName := endpoint.TargetRef.Name


endpoint.TargetRef is an optional pointer (*v1.ObjectReference). According to Kubernetes API specifications, it can be nil. This code will panic if an endpoint has no TargetRef

https://github.com/liqotech/liqo/pull/3115/changes#diff-a3538c69a2c86d46cc2692a9b1e5f24a200f2c05cb99cce1cac12423393248bbR219-R222

Added a check at the start of the block.

cheina97 · 2026-02-10T14:04:25Z

...ntroller-manager/offloading/shadowendpointslice-controller/shadowendpointslice_controller.go

+		err := remoteConnectionsData.FromJSON([]byte(val))
+
+		if err != nil {
+			klog.Errorf("failed to unmarshal direct connection data for shadowendpointslice %q: %v", nsName, err)


Why not returning an error?

In the previous implementation the error was not returned because in case of problems during the unmarshalling, the controller could just skip the remapping of the direct address and use the standard external CIDR address (of the consumer) provided by the ShadowEndpointSlice.
It thought it to be some sort of fallback mechanism.

With the new implementation, since the address in the ShadowEPS is not the one from the external CIDR, it's mandatory to return an error, because there is no "usable" endpoint in case the unmarshalling goes wrong.

cheina97 · 2026-02-10T14:31:07Z

pkg/utils/directconnection/directconnection.go

+type Info struct {
+	ClusterID   string   `json:"ID"`
+	IPs         []string `json:"IPs"`
+	RemappedIPs []string `json:"rIPs"`
+}


It's not clear to me why you need all these info in the annotation. I think just an annotation with the clusterID should be enough.
In theory you should just reflect the endpointslie as it is on the remote cluster, then the remote cluster will reconcile the shadow_endpointslice and will use the "configuration" resource related to the clusterID contained inside the annotation, to remap the IP

Am I missing something?

cheina97 · 2026-02-10T14:40:12Z

...ntroller-manager/offloading/shadowendpointslice-controller/shadowendpointslice_controller.go

+			klog.Errorf("failed to unmarshal direct connection data for shadowendpointslice %q: %v", nsName, err)
+		}
+		// JSON is not propagated to the EndpointSlice
+		delete(shadowEps.Annotations, directConnectionAnnotationLabel)


Can you remove it in a more explicit way? For example create a function that "forge" the endpointslice annotations starting from the shadowendpointslice

https://github.com/liqotech/liqo/pull/3115/changes#diff-f1a76a3c052d25f7d5002fed269741de3338ba5c47f717baed29695a1f9b1c02R158-R167

Done!

…ing logic

Gabbe64 · 2026-02-15T12:20:07Z

Given your feedback I decided to refactor this feature. The core logic remains the same, but the exchanged data has changed slightly.

Changes:

EndpointSlice Reflection: When an endpoint belongs to a cluster other than the local or remote one, we no longer send the address on the consumer's external CIDR. Instead, the address is sent as is (as seen by the consumer cluster).
Annotations: The data payload in the annotation is now smaller since it no longer includes the external CIDR address. This required an update to the data structure used for marshalling/unmarshalling.

I also added a few utilities, including a helper for annotation forging and an optimized lookup for the data structure embedded within the annotations.

…ile method

Gabbe64 added 4 commits August 28, 2025 19:55

feat: add direct connection utilities and types for remote cluster co…

9f201a1

…mmunication

feat: add functions to retrieve local and remapped IP addresses from …

3d1c172

…IP objects

feat: implemented direct connection data transmission in virtual-kubelet

7fbd5f3

feat: implement remapping using direct connection data from the shado…

0159dc7

…wEPS

pull-request-size bot added the size/L label Aug 28, 2025

github-actions bot added the feat Adds a new feature to the codebase label Aug 28, 2025

Gabbe64 marked this pull request as ready for review August 29, 2025 11:47

Gabbe64 and others added 4 commits August 30, 2025 16:58

refactor: rename DirectConnectionInfo to Info + fixed linting errors

9e7f0be

refactor: clean up comments and improve function signature in mapping…

0c46130

… files

Merge branch 'master' into no-fcc-implementation

07a8051

Merge commit '0c4613038f1ebb7e5b25ddd2139bfedbba811410' into HEAD

d906d26

Gabbe64 force-pushed the no-fcc-implementation branch from 07a8051 to 0c46130 Compare August 30, 2025 18:01

github-actions bot added the refactor Reorganizes or optimizes code without changing its behavior label Aug 30, 2025

frisso added the release-1.1 Feature that is planned to be integrated within a given Liqo release label Sep 10, 2025

Gabbe64 mentioned this pull request Feb 6, 2026

[Docs] User docs for the direct connection feature Gabbe64/liqo#1

Draft

cheina97 reviewed Feb 10, 2026

View reviewed changes

refactor: update direct connection handling and improve endpoint mapp…

719f026

…ing logic

Gabbe64 and others added 2 commits February 16, 2026 11:27

Merge branch 'master' into no-fcc-implementation

8737bb2

fix: handle error when unmarshalling direct connection data in Reconc…

cc963ba

…ile method

github-actions bot added the fix Fixes a bug in the codebase. label Feb 16, 2026

refactor: remove commented-out code

1987ee8

Conversation

Gabbe64 commented Aug 28, 2025

Description

Implementation

Transmission - virtual-kubelet

Reception

Uh oh!

adamjensenbot commented Aug 28, 2025

Uh oh!

cheina97 commented Sep 2, 2025

Uh oh!

aleoli commented Sep 10, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Gabbe64 commented Feb 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Gabbe64 commented Feb 15, 2026 •

edited

Loading