Skip to content

Commit 1014b99

Browse files
committed
test(e2e): add controller-runtime client and DWOC config save/restore
- Add ControllerRuntimeClient() method to test client for accessing CRDs - Add controllerv1alpha1 to scheme for DWOC access - Add DWOC save/restore in BeforeAll/AfterAll to prevent config leaks - Use ginkgo.Ordered to ensure sequential test execution - Update copyright year to 2026 This prevents config leaks between custom init container tests and ensures clean state for each test run. Assisted-by: Claude Sonnet 4.5 Signed-off-by: Oleksii Kurinnyi <okurinny@redhat.com>
1 parent 485d377 commit 1014b99

File tree

6 files changed

+161
-25
lines changed

6 files changed

+161
-25
lines changed

AGENTS.md

Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -109,6 +109,9 @@ Use this as a quick reference for common decision points. When you encounter a s
109109
- **IF** testing internal/private functions **THEN** use `package controllers` (internal)
110110
- **IF** test needs async wait **THEN** use `Eventually()`, not `time.Sleep()`
111111
- **IF** documenting test steps **THEN** use `By("step description")`
112+
- **IF** e2e test creates DevWorkspace **THEN** MUST use `DeleteDevWorkspaceAndWait` in cleanup (AfterAll or AfterEach)
113+
- **IF** e2e test suite is `ginkgo.Ordered` **THEN** use `AfterAll` for cleanup
114+
- **IF** e2e test runs multiple times **THEN** use `AfterEach` for cleanup
112115

113116
### Workspace Bootstrapping Decisions
114117

@@ -430,6 +433,70 @@ var _ = Describe("DevWorkspace Controller", func() {
430433
})
431434
```
432435
436+
### E2E Test Cleanup Pattern
437+
438+
**AI Agent Note**: ALWAYS add PVC cleanup to e2e tests to prevent conflicts between test runs, especially in CI environments.
439+
440+
**Critical**: DevWorkspaces use a shared PVC (`claim-devworkspace`) that persists after workspace deletion. Without proper cleanup, subsequent tests can fail due to PVC conflicts or stale data.
441+
442+
**Pattern**: Use `DeleteDevWorkspaceAndWait` in cleanup blocks:
443+
444+
```go
445+
var _ = ginkgo.Describe("[Test Suite Name]", ginkgo.Ordered, func() {
446+
defer ginkgo.GinkgoRecover()
447+
448+
const workspaceName = "test-workspace"
449+
450+
ginkgo.AfterAll(func() {
451+
// Cleanup workspace and wait for PVC to be fully deleted
452+
// This prevents PVC conflicts in subsequent tests, especially in CI environments
453+
_ = config.DevK8sClient.DeleteDevWorkspaceAndWait(workspaceName, config.DevWorkspaceNamespace)
454+
})
455+
456+
ginkgo.It("Test case", func() {
457+
// Test implementation
458+
})
459+
})
460+
```
461+
462+
**Decision Tree for Cleanup**:
463+
- **IF** test suite runs multiple times with different workspaces **THEN** use `AfterEach`
464+
- **IF** test suite uses `ginkgo.Ordered` (sequential tests on same workspace) **THEN** use `AfterAll`
465+
- **IF** test creates workspace **THEN** MUST include cleanup with `DeleteDevWorkspaceAndWait`
466+
467+
**Available Helper Functions** (in `test/e2e/pkg/client/devws.go`):
468+
- `DeleteDevWorkspace(name, namespace)` - Deletes workspace only (fast, may leave PVC)
469+
- `WaitForPVCDeleted(pvcName, namespace, timeout)` - Waits for PVC deletion
470+
- `DeleteDevWorkspaceAndWait(name, namespace)` - Deletes workspace and waits for PVC cleanup (RECOMMENDED)
471+
472+
**Example: AfterEach Pattern** (for tests running multiple times):
473+
474+
```go
475+
ginkgo.Context("Test context", func() {
476+
const workspaceName = "test-workspace"
477+
478+
ginkgo.BeforeEach(func() {
479+
// Setup
480+
})
481+
482+
ginkgo.It("Test case", func() {
483+
// Test implementation
484+
})
485+
486+
ginkgo.AfterEach(func() {
487+
// Cleanup workspace and wait for PVC to be fully deleted
488+
// This prevents PVC conflicts in subsequent tests, especially in CI environments
489+
_ = config.DevK8sClient.DeleteDevWorkspaceAndWait(workspaceName, config.DevWorkspaceNamespace)
490+
})
491+
})
492+
```
493+
494+
**Why This Matters**:
495+
- **CI Flakiness**: Without PVC cleanup, tests can fail intermittently in CI with "PVC already exists" errors
496+
- **Stale Data**: Old PVC data can affect test results and cause false positives/negatives
497+
- **Cloud Environments**: PVC deletion can be slow (30-60+ seconds), requiring explicit wait
498+
- **Test Isolation**: Each test should start with a clean state
499+
433500
### Deep Copy Pattern
434501
435502
**AI Agent Note**: Always DeepCopy objects from cache before modifying to avoid race conditions.
@@ -540,6 +607,8 @@ if err := r.Update(ctx, workspaceCopy); err != nil {
540607
- ❌ Don't commit disabled tests without tracking issues
541608
- ❌ Don't write tests that depend on timing (use Eventually/Consistently)
542609
- ❌ Don't leave test resources unmanaged (always clean up)
610+
- ❌ Don't forget PVC cleanup in e2e tests (use `DeleteDevWorkspaceAndWait`)
611+
- ❌ Don't use `DeleteDevWorkspace` alone in e2e tests (PVC may persist and cause conflicts)
543612
544613
## Debugging
545614

test/e2e/pkg/client/client.go

Lines changed: 13 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -17,21 +17,21 @@ package client
1717

1818
import (
1919
"fmt"
20-
"os"
21-
22-
dw "github.com/devfile/api/v2/pkg/apis/workspaces/v1alpha2"
23-
"k8s.io/apimachinery/pkg/runtime"
24-
utilruntime "k8s.io/apimachinery/pkg/util/runtime"
25-
clientgoscheme "k8s.io/client-go/kubernetes/scheme"
26-
2720
"log"
21+
"os"
2822
"os/exec"
2923
"strconv"
3024
"time"
3125

26+
dw "github.com/devfile/api/v2/pkg/apis/workspaces/v1alpha2"
27+
"k8s.io/apimachinery/pkg/runtime"
28+
utilruntime "k8s.io/apimachinery/pkg/util/runtime"
3229
"k8s.io/client-go/kubernetes"
30+
clientgoscheme "k8s.io/client-go/kubernetes/scheme"
3331
"k8s.io/client-go/tools/clientcmd"
3432
crclient "sigs.k8s.io/controller-runtime/pkg/client"
33+
34+
controllerv1alpha1 "github.com/devfile/devworkspace-operator/apis/controller/v1alpha1"
3535
)
3636

3737
var (
@@ -41,6 +41,7 @@ var (
4141
func init() {
4242
utilruntime.Must(clientgoscheme.AddToScheme(scheme))
4343
utilruntime.Must(dw.AddToScheme(scheme))
44+
utilruntime.Must(controllerv1alpha1.AddToScheme(scheme))
4445
}
4546

4647
type K8sClient struct {
@@ -124,6 +125,11 @@ func (c *K8sClient) Kube() kubernetes.Interface {
124125
return c.kubeClient
125126
}
126127

128+
// ControllerRuntimeClient returns the controller-runtime client for accessing custom resources.
129+
func (c *K8sClient) ControllerRuntimeClient() crclient.Client {
130+
return c.crClient
131+
}
132+
127133
// read a source file and copy to the selected path
128134
func copyFile(sourceFile string, destinationFile string) error {
129135
input, err := os.ReadFile(sourceFile)

test/e2e/pkg/client/devws.go

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
//
2-
// Copyright (c) 2019-2025 Red Hat, Inc.
2+
// Copyright (c) 2019-2026 Red Hat, Inc.
33
// Licensed under the Apache License, Version 2.0 (the "License");
44
// you may not use this file except in compliance with the License.
55
// You may obtain a copy of the License at

test/e2e/pkg/tests/custom_init_container_tests.go

Lines changed: 52 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
//
2-
// Copyright (c) 2019-2025 Red Hat, Inc.
2+
// Copyright (c) 2019-2026 Red Hat, Inc.
33
// Licensed under the Apache License, Version 2.0 (the "License");
44
// you may not use this file except in compliance with the License.
55
// You may obtain a copy of the License at
@@ -23,10 +23,13 @@ import (
2323
"runtime"
2424

2525
dw "github.com/devfile/api/v2/pkg/apis/workspaces/v1alpha2"
26-
"github.com/devfile/devworkspace-operator/test/e2e/pkg/config"
2726
"github.com/onsi/ginkgo/v2"
2827
"github.com/onsi/gomega"
2928
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
29+
"k8s.io/apimachinery/pkg/types"
30+
31+
controllerv1alpha1 "github.com/devfile/devworkspace-operator/apis/controller/v1alpha1"
32+
"github.com/devfile/devworkspace-operator/test/e2e/pkg/config"
3033
)
3134

3235
// getProjectRoot returns the project root directory by navigating up from this file.
@@ -35,9 +38,49 @@ func getProjectRoot() string {
3538
return filepath.Join(filepath.Dir(filename), "..", "..", "..", "..")
3639
}
3740

38-
var _ = ginkgo.Describe("[Custom Init Container Tests]", func() {
41+
var _ = ginkgo.Describe("[Custom Init Container Tests]", ginkgo.Ordered, func() {
3942
defer ginkgo.GinkgoRecover()
4043

44+
var originalConfig *controllerv1alpha1.OperatorConfiguration
45+
46+
ginkgo.BeforeAll(func() {
47+
// Save original DWOC configuration to restore after tests
48+
ctx := context.Background()
49+
dwoc := &controllerv1alpha1.DevWorkspaceOperatorConfig{}
50+
err := config.AdminK8sClient.ControllerRuntimeClient().Get(ctx, types.NamespacedName{
51+
Name: "devworkspace-operator-config",
52+
Namespace: config.OperatorNamespace,
53+
}, dwoc)
54+
if err != nil {
55+
ginkgo.Fail(fmt.Sprintf("Failed to get original DWOC: %s", err))
56+
}
57+
// Deep copy the config to save it
58+
if dwoc.Config != nil {
59+
originalConfig = dwoc.Config.DeepCopy()
60+
}
61+
})
62+
63+
ginkgo.AfterAll(func() {
64+
// Restore original DWOC configuration to prevent config leaks between test runs
65+
ctx := context.Background()
66+
dwoc := &controllerv1alpha1.DevWorkspaceOperatorConfig{}
67+
err := config.AdminK8sClient.ControllerRuntimeClient().Get(ctx, types.NamespacedName{
68+
Name: "devworkspace-operator-config",
69+
Namespace: config.OperatorNamespace,
70+
}, dwoc)
71+
if err != nil {
72+
ginkgo.Fail(fmt.Sprintf("Failed to get current DWOC for restoration: %s", err))
73+
}
74+
75+
// Restore the original config
76+
dwoc.Config = originalConfig
77+
78+
err = config.AdminK8sClient.ControllerRuntimeClient().Update(ctx, dwoc)
79+
if err != nil {
80+
ginkgo.Fail(fmt.Sprintf("Failed to restore original DWOC configuration: %s", err))
81+
}
82+
})
83+
4184
ginkgo.It("Wait DevWorkspace Webhook Server Pod", func() {
4285
controllerLabel := "app.kubernetes.io/name=devworkspace-webhook-server"
4386

@@ -95,8 +138,9 @@ var _ = ginkgo.Describe("[Custom Init Container Tests]", func() {
95138
})
96139

97140
ginkgo.AfterEach(func() {
98-
// Cleanup workspace
99-
_ = config.DevK8sClient.DeleteDevWorkspace(workspaceName, config.DevWorkspaceNamespace)
141+
// Clean up workspace and wait for PVC to be fully deleted
142+
// This prevents PVC conflicts in subsequent tests, especially in CI environments
143+
_ = config.DevK8sClient.DeleteDevWorkspaceAndWait(workspaceName, config.DevWorkspaceNamespace)
100144
})
101145
})
102146

@@ -149,8 +193,9 @@ var _ = ginkgo.Describe("[Custom Init Container Tests]", func() {
149193
})
150194

151195
ginkgo.AfterEach(func() {
152-
// Cleanup workspace
153-
_ = config.DevK8sClient.DeleteDevWorkspace(workspaceName, config.DevWorkspaceNamespace)
196+
// Clean up workspace and wait for PVC to be fully deleted
197+
// This prevents PVC conflicts in subsequent tests, especially in CI environments
198+
_ = config.DevK8sClient.DeleteDevWorkspaceAndWait(workspaceName, config.DevWorkspaceNamespace)
154199
})
155200
})
156201
})

test/e2e/pkg/tests/devworkspace_restart_tests.go

Lines changed: 15 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
// Copyright (c) 2019-2025 Red Hat, Inc.
1+
// Copyright (c) 2019-2026 Red Hat, Inc.
22
// Licensed under the Apache License, Version 2.0 (the "License");
33
// you may not use this file except in compliance with the License.
44
// You may obtain a copy of the License at
@@ -21,9 +21,17 @@ import (
2121
"github.com/onsi/gomega"
2222
)
2323

24-
var _ = ginkgo.Describe("[Create DevWorkspace and ensure data is persisted during restarts]", func() {
24+
var _ = ginkgo.Describe("[Create DevWorkspace and ensure data is persisted during restarts]", ginkgo.Ordered, func() {
2525
defer ginkgo.GinkgoRecover()
2626

27+
const workspaceName = "code-latest"
28+
29+
ginkgo.AfterAll(func() {
30+
// Cleanup workspace and wait for PVC to be fully deleted
31+
// This prevents PVC conflicts in subsequent tests, especially in CI environments
32+
_ = config.DevK8sClient.DeleteDevWorkspaceAndWait(workspaceName, config.DevWorkspaceNamespace)
33+
})
34+
2735
ginkgo.It("Wait DevWorkspace Webhook Server Pod", func() {
2836
controllerLabel := "app.kubernetes.io/name=devworkspace-webhook-server"
2937

@@ -45,7 +53,7 @@ var _ = ginkgo.Describe("[Create DevWorkspace and ensure data is persisted durin
4553
return
4654
}
4755

48-
deploy, err := config.DevK8sClient.WaitDevWsStatus("code-latest", config.DevWorkspaceNamespace, dw.DevWorkspaceStatusRunning)
56+
deploy, err := config.DevK8sClient.WaitDevWsStatus(workspaceName, config.DevWorkspaceNamespace, dw.DevWorkspaceStatusRunning)
4957
if !deploy {
5058
ginkgo.Fail(fmt.Sprintf("DevWorkspace didn't start properly. Error: %s", err))
5159
}
@@ -74,22 +82,22 @@ var _ = ginkgo.Describe("[Create DevWorkspace and ensure data is persisted durin
7482
})
7583

7684
ginkgo.It("Stop DevWorkspace", func() {
77-
err := config.AdminK8sClient.UpdateDevWorkspaceStarted("code-latest", config.DevWorkspaceNamespace, false)
85+
err := config.AdminK8sClient.UpdateDevWorkspaceStarted(workspaceName, config.DevWorkspaceNamespace, false)
7886
if err != nil {
7987
ginkgo.Fail(fmt.Sprintf("failed to stop DevWorkspace container, returned: %s", err))
8088
}
81-
deploy, err := config.DevK8sClient.WaitDevWsStatus("code-latest", config.DevWorkspaceNamespace, dw.DevWorkspaceStatusStopped)
89+
deploy, err := config.DevK8sClient.WaitDevWsStatus(workspaceName, config.DevWorkspaceNamespace, dw.DevWorkspaceStatusStopped)
8290
if !deploy {
8391
ginkgo.Fail(fmt.Sprintf("DevWorkspace didn't start properly. Error: %s", err))
8492
}
8593
})
8694

8795
ginkgo.It("Start DevWorkspace", func() {
88-
err := config.AdminK8sClient.UpdateDevWorkspaceStarted("code-latest", config.DevWorkspaceNamespace, true)
96+
err := config.AdminK8sClient.UpdateDevWorkspaceStarted(workspaceName, config.DevWorkspaceNamespace, true)
8997
if err != nil {
9098
ginkgo.Fail(fmt.Sprintf("failed to start DevWorkspace container"))
9199
}
92-
deploy, err := config.DevK8sClient.WaitDevWsStatus("code-latest", config.DevWorkspaceNamespace, dw.DevWorkspaceStatusRunning)
100+
deploy, err := config.DevK8sClient.WaitDevWsStatus(workspaceName, config.DevWorkspaceNamespace, dw.DevWorkspaceStatusRunning)
93101
if !deploy {
94102
ginkgo.Fail(fmt.Sprintf("DevWorkspace didn't start properly. Error: %s", err))
95103
}

test/e2e/pkg/tests/devworkspaces_tests.go

Lines changed: 11 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
//
2-
// Copyright (c) 2019-2025 Red Hat, Inc.
2+
// Copyright (c) 2019-2026 Red Hat, Inc.
33
// Licensed under the Apache License, Version 2.0 (the "License");
44
// you may not use this file except in compliance with the License.
55
// You may obtain a copy of the License at
@@ -25,9 +25,17 @@ import (
2525
"github.com/onsi/gomega"
2626
)
2727

28-
var _ = ginkgo.Describe("[Create OpenShift Web Terminal Workspace]", func() {
28+
var _ = ginkgo.Describe("[Create OpenShift Web Terminal Workspace]", ginkgo.Ordered, func() {
2929
defer ginkgo.GinkgoRecover()
3030

31+
const workspaceName = "restricted-access"
32+
33+
ginkgo.AfterAll(func() {
34+
// Cleanup workspace and wait for PVC to be fully deleted
35+
// This prevents PVC conflicts in subsequent tests, especially in CI environments
36+
_ = config.DevK8sClient.DeleteDevWorkspaceAndWait(workspaceName, config.DevWorkspaceNamespace)
37+
})
38+
3139
ginkgo.It("Wait DewWorkspace Webhook Server Pod", func() {
3240
controllerLabel := "app.kubernetes.io/name=devworkspace-webhook-server"
3341

@@ -49,7 +57,7 @@ var _ = ginkgo.Describe("[Create OpenShift Web Terminal Workspace]", func() {
4957
return
5058
}
5159

52-
deploy, err := config.DevK8sClient.WaitDevWsStatus("restricted-access", config.DevWorkspaceNamespace, dw.DevWorkspaceStatusRunning)
60+
deploy, err := config.DevK8sClient.WaitDevWsStatus(workspaceName, config.DevWorkspaceNamespace, dw.DevWorkspaceStatusRunning)
5361
if !deploy {
5462
ginkgo.Fail(fmt.Sprintf("OpenShift Web terminal workspace didn't start properly. Error: %s", err))
5563
}

0 commit comments

Comments
 (0)