Skip to content

Commit 5d52371

Browse files
lee-bainesdnitsch
andauthored
fix(AWS SecretsManager): update to v2 (#255) (#272)
* fix: securit update update go 1.21 update AWS SDK v2 for secrets manager * fix: update docs * fix: run fmt locally * fix: update SHAs * fix: set buildvcs false * fix: revert * fix: remove cancel * fix: update go directives * fix: remove deprecated io/ioutil * fix: set buildvcs to false * fix: add GOFLAGS to the docker context * fix: add more unit tests * fix: re-run createcluster workflow... * fix: update crypto pkg Co-authored-by: dnitsch <nitschneiderd@gmail.com>
1 parent 6224347 commit 5d52371

14 files changed

Lines changed: 376 additions & 82 deletions

README.md

Lines changed: 10 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -69,16 +69,21 @@ In order to fetch and store secrets in the AWS Secrets Manager, the user must pr
6969

7070
#### Set up Cloud Backup With AWS Secret Manager
7171

72-
The `secret-agent` expects credentials to be discoverable via standard [AWS mechanisms](https://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/configuring-sdk.html#specifying-credentials). These credentials can be provided in a number of ways, for example:
72+
The `secret-agent` expects credentials to be discoverable via standard [AWS mechanisms](https://aws.github.io/aws-sdk-go-v2/docs/configuring-sdk/#specifying-credentials). These credentials can be provided in a number of ways as per the link.
7373

74-
* Environment Variables: _AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY_
74+
However, when running inside AWS the _preferred_ method should be to attach a serviceAccount to the deployment with properly scope policies on the role.
75+
76+
* Environment Variables:
77+
- _AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY_, _AWS_SESSION_TOKEN_
78+
- _AWS_WEB_IDENTITY_TOKEN_FILE_ -> this will be handled by the IAM controller for each deployment correctly set up annotations on the serviceAccount.
7579
* Shared Credentials file: _~/.aws/credentials_
76-
* Shared Configuration file: _(~/.aws/config_
77-
* EC2 Instance Metadata (preferred): _Obtains credentials from 169.254.169.254_
80+
* Shared Configuration file: _~/.aws/config_
81+
* EC2 Instance Metadata_v2: _Obtains credentials from 169.254.169.254_
82+
- This is not ideal as all the pods on the node will have access to the same policy, _this_ may not be a desired outcome if topologySkew are not properly segragating workloads across different nodes/node pools.
7883

7984
Refer to [AWS documentation](https://docs.aws.amazon.com/secretsmanager/latest/userguide/auth-and-access_overview.html) for instructions on how to obtain credentials and grant necessary permissions to access the AWS Secrets Manager. The `secret-agent` needs to access read/write secrets. This can be achieved by allowing access to the `arn:aws:iam::aws:policy/SecretsManagerReadWrite` AWS managed policy.
8085

81-
Even though the recommended way to obtain credentials is to use the EC2 Instance Metadata service, it is possible to provide custom credentials via a Kubernetes secret. The secret reference is provided in the SAC in `spec.appConfig.credentialsSecretName`. In the default `secret-agent` deployment, the user is expected to publish the cloud credentials' secret in the same namespace as the operator. This target namespace can be changed by changing the runtime argument `--cloud-secrets-namespace=[NS_NAME]` located in the operator's [manifest](/config/manager/manager.yaml). If this argument is omitted completely, the namespace will default to the namespace of each SAC.
86+
When running outside of AWS environments it is possible to provide custom credentials via a Kubernetes secret. The secret reference is provided in the SAC in `spec.appConfig.credentialsSecretName`. In the default `secret-agent` deployment, the user is expected to publish the cloud credentials' secret in the same namespace as the operator. This target namespace can be changed by changing the runtime argument `--cloud-secrets-namespace=[NS_NAME]` located in the operator's [manifest](/config/manager/manager.yaml). If this argument is omitted completely, the namespace will default to the namespace of each SAC.
8287

8388
Once these credentials are posted to a Kubernetes secret, the next step is to configure the AWS Secret Manager using the `SecretAgentConfiguration`.
8489

api/v1alpha1/suite_test.go

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,4 @@
11
//go:build integration
2-
// +build integration
32

43
/*
54

api/v1alpha1/v1alpha1_test.go

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,4 @@
11
//go:build integration
2-
// +build integration
32

43
/*
54

api/v1alpha1/zz_generated.deepcopy.go

Lines changed: 0 additions & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

config/crd/bases/secret-agent.secrets.forgerock.io_secretagentconfigurations.yaml

Lines changed: 12 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,9 @@
1-
21
---
32
apiVersion: apiextensions.k8s.io/v1
43
kind: CustomResourceDefinition
54
metadata:
65
annotations:
7-
controller-gen.kubebuilder.io/version: v0.6.1
8-
creationTimestamp: null
6+
controller-gen.kubebuilder.io/version: v0.15.0
97
name: secretagentconfigurations.secret-agent.secrets.forgerock.io
108
spec:
119
group: secret-agent.secrets.forgerock.io
@@ -48,14 +46,19 @@ spec:
4846
API
4947
properties:
5048
apiVersion:
51-
description: 'APIVersion defines the versioned schema of this representation
52-
of an object. Servers should convert recognized schemas to the latest
53-
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
49+
description: |-
50+
APIVersion defines the versioned schema of this representation of an object.
51+
Servers should convert recognized schemas to the latest internal value, and
52+
may reject unrecognized values.
53+
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
5454
type: string
5555
kind:
56-
description: 'Kind is a string value representing the REST resource this
57-
object represents. Servers may infer this from the endpoint the client
58-
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
56+
description: |-
57+
Kind is a string value representing the REST resource this object represents.
58+
Servers may infer this from the endpoint the client submits requests to.
59+
Cannot be updated.
60+
In CamelCase.
61+
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
5962
type: string
6063
metadata:
6164
type: object
@@ -294,9 +297,3 @@ spec:
294297
storage: true
295298
subresources:
296299
status: {}
297-
status:
298-
acceptedNames:
299-
kind: ""
300-
plural: ""
301-
conditions: []
302-
storedVersions: []

config/rbac/role.yaml

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,7 @@
1-
21
---
32
apiVersion: rbac.authorization.k8s.io/v1
43
kind: ClusterRole
54
metadata:
6-
creationTimestamp: null
75
name: manager-role
86
rules:
97
- apiGroups:

config/webhook/manifests.yaml

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,7 @@
1-
21
---
32
apiVersion: admissionregistration.k8s.io/v1
43
kind: MutatingWebhookConfiguration
54
metadata:
6-
creationTimestamp: null
75
name: mutating-webhook-configuration
86
webhooks:
97
- admissionReviewVersions:
@@ -26,12 +24,10 @@ webhooks:
2624
resources:
2725
- secretagentconfigurations
2826
sideEffects: None
29-
3027
---
3128
apiVersion: admissionregistration.k8s.io/v1
3229
kind: ValidatingWebhookConfiguration
3330
metadata:
34-
creationTimestamp: null
3531
name: validating-webhook-configuration
3632
webhooks:
3733
- admissionReviewVersions:

go.mod

Lines changed: 15 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,10 @@ require (
99
github.com/Azure/azure-sdk-for-go v68.0.0+incompatible
1010
github.com/Azure/go-autorest/autorest v0.11.29
1111
github.com/Azure/go-autorest/autorest/azure/auth v0.5.12
12-
github.com/aws/aws-sdk-go v1.52.6
12+
github.com/aws/aws-sdk-go-v2 v1.27.0
13+
github.com/aws/aws-sdk-go-v2/config v1.27.16
14+
github.com/aws/aws-sdk-go-v2/service/secretsmanager v1.29.1
15+
github.com/aws/smithy-go v1.20.2
1316
github.com/go-logr/logr v1.4.1
1417
github.com/go-playground/validator/v10 v10.15.1
1518
github.com/golang/glog v1.2.1
@@ -42,6 +45,16 @@ require (
4245
github.com/Azure/go-autorest/autorest/validation v0.3.1 // indirect
4346
github.com/Azure/go-autorest/logger v0.2.1 // indirect
4447
github.com/Azure/go-autorest/tracing v0.6.0 // indirect
48+
github.com/aws/aws-sdk-go-v2/credentials v1.17.16 // indirect
49+
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.16.3 // indirect
50+
github.com/aws/aws-sdk-go-v2/internal/configsources v1.3.7 // indirect
51+
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.6.7 // indirect
52+
github.com/aws/aws-sdk-go-v2/internal/ini v1.8.0 // indirect
53+
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.11.2 // indirect
54+
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.11.9 // indirect
55+
github.com/aws/aws-sdk-go-v2/service/sso v1.20.9 // indirect
56+
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.24.3 // indirect
57+
github.com/aws/aws-sdk-go-v2/service/sts v1.28.10 // indirect
4558
github.com/beorn7/perks v1.0.1 // indirect
4659
github.com/cespare/xxhash/v2 v2.2.0 // indirect
4760
github.com/davecgh/go-spew v1.1.1 // indirect
@@ -73,7 +86,6 @@ require (
7386
github.com/googleapis/enterprise-certificate-proxy v0.3.2 // indirect
7487
github.com/googleapis/gax-go/v2 v2.12.4 // indirect
7588
github.com/imdario/mergo v0.3.10 // indirect
76-
github.com/jmespath/go-jmespath v0.4.0 // indirect
7789
github.com/josharian/intern v1.0.0 // indirect
7890
github.com/json-iterator/go v1.1.12 // indirect
7991
github.com/leodido/go-urn v1.4.0 // indirect
@@ -118,4 +130,4 @@ require (
118130
sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd // indirect
119131
sigs.k8s.io/structured-merge-diff/v4 v4.4.1 // indirect
120132
sigs.k8s.io/yaml v1.4.0 // indirect
121-
)
133+
)

go.sum

Lines changed: 28 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -39,8 +39,34 @@ github.com/Azure/go-autorest/logger v0.2.1/go.mod h1:T9E3cAhj2VqvPOtCYAvby9aBXkZ
3939
github.com/Azure/go-autorest/tracing v0.6.0 h1:TYi4+3m5t6K48TGI9AUdb+IzbnSxvnvUMfuitfgcfuo=
4040
github.com/Azure/go-autorest/tracing v0.6.0/go.mod h1:+vhtPC754Xsa23ID7GlGsrdKBpUA79WCAKPPZVC2DeU=
4141
github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
42-
github.com/aws/aws-sdk-go v1.52.6 h1:nw1AMg0wIj5tTnI89KaDe9G5aISqXm4KJEe1DfNbFvA=
43-
github.com/aws/aws-sdk-go v1.52.6/go.mod h1:LF8svs817+Nz+DmiMQKTO3ubZ/6IaTpq3TjupRn3Eqk=
42+
github.com/aws/aws-sdk-go-v2 v1.27.0 h1:7bZWKoXhzI+mMR/HjdMx8ZCC5+6fY0lS5tr0bbgiLlo=
43+
github.com/aws/aws-sdk-go-v2 v1.27.0/go.mod h1:ffIFB97e2yNsv4aTSGkqtHnppsIJzw7G7BReUZ3jCXM=
44+
github.com/aws/aws-sdk-go-v2/config v1.27.16 h1:knpCuH7laFVGYTNd99Ns5t+8PuRjDn4HnnZK48csipM=
45+
github.com/aws/aws-sdk-go-v2/config v1.27.16/go.mod h1:vutqgRhDUktwSge3hrC3nkuirzkJ4E/mLj5GvI0BQas=
46+
github.com/aws/aws-sdk-go-v2/credentials v1.17.16 h1:7d2QxY83uYl0l58ceyiSpxg9bSbStqBC6BeEeHEchwo=
47+
github.com/aws/aws-sdk-go-v2/credentials v1.17.16/go.mod h1:Ae6li/6Yc6eMzysRL2BXlPYvnrLLBg3D11/AmOjw50k=
48+
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.16.3 h1:dQLK4TjtnlRGb0czOht2CevZ5l6RSyRWAnKeGd7VAFE=
49+
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.16.3/go.mod h1:TL79f2P6+8Q7dTsILpiVST+AL9lkF6PPGI167Ny0Cjw=
50+
github.com/aws/aws-sdk-go-v2/internal/configsources v1.3.7 h1:lf/8VTF2cM+N4SLzaYJERKEWAXq8MOMpZfU6wEPWsPk=
51+
github.com/aws/aws-sdk-go-v2/internal/configsources v1.3.7/go.mod h1:4SjkU7QiqK2M9oozyMzfZ/23LmUY+h3oFqhdeP5OMiI=
52+
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.6.7 h1:4OYVp0705xu8yjdyoWix0r9wPIRXnIzzOoUpQVHIJ/g=
53+
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.6.7/go.mod h1:vd7ESTEvI76T2Na050gODNmNU7+OyKrIKroYTu4ABiI=
54+
github.com/aws/aws-sdk-go-v2/internal/ini v1.8.0 h1:hT8rVHwugYE2lEfdFE0QWVo81lF7jMrYJVDWI+f+VxU=
55+
github.com/aws/aws-sdk-go-v2/internal/ini v1.8.0/go.mod h1:8tu/lYfQfFe6IGnaOdrpVgEL2IrrDOf6/m9RQum4NkY=
56+
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.11.2 h1:Ji0DY1xUsUr3I8cHps0G+XM3WWU16lP6yG8qu1GAZAs=
57+
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.11.2/go.mod h1:5CsjAbs3NlGQyZNFACh+zztPDI7fU6eW9QsxjfnuBKg=
58+
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.11.9 h1:Wx0rlZoEJR7JwlSZcHnEa7CNjrSIyVxMFWGAaXy4fJY=
59+
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.11.9/go.mod h1:aVMHdE0aHO3v+f/iw01fmXV/5DbfQ3Bi9nN7nd9bE9Y=
60+
github.com/aws/aws-sdk-go-v2/service/secretsmanager v1.29.1 h1:NSWsFzdHN41mJ5I/DOFzxgkKSYNHQADHn7Mu+lU/AKw=
61+
github.com/aws/aws-sdk-go-v2/service/secretsmanager v1.29.1/go.mod h1:5mMk0DgUgaHlcqtN65fNyZI0ZDX3i9Cw+nwq75HKB3U=
62+
github.com/aws/aws-sdk-go-v2/service/sso v1.20.9 h1:aD7AGQhvPuAxlSUfo0CWU7s6FpkbyykMhGYMvlqTjVs=
63+
github.com/aws/aws-sdk-go-v2/service/sso v1.20.9/go.mod h1:c1qtZUWtygI6ZdvKppzCSXsDOq5I4luJPZ0Ud3juFCA=
64+
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.24.3 h1:Pav5q3cA260Zqez42T9UhIlsd9QeypszRPwC9LdSSsQ=
65+
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.24.3/go.mod h1:9lmoVDVLz/yUZwLaQ676TK02fhCu4+PgRSmMaKR1ozk=
66+
github.com/aws/aws-sdk-go-v2/service/sts v1.28.10 h1:69tpbPED7jKPyzMcrwSvhWcJ9bPnZsZs18NT40JwM0g=
67+
github.com/aws/aws-sdk-go-v2/service/sts v1.28.10/go.mod h1:0Aqn1MnEuitqfsCNyKsdKLhDUOr4txD/g19EfiUqgws=
68+
github.com/aws/smithy-go v1.20.2 h1:tbp628ireGtzcHDDmLT/6ADHidqnwgF57XOXZe6tp4Q=
69+
github.com/aws/smithy-go v1.20.2/go.mod h1:krry+ya/rV9RDcV/Q16kpu6ypI4K2czasz0NC3qS14E=
4470
github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
4571
github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=
4672
github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU=
@@ -144,10 +170,6 @@ github.com/googleapis/gax-go/v2 v2.12.4 h1:9gWcmF85Wvq4ryPFvGFaOgPIs1AQX0d0bcbGw
144170
github.com/googleapis/gax-go/v2 v2.12.4/go.mod h1:KYEYLorsnIGDi/rPC8b5TdlB9kbKoFubselGIoBMCwI=
145171
github.com/imdario/mergo v0.3.10 h1:6q5mVkdH/vYmqngx7kZQTjJ5HRsx+ImorDIEQ+beJgc=
146172
github.com/imdario/mergo v0.3.10/go.mod h1:jmQim1M+e3UYxmgPu/WyfjB3N3VflVyUjjjwH0dnCYA=
147-
github.com/jmespath/go-jmespath v0.4.0 h1:BEgLn5cpjn8UN1mAw4NjwDrS35OdebyEtFe+9YPoQUg=
148-
github.com/jmespath/go-jmespath v0.4.0/go.mod h1:T8mJZnbsbmF+m6zOOFylbeCJqk5+pHWvzYPziyZiYoo=
149-
github.com/jmespath/go-jmespath/internal/testify v1.5.1 h1:shLQSRRSCCPj3f2gpwzGwWFoC7ycTf1rcQZHOlsJ6N8=
150-
github.com/jmespath/go-jmespath/internal/testify v1.5.1/go.mod h1:L3OGu8Wl2/fWfCI6z80xFu9LTZmf1ZRjMHUOPmWr69U=
151173
github.com/josharian/intern v1.0.0 h1:vlS4z54oSdjm0bgjRigI+G1HpF+tI+9rE5LLzOg8HmY=
152174
github.com/josharian/intern v1.0.0/go.mod h1:5DoeVV0s6jJacbCEi61lwdGj/aVlrQvzHFFd8Hwg//Y=
153175
github.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnrnM=

kind

6.61 MB
Binary file not shown.

0 commit comments

Comments
 (0)