chore(ci): add code coverage collection for integration and e2e test suites#25088
chore(ci): add code coverage collection for integration and e2e test suites#25088thomasqueirozb wants to merge 14 commits intomasterfrom
Conversation
thomasqueirozb
added
the
no-changelog
github-actions
bot
added
domain: ci
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
… called from coverage workflow
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
thomasqueirozb
left a comment
thomasqueirozb
left a comment
There was a problem hiding this comment.
/ci-run-coverage
|
@codex review |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4c1b0d972e
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 02162c3cf7
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 999a083914
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e9a4ded61c
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8d9f8664f9
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| for environment in environments { | ||
| ComposeTest::generate(local_config, integration, environment, retries)? | ||
| ComposeTest::generate(local_config, integration, environment, retries, coverage)? | ||
| .test(args.to_owned())?; |
There was a problem hiding this comment.
When coverage is enabled and no specific environment is provided, this loop runs every discovered environment sequentially, but each run writes to the same lcov.info output path (see vdev/src/testing/runner.rs where coverage always uses --output-path .../lcov.info). That means later environments overwrite earlier ones, so vdev int/e2e test --coverage reports only the last environment for multi-environment suites unless callers implement extra merge logic.
Useful? React with 👍 / 👎.
1 participant
Summary
Adds code coverage instrumentation to the integration and e2e test suites, making coverage additive across all test types (unit, component validation, integration, e2e). Results are merged into a single lcov report and uploaded to Datadog.
Key changes:
vdev int/e2e test --coverage: new flag that switches tocargo llvm-cov nextestand mounts a host directory for the coverage outputMakefile:COVERAGE=true/falsecontrols whethermake test/make test-component-validationusecargo nextestorcargo llvm-cov nextest --no-report;make coverage-reportemits the mergedlcov.infoscripts/run-integration-test.sh:-cflag enables coverage and normalizes container paths (SF:/home/vector/) to relative paths for clean merging.github/workflows/unit-tests.yml: new reusable workflow extracted fromtest.yml; used by both the test suite and the coverage workflow.github/workflows/integration.yml: supportsworkflow_callwithcoverageinput, enabling reuse from the coverage workflow.github/workflows/coverage.yml: rewritten as a scheduled (weekly) + on-demand workflow that runs all suites with coverage and uploads the merged report to Datadog.github/workflows/ci-review-trigger.yml: adds/ci-run-coveragetriggerVector configuration
NA
How did you test this PR?
Tested locally with
scripts/run-integration-test.sh -c int mqttto verify the lcov output and path normalization./ci-run-coverageCI Run
Change Type
Is this a breaking change?
Does this PR include user facing changes?
no-changeloglabel to this PR.References
NA