Skip to content

Dependency fetch output from state gcs docs#5503

Open
joshmyers wants to merge 3 commits intogruntwork-io:mainfrom
joshmyers:dependency_fetch_output_from_state_gcs_docs
Open

Dependency fetch output from state gcs docs#5503
joshmyers wants to merge 3 commits intogruntwork-io:mainfrom
joshmyers:dependency_fetch_output_from_state_gcs_docs

Conversation

@joshmyers
Copy link

@joshmyers joshmyers commented Feb 3, 2026

Description

Docs linked to #5499

TODOs

Read the Gruntwork contribution guidelines.

  • I authored this code entirely myself
  • I am submitting code based on open source software (e.g. MIT, MPL-2.0, Apache)]
  • I am adding or upgrading a dependency or adapted code and confirm it has a compatible open source license
  • Update the docs.
  • Run the relevant tests successfully, including pre-commit checks.
  • Include release notes. If this PR is backward incompatible, include a migration guide.

Release Notes (draft)

Docs linked to #5499

Summary by CodeRabbit

  • New Features

    • Added support for Google Cloud Storage (GCS) as a backend for remote-state dependency output fetching.
  • Documentation

    • Updated docs and experiment flag to show GCS compatibility alongside S3.
    • Updated troubleshooting note about fetching outputs from state with GCS and expanded caution text.
  • Tests

    • Added integration tests, fixtures, and example configs covering GCS remote-state output fetching and mock-output scenarios.

@vercel
Copy link

vercel bot commented Feb 3, 2026

@joshmyers is attempting to deploy a commit to the Gruntwork Team on Vercel.

A member of the Team first needs to authorize it.

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Feb 3, 2026

📝 Walkthrough

Walkthrough

Adds Google Cloud Storage (GCS) backend support for fetching dependency outputs from remote state, updates docs and feature flag to reflect GCS support, extends pkg/config remote-state logic for GCS, and adds GCS test fixtures and integration tests.

Changes

Cohort / File(s) Summary
Documentation
docs-starlight/src/content/docs/04-reference/04-experiments.md, docs-starlight/src/content/docs/06-troubleshooting/03-performance.mdx, docs-starlight/src/data/flags/dependency-fetch-output-from-state.mdx
Marked GCS as supported in docs and feature-flag content; updated troubleshooting notes to state dependency-fetch-output-from-state works with S3 and GCS.
GCS backend implementation
pkg/config/dependency.go
Added GCS support to dependency remote-state flow: imports (path, gcphelper, gcs backend), GCS backend recognition, GCS-specific not-found handling, new unexported helper to fetch/parse state from a GCS object, and integrated wiring into existing remote-state and fallback logic.
GCS test fixtures
test/fixtures/output-from-remote-state-gcs/root.hcl, test/fixtures/output-from-remote-state-gcs/env1/app1/..., test/fixtures/output-from-remote-state-gcs/env1/app2/..., test/fixtures/output-from-remote-state-gcs/env1/app3/...
Added Terragrunt/Terraform fixtures configuring GCS backends, outputs, variables, mock outputs, and dependency wiring to exercise GCS remote-state output fetching.
Integration tests
test/integration_gcp_test.go
Added GCP integration tests and a fixture constant to validate dependency-fetch-output-from-state behavior with GCS (normal fetch, disabled fetch, and mock outputs).

Sequence Diagram

sequenceDiagram
    participant TG as Terragrunt (pkg/config/dependency.go)
    participant CFG as Backend Config Parser
    participant GCPH as GCP Helper (gcphelper)
    participant GCS as Google Cloud Storage
    participant JSON as JSON Parser

    TG->>CFG: Parse remote-state backend config (backend="gcs", bucket, key/path)
    CFG->>TG: Return GCS config (bucket, key, optional path)
    TG->>GCPH: Create GCS client (project/creds)
    GCPH->>GCS: Initialize authenticated client
    GCS->>GCPH: Return client
    TG->>GCS: Read object bytes (bucket, key)
    GCS->>TG: Return bytes or not-found error
    TG->>JSON: Parse state JSON into outputs
    JSON->>TG: Return outputs map
    TG->>TG: Provide outputs to dependent modules / fallback as needed
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

🚥 Pre-merge checks | ✅ 1 | ❌ 2
❌ Failed checks (1 warning, 1 inconclusive)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 57.14% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Description check ❓ Inconclusive The PR description is incomplete: it references a linked PR but lacks a substantive description of changes, and the Release Notes section only contains a reference link without actual release note content. Provide a clear description of documentation changes and complete the Release Notes section with a meaningful one-line summary of what was added or updated.
✅ Passed checks (1 passed)
Check name Status Explanation
Title check ✅ Passed The title 'Dependency fetch output from state gcs docs' is directly related to the changeset, which updates documentation to reflect GCS backend support for dependency output fetching.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
docs-starlight/src/content/docs/06-troubleshooting/03-performance.mdx (1)

49-54: ⚠️ Potential issue | 🟡 Minor

Align the S3-only wording with the new GCS support.

Line 53 still says the state file is fetched from S3 only, which now contradicts the updated S3+GCS support described below.

✏️ Suggested wording tweak
-You can significantly improve the performance of dependency blocks by using the [`dependency-fetch-output-from-state`](https://terragrunt.gruntwork.io/docs/reference/experiments/#dependency-fetch-output-from-state) experiment. When the experiment is active, Terragrunt will resolve outputs by directly fetching the backend state file from S3 and parse it directly, avoiding any overhead incurred by calling the `output -json` command of OpenTofu/Terraform.
+You can significantly improve the performance of dependency blocks by using the [`dependency-fetch-output-from-state`](https://terragrunt.gruntwork.io/docs/reference/experiments/#dependency-fetch-output-from-state) experiment. When the experiment is active, Terragrunt will resolve outputs by directly fetching the backend state file from S3 or GCS and parse it directly, avoiding any overhead incurred by calling the `output -json` command of OpenTofu/Terraform.

As per coding guidelines: “Review the documentation for clarity, grammar, and spelling… Make sure that the docs-starlight documentation is accurate and up-to-date with the docs documentation.”

🤖 Fix all issues with AI agents
In `@pkg/config/dependency.go`:
- Around line 1231-1299: getTerragruntOutputJSONFromRemoteStateGCS currently
treats a missing GCS object as a hard error; detect the GCS "object not found"
case (check for storage.ErrObjectNotExist and/or googleapi 404 when calling
obj.NewReader) and return a sentinel/recognizable error so the caller can fall
back to mock outputs like S3 does (mirror isAwsS3NoSuchKey behavior). Update
getTerragruntOutputJSONFromRemoteStateGCS (around the obj.NewReader error
handling) to map the GCS-not-found condition to the same fallback-triggering
signal used for S3 (or add an isGcsObjectNotFound helper and use it in
getTerragruntOutput) so mock-output fallback is preserved.

@joshmyers joshmyers force-pushed the dependency_fetch_output_from_state_gcs_docs branch from fc867a5 to d5161d9 Compare February 4, 2026 11:27
@joshmyers joshmyers force-pushed the dependency_fetch_output_from_state_gcs_docs branch from d5161d9 to 7989616 Compare February 4, 2026 11:35
@joshmyers joshmyers force-pushed the dependency_fetch_output_from_state_gcs_docs branch from 7989616 to a29d650 Compare February 4, 2026 11:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant