Pipelines

Comprehensive, enterprise-grade SDLC pipeline templates for GitHub Actions, GitLab CI, and Azure DevOps with security scanning (SAST), dependency analysis (SCA), supply chain security (SSCA), testing, and deployment automation for multiple programming languages.

Supported Platforms & Languages

Platforms

Platform	Status	Documentation
GitHub Actions	Full Support	Usage Guide
GitLab CI	Full Support	Usage Guide
Azure DevOps	Full Support	Usage Guide

Programming Languages

Language	GitHub Actions	GitLab CI	Azure DevOps	Features
GoLang	yes	yes	yes	Binary, Docker, ARM deployment
Python	yes	yes	yes	PDM, Docker, K8s deployment
Java	yes	yes	yes	Maven, Gradle, Docker
JavaScript/Node.js	yes	yes	yes	npm, Yarn, Docker, K8s deployment
PHP	yes	no	no	Composer, Docker
Ruby	yes	no	no	Bundler, Docker
.NET/C#	yes	yes	yes	Framework, Core, Docker
Terraform	no	yes	yes	Infrastructure as Code
Terra CLI	yes	yes	yes	Terraform/Terragrunt wrapper

Project Structure

pipelines/
├── .github/workflows/          # GitHub Actions reusable workflows
│   ├── go-docker.yaml         # Go with Docker delivery
│   ├── go-binary.yaml         # Go binary compilation
│   ├── pdm-docker.yaml        # Python/PDM with Docker
│   ├── gradle-docker.yaml     # Java/Gradle with Docker delivery
│   ├── maven-docker.yaml      # Java/Maven with Docker delivery
│   ├── yarn-docker.yaml       # JavaScript/Yarn with Docker delivery
│   ├── npm-docker.yaml        # JavaScript/npm with Docker delivery
│   ├── composer-docker.yaml   # PHP/Composer with Docker delivery
│   ├── bundler-docker.yaml    # Ruby/Bundler with Docker delivery
│   ├── dotnet-docker.yaml     # .NET with Docker delivery
│   └── ...
├── gitlab/                     # GitLab CI pipeline templates
│   ├── golang/                # Go language pipelines
│   ├── java/                  # Java language pipelines
│   ├── python/                # Python language pipelines
│   ├── javascript/            # JavaScript/Node.js pipelines
│   ├── dotnet/                # .NET language pipelines
│   ├── terraform/             # Terraform pipelines (raw terraform/terragrunt)
│   ├── terra/                 # Terra CLI pipelines (terraform/terragrunt wrapper)
│   └── global/                # Shared GitLab configurations
├── azure-devops/              # Azure DevOps pipeline templates
│   ├── golang/                # Go language pipelines
│   ├── java/                  # Java language pipelines
│   ├── python/                # Python language pipelines
│   ├── javascript/            # JavaScript/Node.js pipelines
│   ├── dotnet/                # .NET language pipelines
│   ├── terraform/             # Terraform pipelines (raw terraform/terragrunt)
│   ├── terra/                 # Terra CLI pipelines (terraform/terragrunt wrapper)
│   └── global/                # Shared Azure DevOps templates
├── global/                     # Shared resources across platforms
│   ├── scripts/               # Automation scripts
│   │   ├── tools/             # Language-agnostic tools
│   │   │   ├── codeql/        # SAST security scanning (CodeQL)
│   │   │   ├── gitleaks/      # Secret scanning
│   │   │   ├── hadolint/      # Dockerfile linting
│   │   │   ├── semgrep/       # Static analysis
│   │   │   ├── sonarqube/     # Code quality
│   │   │   ├── trivy/         # IaC misconfiguration scanning
│   │   │   └── dependency-track/ # SCA analysis
│   │   ├── languages/         # Language-specific scripts
│   │   │   ├── golang/        # Go scripts (test, cyclonedx, golangci-lint, init)
│   │   │   └── python/        # Python scripts (cyclonedx)
│   │   └── shared/            # Common utilities
│   ├── containers/            # Custom Docker images
│   │   ├── golang.*/          # Go development images
│   │   ├── python.*/          # Python development images
│   │   ├── awscli.latest/     # AWS CLI tools
│   │   └── tor-proxy.latest/  # Network proxy tools
│   └── configs/               # Configuration files
├── makefiles/                  # Includable Makefile fragments for local usage
│   ├── common.mk              # Security tools (sast) and setup
│   ├── golang.mk              # Go targets (lint, test)
│   ├── python.mk              # Python/PDM targets (lint, test)
│   ├── java.mk                # Java/Gradle targets (lint, test)
│   ├── javascript.mk          # JavaScript/Yarn targets (lint, test)
│   ├── dotnet.mk              # .NET/C# targets (lint, test)
│   ├── terraform.mk           # Terraform targets (lint, test)
│   └── terra.mk               # Terra CLI targets (lint, test)
├── .docs/                      # Documentation and examples
│   └── examples/              # Per-provider usage examples
└── .github/tests/              # Validation scripts for this repository

Pipeline Architecture

Each platform follows a consistent 5-stage pipeline architecture:

Code Check (Style/Quality) - Linting, formatting, code quality, basic checks (rebase verification, changelog validation)
Security (SCA/SAST) - Vulnerability scanning, secret detection
Tests - Unit tests, integration tests, coverage reporting
Management - Dependency tracking, SBOM generation
Delivery - Build artifacts, container images, deployments

Installation

Quick Installation

curl -sSL https://raw.githubusercontent.com/rios0rios0/pipelines/main/clone.sh | bash

You can override the installation location with the PIPELINES_HOME environment variable:

PIPELINES_HOME=/opt/pipelines curl -sSL https://raw.githubusercontent.com/rios0rios0/pipelines/main/clone.sh | bash

Manual Installation

mkdir -p $HOME/Development/github.com/rios0rios0
cd $HOME/Development/github.com/rios0rios0
git clone https://github.com/rios0rios0/pipelines.git

Platform Usage

GitHub Actions

GitHub Actions workflows are located in .github/workflows/ and can be used as reusable workflows.

Available Workflows

Workflow	Purpose	Languages
`go.yaml`	Go testing and quality checks	Go
`go-docker.yaml`	Go with Docker image delivery	Go
`go-binary.yaml`	Go binary compilation and release	Go
`pdm.yaml`	Python/PDM testing and quality checks	Python
`pdm-docker.yaml`	Python/PDM with Docker image delivery	Python
`gradle.yaml`	Java/Gradle testing and quality checks	Java
`gradle-docker.yaml`	Java/Gradle with Docker image delivery	Java
`yarn.yaml`	JavaScript/Yarn testing and quality checks	JavaScript
`yarn-docker.yaml`	JavaScript/Yarn with Docker image delivery	JavaScript
`dotnet.yaml`	.NET testing and quality checks	C#
`dotnet-docker.yaml`	.NET with Docker image delivery	C#
`npm.yaml`	JavaScript/npm testing and quality checks	JavaScript
`npm-docker.yaml`	JavaScript/npm with Docker image delivery	JavaScript
`maven.yaml`	Java/Maven testing and quality checks	Java
`maven-docker.yaml`	Java/Maven with Docker image delivery	Java
`composer.yaml`	PHP/Composer testing and quality checks	PHP
`composer-docker.yaml`	PHP/Composer with Docker image delivery	PHP
`bundler.yaml`	Ruby/Bundler testing and quality checks	Ruby
`bundler-docker.yaml`	Ruby/Bundler with Docker image delivery	Ruby
`terra.yaml`	Terra CLI quality, security, and tests	Terraform/HCL

Usage Example (Go with Docker)

name: 'CI/CD Pipeline'

on:
  push:
    branches: [ main ]
    tags: [ '*' ]
  pull_request:
    branches: [ main ]

permissions:
  checks: write      # Required for test results
  contents: write    # Required for releases
  packages: write    # Required for container registry

jobs:
  pipeline:
    uses: 'rios0rios0/pipelines/.github/workflows/go-docker.yaml@main'

Usage Example (Python/PDM with Docker)

name: 'CI/CD Pipeline'

on:
  push:
    branches: [ main ]
    tags: [ '*' ]
  pull_request:
    branches: [ main ]

permissions:
  security-events: 'write'
  contents: 'write'
  packages: 'write'

jobs:
  default:
    uses: 'rios0rios0/pipelines/.github/workflows/pdm-docker.yaml@main'

Usage Example (Java with Docker)

name: 'CI/CD Pipeline'

on:
  push:
    branches: [ main ]
    tags: [ '*' ]
  pull_request:
    branches: [ main ]

permissions:
  security-events: write
  contents: write
  packages: write

jobs:
  pipeline:
    uses: 'rios0rios0/pipelines/.github/workflows/gradle-docker.yaml@main'

Usage Example (JavaScript/Yarn with Docker)

name: 'CI/CD Pipeline'

on:
  push:
    branches: [ main ]
    tags: [ '*' ]
  pull_request:
    branches: [ main ]

permissions:
  security-events: write
  contents: write
  packages: write
  pull-requests: write
  checks: write

jobs:
  pipeline:
    uses: 'rios0rios0/pipelines/.github/workflows/yarn-docker.yaml@main'

Usage Example (.NET with Docker)

name: 'CI/CD Pipeline'

on:
  push:
    branches: [ main ]
    tags: [ '*' ]
  pull_request:
    branches: [ main ]

permissions:
  security-events: write
  contents: write
  packages: write

jobs:
  pipeline:
    uses: 'rios0rios0/pipelines/.github/workflows/dotnet-docker.yaml@main'

Usage Example (JavaScript/npm with Docker)

name: 'CI/CD Pipeline'

on:
  push:
    branches: [ main ]
    tags: [ '*' ]
  pull_request:
    branches: [ main ]

permissions:
  security-events: write
  contents: write
  packages: write
  pull-requests: write
  checks: write

jobs:
  pipeline:
    uses: 'rios0rios0/pipelines/.github/workflows/npm-docker.yaml@main'

Usage Example (Java/Maven with Docker)

name: 'CI/CD Pipeline'

on:
  push:
    branches: [ main ]
    tags: [ '*' ]
  pull_request:
    branches: [ main ]

permissions:
  security-events: write
  contents: write
  packages: write

jobs:
  pipeline:
    uses: 'rios0rios0/pipelines/.github/workflows/maven-docker.yaml@main'

Usage Example (PHP with Docker)

name: 'CI/CD Pipeline'

on:
  push:
    branches: [ main ]
    tags: [ '*' ]
  pull_request:
    branches: [ main ]

permissions:
  contents: write
  packages: write

jobs:
  pipeline:
    uses: 'rios0rios0/pipelines/.github/workflows/composer-docker.yaml@main'

Usage Example (Ruby with Docker)

name: 'CI/CD Pipeline'

on:
  push:
    branches: [ main ]
    tags: [ '*' ]
  pull_request:
    branches: [ main ]

permissions:
  security-events: write
  contents: write
  packages: write

jobs:
  pipeline:
    uses: 'rios0rios0/pipelines/.github/workflows/bundler-docker.yaml@main'

GitLab CI

GitLab CI templates use remote includes and are organized by language in the gitlab/ directory.

Available Templates

Language	Template	Purpose
Go	`go-docker.yaml`	Go with Docker delivery
Go	`go-binary.yaml`	Go binary pipeline
Go	`go-sam.yaml`	Go with AWS SAM deployment
Java	`gradle-docker.yaml`	Gradle with Docker
Java	`maven-docker.yaml`	Maven with Docker
Python	`pdm-docker.yaml`	Python PDM with Docker
JavaScript	`yarn-docker.yaml`	Node.js Yarn with Docker
.NET	`framework.yaml`	.NET Framework pipeline
Terraform	`terra.yaml`	Terraform IaC pipeline

Usage Example (Go with Docker)

include:
  - remote: 'https://raw.githubusercontent.com/rios0rios0/pipelines/main/gitlab/golang/go-docker.yaml'

# Optional: Override delivery stage for custom Docker build
.delivery:
  script:
    - docker build -t "$REGISTRY_PATH$IMAGE_SUFFIX:$TAG" -f .ci/stages/40-delivery/Dockerfile .
  cache:
    key: 'test:all'
    paths: !reference [ .go, cache, paths ]
    policy: 'pull'

Usage Example (Python PDM)

include:
  - remote: 'https://raw.githubusercontent.com/rios0rios0/pipelines/main/gitlab/python/pdm-docker.yaml'

variables:
  PYTHON_VERSION: "3.11"  # Optional: specify a Python version

Usage Example (Terraform -- raw terraform/terragrunt)

include:
  - remote: 'https://raw.githubusercontent.com/rios0rios0/pipelines/main/gitlab/terraform/terra.yaml'

Usage Example (Terra CLI)

The terra CLI wraps Terraform and Terragrunt with a simplified interface, auto-answering prompts, and parallel execution. The terra pipeline provides code check, security, tests, and management stages. Delivery is intentionally excluded because it is project-specific (plan/apply targets, environments, stack ordering). See examples for all providers in the Azure DevOps section below.

Terra Test Stage

Every Terra pipeline (Azure DevOps, GitLab CI, GitHub Actions) exposes a single unified test:all job that delegates to global/scripts/languages/terraform/test-all/run.sh. The runner orchestrates two tiers:

Tier	Inputs	Tooling	Outputs (under `build/reports/`)
`terra-test`	`modules//tests/.tftest.hcl`	`terraform test -junit-xml`	`terra-tests.xml`, `terra-coverage.{md,json,xml}`
`terratest`	`tests/terratest/*.go`	`go test ./...` + `go-junit-report`	`junit-terratest.xml`

The runner auto-detects which tiers the consumer actually has, runs only those, merges both JUnit files into junit-terra-all.xml for the single-artifact upload contract used by GitLab CI and GitHub Actions, and propagates a non-zero exit from either tier so CI correctly fails. When neither tier has tests (e.g., a stack-only repo without modules/ tests or tests/terratest/), the runner emits an empty-but-valid JUnit and exits 0 so the job passes without a bespoke opt-out.

Required GitLab Variables

Configure these in your GitLab project settings:

Variable	Description	Required For
`SONAR_HOST_URL`	SonarQube server URL	Code quality
`SONAR_TOKEN`	SonarQube authentication token	Code quality
`DOCKER_REGISTRY`	Container registry URL	Docker delivery
`DOCKER_USERNAME`	Registry username	Docker delivery
`DOCKER_PASSWORD`	Registry password	Docker delivery

Azure DevOps

Azure DevOps templates are located in the azure-devops/ directory and use template references.

Available Templates

Language	Template	Purpose
Go	`go-docker.yaml`	Go with Docker delivery
Go	`go-arm.yaml`	Go with Azure ARM deployment
Go	`go-function-arm.yaml`	Go Azure Functions
Go	`go-lambda.yaml`	Go AWS Lambda deployment (ZIP)
Go	`go-lambda-sam.yaml`	Go AWS Lambda deployment (SAM)
Java	`kotlin-gradle.yaml`	Kotlin/Gradle with Docker
Python	`pdm-docker.yaml`	Python PDM with Docker
JavaScript	`yarn-docker.yaml`	Node.js Yarn with Docker
.NET	`core.yaml`	.NET Core pipeline
Terraform	`terra.yaml`	Infrastructure as Code pipeline
Terra CLI	`terra/terra.yaml`	Terra CLI wrapper pipeline

Usage Example (Go with Docker)

trigger:
  branches:
    include: [ main ]
  tags:
    include: [ '*' ]

pool:
  vmImage: 'ubuntu-latest'

variables:
  - ${{ if startsWith(variables['Build.SourceBranch'], 'refs/tags/') }}:
      - group: 'production-variables'
  - ${{ else }}:
      - group: 'development-variables'

resources:
  repositories:
    - repository: 'pipelines'
      type: 'github'
      name: 'rios0rios0/pipelines'
      endpoint: 'YOUR_GITHUB_SERVICE_CONNECTION'  # Configure this

stages:
  - template: 'azure-devops/golang/go-docker.yaml@pipelines'

Usage Example (Go with ARM Deployment)

resources:
  repositories:
    - repository: 'pipelines'
      type: 'github'
      name: 'rios0rios0/pipelines'
      endpoint: 'YOUR_GITHUB_SERVICE_CONNECTION'

stages:
  - template: 'azure-devops/golang/go-arm.yaml@pipelines'
    parameters:
      DOCKER_BUILD_ARGS: '--build-arg VERSION=$(Build.BuildNumber)'
      RUN_BEFORE_BUILD: 'echo "Preparing build environment"'

Usage Example (Go with AWS Lambda)

trigger:
  branches:
    include: [ main ]
  tags:
    include: [ '*' ]

pool:
  vmImage: 'ubuntu-latest'

variables:
  - ${{ if startsWith(variables['Build.SourceBranch'], 'refs/tags/') }}:
      - group: 'production-variables'
  - ${{ else }}:
      - group: 'development-variables'

resources:
  repositories:
    - repository: 'pipelines'
      type: 'github'
      name: 'rios0rios0/pipelines'
      endpoint: 'YOUR_GITHUB_SERVICE_CONNECTION'

stages:
  - template: 'azure-devops/golang/go-lambda.yaml@pipelines'
    parameters:
      LAMBDA_FUNCTION_NAME: 'my-go-lambda-function'
      AWS_REGION: 'us-east-1'
      AWS_SERVICE_CONNECTION: 'AWS-Service-Connection'  # Configure in Azure DevOps
      DEPLOY_STRATEGY: 'zip'  # or 'sam'
      GOARCH: 'amd64'  # or 'arm64'
      LAMBDA_TIMEOUT: '30'
      LAMBDA_MEMORY_SIZE: '128'

For SAM-based deployments:

stages:
  - template: 'azure-devops/golang/go-lambda-sam.yaml@pipelines'
    parameters:
      S3_BUCKET: 'my-deployment-bucket'
      AWS_REGION: 'us-east-1'
      AWS_SERVICE_CONNECTION: 'AWS-Service-Connection'
      SAM_CONFIG_ENV: 'default'  # References samconfig.toml environment

Required Variable Groups

Create these variable groups in Azure DevOps Library:

Shared Variables (All Projects):

Variable	Description
`SONAR_HOST_URL`	SonarQube server URL
`SONAR_TOKEN`	SonarQube authentication token

Project-Specific Variables (.NET Example):

Variable	Description
`SONAR_PROJECT_NAME`	SonarQube project display name
`SONAR_PROJECT_KEY`	SonarQube project unique key

AWS Lambda Deployment Variables (Optional):

Variable	Description	Required For
`AWS_ACCESS_KEY_ID`	AWS access key (if not using service connection)	Lambda deployment
`AWS_SECRET_ACCESS_KEY`	AWS secret key (if not using service connection)	Lambda deployment
`LAMBDA_ROLE_ARN`	IAM role ARN for Lambda function	Creating new functions

Note: For AWS deployments, it is recommended to use Azure DevOps AWS Service Connection instead of storing credentials in variable groups. Configure the service connection in Azure DevOps Project Settings > Service Connections.

Available Tools & Scripts

Security & Analysis Tools

SAST (Static Application Security Testing)

Tool	Purpose	Script Location	Configuration
Gitleaks	Secret detection	`global/scripts/tools/gitleaks/`	`.gitleaks.toml`
CodeQL	SAST security scanning	`global/scripts/tools/codeql/`	Auto-configured
Semgrep	Static analysis	`global/scripts/tools/semgrep/`	Auto-configured
Hadolint	Dockerfile linting	`global/scripts/tools/hadolint/`	`.hadolint.yaml`
Trivy IaC	IaC misconfiguration scanning	`global/scripts/tools/trivy/run.sh`	`.trivyignore` (global + project, merged)

SCA (Software Composition Analysis)

Tool	Purpose	Languages	Script / Integration
Trivy SCA	Dependency vulnerability scanning	All	`global/scripts/tools/trivy/run-sca.sh`
govulncheck	Go vulnerability scanning	Go	`global/scripts/languages/golang/govulncheck/`
Safety	Python dependency scanning	Python	`pdm run safety-scan`
OWASP Dependency-Check	Java dependency scanning	Java	`global/scripts/languages/java/dependency-check/`
yarn npm audit	JS/Node.js dependency scanning	JavaScript	`yarn npm audit --recursive`
npm audit	JS/Node.js dependency scanning	JavaScript	`npm audit --audit-level=high`
Composer Audit	PHP dependency scanning	PHP	`composer audit`
bundler-audit	Ruby dependency scanning	Ruby	`bundle-audit check --update`

Quality & Management

Tool	Purpose	Script Location	Configuration
Basic Checks	PR/MR rebase and changelog verification	`global/scripts/shared/rebase-check.sh`, `changelog-check.sh`	Auto-configured
SonarQube	Code quality & security	`global/scripts/tools/sonarqube/`	Project settings
Dependency Track	SBOM tracking	`global/scripts/tools/dependency-track/`	Environment variables

Basic Checks

Every pipeline includes basic checks that run in parallel with linting during the Code Check stage. These checks verify:

Rebase verification — the PR/MR branch is rebased on top of the target branch (usually main). If the branch is behind, the pipeline fails with clear instructions to rebase. This enforces a linear commit history and prevents merge conflicts from reaching the test and delivery stages.
Changelog validation — the CHANGELOG.md file was modified and new entries are placed under the [Unreleased] section. If entries appear below an existing version section (e.g., due to an erroneous rebase), the pipeline fails with instructions to fix the placement.

OWASP Dependency-Check and the NVD Database

The Java sca:dependency-check job scans dependencies against a local copy of the NVD (~350,000 CVE records). Building that copy is the only slow part of the job, and the NVD API rate limits it per source IP: 5 requests per rolling 30 seconds anonymously, 50 with an API key. Hosted CI runners share their egress IPs with every other project on the platform, so an unauthenticated first run spends most of its time in 429 backoff.

Set NVD_API_KEY. Request a free key at nvd.nist.gov/developers/request-an-api-key, then expose it to the pipeline:

Platform	How to provide it
GitHub Actions	Repository secret `NVD_API_KEY`; the wrapper workflows already forward it
GitLab CI	Masked CI/CD variable `NVD_API_KEY`
Azure DevOps	Pipeline variable (or variable group) `NVD_API_KEY`

Without a key the scan still works: it falls back to NIST's gzipped JSON data feeds, which are not rate limited, and logs a warning. A key is still recommended — it is faster and keeps findings fresher.

The database is cached at .owasp/ on every platform and reused for 24 hours before Dependency-Check refreshes it, so the usual run is an incremental update rather than a rebuild. On GitHub Actions the cache key rotates daily and falls back to the most recent snapshot; note that a cache written on a PR branch is visible only to that branch, so the snapshot every PR restores from is the one written by the default branch. The job is capped at 30 minutes on all three platforms.

Optional environment variables:

Variable	Default	Purpose
`NVD_API_KEY`	(unset)	Authenticates against the NVD API, raising the rate limit tenfold
`NVD_DATAFEED_URL`	NIST's public feeds when no key is set	Points at a self-hosted `vulnz` mirror; `{0}` expands to each year
`NVD_VALID_FOR_HOURS`	`24`	How long a cached database is reused before refreshing
`DEPENDENCY_CHECK_DATA_DIR`	`./.owasp`	Where the CVE database lives — this is the directory to cache

Language-Specific Tools

Go Tools

Tool	Purpose	Script Location
golangci-lint	Go linting suite	`global/scripts/languages/golang/golangci-lint/`
Go Test Runner	Comprehensive testing	`global/scripts/languages/golang/test/`
CycloneDX	SBOM generation	`global/scripts/languages/golang/cyclonedx/`

Terraform / Terra Tools

Tool	Purpose	Script Location
order-check	File-ordering checker/auto-fixer (see below)	`global/scripts/languages/terraform/order-check/`
tftest-gen	Smoke-test generator for single-module repos	`global/scripts/languages/terraform/tftest-gen/`
terra-test	`terraform test` runner over module test suites	`global/scripts/languages/terraform/terra-test/`
CycloneDX	SBOM generation for Terraform projects	`global/scripts/languages/terraform/cyclonedx/`

File-Ordering Standard (`order-check`)

Dense Terragrunt monorepos keep their *.hcl / *.tf files in a consistent order. The order-check job runs in the Code Check stage of the terra and terraform pipelines (all three platforms) and enforces:

environments/**/root.hcl — dependency blocks and the inputs block ordered ascending by dependency number (the environments/NN_ prefix).
stacks/*/variables.tf — a // SET ON .HCL section before a // SET ON .ENV section; dependency-derived variables ordered by dependency number inside .HCL.
**/providers.tf (stacks + modules) — required_providers and provider blocks ordered heaviest → lightest (cloud → data → orchestration → network → utility → trivial like random/null/local).
stacks/*/outputs.tf — outputs ordered to follow the declaration order of the modules/resources they reference in main*.tf.
dead inputs — every inputs = {} key (in a root.hcl or a leaf terragrunt.hcl) must be declared as a variable in the target stack. An undeclared input is dead code: Terraform silently drops the TF_VAR_ Terragrunt exports for it, so the value is passed and never read. These are reported only, never auto-removed — the finding names the exact keys so you can delete them before pushing.

# check (CI gate; writes build/reports/junit-order-check.xml)
"$SCRIPTS_DIR/global/scripts/languages/terraform/order-check/run.sh"

# auto-sort, then re-align with your formatter
"$SCRIPTS_DIR/global/scripts/languages/terraform/order-check/run.sh" --fix
terra format   # or: terraform fmt -recursive && terragrunt hcl format

The provider ranking and path exclusions can be overridden per-repo with an optional .terraform-order.json in the repo root:

{
  "provider_order": ["azurerm", "helm", "kubernetes", "random"],
  "ignore": ["modules/legacy/**"]
}

Only python3 is required (no Terraform binary). The --fix rewriter is round-trip-safe: it only reorders existing blocks and leaves any file it cannot parse cleanly untouched. Dead inputs are the one exception to --fix — they are reported but never deleted, since removing content would break that invariant.

Usage Examples

Run Security Scanning Locally (via Makefile)

make setup      # Clone/update pipelines repo
make lint       # Run golangci-lint
make test       # Run Go tests with coverage
make security   # Run all security tools (CodeQL, Gitleaks, Hadolint, Trivy, Semgrep)

Configure Go Linting Globally

# Symlink the shared golangci-lint config for IDE integration
SCRIPTS_DIR=$HOME/Development/github.com/rios0rios0/pipelines
ln -s $SCRIPTS_DIR/global/scripts/languages/golang/golangci-lint/.golangci.yml ~/.golangci.yml

Container Images

Pre-built container images optimized for CI/CD environments:

Image	Purpose	Registry
`golang.1.26-awscli`	Go 1.26 + AWS CLI	`ghcr.io/rios0rios0/pipelines`
`python.3.9-pdm-buster`	Python 3.9 + PDM	`ghcr.io/rios0rios0/pipelines`
`python.3.10-pdm-bullseye`	Python 3.10 + PDM	`ghcr.io/rios0rios0/pipelines`
`awscli.latest`	AWS CLI tools	`ghcr.io/rios0rios0/pipelines`
`tor-proxy.latest`	Network proxy with health check	`ghcr.io/rios0rios0/pipelines`

Building Custom Images

# Build and push a custom container
make build-and-push NAME=awscli TAG=latest

# Local build for testing
docker build -t my-image -f global/containers/awscli.latest/Dockerfile global/containers/awscli.latest/

Makefile Integration

The recommended way to use this repository locally is through the includable .mk files. GNU Make's -include directive imports targets from the pipelines repository, so your project Makefile only needs to declare SCRIPTS_DIR and the includes:

Before (repeated in every project):

SCRIPTS_DIR = $(HOME)/Development/github.com/rios0rios0/pipelines

.PHONY: lint
lint:
	${SCRIPTS_DIR}/global/scripts/languages/golang/golangci-lint/run.sh --fix .

.PHONY: test
test:
	${SCRIPTS_DIR}/global/scripts/languages/golang/test/run.sh .

.PHONY: sast
sast:
	${SCRIPTS_DIR}/global/scripts/tools/codeql/run.sh "go"

After (include once, get all targets):

# Pipeline targets: setup, sast, lint, test
SCRIPTS_DIR ?= $(HOME)/Development/github.com/rios0rios0/pipelines
-include $(SCRIPTS_DIR)/makefiles/common.mk
-include $(SCRIPTS_DIR)/makefiles/golang.mk

build:
	go build -o bin/app .

run:
	go run .

This gives you the following targets for free:

Target	Source	Description
`make setup`	`common.mk`	Clone or update the pipelines repository
`make sast`	`common.mk`	Run all security SAST tools
`make lint`	`<language>.mk`	Run language-specific linter
`make test`	`<language>.mk`	Run language-specific tests

Available language files:

File	Language	`lint`	`test`
`golang.mk`	Go	`golangci-lint --fix`	Go test + coverage
`python.mk`	Python (PDM)	`isort` + `black` + `flake8` + `mypy`	`pytest`
`java.mk`	Java (Gradle)	`./gradlew check`	`./gradlew test`
`javascript.mk`	JavaScript (Yarn)	`yarn lint`	`yarn test`
`dotnet.mk`	.NET/C#	`dotnet format`	`dotnet test`
`terraform.mk`	Terraform	`terraform fmt` + `validate`	`terraform plan`
`terra.mk`	Terra CLI	`terra format` + git diff check	unified `test-all` runner (`terraform test` on all modules + Terratest suite when present)

The -include prefix means Make silently skips the includes if the repository is not cloned yet. Run make setup (or curl ... | bash) to bootstrap.

See the .docs/examples/ directory for complete per-provider examples including Makefiles.

Direct Script Usage

If you prefer calling scripts directly without Makefile includes:

export SCRIPTS_DIR=$HOME/Development/github.com/rios0rios0/pipelines

# Go linting
$SCRIPTS_DIR/global/scripts/languages/golang/golangci-lint/run.sh --fix

# Go tests
$SCRIPTS_DIR/global/scripts/languages/golang/test/run.sh

# Security scans
$SCRIPTS_DIR/global/scripts/tools/gitleaks/run.sh
$SCRIPTS_DIR/global/scripts/tools/codeql/run.sh go
$SCRIPTS_DIR/global/scripts/tools/hadolint/run.sh
$SCRIPTS_DIR/global/scripts/tools/trivy/run.sh
$SCRIPTS_DIR/global/scripts/tools/semgrep/run.sh

Testing Pipeline Changes

When developing pipeline modifications, you can test against development branches:

Switch to Development Branch

export BRANCH=your-feature-branch-name

# Update all pipeline references to use your branch
find . -type f -name "*.yaml" -exec sed -i.bak -E "s|(remote: 'https://raw.githubusercontent.com/rios0rios0/pipelines/)[^/]+(/.*)|\\1$BRANCH\\2|g" {} +

Test Your Changes

# Update your project's pipeline reference
# Before:
include:
  - remote: 'https://raw.githubusercontent.com/rios0rios0/pipelines/main/gitlab/golang/go-docker.yaml'

# After:
include:
  - remote: 'https://raw.githubusercontent.com/rios0rios0/pipelines/your-feature-branch/gitlab/golang/go-docker.yaml'

Release Reconciliation

Releases are cut by the delivery-release job, which runs only on a push to main whose commit message is a bump (chore(bump) / chore/bump-) and depends on the quality gate (go / composer / maven). When a bump PR merges but that main run fails the gate, the tag and GitHub Release are never created — yet the PR already committed [X.Y.Z] to CHANGELOG.md. The changelog then runs ahead of the tags: bumped, but never released.

Two mechanisms guard against this:

Tag-push recovery. Pushing a version tag runs only the delivery step (the quality-gate jobs skip on tags), and the release stage derives the version from the tag ref — so a failed bump is recovered by re-pushing its tag. This is wired across all three platforms: GitHub Actions (github/global/stages/40-delivery/release + the go-library/composer-library/maven-library workflows; go-binary already delivered on tags via GoReleaser), GitLab CI (gitlab/global/stages/40-delivery/release.yaml, which now fires on a $CI_COMMIT_TAG), and Azure DevOps (azure-devops/global/stages/40-delivery/release.yaml, whose condition now also matches refs/tags/*). Recover a failed bump with:
```
git tag 1.2.3 <bump-commit-sha> && git push origin 1.2.3
```
Scheduled reconciliation. global/scripts/shared/reconcile-releases.sh diffs the released CHANGELOG.md versions against the git tags and resolves each gap to its bump commit. It is run org-wide on a schedule by config-automation — the same place the compliance audit and config/docs refresh already run — which enumerates every rios0rios0 repo, (re-)pushes any missing tag at its bump commit (triggering the recovery path above), and reports the result. Run it against any repo locally with:
```
global/scripts/shared/reconcile-releases.sh /path/to/repo
```
It prints one version<TAB>commit<TAB>status row per gap (empty output means the changelog and tags agree). A tag re-pushed to recover a release must be pushed with a PAT, not the default GITHUB_TOKEN, for it to re-trigger delivery; a GITHUB_TOKEN-pushed tag is still created (enough for tag-driven ecosystems such as Go modules and Packagist) but starts no workflow.

Troubleshooting

Common Issues & Solutions

Pipeline Failures

Issue: "No directories found to test it" (Go projects)

Cause: a Go module that uses none of the cmd/, pkg/, or internal/ directories -- for example one that keeps its packages at the repository root or under a differently named directory
Solution: none required -- the test runner now falls back to testing the whole module (./...) when none of those directories exist, so the run proceeds instead of aborting
Note: modules that do use cmd/, pkg/, or internal/ keep their existing, narrower test scope

Issue: "golangci-lint: command not found"

Cause: golangci-lint not installed or not in PATH
Solution: The script automatically downloads golangci-lint, ensure Docker is available

Issue: Docker build fails with SSL certificate errors

Cause: Network restrictions in CI environment
Solution: This is expected in restricted environments; contact your platform administrator

Security Tool Issues

Issue: CodeQL analysis fails

Cause: CodeQL CLI not installed or language not supported
Solution: Ensure network access to download CodeQL CLI bundle; supported languages: go, python, java, javascript, csharp, ruby (PHP is not supported)

Issue: Gitleaks takes too long or fails

Cause: Large repository or network issues
Solution: Increase timeout values, ensure network access to GitHub releases for the Gitleaks binary download

Issue: Semgrep timeout or hangs

Cause: Large codebase, downloading security rules
Solution: Allow 10+ minutes for completion, do not cancel the operation

Issue: Hadolint skips analysis

Cause: No Dockerfiles found in the project
Solution: This is expected for projects without Dockerfiles; Hadolint auto-skips gracefully

Issue: Trivy IaC scan finds false positives

Cause: Trivy flags misconfigurations in Terraform, Kubernetes, or Dockerfiles
Solution: Add entries to .trivyignore in the project root to suppress known false positives. The project file is merged with (appended to) the always-applied global ignore in global/scripts/tools/trivy/.trivyignore — put fleet-wide, well-understood false positives there so every project inherits them

Platform-Specific Issues

GitHub Actions:

Issue: Workflow does not trigger
Solution: Check repository permissions, ensure workflow file is in .github/workflows/

GitLab CI:

Issue: "Remote file could not be fetched"
Solution: Verify the remote URL is accessible, check branch name in URL

Azure DevOps:

Issue: "Template not found"
Solution: Ensure GitHub service connection is configured correctly

Environment Requirements

Minimum Requirements:

Docker (for container builds and security tools)
Git (for repository operations)
Network access (for downloading tools and dependencies)

Language-Specific Requirements:

Go: Go 1.18+ (automatically installed in CI)
Python: Python 3.8+ (automatically managed in CI)
Java: JDK 11+ (automatically managed in CI)
Node.js: Node 16+ (automatically managed in CI)

Performance Expectations

Operation	Expected Duration	Notes
Script downloads	1-5 seconds	First-time tool downloads
Go linting	10-30 seconds	Depends on codebase size
Security scanning	2-10 minutes	Depends on tools and project size
Container builds	5-30 minutes	Depends on base image and dependencies
Semgrep analysis	5-15 minutes	Downloads large rule sets

Important: Never cancel operations that appear to be hanging - they may be downloading large Docker images or rule sets.

Contributing

Contributions are welcome. See CONTRIBUTING.md for guidelines.

License

This project is licensed under the MIT License.

Note: This repository provides pipeline templates and automation scripts, not a runnable application. Users consume these templates in their own projects to establish comprehensive CI/CD pipelines with security, quality, and testing automation.

Name		Name	Last commit message	Last commit date
Latest commit History 1,178 Commits
.docs		.docs
.github		.github
azure-devops		azure-devops
github		github
gitlab		gitlab
global		global
makefiles		makefiles
.editorconfig		.editorconfig
.gitignore		.gitignore
CHANGELOG.md		CHANGELOG.md
CLAUDE.md		CLAUDE.md
CODE_OF_CONDUCT.md		CODE_OF_CONDUCT.md
CONTRIBUTING.md		CONTRIBUTING.md
LICENSE		LICENSE
Makefile		Makefile
README.md		README.md
clone.sh		clone.sh

Uh oh!

Folders and files

Latest commit

History

Repository files navigation

Pipelines

Supported Platforms & Languages

Platforms

Programming Languages

Project Structure

Pipeline Architecture

Installation

Quick Installation

Manual Installation

Platform Usage

GitHub Actions

Available Workflows

Usage Example (Go with Docker)

Usage Example (Python/PDM with Docker)

Usage Example (Java with Docker)

Usage Example (JavaScript/Yarn with Docker)

Usage Example (.NET with Docker)

Usage Example (JavaScript/npm with Docker)

Usage Example (Java/Maven with Docker)

Usage Example (PHP with Docker)

Usage Example (Ruby with Docker)

GitLab CI

Available Templates

Usage Example (Go with Docker)

Usage Example (Python PDM)

Usage Example (Terraform -- raw terraform/terragrunt)

Usage Example (Terra CLI)

Terra Test Stage

Required GitLab Variables

Azure DevOps

Available Templates

Usage Example (Go with Docker)

Usage Example (Go with ARM Deployment)

Usage Example (Go with AWS Lambda)

Required Variable Groups

Available Tools & Scripts

Security & Analysis Tools

SAST (Static Application Security Testing)

SCA (Software Composition Analysis)

Quality & Management

Basic Checks

OWASP Dependency-Check and the NVD Database

Language-Specific Tools

Go Tools

Terraform / Terra Tools

File-Ordering Standard (order-check)

Usage Examples

Run Security Scanning Locally (via Makefile)

Configure Go Linting Globally

Container Images

Building Custom Images

Makefile Integration

Direct Script Usage

Testing Pipeline Changes

Switch to Development Branch

Test Your Changes

Release Reconciliation

Troubleshooting

Common Issues & Solutions

Pipeline Failures

Security Tool Issues

Platform-Specific Issues

Environment Requirements

Performance Expectations

Contributing

License

About

Topics

Resources

Code of conduct

Contributing

Security policy

Stars

Watchers

Forks

File-Ordering Standard (`order-check`)