Skip to content

jux-tools/pytest-jux

BranchesTags

Repository files navigation

pytest-jux

License Python Version pytest codecov SLSA 2 Security

A pytest plugin for signing and publishing JUnit XML test reports to the Jux REST API

Overview

pytest-jux is a client-side pytest plugin that automatically signs JUnit XML test reports using XML digital signatures (XMLDSig) and publishes them to a Jux REST API backend for storage and analysis. It enables system administrators, integrators, and infrastructure engineers to maintain a chain-of-trust for test results across local and distributed environments.

Architecture

pytest-jux is the client component in a client-server architecture:

  • pytest-jux (this project): Client-side plugin for signing and publishing test reports
  • Jux API Server (separate project): Server-side backend for storing, querying, and visualizing reports

This separation allows pytest-jux to be lightweight and focused on test report signing, while the Jux API Server handles data persistence, deduplication, and web interfaces.

Features

Client-Side Features (pytest-jux)

  • Automatic Report Signing: Signs JUnit XML reports with XML digital signatures after test execution
  • XML Canonicalization: Uses C14N for generating canonical hashes
  • Chain-of-Trust: Cryptographic signatures ensure report integrity and provenance
  • REST API Publishing: Publishes signed reports to Jux API backend
  • Local Storage & Caching: XDG-compliant storage with offline queue for unreliable networks
  • Flexible Storage Modes: LOCAL (filesystem only), API (server only), BOTH (dual), CACHE (offline queue)
  • Configuration Management: Multi-source configuration (CLI, environment, files) with precedence
  • pytest Integration: Seamless integration via pytest hooks (post-session processing)
  • Standalone CLI Tools: Key generation, signing, verification, inspection, cache, and config utilities
  • Environment Metadata: Captures test environment context (hostname, user, platform)
  • Custom Metadata Support: Add custom metadata to reports via pytest-metadata integration
  • Security Framework: Comprehensive security with automated scanning and threat modeling

Server-Side Features (Jux API Server)

The following features are provided by the Jux API Server (separate project):

  • Report Storage: Persistent storage in SQLite (local) or PostgreSQL (distributed)
  • Duplicate Detection: Canonical hash-based deduplication prevents redundant storage
  • Signature Verification: Server-side validation of XMLDSig signatures
  • Query API: REST API for retrieving and filtering stored reports
  • Web UI: Browser-based interface for visualizing test results
  • Multi-tenancy: Support for multiple projects and users

Security

pytest-jux implements a comprehensive security framework with SLSA Build Level 2 compliance:

Supply Chain Security (SLSA L2)

SLSA 2

All pytest-jux releases include cryptographic provenance attestations:

  • ✅ Build Integrity: Packages built on GitHub Actions (not developer workstations)
  • ✅ Source Traceability: Cryptographic link to exact source code commit
  • ✅ Tamper Detection: Any modification invalidates the signature
  • ✅ Independent Verification: Verify packages with slsa-verifier

Verify a release:

# Install SLSA verifier
go install github.com/slsa-framework/slsa-verifier/v2/cli/slsa-verifier@latest

# Download and verify
pip download pytest-jux==0.4.1 --no-deps
curl -L -O https://github.com/jux-tools/pytest-jux/releases/download/v0.4.1/pytest-jux-0.4.1.intoto.jsonl

slsa-verifier verify-artifact \
  --provenance-path pytest-jux-0.4.1.intoto.jsonl \
  --source-uri github.com/jux-tools/pytest-jux \
  pytest_jux-0.4.1-py3-none-any.whl

See SLSA Verification Guide for complete instructions.

Security Framework

  • Automated Scanning: pip-audit, Ruff (security rules), Safety, Trivy
  • Threat Modeling: STRIDE methodology with 19 identified threats
  • Cryptographic Standards: NIST-compliant algorithms (RSA-SHA256, ECDSA-SHA256)
  • Supply Chain: SLSA L2, Dependabot, OpenSSF Scorecard
  • Vulnerability Reporting: Coordinated disclosure with 48-hour response time

See Security Policy for vulnerability reporting and Security Framework for complete details.

Architecture Documentation

This project uses Architecture Decision Records (ADRs) to track significant architectural decisions:

  • ADR-0001: Record architecture decisions
  • ADR-0002: Adopt development best practices
  • ADR-0003: Use Python 3 with pytest, lxml, signxml, and SQLAlchemy stack
  • ADR-0004: Adopt Apache License 2.0
  • ADR-0005: Adopt Python Ecosystem Security Framework
  • ADR-0006: Adopt SLSA Build Level 2 Compliance

See the docs/adr/ directory for complete decision history.

Requirements

  • Python 3.11+
  • pytest 7.4+ or 8.x
  • Supported OS: Debian 12/13, latest openSUSE, latest Fedora

Installation

# From PyPI (when published)
uv pip install pytest-jux

# From source (development)
cd pytest-jux
uv pip install -e ".[dev,security]"

Usage

Basic Usage

# Run tests with JUnit XML generation and auto-publish
pytest --junit-xml=report.xml \
       --jux-publish \
       --jux-api-url=https://jux.example.com/api/v1 \
       --jux-key=~/.jux/private_key.pem

Configuration via pytest.ini

[pytest]
addopts = --junit-xml=report.xml
jux_api_url = https://jux.example.com/api/v1
jux_key_path = ~/.jux/private_key.pem

Plugin Options

  • --jux-publish: Enable pytest-jux plugin (default: disabled)
  • --jux-api-url URL: Jux REST API endpoint
  • --jux-key PATH: Path to private key for signing (PEM format)
  • --jux-storage-mode MODE: Storage mode (local, api, both, cache)
  • --jux-storage-path PATH: Custom storage directory path

Adding Custom Metadata

pytest-jux integrates with pytest-metadata to add custom metadata to your test reports:

# Add metadata via command line
pytest --junit-xml=report.xml \
       --jux-sign \
       --jux-key ~/.jux/signing_key.pem \
       --metadata ci_build_id 12345 \
       --metadata environment production

The metadata appears as property tags in the JUnit XML:

<testsuite>
  <properties>
    <property name="ci_build_id" value="12345"/>
    <property name="environment" value="production"/>
  </properties>
  ...
</testsuite>

See How to Add Metadata to Reports for complete documentation including CI/CD integration, JSON metadata, and programmatic usage.

Storage & Caching

pytest-jux provides flexible storage options for test reports, supporting both local filesystem storage and API publishing. This enables offline operation, network-resilient workflows, and local inspection of test results.

Storage Modes

pytest-jux supports four storage modes:

  • LOCAL: Store reports locally only (no API publishing)
  • API: Publish to API only (no local storage)
  • BOTH: Store locally AND publish to API
  • CACHE: Store locally, publish when API available (offline queue)

Storage Location

Reports are stored in XDG-compliant directories:

  • macOS: ~/Library/Application Support/jux/reports/
  • Linux: ~/.local/share/jux/reports/
  • Windows: %LOCALAPPDATA%\jux\reports\

Custom storage paths can be specified via --jux-storage-path or JUX_STORAGE_PATH environment variable.

Storage Examples

Local storage only (no API required):

pytest --junit-xml=report.xml \
       --jux-enabled \
       --jux-sign \
       --jux-key=~/.jux/private_key.pem \
       --jux-storage-mode=local

API publishing with local backup:

pytest --junit-xml=report.xml \
       --jux-enabled \
       --jux-sign \
       --jux-key=~/.jux/private_key.pem \
       --jux-api-url=https://jux.example.com/api/v1 \
       --jux-storage-mode=both

Offline queue mode (auto-publish when API available):

pytest --junit-xml=report.xml \
       --jux-enabled \
       --jux-sign \
       --jux-key=~/.jux/private_key.pem \
       --jux-api-url=https://jux.example.com/api/v1 \
       --jux-storage-mode=cache

Cache Management

The jux-cache command provides tools for inspecting and managing cached reports.

List cached reports:

# Text output
jux-cache list

# JSON output
jux-cache list --json

Show report details:

# View specific report
jux-cache show sha256:abc123...

# JSON output
jux-cache show sha256:abc123... --json

View cache statistics:

# Text output
jux-cache stats

# JSON output
jux-cache stats --json

Clean old reports:

# Dry run (preview what would be deleted)
jux-cache clean --days 30 --dry-run

# Actually delete reports older than 30 days
jux-cache clean --days 30

Custom storage path:

jux-cache --storage-path /custom/path list

Configuration Management

pytest-jux supports multi-source configuration with precedence: CLI arguments > environment variables > configuration files > defaults.

Configuration Sources

Configuration can be loaded from:

  1. Command-line arguments (highest precedence)
  2. Environment variables (prefixed with JUX_)
  3. Configuration files:
    • ~/.jux/config (user-level)
    • .jux.conf (project-level)
    • pytest.ini (pytest configuration)
    • /etc/jux/config (system-level, Linux/Unix)

Configuration File Format

Configuration files use INI format with a [jux] section:

[jux]
# Core settings
enabled = true
sign = true
publish = false

# Storage settings
storage_mode = local
# storage_path = ~/.local/share/jux/reports

# Signing settings
# key_path = ~/.jux/signing_key.pem
# cert_path = ~/.jux/signing_key.crt

# API settings
# api_url = https://jux.example.com/api/v1
# api_key = your-api-key-here

Configuration Management Commands

The jux-config command provides tools for managing configuration.

List all configuration options:

# Text output
jux-config list

# JSON output
jux-config list --json

Dump current effective configuration:

# Text output (shows sources)
jux-config dump

# JSON output
jux-config dump --json

View configuration files:

# View specific file
jux-config view ~/.jux/config

# View all configuration files
jux-config view --all

Initialize configuration file:

# Create minimal config at default path (~/.jux/config)
jux-config init

# Create full config with all options
jux-config init --template full

# Create at custom path
jux-config init --path /custom/path/config

# Force overwrite existing file
jux-config init --force

Validate configuration:

# Basic validation
jux-config validate

# Strict validation (check dependencies)
jux-config validate --strict

# JSON output
jux-config validate --json

Environment Variables

All configuration options can be set via environment variables:

export JUX_ENABLED=true
export JUX_SIGN=true
export JUX_KEY_PATH=~/.jux/private_key.pem
export JUX_STORAGE_MODE=local
export JUX_API_URL=https://jux.example.com/api/v1
export JUX_API_KEY=your-api-key-here

Configuration Examples

Minimal configuration (local storage only):

[jux]
enabled = true
sign = false
storage_mode = local

Development configuration (local storage + API publishing):

[jux]
enabled = true
sign = true
key_path = ~/.jux/dev_key.pem
storage_mode = both
api_url = http://localhost:4000/api/v1

Production configuration (signed reports with offline queue):

[jux]
enabled = true
sign = true
key_path = ~/.jux/prod_key.pem
cert_path = ~/.jux/prod_key.crt
storage_mode = cache
api_url = https://jux.example.com/api/v1

CLI Tools

pytest-jux provides standalone CLI commands for key management, signing, verification, and administration:

Key generation:

jux-keygen --output ~/.jux/private_key.pem --algorithm rsa

Offline signing:

jux-sign report.xml --key ~/.jux/private_key.pem --output signed_report.xml

Signature verification:

jux-verify signed_report.xml

Report inspection:

jux-inspect report.xml

Cache management:

jux-cache list
jux-cache show sha256:abc123...
jux-cache stats
jux-cache clean --days 30

Configuration management:

jux-config list
jux-config dump
jux-config init
jux-config validate

Manual publishing:

# Publish single report to Jux API
jux-publish --file report.xml --api-url https://jux.example.com/api/v1

# Publish all queued reports
jux-publish --queue --api-url https://jux.example.com/api/v1

# Dry-run mode (preview without publishing)
jux-publish --queue --api-url https://jux.example.com/api/v1 --dry-run

# JSON output for scripting
jux-publish --queue --api-url https://jux.example.com/api/v1 --json

See CLI tool documentation for complete usage details.

Documentation

Complete Documentation Index: See docs/INDEX.md for a complete map of all available documentation.

This project follows the Diátaxis framework, organizing documentation into four categories:

📚 Tutorials (Learning-Oriented)

Step-by-step guides to learn pytest-jux from beginner to advanced:

🛠️ How-To Guides (Problem-Oriented)

Practical solutions to specific problems:

Key Management:

Storage & Configuration:

Integration:

Troubleshooting:

📖 Reference (Information-Oriented)

Complete technical reference documentation:

API Reference:

CLI Reference:

Configuration:

💡 Explanation (Understanding-Oriented)

Conceptual understanding and design discussions:

🔒 Security Documentation

🆘 Quick Navigation

I want to...

Development

Setup

# Clone repository
git clone https://github.com/jux-tools/pytest-jux.git
cd pytest-jux

# Create virtual environment
uv venv
source .venv/bin/activate  # or: .venv\Scripts\activate on Windows

# Install development dependencies
uv pip install -e ".[dev,security]"

# Install pre-commit hooks
pre-commit install

Development Commands

Code Quality

# Format code
make format

# Lint code
make lint

# Type checking
make type-check

# All quality checks
make quality

Testing

# Run all tests
make test

# Run tests with coverage
make test-cov

# Run security tests
make test-security

Security

# Run security scanners
make security-scan

# Run security test suite
make security-test

# Complete security validation
make security-full

Architecture Validation

# Validate C4 DSL architecture model
podman run --rm -v "$(pwd)/docs/architecture:/usr/local/structurizr" \
  structurizr/cli validate -workspace workspace.dsl

# Generate architecture diagrams
podman run --rm -p 8080:8080 \
  -v "$(pwd)/docs/architecture:/usr/local/structurizr" structurizr/lite
# Open http://localhost:8080

Project Structure

pytest-jux/
├── pytest_jux/              # Plugin source code
│   ├── __init__.py
│   ├── plugin.py           # pytest hooks
│   ├── signer.py           # XML signing
│   ├── verifier.py         # Signature verification
│   ├── canonicalizer.py    # C14N operations
│   ├── config.py           # Configuration management (Sprint 3)
│   ├── metadata.py         # Environment metadata (Sprint 3)
│   ├── storage.py          # Local storage & caching (Sprint 3)
│   ├── api_client.py       # REST API client (Sprint 4)
│   └── commands/           # CLI commands
│       ├── keygen.py       # Key generation
│       ├── sign.py         # Offline signing
│       ├── verify.py       # Signature verification
│       ├── inspect.py      # Report inspection
│       ├── cache.py        # Cache management (Sprint 3)
│       ├── config_cmd.py   # Config management (Sprint 3)
│       └── publish.py      # Manual publishing (Sprint 4)
├── tests/                   # Test suite
│   ├── test_plugin.py
│   ├── test_signer.py
│   ├── test_verifier.py
│   ├── test_canonicalizer.py
│   ├── test_config.py      # Config tests (Sprint 3)
│   ├── test_metadata.py    # Metadata tests (Sprint 3)
│   ├── test_storage.py     # Storage tests (Sprint 3)
│   ├── test_api_client.py  # API client tests (Sprint 4)
│   ├── commands/           # CLI command tests
│   │   ├── test_cache.py   # Cache command tests (Sprint 3)
│   │   ├── test_config_cmd.py  # Config command tests (Sprint 3)
│   │   └── test_publish.py # Publish command tests (Sprint 4)
│   ├── security/           # Security tests
│   └── fixtures/           # JUnit XML fixtures & test keys
├── docs/                    # Documentation
│   ├── tutorials/          # Learning-oriented
│   ├── howto/             # Problem-oriented
│   ├── reference/         # Information-oriented
│   ├── explanation/       # Understanding-oriented
│   ├── adr/              # Architecture decisions
│   ├── architecture/     # C4 DSL models
│   ├── sprints/          # Sprint documentation
│   └── security/         # Security documentation
├── .github/
│   └── workflows/
│       └── security.yml    # Security scanning workflow
├── LICENSE                 # Apache License 2.0
├── NOTICE                  # Copyright notices
├── Makefile                # Development commands
├── pyproject.toml          # Project metadata
└── README.md              # This file

Note: This project does not include database models (models.py) or database integration. These are handled by the Jux API Server.

Architecture Overview

Client-Server Architecture

┌─────────────────────────────────┐         ┌──────────────────────────┐
│   pytest-jux (Client)           │         │  Jux API Server          │
│                                 │         │                          │
│  1. Run tests                   │         │  6. Receive signed XML   │
│  2. Generate JUnit XML          │         │  7. Verify signature     │
│  3. Canonicalize (C14N)         │  HTTP   │  8. Check for duplicates │
│  4. Sign with XMLDSig           │ ─POST─> │  9. Store in database    │
│  5. Publish to API              │         │ 10. Return status        │
│                                 │         │                          │
│  • XML signing                  │         │  • Report storage        │
│  • Environment metadata         │         │  • Duplicate detection   │
│  • API client                   │         │  • Signature verification│
│                                 │         │  • Query API             │
│                                 │         │  • Web UI                │
└─────────────────────────────────┘         └──────────────────────────┘

pytest Plugin Integration

pytest-jux integrates with pytest via the pytest_sessionfinish hook, processing JUnit XML reports after test execution completes.

Client-Side Workflow

  1. Generate: pytest creates JUnit XML report (--junit-xml)
  2. Canonicalize: Convert XML to canonical form (C14N) and compute SHA-256 hash
  3. Sign: Generate XMLDSig signature using private key
  4. Capture Metadata: Collect environment information (hostname, platform, etc.)
  5. Publish: POST signed report + metadata to Jux REST API

Server-Side Processing (Jux API Server)

  1. Receive: Accept signed XML report via REST API
  2. Verify: Validate XMLDSig signature
  3. Deduplicate: Check canonical hash against existing reports
  4. Store: Save to database (SQLite or PostgreSQL)
  5. Index: Make report queryable via API and Web UI

C4 DSL Architecture Models

The project's architecture is documented using C4 DSL models in docs/architecture/. See the architecture documentation for:

  • System context: pytest-jux in the Jux ecosystem
  • Container view: Plugin components and REST API integration
  • Component view: Internal module structure

AI-Assisted Development Notes

AI Collaboration Context

  • This project follows AI-Assisted Project Orchestration patterns
  • ADRs provide AI context across development sessions
  • Sprint documentation maintains development continuity (see docs/sprints/)
  • C4 DSL models provide visual architecture context
  • TDD approach guides AI-assisted test and implementation generation

Development Practices Integration

  • AI assistance for boilerplate generation (pytest hooks, SQLAlchemy models)
  • Human review required for cryptographic code (security-critical)
  • Quality gates: all tests pass, type checking clean, code coverage >85%
  • Context management: ADRs and sprint docs enable session continuity

Contributing

Contributions welcome! Please:

  1. Read the Architecture Decision Records to understand project direction
  2. Follow the development best practices
  3. Review security guidelines for security-sensitive changes
  4. Ensure all tests pass and coverage remains >85%
  5. Run security scanners: make security-full
  6. Update documentation for new features
  7. Use conventional commits for clear changelog generation

License

Copyright 2025 Georges Martin

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

See LICENSE for the full license text.

API Specification

Source of Truth: The canonical API specifications live in jux-openapi.

  • OpenAPI specs: jux-openapi/specs/v1/
  • API changelog: jux-openapi/docs/CHANGELOG.md

pytest-jux v0.4.0+ is compliant with jux-openapi API v1.0.0.

Related Projects

Jux API Server (Separate Project)

The Jux API Server is the server-side component that works with pytest-jux. It is a separate project that provides:

Core Functionality:

  • REST API endpoints for receiving signed test reports
  • XMLDSig signature verification
  • Report storage in SQLite (local) or PostgreSQL (distributed)
  • Canonical hash-based duplicate detection
  • Query API for retrieving stored reports

User Interfaces:

  • Web UI for browsing and visualizing test results
  • CLI tools for querying reports
  • API documentation (OpenAPI/Swagger)

Technology Stack:

  • Elixir/Phoenix framework
  • PostgreSQL or SQLite database
  • RESTful API design

Deployment Options:

  • Local development (SQLite)
  • Single-server deployment (PostgreSQL)
  • Distributed/multi-tenant deployment (PostgreSQL cluster)

For more information about the Jux API Server, refer to its separate repository and documentation.

Releases

Latest Release: v0.4.1 - Documentation & C4 Model Updates (2026-01-08)

Release Notes: See docs/release-notes/ for detailed release notes for all versions.

Changelog: See CHANGELOG.md for a complete version history.

Support

About

A pytest plugin for signing and publishing JUnit XML test reports to the Jux REST API

Resources

Readme

License

Apache-2.0 license

Security policy

Security policy
Activity
Custom properties

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases 8

v0.4.1 - Documentation & C4 Model Updates Latest
Jan 8, 2026
+ 7 releases

Packages

 
 
 

Contributors

Languages