Refresh rplog: Go 1.25.12, dep bump, hardening, tests, benchmarks + leaner logs - #2
Open
daveseddon-runpod wants to merge 3 commits into
Open
Refresh rplog: Go 1.25.12, dep bump, hardening, tests, benchmarks + leaner logs#2daveseddon-runpod wants to merge 3 commits into
daveseddon-runpod wants to merge 3 commits into
Conversation
Toolchain & deps: - go.mod 1.21.6 -> 1.25.12; enve v1.0.2 -> v1.2.2; go mod tidy - CI: Go 1.25.12, add go vet, run tests with -race, add benchmark step Security hardening: - trace.FromHeaderOrNew validates untrusted X-Trace-ID/X-Request-ID as UUIDs (regenerate on garbage/oversized) and sanitizes X-*-Source (strip control/CRLF, cap length) -> closes log-injection / unbounded input - Handler.Handle clamps negative elapsed times to 0 (clock skew / hostile ctx) Tests: - New trace/trace_test.go + expanded log_test.go: table-driven positive/negative/boundary/corner/security cases, race tests - trace coverage 98.1% Benchmarks & optimization: - Benchmarks for trace + log hot paths - FromHeaderOrNew skips time.Parse on absent X-Trace-Start: ~7-8% faster, ~30% fewer allocs on missing/invalid paths - Analysis recorded in BENCHMARKS.md Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- log.go: extract metadataFromBuildInfo() helper, removing the goto FILLED label; behavior is unchanged. Correct the Init godoc, which falsely claimed the package self-initializes with os.Stderr. - GO_README.md: rewrite to describe the real API (Init + standard log/slog, Metadata, the trace middlewares). Removes references to non-existent Log()/DebugContext/InfoContext/... functions, fixes the ../README.md link and the broken code blocks. - README.md: fix Go links (Go lives at the repo root, not ./go; buildmeta docs are ./cmd/README.md). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Every log record previously carried vcs_name, vcs_commit, vcs_tag, vcs_time plus a full source file/line block (AddSource:true). On a representative line that is ~475 bytes; ~195 of them (~40%) are redundant on every record. Changes to rplog.Init: - Per-line VCS metadata reduced to vcs_commit only. vcs_name (~always "git"), vcs_tag, and vcs_time (derivable from the commit) are now emitted once in a new structured "rplog initialized" startup record and can be joined back via vcs_commit. (~65 bytes/line) - AddSource now defaults to false: a source block on every record is a large, mostly-redundant cost. (~130 bytes/line) - Replace the stray fmt.Println debug print with the structured startup record. Metadata.Fields() is unchanged (returns the full set), so Datadog-style tag exporters keep every field. Rationale (see GO_README.md "Downstream usage"): neither runpod/host nor runpod/ai-api emits the full vcs_* set per line today -- host logs a single `ver`, ai-api a single `version`. This aligns the library default with real consumers. The savings are opt-in for anyone building their own slog.Handler. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
A refresh of the Go implementation: modernize the toolchain and dependencies, replace the near-absent test suite with rigorous table-driven + security + race tests, harden the untrusted-input paths, add benchmarks, and act on the benchmark analysis. Also trims redundant per-line metadata for a ~40% smaller log line.
Best reviewed commit-by-commit.
What changed
1. Toolchain & dependencies
go.mod: go 1.21.6 → 1.25.12;envev1.0.2 → v1.2.2;go mod tidy(uuid was already latest).go vet, tests now run with-race, plus a benchmark step. Test bodies modernized to Go 1.25 idioms (b.Loop(), range-over-int,WaitGroup.Go,SplitSeq).2. Security hardening (
trace,log)trace.FromHeaderOrNewingests untrusted HTTP headers. It now:X-Trace-ID/X-Request-IDas UUIDs (regenerating on garbage/oversized input), andX-Trace-Source/X-Request-Source(strips control/CRLF chars, caps length) — closing a log-injection and unbounded-input hole.Handler.Handleclamps negative elapsed times to 0 (clock skew / hostileCtxWith). The security tests were confirmed to fail against the pre-hardening code and pass after — i.e. they catch real issues (CRLF echoed verbatim, hostile IDs passed through, 10 KB input accepted).Valid-UUID traffic (what
hostgenerates itself) is unaffected — the changes are backward compatible for the normal path.3. Tests
trace/trace_test.go+ expandedlog_test.go: table-driven positive / negative / boundary / corner / security cases, plus concurrency tests under-race.4. Benchmarks & optimization
BENCHMARKS.md).FromHeaderOrNewno longer callstime.Parseon an absentX-Trace-Start(the common first-hop case), which was allocating a*ParseErrorevery request: ~7–8% faster, ~30% fewer allocations on the missing/invalid paths.5.
Initrefactor + doc fixesgoto FILLEDpattern inInit, extractingmetadataFromBuildInfo()(behavior unchanged).GO_README.mdto describe the real API (Init+ standardlog/slog) — it previously documentedLog()/DebugContext/… functions that don't exist — and fixed broken links in both READMEs.Every record previously carried
vcs_name,vcs_commit,vcs_tag,vcs_timeand a fullsourcefile/line block. On a representative line that's ~475 bytes, ~195 of them (~40%) redundant on every record:sourceblock (AddSource:false)vcs_name(always"git"),vcs_tag,vcs_timeInitnow stampsvcs_commitonly per line (it uniquely identifies the build); the other VCS fields are emitted once in a new structured"rplog initialized"startup record and join back viavcs_commit.This is optional to adopt — anyone building their own
slog.Handlerkeeps full control.Metadata.Fields()is unchanged (returns the complete set), so Datadog-style tag exporters keep every field.Why we recommend it (downstream analysis): neither
runpod/hostnorrunpod/ai-apiactually emits the fullvcs_*set per line today —hostlogs a singlever,ai-apia singleversion, and neither callsrplog.Init(host builds its own handler; ai-api doesn't use rplog). Logging that it's"git"on literally every line — along with a redundant timestamp and source block — is pure overhead. This aligns the library default with what real consumers already do. Full write-up inGO_README.md→ "Downstream usage".Verification
go build ./...,go vet ./...clean.go test -race -cover ./...green (trace 98.1%).buildmetasanity-checked end to end.