Splitting a large function into smaller ones causes >8GB memory usage #411
Description
Problem
When a large function (623 CFG blocks, over the 500 block limit) is split into smaller named helper functions that are each under the limit, NilAway's memory usage jumps from ~428MB to over 8GB, eventually getting OOM-killed.
Before the refactoring, NilAway skipped the large function with an INTERNAL ERROR ("function too large") and the package analyzed successfully at 428MB in 8 seconds. After the refactoring, the extracted functions are within NilAway's stated limits, but analyzing them causes unbounded memory growth.
Reproduction
Repository: https://github.com/fleetdm/fleet
Before (works): At commit 35f1b2d96c42, ListHostSoftware has 623 CFG blocks (over the 500 block limit), so NilAway skips it. The package completes at ~428MB.
git checkout 35f1b2d96c42
# Using golangci-lint with NilAway plugin (see .custom-gcl.yml):
./custom-gcl run -c .golangci-incremental.yml --tests=false --timeout 5m --concurrency=1 ./server/datastore/mysql/
# Completes in ~8s, ~428MB peak RSS
# Log shows: INTERNAL ERROR: skipping function ListHostSoftware(): function too large (623 CFG blocks, exceeds limit of 500 blocks)
After (OOM): At commit b03eefb6ddff on the victor/nilaway-memory-repro branch, ListHostSoftware has been split into three named functions, each under 500 CFG blocks:
gatherHostSoftwareState(~50 branches, ~266 lines)hydrateHostSoftwareResults(~64 branches, ~288 lines)ListHostSoftware(remaining, ~177 branches)
git checkout b03eefb6ddff
./custom-gcl run -c .golangci-incremental.yml --tests=false --timeout 5m --concurrency=1 ./server/datastore/mysql/
# Memory grows past 8GB, gets OOM-killed
The only change between the two commits is this diff in server/datastore/mysql/software.go.
Environment
- Go 1.26.1
- NilAway v0.0.0-20260126174828-99d94caaf043
- golangci-lint v2.11.3
- macOS darwin/arm64, 128GB RAM
Notes
GOGC=25,GOMEMLIMIT=4GiB, and--concurrency=1do not help.- The extracted functions compile and pass all existing tests.
- Other packages in the same repo with similar dependency counts (1023-1055 transitive deps) analyze fine at 300-400MB. The issue is specific to this package after the refactoring.
- We split the function specifically so NilAway could analyze it (since it was being skipped due to the CFG block limit). The irony is that making the code analyzable causes NilAway to run out of memory.
Related issues:
- Optimize memory usage for large projects #127 - Memory optimization for large projects (general discussion of NilAway's memory usage on large codebases)
nilawayconsume all memory of system eventually on ~11 MB project #359 - Report of NilAway consuming all system memory on an ~11MB project- objectpath performance #385 -
objectpath.Encoder.For()accounts for 52% of allocations in NilAway; upstream fix pending - Adopt go/analysis/checker to fix memory explosion on large codebases #410 - Our request to adopt
go/analysis/checkerfor per-package fact caching (separate from this bug, but would reduce baseline memory)