Add adaptive Jacobian reuse to first-order solvers

[ChrisRackauckas-Claude]

Ignore this PR until it has been reviewed by @ChrisRackauckas.

Prompt

Add OrdinaryDiffEq-style Jacobian reuse heuristics to NonlinearSolve's first-order methods, benchmark the standalone nonlinear-solver behavior before choosing a default, and keep OrdinaryDiffEq's separately owned Jacobian/iteration-matrix lifecycle unaffected.

Summary

• add a public, documented JacobianReuse(max_age = 10, max_residual_ratio = 1) policy
• keep exact Newton behavior as the default; opt in with jacobian_reuse = true or a configured policy
• reuse a concrete factorization only while its Jacobian is unchanged
• refresh after the age limit, residual deterioration/non-finite progress, a stale linear failure, a stale line-search failure, or a stale rejected descent/trust step
• preserve a fresh Jacobian across repeated trust-region rejections at an unchanged state
• reset policy state on reinit!; explicit step!(...; recompute_jacobian = true/false) decisions take precedence
• expose the keyword on NewtonRaphson, TrustRegion, GaussNewton, LevenbergMarquardt, PseudoTransient, and the first-order polyalgorithms
• bump NonlinearSolveFirstOrder to 2.3.0 and NonlinearSolve to 4.22.0, with the root compat floor raised to NonlinearSolveFirstOrder 2.3

NewtonRaphson already constructs GeneralizedFirstOrderAlgorithm, so a second solver alias/type is unnecessary: the existing solver becomes adaptive only when the policy is supplied.

Design and prior art

This builds on the iterator and recomputation controls from #345 and the manual reuse request in #247. The closed WIP in #206 found that unconditional/threshold reuse was not consistently beneficial on small problems. The bounded age follows the established modified-Newton shape used by KINSOL; explicit lag configuration is also established in PETSc SNES.

The policy is intentionally immutable and stateless; per-solve residual/age state lives in the cache. This lets one configured policy be shared safely by polyalgorithm members.

Prerequisite stack and baseline findings

The published branch is intentionally stacked, in this order:

1. current upstream master, including merged Align source-tree downgrade dependency floors #1070 (correct source-tree downgrade floors)
2. #1057, which restores the generalized-Rosenbrock bad-Broyden trajectory
3. #1073, which removes a non-public FunctionWrappersWrappers access while preserving continuation inference
4. the unchanged Jacobian-reuse feature patch

The old CI failures were reproduced on clean master and bisected separately before being included here. The apparent Brown bad-Broyden “fix” was disproved: identical Julia/package/source/Ubuntu-x86 graphs diverge between EPYC 7502 and EPYC 9354. The invalid expectation PR #1077 was reverted to a zero diff and closed; corrected evidence is recorded in #1056. The pre-existing global broken expectation remains unchanged.

Benchmark/default decision

BenchmarkTools medians, one BLAS thread, warmed solves:

Case	Exact Newton	Reuse result
dense diagonal, n=2	0.0089 ms	0.0096 ms (age 2)
dense diagonal, n=256	6.01 ms	2.65 ms (age 5)
trust region, n=256	7.01 ms	1.99 ms (age 5)
sparse Brusselator, 2048 unknowns	36.8 ms	26.6 ms (age 5)
23-problem Newton library	2.88 ms, 22/23 success	1.76 ms (age 2), same 22/23 success

For the sparse Brusselator, age 5 reduced Jacobian evaluations 4→3 and factorizations 3→2, while nonlinear steps increased 3→7. Policy sensitivity is real: the 23-problem age-5 run regressed to 3.36 ms, and the n=2 case showed pure overhead. Matrix-free runs had identical Jacobian/factorization work counts and inconsistent timing noise, so no matrix-free benefit is claimed.

Therefore this PR keeps exact Newton and the default polyalgorithm unchanged. jacobian_reuse = true selects a conservative age-10 bounded policy; users with expensive concrete Jacobians can tune age and residual ratio.

OrdinaryDiffEq isolation

OrdinaryDiffEqNonlinearSolve owns distinct J and W refresh decisions, including mass-matrix/γdt changes and convergence history across time steps. Its explicit recompute_jacobian Boolean overrides this standalone policy, while reinit! resets standalone reuse state.

Local integration checks ran the existing Robertson FBDF/TRBDF2 Jacobian-reuse test (8/8) and repeated both methods with NewtonRaphson(jacobian_reuse = true). Both remained accurate and successful; FBDF observed 10 Jacobian calls versus 115 W builds, and TRBDF2 observed 7 versus 129.

Validation

Final stacked commit: 99348eb0d.

• Runic 1.7.0 --check --diff . across the repository: passed
• GROUP=Core Pkg.test() for NonlinearSolveFirstOrder: passed
  • adaptive Jacobian reuse: 64/64
  • General NLLS: 472/472
  • NewtonRaphson: 309/309
  • TrustRegion: 1176/1176
  • all remaining least-squares, sparse, mass-matrix, structured/operator-Jacobian, iterator, and termination groups passed
• GROUP=QA Pkg.test() for NonlinearSolveFirstOrder: 18/18
• focused reuse regression on Julia 1.10.11 after a clean Julia-1.10 resolve: 64/64
• GROUP=PolyAlgorithms Pkg.test() for NonlinearSolve: passed, including 24/24 basic and 46/46 problem-library assertions
• GROUP=QA Pkg.test() for NonlinearSolve: 17/17
• GROUP=Core Pkg.test() for NonlinearSolveBase: passed
• GROUP=QA Pkg.test() for NonlinearSolveBase: 18/18
• exact continuation inference regression after the public-cache follow-up: 26/26
• OrdinaryDiffEq integration described above: passed

A local full root Core run cannot be a green signal on this EPYC 7502 host because the pre-existing Brown bad-Broyden @test_broken unexpectedly passes here; the identical graph fails with residual 5.5 on the EPYC 9354 CI host. That master failure was investigated under #1056, and no platform/version gate or weakened expectation is included.

Documenter expanded the new @docs JacobianReuse entry and completed document checks. The strict full docs build then stopped at pre-existing external link checks: numerical.recipes timed out and GitHub returned HTTP 429 for a pinned CMINPACK source link. Link checking and failure behavior were left unchanged.

Process

1. inspected current NonlinearSolve Jacobian/factorization lifecycle and OrdinaryDiffEq's separate J/W lifecycle
2. reviewed Completeness for use in OrdinaryDiffEq.jl #247, [WIP] reuse option in Newton Raphson method #206, What is a Nonlinear Solver? And How to easily build Newer Ones #345, KINSOL, and PETSc prior art
3. implemented an opt-in residual-aware, age-bounded policy with explicit override precedence
4. added accepted-step, deterioration, manual override, reinit, line-search retry, trust rejection, constructor, and polyalgorithm tests
5. benchmarked small, dense, sparse, trust-region, matrix-free, and 23-problem cases
6. ran minimum-version, grouped Core/QA, root polyalgorithm, ODE integration, formatting, and docs checks

[ChrisRackauckas-Claude]

Implementation scratchpad for commit 6dd21578d:

• preserve exact-Newton defaults and OrdinaryDiffEq's explicit J/W refresh ownership
• add a stateless public policy plus per-cache residual/age tracking
• refresh on bounded age, lost residual progress, or stale solve/globalization failures
• preserve current Jacobians across unchanged-state trust-region rejections
• forward the opt-in policy through first-order constructors and polyalgorithms
• validate with Runic, Julia 1.10/1.12 targeted tests, FirstOrder Core/QA, root PolyAlgorithms, and FBDF/TRBDF2 integration
• retain strict docs link checking; the local build reached document checks and then hit transient failures on pre-existing external links (timeout/HTTP 429)

Benchmarking showed wins for expensive dense/sparse factorizations but overhead and policy sensitivity on small/problem-library cases, so the solver default remains exact Newton.

[ChrisRackauckas-Claude]

Published final stacked head 99348eb0d.

Safety and history checks:

• fetched upstream, fork, and origin immediately before push
• upstream remained at 6264446f4
• remote feature branch remained at the original single-author head 6dd21578d; no contributor commits intervened
• git range-diff reports 6dd21578d = 99348eb0d for the feature patch
• updated with an exact --force-with-lease against 6dd21578d
• every stacked commit retains the required Chris Rackauckas co-author trailer
• invalid Brown expectation commit is absent

Final local outputs include:

• NonlinearSolveFirstOrder Core: passed (feature 64/64, Newton 309/309, TrustRegion 1176/1176, remaining groups passed)
• NonlinearSolveFirstOrder QA: 18/18
• Julia 1.10 focused feature suite after clean resolve: 64/64
• root PolyAlgorithms: passed, including 46/46 problem-library assertions
• root QA: 17/17
• NonlinearSolveBase Core: passed
• NonlinearSolveBase QA: 18/18
• continuation inference: 26/26
• Runic and git diff --check: passed

CI is now running on the combined stack, which includes both focused prerequisite fixes before the unchanged feature commit.