On Levenberg-Marquardt Algorithm when the evaluation of the Jacobian is costly

[jbensabat]

Hello
I am running pestpp-glm with the wrf_hydro model with a substantial number of parameters. The evaluation of the jacobian is costly and requires between (n+1) to 2n simulations (n is the number of parameters). My question is whether anyone has tried alternative approaches such as low rank updates of the jacobian between iterations (instead of fully calculating the jacobian at each iteration) with a strategy for recalculating the jacobian every few iterations. I would like to know is anyone has tried that before I get deeper into the code and integrate the alternative algorithm.
best
jac

[cnicol-gwlogic]

Have you tried SVD-Assist? Otherwise, there are a few different approaches available in the PEST-HP suite.

[doherty1234]

Ensemble space inversion (ENSI) may help here. You can set it up using ENSIPREP, then solve the inverse problem (calibration or stochastic history matching) using whatever package is best for you.

[jbensabat]

I tried the definition of super-parameters but the decision on the number of super-parameters is to my level of understanding arbitrary.

regarding ensemble space inversion, I must admit I am not acquainted with the approach and will look into it.

But these two approaches are somewhat compatible with the suggested potential improvement of the iterative process.
best
jac

[jtwhite79]

I've been thinking about adding the quasi-newton rank-1 update (possibly using all the tested upgrades, which would be more than rank-1 haha) to pestpp-glm, so if you want to work on this, that would be sweet! Lemme know if you wanna collab on this...

Also if you aren't committed to derivative methods, ensemble methods can help cut down the number of model runs. pestpp-ies is a drop-in replacement for pestpp-glm and it solves the same GLM equation, but in ensemble form. It seems to work pretty well in a lot of settings (including with wrf-hydro https://agupubs.onlinelibrary.wiley.com/doi/full/10.1029/2024WR038048). But you lose some measure of causality in the parameter-observation relation with ensemble methods compared to derivative methods (there is no free lunch!)

[jbensabat]

hi
Thanks for your comments.
I will first assess the amount of work that needs to be done in order to integrate the low-rank version of LM. I personally prefer the traditional way of calibration (comparing measured and simulated values). But sometimes certain loss function can lead to a faster convergence (when the hypersurface is relatively flat).
best
jac