pestpp-glm: stdv in external obs not being used by FOSM

[Yasin7795]

In pestpp-glm (v5.2.23), I am using external observation data that include a standard_deviation column. The 5.2.24 manual (section 4.18.2, external file support) states that when a standard_deviation column is present, those values “will be used as the observation noise standard deviation instead of the weight.” The .rec file from my runs also reports:
“observations have 'standard_deviation' defined – this will be used instead of weight for the observation noise covariance matrix.”

However, from my testing it appears that the FOSM noise covariance is still derived from calibration weights, not from the standard_deviation values.
I tested this by changing standard_deviation values for certain influential observations groups (while keeping calibration weights fixed). The results do not show any apparent change in the posterior parameter distributions calculated using FOSM.
Also, the fosm_reweight file contains values that appear consistent with a reweighting based on calibration weights and residuals (e.g., something like max(w_calib, 1/|res|)), and do not reflect values like max(1/stdv, 1/|res|) as it seems to be implied by the manual.

so despite the manual text and the .rec message, the standard_deviation column in external observation files appears not to be used in GLM’s FOSM noise covariance construction, and the process still seems to rely on calibration weights.

Is pestpp-glm currently intended to use standard_deviation from external observation files for FOSM/posterior noise, or is this functionality not yet implemented/fully wired in the GLM FOSM pipeline?

Thank you in advance for your time.

[jtwhite79]

I just had a look at this and at least for a simple test problem, glm seems to be picking up the standard_deviation info in both parameter and observation data external sections. Can you check that rec, first to make sure you see something like this near the top:

..reading external file 'pest.obs_data.csv'
...header line: OBSNME,OBSVAL,WEIGHT,OBGNME,EXTRA,STANDARD_DEVIATION

and then you should see something like this whenever the fosm calculations are being done:

Note: the following observations have 'standard_deviation' defined - this will be used
      instead of weight for the observation noise covariance matrix : 
H01_04 0.1
H01_06 0.1

[Yasin7795]

Hi Jeremy,

Thank you for looking into this. I retested and yes my rec file has both of those stetements, and now it seems like the posterior is responding to the change in stdv values. I may have been reading in the wrong file or just haven't increased my stdv values enough to see a change. However, the fosm_reweight.rei file still shows something like min(calibration_weights, 1/|res|) in the weight column. Are these weights not used for FOSM when there is a standard deviation column in the observation data? if so, would that mean the Morozov’s discrepancy scaling is not applied when standard deviation is used for the noise matrix?

[jtwhite79]

got it. I can update that rei file to report that "implied" weight by the standard deviation(s)

[Yasin7795]

Thanks Jeremy — just to be sure I understand how it works internally:
When a standard_deviation column is present, does FOSM use only those standard deviation values to define the observation noise, or does it still increase them based on the size of the residuals? (I think in the fosm_reweight.rei file this would appear as weights based on either the stdv or the residuals, whichever is larger.)

[jtwhite79]

I see what you are getting at. Currently, glm defaults to the user-supplied standard_deviation over the residual-adjusted weights. Do you think it would be appropriate to adjust the standard deviation values - is this the behavior you were expecting?

[Yasin7795]

Yes, that’s what I had in mind.
I expected that if an observation’s residual exceeded its user-supplied standard deviation, the stdv would be increased to match the residual — following the same Morozov discrepancy logic GLM currently uses when working from calibration weights.
So I thought that residual-based inflation would still be applied even when standard_deviation is provided.