Best practice using IES with highly refined pilot point fields

[jesshe]

Originally and probably naively, I thought ies would be great for models using many pilot points over traditional PEST/GLM. However, I'm starting to realize that you can't just throw tons of adjustable pilot points at ies and expect it to take full advantage of the higher degree of parameterization/refinement.

For some context, I've been recalibrating a model where poor fits at certain targets are caused by lack of resolution in the original pilot point fields - after increasing pilot point refinement I was able to improve fits by making manual adjustments to a few local pilot points around poorly fit targets. However, putting the refined model through ies hasn't resulted in an improvement so far and I suspect there are things I could be doing a lot better with how my calibration is being set up:

• I've now got tens of thousands of pilot points which are initially randomly sampled and I'm using an ensemble size of 300. Given this, issues like spurious/coincidental correlation are probably huge. I'd guess that its basically impossible for the calibration to specifically target and just a few local pilot points as I did manually. I was pretty heavy handed with the pilot point refinement and could cut the parameter count significantly - perhaps this would be useful?
• As mentioned, initial pilot point values are sampled randomly. As I understand, pilot points which happen to be correlated from sampling won't really be adjusted independently and will behave almost like a "zone". Wondering if there's a better way to come up with initial parameter values and make the local pilot points that need adjusting "distinct".
• I've tried using manual localization but haven't had much success.

Wondering if anyone has thoughts w.r.t. the above, or has had a similar experience using ies with models parameterized by big pilot point fields?

[cnicol-gwlogic]

Others can no doubt be of more help, but here are my thoughts:

• I often find I need to get quite granular with my objective function. I.e., define local-ish obs groups where you've got prediction- / confidence-critical observations.
• Are there any other obs groups that could be limiting your (presumably) head obs group history match? Eg temporal trends. Is one obs type more important than the other (to model end use / forecast uncertainty)? Reweight as necessary.
• Are you using spatial covariances? If not, maybe that's messing with things. If so, have those been defined correctly / flexibly enough to achieve your local param refinement?

I'd try / think through the above suggestions first, then if they don't help, you could also try the following:

• Try auto localisation. I always use this, despite it being slow for some cases.
• Try increase that ensemble size (eg to 600), and see if that helps.

Chris.

[cnicol-gwlogic]

One other thing I just thought of: initial par values (around which the initial random sampling is centred) - how are they defined? Literature/field values? Previous history-matched values? If this is only a problem in a couple of areas, you could always define the local initial values as your manually tweaked values, which you know give you a good starting point.

One other thing to check: are param bounds set sufficiently wide?

[jtwhite79]

I think @cnicol-gwlogic is right on it as far as how to fix the problem. Just some thoughts on why this might be happening:

• too few realizations to occupy the solution space. If you have heaps of high quality and diverse data, then your solution space might be close to 300 dimensions (I havent seen a solution space this large in practice very often tho). In this case, adding more realizations will help by increasing the degrees of freedom.
• the objective function is not tuned to focus on these observations that are important to you. If there are observations that are important for you to match, then have you given then sufficiently high weight so that they are a substantial portion of the objective function? Is the mean phi still decreasing across iterations? If the phi is still decreasing across iterations, the reweighting will probably help...The latest versions of ies reports the obs group info so that you can see what portion of the objective function each group occupies across the realizations
• The prior parameter ensemble is too uninformed so that pilot points near each other can have wildly different realized values, which might obscure that par-obs gradient. I think what you are talking about with "zones" is spatial correlation in the pilot points so that pilot points near each other will have similar values. And as @cnicol-gwlogic says, if you have a good feel for what values are needed in the area of interest, then by all means set those values as the prior mean values so that the prior ensemble samples around those values
• The prior parameter ensemble is too restrictive. This might show up as lots of parameter values going to their bounds across iterations. You can see this in the parameter change summary

In theory localization should help increase the degrees of freedom but Ive not seen that in practice (yet). And the spurious correlation thing is really only a problem with estimating posterior uncertainty - the large correlations (needed to match data) will usually feature strongly in the first several eigen components of the approximate jacobian so you can usually fit data pretty well with a small ensemble (unless you have heaps of high quality and diverse data...). I pretty regularly use 100K's of parameters with ensembles of size 100-300 and can find a decent fit to the important obs most of the time (as long as I define the objective function appropriately)

And a word of caution on the reweighting - if you are using noise realizations, be careful not to set weights so low that highly improbable/implausible values of observation+noise are realized (the converse can also be a problem: the weights are so high, the implied noise is under represented). You can prevent this in several ways (like providing a "standard_deviation" column in the observation data of version 2 control file, supplying your own obs+noise ensemble, etc).

[jesshe]

@cnicol-gwlogic @jtwhite79 Many thanks to both of you for your suggestions! I definitely agree that the objective weighting should be revisited - the observation dataset is rich and the critical periods/locations are probably under emphasized. It seems like the prior ensemble can really make or break the calibration. I was sampling the prior ensemble just based on values from a previous calibration with no defined spatial covariances, realizing that wasn't too smart either.