Expose underlying model of campaign

[brandon-holt]

Hi, is it/would it be possible to expose the underlying model of a campaign in order to calculate predicted means and variances of a a set of new measurements (not necessarily those recommended by the campaign, but any user-specified measurement that exists within the search space)?

Ideally I'd like to be able to quantify the performance of the model on a set of known measurements as well.

Thanks!

[Scienfitz]

Hi @brandon-holt
at the moment unfortunately it is only possible by injecting your own code eg in acquisition.py to store the otherwise non-persistent model and then further analyze it

Having said that this exact functionality is already int he works and has also been requested e.g. here #78
We will enable custom callables / extractors with pre-provided examples that do exactly what you want and more

Would you mind registering your interest in this feature in the other mentioned Issue? I would then close this one then to avoid duplicates

[brandon-holt]

Got it, will do! Thanks for the reply, looking forward to this feature!

[brandon-holt]

@AdrianSosic @Scienfitz Will the update allowing for model exposure also include some kind of feature importance analysis, to give us insight into which features/feature values were most useful for the model and how they influence the predictions?

[Scienfitz]

Hi @brandon-holt

yes