[ENH] Bayesian Linear Regression using Normal Conjugate Prior

[meraldoantonio]

What does this implement/fix? Explain your changes.

Implementing a new Bayesian linear regression estimator module using normal conjugate prior
with known noise variance.

Does your contribution introduce a new dependency? If yes, which one?

No

What should a reviewer concentrate their feedback on?

Correctness of implementation

Did you add any tests for the change?

No

Any other comments?

No

PR checklist

For all contributions

• [] I've added myself to the list of contributors with any new badges I've earned :-)
  How to: add yourself to the all-contributors file in the skpro root directory (not the CONTRIBUTORS.md). Common badges: code - fixing a bug, or adding code logic. doc - writing or improving documentation or docstrings. bug - reporting or diagnosing a bug (get this plus code if you also fixed the bug in the PR).maintenance - CI, test framework, release.
  See here for full badge reference
• The PR title starts with either [ENH], [MNT], [DOC], or [BUG]. [BUG] - bugfix, [MNT] - CI, test framework, [ENH] - adding or improving code, [DOC] - writing or improving documentation or docstrings.

For new estimators

• I've added the estimator to the API reference - in docs/source/api_reference/taskname.rst, follow the pattern.
• I've added one or more illustrative usage examples to the docstring, in a pydocstyle compliant Examples section.
  (https://www.sktime.net/en/latest/developer_guide/dependencies.html#adding-a-soft-dependency).

[fkiraly]

Great, only some smaller comments:

• the BayesianLinearRegressor is not in the folder linked to in the API ref
• non-blocking suggestions about improving the docstring

Non-blocking, to think about:

The notebook could be about Bayesian linear regression, not just about the conjugate approach. It would be informative to separately showcase the Bayes rule, and different ways to estimate/approximate the actual posterior. Conjucate and MCMC are two of a longer list (other strategies are variational and approximate).

[fkiraly]

Excellent work! My comments are mainly on documentation, PR structure, nothing content-wise.

• the notebooks should be reconciled with PR [DOC] Adding a Notebook companion to BayesianMCMCLinearRegressor, Restructuring Estimator Folder #480, otherwise I see some inconsistencies that could cause merge headaches. I would suggest to split off the notebooks in a separate PR, with dependency graph either 500, 480 -> new PR, or 500 -> 480 -> new PR (left side of arrow gets merged earlier).
• in the docstring, I think it is essential to say what exactly the model assumptions are, a little bit of math would be great there, and how these map onto python parameters.
• the changes in [DOC] Adding a Notebook companion to BayesianMCMCLinearRegressor, Restructuring Estimator Folder #480 are inconsistent with the directory structure here - I would recommend to make all submodues of bayesian private, as long as we are not certain about the exact end state desired.

[meraldoantonio]

Hi @fkiraly,

Thanks for the review! I've made the following updates:

1. Removed the notebooks and notebook artefacts from this PR. They will be included in a separate PR. I propose the following merge order:

   • Merge this PR first.
   • Followed by the MCMC PR ([DOC] Adding a Notebook companion to BayesianMCMCLinearRegressor, Restructuring Estimator Folder #480). This PR will simply restructure the MCMC imports. I'll also remove the outdated example notebook from this PR.
   • Finally, merge the notebook PR, which will contain the two example notebooks and the artefacts.

2. Made the submodule private.

3. Updated the docstring to include mathematical details and model assumptions.

Let me know if this works!

[fkiraly]

Works for me, all perfect.

Maybe call the new API section not "Bayesian" but "Bayesian regressors" or similar? "Bayesian" is just an adjective (and/or the name of a sunken yacht).

[meraldoantonio]

Hi @fkiraly, I've modified the API name in regression.rst and I think this PR is ready to be merged. If all OK, could you help with the merge? Thanks :)