Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove Turing integration tests #733

Merged
merged 15 commits into from
Dec 13, 2024
Merged

Remove Turing integration tests #733

merged 15 commits into from
Dec 13, 2024

Conversation

penelopeysm
Copy link
Member

@penelopeysm penelopeysm commented Nov 29, 2024

Closes #703

General breaking changes

Breaking changes in TestUtils.DEMO_MODELS

  • What was originally demo_assume_observe_literal has been renamed to demo_assume_multivariate_observe_literal to avoid name clash with new model
  • demo_assume_literal_dot_observe renamed to demo_assume_dot_observe_literal to match the naming pattern of the other models
  • Added a new demo_assume_observe_literal, which is a univariate assume
@model function demo_assume_observe_literal()
    # univariate `assume` and literal `observe`
    s ~ InverseGamma(2, 3)
    m ~ Normal(0, sqrt(s))
    1.5 ~ Normal(m, sqrt(s))
    2.0 ~ Normal(m, sqrt(s))

    return (; s=s, m=m, x=[1.5, 2.0], logp=getlogp(__varinfo__))
end

This is a slightly annoying change, because anybody who uses demo_assume_observe_literal will have the behaviour changed silently. But the only code that uses it is DynamicPPL and Turing, so I think it's fine, as long as I fix it upstream too. (See GitHub-wide search for demo_assume_observe_literal)

Changes to test-internal utils

  • Added a function make_chain_from_prior([rng,] model, n_iters) which constructs an MCMCChains.Chains object by sampling from the prior of modelforn_iters` iterations.

Note this is in test/test_util.jl rather than src/test_utils/foo.jl, hence only accessible inside the test suite.

Changes to test suite

  • Removed the old Turing integration tests (see Turing integration tests should be moved to (or included in) Turing repo #703).
  • Most of them have been incorporated back into the DynamicPPL test suite itself. The functionality that required Turing.jl was either sampling (which has been replaced with the make_chain_from_prior function above), or some sampling algorithm with some space parameter (which has been replaced with a dummy algorithm MyAlg in test/varinfo.jl).
  • The remaining tests that haven't been added back are being tracked here: Update for proposed DynamicPPL 0.32 Turing.jl#2419 However, I think this can be part of a different PR as it would require more extensive modifications to our DemoModels module.

Feedback needed:

  • make_chain_from_prior seems too useful to languish in test/test_util.jl. Should this be in DynamicPPLMCMCChainsExt?

@coveralls
Copy link

coveralls commented Nov 29, 2024

Pull Request Test Coverage Report for Build 12302036826

Details

  • 22 of 22 (100.0%) changed or added relevant lines in 1 file are covered.
  • 22 unchanged lines in 5 files lost coverage.
  • Overall coverage decreased (-0.5%) to 86.025%

Files with Coverage Reduction New Missed Lines %
ext/DynamicPPLForwardDiffExt.jl 1 66.67%
src/logdensityfunction.jl 1 79.41%
src/threadsafe.jl 2 58.47%
src/varinfo.jl 8 84.72%
src/model_utils.jl 10 19.64%
Totals Coverage Status
Change from base Build 12295360310: -0.5%
Covered Lines: 3675
Relevant Lines: 4272

💛 - Coveralls

@penelopeysm penelopeysm force-pushed the py/remove-extra-tests branch from 556cd56 to 696bb3d Compare November 29, 2024 14:17
@TuringLang TuringLang deleted a comment from github-actions bot Nov 29, 2024
@TuringLang TuringLang deleted a comment from github-actions bot Nov 29, 2024
@TuringLang TuringLang deleted a comment from github-actions bot Nov 29, 2024
@TuringLang TuringLang deleted a comment from github-actions bot Nov 29, 2024
@TuringLang TuringLang deleted a comment from github-actions bot Nov 29, 2024
@TuringLang TuringLang deleted a comment from github-actions bot Nov 29, 2024
@penelopeysm penelopeysm force-pushed the py/remove-extra-tests branch from e06591b to 1055a10 Compare November 29, 2024 14:47
test/model_utils.jl Outdated Show resolved Hide resolved
@penelopeysm penelopeysm force-pushed the py/remove-extra-tests branch 5 times, most recently from 21889d7 to 0d7b2d1 Compare December 4, 2024 15:13
@TuringLang TuringLang deleted a comment from github-actions bot Dec 4, 2024
@TuringLang TuringLang deleted a comment from github-actions bot Dec 4, 2024
@TuringLang TuringLang deleted a comment from github-actions bot Dec 4, 2024
@TuringLang TuringLang deleted a comment from github-actions bot Dec 4, 2024
@penelopeysm penelopeysm force-pushed the py/remove-extra-tests branch from 0d7b2d1 to 675b40f Compare December 4, 2024 15:21
@penelopeysm penelopeysm marked this pull request as ready for review December 4, 2024 15:22
@penelopeysm penelopeysm force-pushed the py/remove-extra-tests branch 2 times, most recently from d023c04 to 543a97b Compare December 5, 2024 00:38
@TuringLang TuringLang deleted a comment from github-actions bot Dec 5, 2024
@TuringLang TuringLang deleted a comment from github-actions bot Dec 5, 2024
@TuringLang TuringLang deleted a comment from github-actions bot Dec 5, 2024
@TuringLang TuringLang deleted a comment from github-actions bot Dec 5, 2024
@penelopeysm penelopeysm force-pushed the py/remove-extra-tests branch from 7bfd233 to 7a93e4b Compare December 8, 2024 20:14
@penelopeysm penelopeysm force-pushed the py/remove-extra-tests branch from 7a93e4b to eafcf3c Compare December 8, 2024 20:32
@penelopeysm penelopeysm requested a review from mhauru December 10, 2024 12:21
docs/src/api.md Show resolved Hide resolved
src/varinfo.jl Show resolved Hide resolved
test/model.jl Outdated Show resolved Hide resolved
test/model_utils.jl Show resolved Hide resolved
test/turing/varinfo.jl Show resolved Hide resolved
test/model.jl Show resolved Hide resolved
@mhauru
Copy link
Member

mhauru commented Dec 12, 2024

What do you think is the right order to merge this and TuringLang/Turing.jl#2419?

@penelopeysm
Copy link
Member Author

What do you think is the right order to merge this and TuringLang/Turing.jl#2419?

This one first, as it's not just tests being modified but also source code so the Turing PR will have to handle the breaking changes here.

Copy link
Member

@mhauru mhauru left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one first, as it's not just tests being modified but also source code so the Turing PR will have to handle the breaking changes here.

Sounds good. Given that this creates a moment when the tests are in neither package, we should make sure to readd them in Turing.jl soon.

Great stuff, thanks @penelopeysm!

@penelopeysm penelopeysm disabled auto-merge December 13, 2024 02:47
@penelopeysm penelopeysm merged commit 5c8fc59 into master Dec 13, 2024
11 of 14 checks passed
@penelopeysm penelopeysm deleted the py/remove-extra-tests branch December 13, 2024 02:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Turing integration tests should be moved to (or included in) Turing repo
3 participants