Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Rework of how transformations are handled (#575)
* initial implementation of VarNameVector * added some hacky getval and getdist get things to work for VarInfo * Apply suggestions from code review Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * added arbitrary metadata field as discussed * renamed idcs to varname_to_index * renamed vns to varnames for VarNameVector * added keys impl for Metadata * added push! and update! for VarNameVector * added getindex_raw! and setindex_raw! for VarNameVector * added `iterate` and `convert` (for `AbstractDict) impls for `VarNameVector` * make the key and eltype part of the `VarNameVector` type * added more tests for VarNameVector * formatting * more testing for VarNameVector * minor changes to some comments * added a bunch more tests for VarNameVector + several bugfixes in the process * formatting * added `similar` implementation for `VarNameVector` * formatting * removed debug statement * made VarInfo slighly more generic wrt. underlying metadata * fixed incorrect behavior in `keys` for `Metadata` * minor style changes to VarNameVector tests * style * added testing of `update!` with smaller sizes and fixed bug related to this * formatting * move functionality related to `push!` for `VarNameVector` into `push!` * Update src/varnamevector.jl Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * several fixes to make sampling with VarNameVector + initiall tests for sampling with VarNameVector * VarInfo + VarNameVector tests for all demo models * Apply suggestions from code review Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * added docs on the design of `VarNameVector` * Apply suggestions from code review Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * added note on `update!` * further elaboration of the design of `VarInfo` and `VarNameVector` * more writing improvements * added docstring to `has_inactive_ranges` and `inactive_ranges_sweep!` * moved docs on `VarInfo` design to a separate internals section * writing improvements for internal docs * further motivation of the design choices made in `VarNameVector` * improved writing * VarNameVector is now grown as much as needed * updated `delete!` * Significant changes to implementation of `VarNameVector`: - "delete-by-mark" is now replaced by proper deletion. - `inactive_ranges` replaced by `num_inactive`, which only keeps track of the number of inactive entries for a given `VarName. - `VarNameVector` is now a "grow-as-needed" structure where the underlying also mimics the order that the user experiences.` * added `copy` when constructing `VectorVarInfo` from `VarInfo` * added missing `isempty` impl * remove impl of `iterate` and instead implemented `pairs` and `values` iterators * added missing `empty!` for `num_inactive` * removed redundant `shift_left!` methd * fixed `delete!` for `VarNameVector` * added `is_contiguous` as an alterantive to `!has_inactive` * updates to internal docs * renamed `sweep_inactive_ranges!` to `contiguify!` * improvements to internal docs * more improvements to internal docs * moved additional methods description in internals to earlier in the doc * moved internals docs to a separate directory and split into files * more improvements to internals doc * formatting * added tests for `delete!` and fixed reference to old method * addition to `delete!` test * added `values_as` impls for `VarNameVector` * added docs for `replace_valus` and `values_as` for `VarNameVector` * fixed doctest * formatting * temporarily disable doctests so we can build docs * added missing compat entry for ForwardDiff in docs * moved some shared code into methods to make things a bit cleaner * added impl of `merge` for `VarNameVector` * renamed a few variables in `merge` impl for `VarNameVector` * forgot to include some changes in previous commit * added impl of `subset` for `VarNameVector` * fixed `pairs` impl for `VarNameVector` * added missing impl of `subset` for `VectorVarInfo` * added missing impl of `merge_metadata` for `VarNameVector` * added a bunch of `from_vec_transform` and `tovec` impls to make `VarNameVector` work with `Cholesky`, etc. * make default args use `from_vec_transform` rather than `FromVec` * fixed `values_as` fro `VarInfo` with `VarNameVector` as `metadata` * fixed impl of `getindex_raw` when using integer index for `VarNameVector` * added tests for `getindex` with `Int` index for `VarNameVector` * fix for `setindex!` and `setindex_raw!` for `VarNameVector` * introduction of `from_vec_transform` and `tovec` and its usage in `VarInfo` * moved definition of `is_splat_symbol` to the file where it's used * added `VarInfo` constructor with vector input for `VectorVarInfo` * make `extract_priors` take the `rng` as an argument * added `replace_values` for `Metadata` * make link and invlink act on the `metadata` field for `VarInfo` + implementations of these for `Metadata` and `VarNameVector` * added temporary defs of `with_logabsdet_jacobian` and `inverse` for `transpose` and `Bijectors.vec_to_triu` * added invlink_with_logpdf overload for `ThreadSafeVarInfo` * added `is_transformed` field to `VarNameVector` * removed unnecessary defintions of `with_logabsdet_jacobian` and `inverse` for `transpose` * fixed issue where we were storing the wrong transformations in `VarNameVector` * make sure `extract_priors` doesn't mutate the `varinfo` * updated `similar` for `VarNameVector` and fixed `invlink` for `VarNameVector` * added handling of `is_transformed` in `merge` for `VarNameVector` * removed unnecesasry `deepcopy` from outer `link` * updated `push!` to also `push!` on `is_transformed` * skip tests for mutating linking when using VarNameVector * use same projection for `Cholesky` in `VarNameVector` as in `VarInfo` * fixed `settrans!!` for `VarInfo` with `VarNameVector` * fixed bug in `set_flag!` * fixed another typo * fixed return values of `settrans!!` * updated static transformation tests * Update test/simple_varinfo.jl Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * removed unnecessary impl of `extract_priors` * make `short_varinfo_name` of `TypedVarInfo` a bit more informative * moved impl of `has_varnamevector` for `ThreadSafeVarInfo` * added back `extract_priors` impl as we do need it * forgot to include tests for `VarNameVector` in `runtests.jl` * fix for `relax_container_types` in `test/varnamevector.jl` * fixed `need_transforms_relaxation` * updated some tests to not refer directly to `FromVec` * introduce `from_internal_transform` and its siblings * remove `with_logabsdet_jacobian_and_reconstruct` in favour of `with_logabsdet_jacobian` with `from_linked_internal_transform`, etc. * added `internal_to_linked_internal_transform` + fixed a few bugs in the linking as a resultt * added `linked_internal_to_internal_transform` as a complement to `interanl_to_linked_interanl_transform` * fixed bugs in `invlink` for `VarInfo` using `linked_internal_to_internal_transform` * more work on removing calls to `reconstruct` * removed redundant comment * added `from_linked_vec_transform` specialization for `LKJ` * more work on removing references to `reconstruct` * added `copy` in `values_from_metadata` to preserve behavior and avoid refs to internal representation * remove `reconstruct_and_link` and `invlink_and_reconstruct` * replaced references to `link_and_reconstruct` and `invlink_and_reconstruct` * introduced `recombine` and replaced calls to `reconstruct` with `n` samples * completely removed `reconstruct` * renamed `maybe_reconstruct_and_link` to `to_maybe_linked_internal` and `maybe_invlink_and_reconstruct` to `from_maybe_linked_internal` * added impls of `from_*_internal_transform` for `ThreadSafeVarInfo` * removed `reconstruct` from docs and from exports * renamed `getval` to `getindex_internal` and made `dist` an optional argument for all the transform-related methods * updated docs + added description of how internals of transforms work * added a bunch of illustrations for the transforms docs + dot files used to generated * temporarily removed `VarNameVector` completely * formatting * Update docs/src/internals/transformations.md Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Update docs/src/internals/transformations.md Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * removed refs to VectorVarInfo * added impls of `from_internal_transform` for `ThreadSafeVarInfo` * reverted accidental removal of old `VarInfo` constructor * fixed incorrect `recombine` call * removed undefined refs to `VarNameVector` stuff in `setup_varinfos` * bump minior version because Turing breaks * fix: was using `from_linked_internal_transform` in `from_internal_transform` for `ThreadSafeVarInfo` * removed `getindex_raw` * removed redundant docstrings * fixed tests * fixed comparisons in tests * try relative references for images in transformation docs * another attempt at fixing asset-references * fixed getindex diagrams in docs * minor changes to comments * remove Combinatorics as a test dep, as it's not needed for this PR * reverted unnecessary change * disable type-stability tests for models on older Julia versions * removed seemingly completely unused impl of `setval!` * Update test/model.jl Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Markus Hauru <[email protected]> * Type-stability tests are now correctly using `rand_prior_true` instead of `rand` * `getindex_internal` now calls `getindex` instead of `view`, as the latter can result in type-instability since transformed variables typically result in non-view even if input is a view * Removed seemingly unnecessary definition of `getindex_internal` * Fixed references to `newmetadata` which has been replaced by `replace_values` * Made implementation of `recombine` more explicit * Added docstrings for `untyped_varinfo` and `typed_varinfo` * Added TODO comment about implementing `view` for `VarInfo` * Fixed potential infinite recursion as suggested by @mhauru * added docstring to `from_vec_trnasform_for_size * Replaced references to `vectorize(dist, x)` with `tovec(x)` * Fixed docstring * Update src/extract_priors.jl Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Bump minor version since this is a breaking change * Apply suggestions from code review Co-authored-by: Markus Hauru <[email protected]> * Update src/varinfo.jl Co-authored-by: Tor Erlend Fjelde <[email protected]> * Apply suggestions from code review * Apply suggestions from code review * Update src/extract_priors.jl Co-authored-by: Xianda Sun <[email protected]> * Added fix for product distributions of targets with changing support + tests * Addeed tests for product of distributions with dynamic support * Apply suggestions from code review Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Empty commit to trigger CI * Update test/model.jl Co-authored-by: Markus Hauru <[email protected]> * Increase HTML page size threshold for docs --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Markus Hauru <[email protected]> Co-authored-by: Xianda Sun <[email protected]> Co-authored-by: Markus Hauru <[email protected]>
- Loading branch information
5 people
authored
Aug 21, 2024
1 parent
cdd3407
commit 138bd40
