Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DM-33027: add PipelineGraph class #347

Merged
merged 16 commits into from
Aug 4, 2023
Merged

DM-33027: add PipelineGraph class #347

merged 16 commits into from
Aug 4, 2023

Commits on Aug 4, 2023

  1. Add test-utility PipelineTask with fully dynamic connections.

    @TallJimbo
    TallJimbo committed Aug 4, 2023
    Configuration menu
    Copy the full SHA
    c0486b3 View commit details
    Browse the repository at this point in the history

  2. Minor cleanups to docs and signatures in pipeTools.

    @TallJimbo
    TallJimbo committed Aug 4, 2023
    Configuration menu
    Copy the full SHA
    3f6da4c View commit details
    Browse the repository at this point in the history

  3. Add PipelineGraph package.

    @TallJimbo
    TallJimbo committed Aug 4, 2023
    Configuration menu
    Copy the full SHA
    bac56ea View commit details
    Browse the repository at this point in the history

  4. Integrate PipelineGraph with Pipeline and use it for sorting. 

    Much of the code changed here is actually stuff I want to deprecate in
the future, once PipelineGraph has been integrated with more things.
In the meantime, this addresses much the duplication caused by adding
PipelineGraph.
    @TallJimbo
    TallJimbo committed Aug 4, 2023
    Configuration menu
    Copy the full SHA
    7f0d750 View commit details
    Browse the repository at this point in the history

  5. Use PipelineGraph instead of PipelineDatasetTypes in step tester.

    @TallJimbo
    TallJimbo committed Aug 4, 2023
    Configuration menu
    Copy the full SHA
    22cbfa8 View commit details
    Browse the repository at this point in the history

  6. Guard against some optional dependencies not being present in tests.

    @TallJimbo
    TallJimbo committed Aug 4, 2023
    Configuration menu
    Copy the full SHA
    6789be5 View commit details
    Browse the repository at this point in the history

  7. Add changelog entry.

    @TallJimbo
    TallJimbo committed Aug 4, 2023
    Configuration menu
    Copy the full SHA
    6334faa View commit details
    Browse the repository at this point in the history

  8. Add edges to DatasetTypeNode. 

    This provides some symmetry with TaskNode and a bit of convenience.
    @TallJimbo
    TallJimbo committed Aug 4, 2023
    Configuration menu
    Copy the full SHA
    69bf1e6 View commit details
    Browse the repository at this point in the history

  9. Add, improve simple graph-traversal methods on PipelineGraph.

    @TallJimbo
    TallJimbo committed Aug 4, 2023
    Configuration menu
    Copy the full SHA
    02bbc62 View commit details
    Browse the repository at this point in the history

  10. Add pipeline_graph module to API reference docs. 

    We can't tell Sphinx about having lifted the PipelineGraph symbol
to lsst.pipe.base, unfortunately, as it doesn't like duplicates and
can't do aliases.
    @TallJimbo
    TallJimbo committed Aug 4, 2023
    Configuration menu
    Copy the full SHA
    8332832 View commit details
    Browse the repository at this point in the history

  11. Add option to resolve PipelineGraph when making it from a Pipeline. 

    This is what most uses will want to do anyway, unless they don't have
a data repository.
    @TallJimbo
    TallJimbo committed Aug 4, 2023
    Configuration menu
    Copy the full SHA
    53cb0d5 View commit details
    Browse the repository at this point in the history

  12. Provide user guide for PipelineGraph and reorder its API accordingly.

    @TallJimbo
    TallJimbo committed Aug 4, 2023
    Configuration menu
    Copy the full SHA
    0ffd43b View commit details
    Browse the repository at this point in the history

  13. Mark PipelineGraph serialization as experimental.

    @TallJimbo
    TallJimbo committed Aug 4, 2023
    Configuration menu
    Copy the full SHA
    497ac24 View commit details
    Browse the repository at this point in the history

  14. Make SerializedPipelineGraph compatible with pydantic v2.

    @TallJimbo
    TallJimbo committed Aug 4, 2023
    Configuration menu
    Copy the full SHA
    8f0d3fa View commit details
    Browse the repository at this point in the history

  15. Minor review-inspired cleanups for PipelineGraph.

    @TallJimbo
    TallJimbo committed Aug 4, 2023
    Configuration menu
    Copy the full SHA
    14334a4 View commit details
    Browse the repository at this point in the history

  16. Use enum to make PipelineGraph load options clearer.

    @TallJimbo
    TallJimbo committed Aug 4, 2023
    Configuration menu
    Copy the full SHA
    8091ec1 View commit details
    Browse the repository at this point in the history