Nightly Build Failure 2024-07-03

[zaneselvans]

Overview

It looks like #3687 moved pudl_etl_job_factory to conftest.py but it was actually also being used in the PUDL ETL CLI.

Next steps

Some options:

• Move pudl_etl_job_factory back to pudl/etl/cli.py or somewhere else that it can be imported to the CLI.
• Abandon the PUDL ETL, and go to using the Dagster CLI instead. (bigger change)

Verify that everything is fixed!

Tasks

Beta Give feedback

1. Try running a fast ETL locally using the CLI

Relevant logs

/home/mambauser/pudl/src/pudl/etl/cli.py:91: ExperimentalWarning: Function `build_reconstructable_job` is experimental. It may break in future versions, even between dot releases. To mute warnings for experimental functionality, invoke warnings.filterwarnings("ignore", category=dagster.ExperimentalWarning) or use one of the other methods described at https://docs.python.org/3/library/warnings.html#describing-warning-filters.
  pudl_etl_reconstructable_job = build_reconstructable_job(
Traceback (most recent call last):
  File "/home/mambauser/env/bin/pudl_etl", line 8, in <module>
    sys.exit(pudl_etl())
             ^^^^^^^^^^
  File "/home/mambauser/env/lib/python3.12/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mambauser/env/lib/python3.12/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/home/mambauser/env/lib/python3.12/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mambauser/env/lib/python3.12/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mambauser/pudl/src/pudl/etl/cli.py", line 91, in pudl_etl
    pudl_etl_reconstructable_job = build_reconstructable_job(
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mambauser/env/lib/python3.12/site-packages/dagster/_core/decorator_utils.py", line 203, in wrapped_with_pre_call_fn
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "/home/mambauser/env/lib/python3.12/site-packages/dagster/_core/definitions/reconstruct.py", line 549, in build_reconstructable_job
    job_def = job_def_from_pointer(pointer)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mambauser/env/lib/python3.12/site-packages/dagster/_core/definitions/reconstruct.py", line 671, in job_def_from_pointer
    target = def_from_pointer(pointer)
             ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mambauser/env/lib/python3.12/site-packages/dagster/_core/definitions/reconstruct.py", line 639, in def_from_pointer
    target = pointer.load_target()
             ^^^^^^^^^^^^^^^^^^^^^
  File "/home/mambauser/env/lib/python3.12/site-packages/dagster/_core/code_pointer.py", line 303, in load_target
    reconstructor = cast(Callable, self.reconstructor_pointer.load_target())
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mambauser/env/lib/python3.12/site-packages/dagster/_core/code_pointer.py", line 220, in load_target
    return _load_target_from_module(
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mambauser/env/lib/python3.12/site-packages/dagster/_core/code_pointer.py", line 197, in _load_target_from_module
    raise DagsterInvariantViolationError(f"{fn_name} not found {error_suffix}")
dagster._core.errors.DagsterInvariantViolationError: pudl_etl_job_factory not found in module pudl.etl.cli. dir: ['DagsterInstance', 'EpaCemsSettings', 'EtlSettings', 'PudlPaths', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', '__warningregistry__', 'build_reconstructable_job', 'click', 'execute_job', 'fsspec', 'get_dagster_execution_config', 'logger', 'pathlib', 'pudl', 'pudl_etl', 'sys']
Copying outputs to GCP bucket gs://builds.catalyst.coop/2024-07-03-0602-5a4f07e0d-main