Updating or creating a new workflow_generator

Introduction

This is a guide for developer wishing to update or create a new workflow_generator. The workflow_generators are classes that have create_osw function that generate openstudio osw file based on user input in the project yaml file.

Example use from here

In the example above, residential_hpxml version 2024.07.19is being used. The workflow_generator will be passed the whole yaml file as a dictionary.

The directory structure for the workflow_generator looks like this:

buildstockbatch/
└── workflow_generator/
    ├── residential/
    │   ├── latest/
    │   │   ├── __init__.py  # __version__="2024.07.20"
    │   │   ├── other_workflow_generator_files.py
    │   │   └── tests/
    │   │       ├── test_workflow_generator.py
    │   │       └── testing_data/
    │   │           └── stock_snapshot_data
    │   └── v2024_07_19/
    │       ├── __init__.py  # __version__="2024.07.19"
    │       ├── other_workflow_generator_files.py
    │       └── tests/
    │           ├── test_workflow_generator.py
    │           └── testing_data/
    │               └── stock_snapshot_data
    └── commercial/
    │   └── latest/
    │       ├── __init__.py  # __version__="2024.07.18"
    │       ├── other_workflow_files.py
    │       └── tests/
    │           ├── test_workflow_generator.py
    │           └── testing_data/
    │               └── stock_snapshot_data
    └── __init__.py  # main_init that imports all versions

The latest version is always placed in the folder named "latest" and the other version files are placed inside folder named after the version. Make sure to use underscore instead of a period for the folder names - otherwise it will cause problem during import.

Creating a new version vs updating existing version

You must create a new version of the workflow generator if any of the following conditions apply:

• There is an update in the underlying stock modelling tool (ResStock or ComStock) that would make old osw files incompatible. In addition, even if old osw files are still compatible, unless the new osw file would look exactly the same for the existing input, a new workflow generator version should be created.

• There is a significant refactoring of the workflow generator code beyond a simple fix / feature.

You can update the latest workflow generator without creating a new version if all of the following conditions apply:

• The change is small enough that someone can quickly review the changes and have high degree of confidence on the changes doing the intended behavior
• It will continue to generate exact same osw files for the existing input. This means, you shouldn't have to rewrite or touch the existing tests to make them pass.

Process for creating a new version of workflow_generator

Here are the step by step process for creating a new version of the residential workflow_generator. We are going use the example file structure above. Say, you plan to release the next version by 2024.08.23

• Checkout a new branch.
• Copy the latest version to a new folder - say latest - copy.
• Rename the latest - copy folder name to v2024_07_20 based on the version info stored at latest - copy\__init__.py
• Go to test_workflow_generator.py inside v2024_07_20\tests and replace all latest module in the import statements with v2024_07_20. We could have use relative import to avoid having to do this step, but pytest doesn't work well with relative import. So, we are stuck with this.
• Edit the main_init file at workflow_generator\__init__.py to import the v2024_07_20 workflow_generator following the pattern of other versions. This step might be unnecessary in the future when refactor this to import all versions automatically.
• We won't be touching this version - this is now frozen for life - except for trivial bug fixes.
• Edit the version in latest\__init__.py to 2024.08.23. This is the latest version now.
• Edit the version_info in latest\__init__.py to describe what's new in this version.
• Replace the testing_data inside latest\tests with most up to date snapshot of the stock modelling tool that this generator is supposed to work with. For ResStock, you would download the xml files from the branch of ResStock this new generator is compatible with.
• Edit other files inside latest as required to support the new behavior of the generator. You don't have to worry about backward compatibility because older stock models will be using older version of the workflow generator.
• Commit your changes to latest and also commit the new 2024_07_20 folder.
• Push your branch, create a PR and pray that all the tests pass!

After this process is completed, the new directory structure will look like:

buildstockbatch/
└── workflow_generator/
    ├── residential/
    │   ├── latest/
    │   │   ├── __init__.py  # __version__="2024.08.23" # updated version
    │   │   ├── other_workflow_generator_files.py  # updated code
    │   │   └── tests/
    │   │       ├── test_workflow_generator.py  # updated tests
    │   │       └── testing_data/
    │   │           └── stock_snapshot_data  # Updated with most up-to-date snapshot
    │   ├── v2024_07_19/
    │   │   ├── __init__.py  # __version__="2024.07.19"
    │   │   ├── other_workflow_generator_files.py
    │   │   └── tests/
    │   │       ├── test_workflow_generator.py
    │   │       └── testing_data/
    │   │           └── stock_snapshot_data
    │   └── v2024_07_20/
    │       ├── __init__.py  # __version__="2024.07.20"
    │       ├── other_workflow_generator_files.py
    │       └── tests/
    │           ├── test_workflow_generator.py  # Updated import statements
    │           └── testing_data/
    │               └── stock_snapshot_data
    └── commercial/
    │   ├── latest/
    │   │   ├── __init__.py  # __version__="2024.07.18"
    │   │   ├── other_workflow_files.py
    │   │   └── tests/
    │   │       ├── test_workflow_generator.py
    │   │       └── testing_data/
    │   │           └── stock_snapshot_data
    └── __init__.py  # main_init that imports all versions, including v2024_07_20