Merge branch 'develop' into geothermal_loop_baseline

.github/pull_request_template.md

@@ -9,7 +9,6 @@ Not all may apply:
 - [ ] Tests (and test files) have been updated
 - [ ] Documentation has been updated
   - [ ] If related to resstock-estimation, checklist includes [data dictionary](https://github.com/NREL/resstock/tree/develop/resources/data/dictionary), [source report](https://github.com/NREL/resstock/tree/develop/project_national/resources/source_report.csv), [options saturation](https://github.com/NREL/resstock/tree/develop/project_national/resources/options_saturations.csv), [options_lookup](https://github.com/NREL/resstock/blob/develop/resources/options_lookup.tsv).
-  - [ ] If changes to project_testing tsvs, checklist includes [yml_precomputed](https://github.com/NREL/resstock/tree/develop/test/tests_yml_files/yml_precomputed), [yml_precomputed_outdated](https://github.com/NREL/resstock/tree/develop/test/tests_yml_files/yml_precomputed_outdated), [yml_precomputed_weight](https://github.com/NREL/resstock/tree/develop/test/tests_yml_files/yml_precomputed_weight)
 - [ ] Changelog has been updated
 - [ ] `openstudio tasks.rb update_measures` has been run
 - [ ] No unexpected regression test changes on CI (checked comparison artifacts)

.github/workflows/config.yml

@@ -46,7 +46,7 @@ jobs:
         with:
           path: resources
           name: options_lookup
-
+      
       - name: Run integrity checks for all projects
         run: |
           bundle exec rake unit_tests:project_integrity_checks
@@ -73,7 +73,7 @@ jobs:
 
   analysis-tests:
     runs-on: ubuntu-latest
-    needs: [format-files]
+    needs: [unit-tests]
     container:
       image: docker://nrel/openstudio:3.6.1
     steps:
@@ -84,17 +84,39 @@ jobs:
       - name: Install gems
         run: |
           rm -f Gemfile.lock && bundle install
+      
+      - name: Install python
+        shell: bash
+        run: |
+          export DEBIAN_FRONTEND=noninteractive
+          sudo apt update
+          sudo -E apt-get install -y python3-pip
+
+      - name: Install python dependencies
+        run : |
+          python3 -m pip install --progress-bar off --upgrade pip
+          pip3 install --progress-bar off pandas pyyaml
 
       - name: Download formatted options_lookup
         uses: actions/download-artifact@v3
         with:
           path: resources
           name: options_lookup
 
+      - name: Generate precomputed buildstocks
+        run: |
+          python3 test/update_yml_precomputed_files.py
+
       - name: Run run_analysis.rb
         run: |
           bundle exec rake workflow:analysis_tests
 
+      - name: Upload precomputed buildstocks
+        uses: actions/upload-artifact@v3
+        with:
+          path: test/tests_yml_files/yml_precomputed*/buildstock*.csv
+          name: precomputed_buildstocks
+
       - name: Upload run_analysis.rb results
         uses: actions/upload-artifact@v3
         with:
@@ -107,7 +129,7 @@ jobs:
 
   integration-tests:
     runs-on: ubuntu-latest
-    needs: [format-files]
+    needs: [unit-tests]
     steps:
       - uses: actions/checkout@v3
         with:
@@ -153,7 +175,6 @@ jobs:
       - name: Process results
         run: |
           pip install plotly
-          
           python test/process_bsb_analysis.py
 
       - name: Run tests
@@ -322,6 +343,25 @@ jobs:
           path: resources
           name: options_lookup
 
+      - name: Download precomputed buildstocks
+        uses: actions/download-artifact@v3
+        with:
+          path: test/tests_yml_files
+          name: precomputed_buildstocks
+
+      - name: Build documentation
+        run: |
+          ruby docs/read_the_docs/source/workflow_inputs/create_characteristics_rst.rb
+          ruby docs/read_the_docs/source/workflow_outputs/csv_tables.rb
+          cd docs/read_the_docs
+          make html SPHINXOPTS="-W --keep-going -n"
+
+      - name: Save Docs
+        uses: actions/upload-artifact@v3
+        with:
+          name: documentation
+          path: docs/read_the_docs/_build/html/
+
       - name: Commit latest results
         shell: bash        
         run: |
@@ -335,6 +375,7 @@ jobs:
           git add test/base_results
           git add docs
           git add resources/options_lookup.tsv
+          git add test/tests_yml_files
           git status
           if [[ $(git diff --cached --exit-code) ]]; then
             git config --global user.email "[email protected]"
@@ -343,16 +384,3 @@ jobs:
             echo "Pushing to branch: $branch_name"
             git push -u origin $branch_name
           fi
-
-      - name: Build documentation
-        run: |
-          ruby docs/read_the_docs/source/workflow_inputs/create_characteristics_rst.rb
-          ruby docs/read_the_docs/source/workflow_outputs/csv_tables.rb
-          cd docs/read_the_docs
-          make html SPHINXOPTS="-W --keep-going -n"
-
-      - name: Save Docs
-        uses: actions/upload-artifact@v3
-        with:
-          name: documentation
-          path: docs/read_the_docs/_build/html/

.readthedocs.yml

@@ -3,7 +3,7 @@ version: 2
 sphinx:
   configuration: docs/read_the_docs/source/conf.py
 
-build:
-  os: ubuntu-22.04
-  tools:
-    python: "3.8"
+python:
+  version: "3.8"
+  install:
+    - requirements: docs/read_the_docs/requirements.txt

CHANGELOG.md

@@ -6,6 +6,8 @@ Features
 - Include HVAC secondary heating capabilities for project_testing ([#1090](https://github.com/NREL/resstock/pull/1090))
 - For heat pump upgrades, adds the ability to set the existing primary (non-shared) heating system as the backup system using only a single option from the lookup ([#1074](https://github.com/NREL/resstock/pull/1074))
 - options_saturations.csv is added to project_*/resources/ folder ([#1132](https://github.com/NREL/resstock/pull/1132))
+- Update `run_analysis.rb` to map datapoints to run folder names when the `-k` argument is supplied ([#1138](https://github.com/NREL/resstock/pull/1138))
+- Add Water Heater Location and Geometry Space Combination, update Geometry Garage and Geometry Floor Area Bin to RECS2020, update RECS2020 microdata from v2 to v4, auto-generate buildstocks for yml_precomputed tests ([#1125](https://github.com/NREL/resstock/pull/1125))
 
 Fixes
 - Set standard format for options_lookup ([#962](https://github.com/NREL/resstock/pull/962))
@@ -15,6 +17,8 @@ Fixes
 - Fix/clarify duct location assignment by defining Duct Location.tsv, making Duct Leakage and Insulation (formerly Duct) depend on Duct Location, and making HVAC Has Ducts depend on HVAC Has Shared Systems. Includes fixes on standalone and shared heating system assignment for Other Fuel. ([#1104](https://github.com/NREL/resstock/pull/1104), [#1112](https://github.com/NREL/resstock/pull/1112))
 - Correct refrigerator rated annual kWh based on EF and an assumed volume of 20.9cft. ([#1118](https://github.com/NREL/resstock/pull/1118))
 - Minor changes to heating and cooling setpoint TSV due to refactoring of prune_rules handling in resstock-estimation ([#1132](https://github.com/NREL/resstock/pull/1132))
+- Minor changes to heating and cooling setpoint TSV after a bug fix ([#1136](https://github.com/NREL/resstock/pull/1136))
+- Update `run_analysis.rb` to handle illegal path characters in upgrade names ([#1138](https://github.com/NREL/resstock/pull/1138))
 
 ## ResStock v3.1.0
 ###### May 25, 2023 - [Diff](https://github.com/NREL/resstock/compare/v3.0.0...v3.1.0)

docs/read_the_docs/requirements.txt

@@ -0,0 +1 @@
+sphinx-rtd-theme

docs/read_the_docs/source/basic_tutorial/run_project.rst

@@ -54,7 +54,7 @@ Run ``openstudio workflow/run_analysis.rb -h`` to see all available commands/arg
       -n, --threads N                  Number of parallel simulations (defaults to processor count)
       -m, --measures_only              Only run the OpenStudio and EnergyPlus measures
       -i, --building_id ID             Only run this building ID; can be called multiple times     
-      -k, --keep_run_folders           Preserve run folder for all datapoints
+      -k, --keep_run_folders           Preserve run folder for all datapoints; also populates run folder in cli_output.log and results-xxx.csv files
       -s, --samplingonly               Run the sampling only
       -d, --debug                      Preserve lib folder and "existing" xml/osw files
       -o, --overwrite                  Overwrite existing project directory