v0.9.9

• Access NEI source files from DataCommons (2011-2018)
• Automatically generate inventories if they do not exist when calling getInventory, getInventoryFlows, or getInventoryFacilities
• Refactored filtering of processed inventory files
• Moved some RCRAInfo source data processing steps to filter (e.g. limiting to National Biennial Reporting data); stored inventory files maintain more data
• Improvements to support processing of DMR and RCRAInfo source data
• Improvements to dependency management, command line operations, and error reporting

v0.9.8

Processed stewi files are no longer maintained in the repository and must be generated locally

NEI data are temporarily unavailable in v0.9.8

In preparation of v1.0, this release provides substantial updates:

• Use of esupy for local data storage.
• Removal of all inventory files within the repository
• Source files downloaded to local user directory
• Inventory files generated and stored to local user directory
• Ability to store generated and named stewicombo inventories
• Metadata json files generated with all files
  -FlowbySCC updated to Flowbyprocess

v0.9.7

Add processing of GHGRP with initial release of 2013 - 2018 data years. Includes inventories for flow by facility by subpart, currently categorized as a FlowBySCC.

v0.9.6

-Updates for processing DMR (2014-2017) in stewi and stewicombo
-Minor update to stewicombo to update flowname for PM10/PM2.5 overlap to improve mapping (NEI)

v0.9.5

• Updates for processing NEI 2017 and eGRID 2018.
• Includes new flowbySCC format for NEI data
• Updates to field names for more consistency across ecosystem of tools
• Refactoring of NEI and eGRID scripts, and updates to config management
• Minor revisions to stewicombo to enable access from other packages
• Performance improvements to overlap handler

v0.9.4

Fixed issue causing PM overlap to be ignored

v0.9.3

Fixes issue of config.yaml files in chemicalmatcher and facilitymatcher not being installed with pip install

v0.9.2

standardizedinventories-master folder

1. The setup.py and requirements were modified to include BeautifulSoup, argparse, selenium, regex, PyYAML, and webdriver-manager in the installation requirements.

stewi folder

1. TRI.py was modified
2. The following packages were included:
   a. argparse: to write the inputs directly from Windows CMD
   b. beautifulsoup: to retrieve information directly from non-dynamic html
   c. regex: to handle regular expressions
   d. io: to deal with various type of input/output
3. The files TRI_File_1a_columns.txt and TRI_File_3a_columns.txt were added in stewi/data folder. These include all the names of the columns for TRI Data Plus File 1a and File 3a. In 2018, the TRI program administrators split File 1 in two new ones called File 1a and File 1b. File 1a is the file StEWI needs. I used TRI Fila 3a because it has information about the off-site transfer. I could note that StEWI did not include information about the basis of estimate for off-site land treatment and other off-site land disposals.
4. TRI_required_fields.txt was modified to add new information
5. TRI_keys.txt was added in stewi/data folder to include new TRI keys easier.
6. Files called TRI_chem_release_Year.csv were added to include new validation source
7. A new file config.yaml was added. This file is useful to handle changes in a web site localization and/or the code source of a website is modified.
8. To use TRI.py you can navigate to standardizedinventories-master (or stewi) in Windows CMD and write:

At this moment only, there is information about RCRA, TRI, FRS, and SRS.
python “stewi/”TRI.py Option Year -F File1a File2 … FileN

Where the options are A, B, C: A is for extracting files from the TRI Data Plus web site. B for organizing TRI National Totals files from TRI_chem_release_Year.csv (this is expected to be download before and to be held as it is described in TRI.py). C for organizing TRI as required by StEWI

For instance, if you want to use File 1a and File 3a of TRI Data Plus (as it is our case), retrieve information for TRI 2018. Therefore, you write in Windows CMD:

python “stewi/”TRI.py A 2018 -F 1a 3a

After, if you want to create TRI_2018_NationalTotals.csv for validation:

python “stewi/”TRI.py B 2018

The flag -F and the files are not needed, but you need to have TRI_chem_release_2018.csv in the data folder.

Finally, if you want to organize this for StEWI:

python “stewi/”TRI.py C 2018 -F 1a 3a

Note: as you know, TRI may include new columns. Therefore, you only need to add them to TRI_File_1a_columns.txt and TRI_File_3a_columns.txt (or other files you want).

8. RCRAInfo.py was modified
9. Selenium package was used to handle dynamic html due to requests, and urllib3 does not control this.
10. To use RCRAInfo.py, you can navigate to standardizedinventories-master (or stewi) in Windows CMD and write:

python “stewi/”RCRAInfo.py Option Year -T Table1 Table2 … TableN

Where the options are E, O, C: E is for extracting files from the RCRAInfo web site. O for organizing Biennial Report for each year due to the current flat file has information of all year mixed. C for creating the files StEWI needs.

For instance, if you want to retrieve table BR_REPORTING. Therefore, you write in Windows CMD:

python “stewi/” RCRAInfo.py E -T BR_REPORTING

After, if you organize for each year the table BR_REPORTING (This not take data for existing RCRAInfo report)

python “stewi/” RCRAInfo.py O -T BR_REPORTING

Finally, you want to organize Biennial Report 2017 for StEWI

python “stewi/” RCRAInfo.py C 2017

The Flag -T and the files are not needed, but you need to have RCRAInfo_2017_NationalTotals.csv in the data folder.

11. The RCRAInfo National Totals were obtained from https://rcrapublic.epa.gov/rcrainfoweb/action/modules/br/trends/view
12. RCRA_FlatFile_LineComponents_2019.csv was added due to changes in the specification of flat files https://rcrainfo.epa.gov/rcrainfo-help/application/publicHelp/index.htm
13. ValidationSets_Sources.csv file was modified to include the source for TRI National Totals and RCRAInfo National Total. For TRI for the year between 2001 and 2017 and RCRAInfo from 2001 to 2009 and 2017.

chemicalmatcher folder

1. Some modifications in globals.py, programsynonymlookupbyCAS.py, and writeStEWIchemicalmatchesbyinventory.py were made.

facilitymatcher folder

1. Some alterations in globals.py, WriteFacilityMatchesforStEWI.py, and WriteFRSNAICSforStEWI.py were made.

Beta release

-Cleared review in EPA ORD
-Adds requirements.txt file
-Fixes issues in setup.py to install data with pip installation
-PEP8 compliance update for all 4 init files.
-Cleaned up readme to make for error-free pip install
-Add VOC handling to stewicombo
-Other general fixes. See commits to master since Sept 14, 2018.

StEWI v0.9beta

Reviewed version of 0.9a...minor addition of two fields to eGRID facility output.