NYISO fails on a KeyError when calling get_generation

[Rdbaker]

Python version: 3.6.8
pyiso version: 0.4.0

When trying to get the generation for the NYISO BA, I got hit with a KeyError. It looks like pyiso is expecting a column name that no longer exists on the data

In [1]: import pyiso
In [2]: nyiso = pyiso.client_factory('NYISO')
In [3]: nyiso.get_generation(latest=True)
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
~/.virtualenvs/energy-scrapers/lib/python3.6/site-packages/pandas/core/indexes/base.py in get_loc(self, key, method, tolerance)
   2896             try:
-> 2897                 return self._engine.get_loc(key)
   2898             except KeyError:

pandas/_libs/index.pyx in pandas._libs.index.IndexEngine.get_loc()

pandas/_libs/index.pyx in pandas._libs.index.IndexEngine.get_loc()

pandas/_libs/hashtable_class_helper.pxi in pandas._libs.hashtable.PyObjectHashTable.get_item()

pandas/_libs/hashtable_class_helper.pxi in pandas._libs.hashtable.PyObjectHashTable.get_item()

KeyError: 'Gen MWh'

During handling of the above exception, another exception occurred:

KeyError                                  Traceback (most recent call last)
<ipython-input-3-a5d8fee47018> in <module>
----> 1 nyiso.get_generation(latest=True)

~/.virtualenvs/energy-scrapers/lib/python3.6/site-packages/pyiso/nyiso.py in get_generation(self, latest, start_at, end_at, **kwargs)
     99
    100         # get data
--> 101         df = self.get_any('rtfuelmix', self.parse_genmix)
    102         extras = {
    103             'ba_name': self.NAME,

~/.virtualenvs/energy-scrapers/lib/python3.6/site-packages/pyiso/nyiso.py in get_any(self, label, parser, dates_list)
    122
    123                 try:
--> 124                     pieces.append(parser(csv))
    125                 except AttributeError:
    126                     pass

~/.virtualenvs/energy-scrapers/lib/python3.6/site-packages/pyiso/nyiso.py in parse_genmix(self, content)
    269
    270         # assemble final
--> 271         final_df = pd.DataFrame({'gen_MW': df['Gen MWh'], 'fuel_name': df['fuel_name']})
    272
    273         # return

~/.virtualenvs/energy-scrapers/lib/python3.6/site-packages/pandas/core/frame.py in __getitem__(self, key)
   2978             if self.columns.nlevels > 1:
   2979                 return self._getitem_multilevel(key)
-> 2980             indexer = self.columns.get_loc(key)
   2981             if is_integer(indexer):
   2982                 indexer = [indexer]

~/.virtualenvs/energy-scrapers/lib/python3.6/site-packages/pandas/core/indexes/base.py in get_loc(self, key, method, tolerance)
   2897                 return self._engine.get_loc(key)
   2898             except KeyError:
-> 2899                 return self._engine.get_loc(self._maybe_cast_indexer(key))
   2900         indexer = self.get_indexer([key], method=method, tolerance=tolerance)
   2901         if indexer.ndim > 1 or indexer.size > 1:

pandas/_libs/index.pyx in pandas._libs.index.IndexEngine.get_loc()

pandas/_libs/index.pyx in pandas._libs.index.IndexEngine.get_loc()

pandas/_libs/hashtable_class_helper.pxi in pandas._libs.hashtable.PyObjectHashTable.get_item()

pandas/_libs/hashtable_class_helper.pxi in pandas._libs.hashtable.PyObjectHashTable.get_item()

KeyError: 'Gen MWh'

[mike-welch]

The dataset was changed at some point in 2017 to use Gen MW instead of Gen MWh.

see: http://mis.nyiso.com/public/P-63list.htm