CAISO - Changing frequency from 5m to 1hr?

[vlepore]

Hi - I'm having trouble changing the freq, from 5 min to 1 hr for CAISO feed. Any suggestions?

Thanks!

[r24mille]

Thanks for the question. Which method are you using (e.g. get_load(...), get_generation(...), etc)? Are you using a historical time range, forecast time range, or both?

I wrote up a quick check and got the same issue. I have to admit, I'm a bit confused as well.

c = client_factory('CAISO')
start_at = datetime(year=2017, month=10, day=5, hour=0)
end_at = datetime(year=2017, month=10, day=5, hour=12)
results = c.get_load(start_at=start_at, end_at=end_at, freq=c.FREQUENCY_CHOICES.hourly)

I hooked a debugger into the CAISO client and can see the options are handled as options['freq']='hourly' and options['market']='RT5M' which is what's reflected in the results. I also tried:

results = c.get_load(start_at=start_at, end_at=end_at, 
                     freq=c.FREQUENCY_CHOICES.hourly, market=c.MARKET_CHOICES.hourly)

However, that yielded no results. @xinyue-luna or @ajdonnison, do either of you know why the options are being handled in this way? Is it a bug? Are we misunderstanding the meaning of market/frequency.

@vlepore I did notice that requests for forecasts could have hourly frequency. The issue, for me, was with historical data.

[vlepore]

@r24mille thank you for the help, I'm looking at get_lmp(....)

start = "08/2017"
end = "09/2017"
date1 = datetime.strptime(start, "%m/%Y")
date2 = datetime.strptime(end, "%m/%Y")
caiso = client_factory("CAISO", timeout_seconds=360)
data = caiso.get_lmp(start_at=date1, end_at=date2, node_id=node)

This will pull the historical data within that month at a 5m interval, and I've tried the work around you've mentioned freq=c.FREQUENCY_CHOICES.hourly with no results. Appreciate the help!

[xinyue-luna]

@r24mille thanks for checking it. For CAISO load data, from the code, it seems that it was designed to retrieve either Day-Ahead (DAM) Hourly Load Forecast, or Real-time 5-min (RT5M) Load Forecast (CAISO doesn't provide Hourly-Ahead Hourly Load Forecast). freq actually is ignored in get_load(), and only market is used to decide if to return dam or fivemin data. Unless user specify market=c.MARKET_CHOICES.dam, market will be set to fivemin.
Reading your code:

results = c.get_load(start_at=start_at, end_at=end_at, 
                     freq=c.FREQUENCY_CHOICES.hourly, market=c.MARKET_CHOICES.hourly)

I think it should return RT5M load forecast, the same result as from this line of code:
results = c.get_load(start_at=start_at, end_at=end_at, freq=c.FREQUENCY_CHOICES.hourly)
Need further investigation into why the former returned nothing while the latter returned results.

[r24mille]

Thanks for the response, @xinyue-luna.

freq actually is ignored in get_load(), and only market is used to decide if to return dam or fivemin data. Unless user specify market=c.MARKET_CHOICES.dam, market will be set to fivemin.

So, is another way of stating this, "Only fivemin market (and frequency) are supported for historical load data?" Because the day-ahead market doesn't make sense for historical data, right?

When I request the forecast data with:

c = client_factory('CAISO')
now = datetime.utcnow().replace(tzinfo=pytz.utc)
start_at = now + timedelta(hours=1)
end_at = now + timedelta(hours=10)
results = c.get_load(start_at=start_at, end_at=end_at, market=c.MARKET_CHOICES.dam)

The day-ahead market freq is 1hr. So really, it doesn't seem like there's an option at all. Historical is always 5min and forecast is always 1hr. Am I understanding this correctly?

If so:

• Why even allow end-users to pass in market or freq arguments? There's only one valid market for historical data and one valid market for forecast data.
• Does the same hold true for get_lmp(...)? As @vlepore clarified, the original issue arose when requesting historical get_lmp(...) data.
• Can we work on a list of supported markets / frequencies for CAISO for its implementation of each BaseClient method? We should log warning messages if end-users pass unsupported combinations.

[xinyue-luna]

@r24mille Good questions.
Regarding CAISO load, there are just two markets actually provide data: day-ahead hourly market and real-time fivmin market.
Regarding LMP, there are different electricity markets at different frequencies. See more here. PYISO is not fetching data for all the markets at all frequencies.