Error running Scheduler_intro notebook

[susmitastro]

Hello,

I am trying to run scheduler_intro.ipynb to generate a list of observations and then schedule the simulated observations.

The generation of the list of observations is done in the following way:

class MySurvey(BaseSurvey):
   def generate_observations_rough(self, conditions):
       obs = empty_observation()
       obs['RA'] = np.radians(273.0)
       obs['dec'] = np.radians(-78.0)
       #obs['RA'] = 0
       #obs['dec'] = np.radians(-87)
       obs['exptime'] = 30.  # Seconds
       obs['nexp'] = 2
       obs['filter'] = 'r'
       # If it's been 12 hours or more, don't try to execute the observation anymore
       obs['flush_by_mjd'] = conditions.mjd + 1
       obs['note'] = 'mysurvey'  # Always good to set the note to which survey generated the observation
       # Make it a list of n
       result = [obs] * 7
       return result
       #print(empty_observation)

The simulated observations are scheduled using the following:

# Now to build our scheduler object, and run it for a day
scheduler = CoreScheduler([my_survey], nside=nside)
# sim_runner will return the updated model observatory, the updated scheduler, and an array of
# completed observations
mo, scheduler, observations = sim_runner(mo, scheduler,survey_length=1,
                                                verbose=True)

When the notebook is ran for the default RA and DEC given in the tutorial, it seems to be working fine. However, if the RA and DEC are changed to a different coordinates (described below) it seems to run into problems.

There are two different issues when the RA and DEC are changed:

The notebook runs perfectly fine when the RA is 228 deg or more and DEC is < -77, but seems to break once the RA and DEC are outside these limits.
Secondly, at times for some of the values of the RA and DEC that are within the above mentioned limits seems to not work for the notebook. For example:

• Run 1: RA 360 DEC -77 works fine
• Run 2: RA 350 DEC -77 works fine
• Run 3: RA 360 DEC -77 doesnt work

The block above produces the following error:

RuntimeError                              Traceback (most recent call last)
[scheduler.ipynb](about:blank) Cell 10 line 7
      2 scheduler = CoreScheduler([my_survey], nside=nside)
      3 # sim_runner will return the updated model observatory, the updated scheduler, and an array of
      4 # completed observations
----> 7 mo, scheduler, observations = sim_runner(mo, scheduler,survey_length=1,
      8                                                  verbose=True)

File[ ~/.conda/envs/rubin-sim/lib/python3.11/site-packages/rubin_sim/scheduler/sim_runner.py:105](about:blank), in sim_runner(observatory, scheduler, filter_scheduler, mjd_start, survey_length, filename, delete_past, n_visit_limit, step_none, verbose, extra_info, event_table, record_rewards)
    102 if observatory.mjd == mjd_last_flush:
    103     #sush
    104     print(mjd_last_flush)
-->[ 105](about:blank)     raise RuntimeError("Scheduler has failed to provide a valid observation multiple times.")
    106 # if this is a first offence, might just be that targets set. Flush queue and get some new targets.
    107 scheduler.flush_queue()

RuntimeError: Scheduler has failed to provide a valid observation multiple times

I am running these in rubin_sim version 1.3.3 and with opsim baseline v3.3_10yrs
I was wondering if someone will be able to help me with this issue.

[rhiannonlynne]

Hi - my apologies for the delay in replying to your issue. I've answered this in the community post (https://community.lsst.org/t/scheduler-not-working-for-given-ra-dec-and-mjd-start/8015/6) but basically .. for a bit more insight, check this out:

class MySurvey(BaseSurvey):
   def generate_observations_rough(self, conditions):
       obs = empty_observation()
       obs['RA'] = np.radians(273.0)
       obs['dec'] = np.radians(-78.0)
       obs['exptime'] = 30.  # Seconds
       obs['nexp'] = 2
       obs['filter'] = 'r'
       # If it's been 12 hours or more, don't try to execute the observation anymore
       obs['flush_by_mjd'] = conditions.mjd + 1
       obs['note'] = 'mysurvey'  # Always good to set the note to which survey generated the observation
       # Make it a list of n
       result = [obs] * 7
       return result

# load up a model observatory.
nside=32  # Specify the HEALpix resolution we want to do everything in.
mjd_start = 60218
mo = ModelObservatory(nside=nside, mjd_start=mjd_start)
conditions = mo.return_conditions()

my_survey = MySurvey([])
print("what's the current time?", conditions.mjd)
print("Is the current survey feasible and has a reward? (i.e. it's not -Infinity)", my_survey.calc_reward_function(conditions))
print("Ok, try the observations")
obs = my_survey.generate_observations(conditions)
print("did the conditions change? (they should not have here yet)", conditions.mjd)
pd.DataFrame(np.hstack(obs))[['RA', 'dec', 'mjd', 'exptime', 'filter', 'alt', 'az', 'airmass', 'night', 'slewtime', 'sunAlt']]

# Now to build our scheduler object, and run it for a day

scheduler = CoreScheduler([my_survey], nside=nside)
# sim_runner will return the updated model observatory, the updated scheduler, and an array of
# completed observations
print('starting time', mo.mjd)
mo, scheduler, observations = sim_runner(mo, scheduler,survey_length=0.3,
                                                verbose=True)
print('ending time', mo.mjd)
# Show what observations were acquired
pd.DataFrame(np.hstack(observations))[['RA', 'dec', 'mjd', 'exptime', 'filter', 'alt', 'az', 'airmass', 'night', 'slewtime', 'sunAlt']]

So basically, it looks like the field set during the run of sim_runner, but you couldn't see the observations which had been acquired because the way sim_runner exited did not return those observations.
You can see what time that sim_runner ended, even with an error, by checking on mo.mjd.

And because of this particular setup of MySurvey, this particular field is the only proposed observation even if it's not possible to observe because it's set already. MySurvey should check to see if the field is visible before proposing it with a valid reward value. This isn't really explored in this very basic introductory notebook .. and it doesn't really look like any of the other notebooks in this introductory set explore that problem very well either. However the "SlewtimeBasisFunction" does include an altitude limit by default and you can use that to mask off the targets when they're not visible.