move_to starting too early during sun wait time

[kmharrington]

Credit to the sun safety checker script for catching this before running it on the telescope🎉

There's a period of cut scans that has two move_tos at the end. The first one is not sun safe but the second one is.

run.wait_until('2024-10-09T13:35:36+00:00')
run.acu.move_to(az=180.0, el=60.0)
run.wait_until('2024-10-09T13:40:36+00:00')
run.acu.move_to(az=180.0, el=60.0)
#run.wait_until('2024-10-09T17:33:47.970000+00:00') <--- not sun safe yet
#run.acu.move_to(az=49.108, el=60.0)
run.wait_until('2024-10-09T17:38:47.970000+00:00')
run.acu.move_to(az=49.108, el=60.0)

I'm running on the sun safety checker branch of both scheduler and scheduler-scripts
configuration to generate schedule

platform: satp1

# yaml loads iso format automatically into datetimes
t0: 2024-10-08T20:00:00+00:00
t1: 2024-10-09T20:00:00+00:00
t0_state_file: None

elevation: 60
boresight: 0
az_speed: 0.5
az_accel: 1.0

# optional, only needed if running non-defaults
hwp_dir: False # True: forwards, False: backwards
run_relock: True # Relock detectors at the beginning of the schedule
home_at_end: False # Home: spin down HWP and go to (180,60)

[mhasself]

So if I reduce the "min_angle" in the checking script to 40.9, then this failure goes away.

It would be nice to adopt nested values for the min_angle / exclusion_radius, such as:

(a) schedule design + generation (most conservative, e.g. radius=42 deg)
(b) checking the schedule (slightly less tolerant than the ACU agent, e.g. radius=41.5 deg)
(c) running the schedule (ACU agent limits, 41 deg)

This is the cheapest way to get a bit of wiggle room for rounding errors.

These could be separate config entries, e.g. in the sun_policy, 'design_buffer': 1.0, 'check_buffer': 0.5.

[kmharrington]

if we want another example of this showing up (hanging out on some satp2 branches) running this:

platform: satp2

# yaml loads iso format automatically into datetimes
t0: 2024-10-20T13:30:00+00:00
t1: 2024-10-21T20:00:00+00:00
t0_state_file: None

elevation: 60
boresight: 45 
az_speed: 0.5
az_accel: 1.0

# optional, only needed if running non-defaults
bad_smurf_slots: 
  - det-controller-c4s4
  - det-controller-c5s5
apply_boresight_rotation: True 
disable_hwp: False
no_cmb: False
hwp_dir: True # True: forwards, False: backwards
home_at_end: True # Home: spin down HWP and go to (180,60)
run_relock: True # Relock detectors at the beginning of the schedule
iv_cadence: 3600 ## take lots of IVs

cal_targets:
  - source: jupiter
    # boresight: # if not specified, use scan boresight
    elevation: 50
    focus: 'ws2'
    allow_partial: True
    az_branch: 270
  - source: saturn
    # boresight: # if not specified, use scan boresight
    elevation: 50
    focus: 'ws5'
    allow_partial: True

gets this

run.wait_until('2024-10-20T13:45:01.030000+00:00')
run.acu.move_to(az=180.0, el=48.0)
run.wait_until('2024-10-20T13:50:01.030000+00:00')
run.acu.move_to(az=180.0, el=48.0)
run.wait_until('2024-10-20T16:59:12+00:00')
run.acu.move_to(az=111.483, el=48.0)
run.acu.move_to(az=111.483, el=60.0) <<--- flagged by SunCrawler as not sun safe
run.wait_until('2024-10-20T16:59:12+00:00')

[kmharrington]

I'm still seeing this error show up and it is not just an edge case. The config file below ends up with the telescope trying to move to az,el = (49.1,60) at 17:03 UTC. That position is not sun safe until 17:21 UTC at the earliest.

Config file:

platform: satp1

# yaml loads iso format automatically into datetimes
t0: 2024-11-25T17:00:00+00:00
t1: 2024-11-26T17:00:00+00:00
t0_state_file: None
#t0_state_file: satp1/state_files/state_2024-11-25T17:00:00+00:00.npy

elevation: 60
boresight: 0
az_speed: 0.5
az_accel: 0.25

# optional, only needed if running non-defaults
hwp_dir: True # True: forwards, False: backwards
run_relock: False # Relock detectors at the beginning of the schedule
home_at_end: True # Home: spin down HWP and go to (180,60)
bias_step_cadence: 900

[mmccrackan]

While working on the comments on #140, I looked into this a bit since I need to implement #133, I looked into this issue and I think it comes from this line:

scheduler/src/schedlib/policies/stages/build_op.py

Line 657 in d27de6e

t0=block.t0-dt.timedelta(seconds=pre_dur),

As Katie hypothesized in #135, this occurs because the scan is supposed to start at 17:37, but the det setup is triggered and it runs for 34 minutes. The above line decrements the time to 17:03 for the IR block, forcing a move to the position before it is sun-safe.