Reproduce heartbeats

[ndtretyak]

Test reproducing #1706

[Quinn-With-Two-Ns]

Hi #1706 is now resolved and a fix will be included in the next release. The SDK will not correctly wait for the workflow tasks to finish before shutting down the local activity worker (assuming a graceful stop time is provided)

[ndtretyak]

In my opinion, it's fine not to wait for the workflow tasks to finish and let them fail. However, they currently end up in an invalid state and continue heartbeating. Is it possible to stop heartbeating if the local activity worker has already stopped?

Also, there is no WorkerStopTimeout in the server's code https://github.com/search?q=repo%3Atemporalio%2Ftemporal%20WorkerStopTimeout&type=code, so this fix alone won't prevent server's scheduler from getting stuck.

[Quinn-With-Two-Ns]

However, they currently end up in an invalid state and continue heartbeating.

The worker is not in any invalid state, it is waiting for the local activity to finish or timeout

[ndtretyak]

I added another commit setting ScheduleToCloseTimeout to 2 seconds and StartToCloseTimeout to 1 second, but the workflow task continues heartbeating for the entire duration of the test (15 seconds).

[Quinn-With-Two-Ns]

I added another commit setting ScheduleToCloseTimeout to 2 seconds and StartToCloseTimeout to 1 second, but the workflow task continues heartbeating for the entire duration of the test (15 seconds).

Ah okay that is not correct I agree. Thank you for explaining. @yuandrew can you please take a look