Reboot when exception is raised after cancelling the asyncio task

[bixb922]

I use asyncio a lot (it makes life easy, especially when I think back to coding event loops in assembly language in the 1970ies ;-)

What I was trying to do is to await end task that had been cancelled, but an exception occurred in the task after cancelling it. The microcontroller reboots always. Issue #8929 seems to be related.

The microcontroller is a ESP32-WROOM-32D packaged on ESP32-DEVKITC-V4. I also tried with a ESP32-S3 with the same version 1.19.1, same result. I attached a test program and the output.

Please tell me if I am doing something wrong. Code and output follows:

import sys
print(sys.implementation)
import uasyncio

async def fun_with_exception():
    await uasyncio.sleep(1)
    raise ValueError
        
async def main():
    task = uasyncio.create_task( fun_with_exception() )
    await uasyncio.sleep(2)
	# At this point the task has already processed the CancelledError exception.
    task.cancel()
    await task
        
uasyncio.run( main() )

The output is as follows:

C\:>mpremote run asyncio_error.py
(name='micropython', version=(1, 19, 1), _machine='ESP32 module with ESP32', _mpy=10246)
Task exception wasn't retrieved
future: <Task> coro= <generator object 'fun_with_exception' at 3ffe5fb0>
Traceback (most recent call last):
  File "uasyncio/core.py", line 1, in run_until_complete
  File "<stdin>", line 8, in fun_with_exception
ValueError:
Guru Meditation Error: Core  1 panic'ed (LoadProhibited). Exception was unhandled.

Core  1 register dump:
PC      : 0x40084c01  PS      : 0x00060f30  A0      : 0x800dc148  A1      : 0x3ffccda0
A2      : 0x3ffe54fe  A3      : 0x3ffe5b18  A4      : 0x0000004b  A5      : 0x00000006
A6      : 0x3ffe5b18  A7      : 0x3f40b7d8  A8      : 0x80083849  A9      : 0x3ffccd10
A10     : 0x00000001  A11     : 0x3f402f74  A12     : 0x0000ffff  A13     : 0x00000000
A14     : 0x3ffcce30  A15     : 0x3ffccd90  SAR     : 0x00000011  EXCCAUSE: 0x0000001c
EXCVADDR: 0x00000006  LBEG    : 0x4000c46c  LEND    : 0x4000c477  LCOUNT  : 0x00000000

Backtrace:0x40084bfe:0x3ffccda0 0x400dc145:0x3ffcce40 0x400dc1a3:0x3ffcce60 0x400dc206:0x3ffcce90 0x400dbc76:0x3ffcceb0 0x400e2679:0x3ffcced0 0x400e27a9:0x3ffccef0 0x40084789:0x3ffccf10 0x400dbd66:0x3ffccfb0 0x400e2679:0x3ffccfe0 0x400846f6:0x3ffcd000 0x400dbd66:0x3ffcd0a0 0x400e2679:0x3ffcd100 0x400e27a9:0x3ffcd120 0x40084789:0x3ffcd140 0x400dbd66:0x3ffcd1e0 0x400e2679:0x3ffcd230 0x400e26a2:0x3ffcd250 0x400efbca:0x3ffcd270 0x400efd5d:0x3ffcd300 0x400d4d12:0x3ffcd350


ELF file SHA256: 00495403985fb7e0

Rebooting...
ets Jun  8 2016 00:22:57

rst:0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0030,len:4540
ho 0 tail 12 room 4
load:0x40078000,len:12344
ho 0 tail 12 room 4
load:0x40080400,len:4124
entry 0x40080680
MicroPython v1.19.1 on 2022-06-18; ESP32 module with ESP32
Type "help()" for more information.
>>>

[Carglglz]

@bixb922 I think you are right about #8929, I confirm this happens with a pyboard too PYBv1.1.
right now what works is:

...
await task()

or if you create a task with uasyncio.create_task()

ret = await uasyncio.gather(task)

Here is my working code

import sys
import uasyncio

print(sys.implementation)


async def task():
    await uasyncio.sleep(10)
    raise ValueError("wrong value")
    print("hello")
    return 0


async def main():
    task1 = uasyncio.create_task(task())

    await uasyncio.sleep(2)
    print("Waiting...")

    try:
        ret = await uasyncio.gather(task1)
        print(ret)
    except Exception as e:
        print(e)
 
uasyncio.run(main())

(name='micropython', version=(1, 19, 1), _machine='PYBv1.1 with STM32F405RG', _mpy=7430)
Waiting...
wrong value

[bixb922]

Thanks! Yes, that helped! If I use await uasyncio.gather(task) instead of await task in the OP then there is no crash anymore. With this change, the await-gather raises a "RuntimeError: can't gather", which probably means "this task is not longer valid since it aborted with an exception, it can't be awaited".

In CPython the await of the OP returns the exception in all cases. I understand that there is a difference in the handling of exceptions in Micropython asyncio tasks, since in Micropython the exception is processed by the task the next time it is scheduled, and then the task is not scheduled any more, while in CPython, a task with an unhandled exception is kept in the loop until the loop ends (i.e. normally at the program exit), is that right?

[Carglglz]

the await-gather raises a "RuntimeError: can't gather", which probably means "this task is not longer valid since it aborted with an exception, it can't be awaited"

Yes, I think the task must be still running to be gathered?, that's why I increased sleep time, otherwise I got the same error.

In CPython the await of the OP returns the exception in all cases. I understand that there is a difference in the handling of exceptions in Micropython asyncio tasks, since in Micropython the exception is processed by the task the next time it is scheduled, and then the task is not scheduled any more, while in CPython, a task with an unhandled exception is kept in the loop until the loop ends (i.e. normally at the program exit), is that right?

I've no idea how this compares to CPython, but yes I think it worth exploring and I think this tutorial is the goto place for uasyncio MicroPython peterhinch/micropython-async

[peterhinch]

It is possible to create a global exception handler which traps untrapped exceptions:

import uasyncio as asyncio
import sys

def _handle_exception(loop, context):
    print('Global handler')
    sys.print_exception(context["exception"])
    #loop.stop()
    sys.exit()  # Drastic - loop.stop() does not work when used this way

async def bar():
    await asyncio.sleep(0)
    1/0  # Crash

async def main():
    loop = asyncio.get_event_loop()
    loop.set_exception_handler(_handle_exception)
    asyncio.create_task(bar())
    for _ in range(5):
        print('Working')
        await asyncio.sleep(0.5)

asyncio.run(main())

This is discussed here

[bixb922]

Thank you @peterhinch. Every time I read that tutorial I learn something new!