UDP packet loss when memory is fragmented

[jfcliche]

I have implemented on my ESP32C3 a simple SSDP discovery function that multicasts a UDP requests and receives unicast responses from all servers on the subnet. UDP responses are 400-ish bytes long and are coming in a very rapid burst of 3 packets, sometimes separated by a few microseconds.

I observe that if my memory is heavily used and is fragmented, I almost systematically lose the 3rd UDP packet.
I have tried reading my packets in a tight loop with socket.recv(), socket.read(), socket.readinto(), blocking socket or not, with or without select.poll(), but I always have the same packet loss. This does not happen if I load only the discovery module and not the rest of the application.

To me it looks like the UDP packets are arriving faster than I can process them and the underlying LWIP code cannot allocate PBUF space for it and just silently drops the packet. Is that expected? I would not have thought that LWIP was sharing heap memory with Python, but there seems somehow to be a link with python memory utilization. I thought that maybe the garbage collector kicks in more often and slows down things enough to lose packets, but calling gc.collect() before the packet receiving loop does not help.

Any thoughts?

[sophiedegran]

Hi
I would suggest you to look at the asyncio methods for that with StreamReader and StreamWriter, also as you have many exchange it seemed you have to Queue.
Look at Peter example