Improve Windows Support

[zsquareplusc]

The current implementation does not support Windows.
TODO: provide one
pull requests are welcome

[jabdoa2]

We would like to use pyserial-asyncio on windows. What needs to be done to support it? Currently, we use add_reader/add_writer with pyserial which seems to work fine on windows.

[SzieberthAdam]

What is the reason of not supporting Windows?

[rob-smallshire]

Simple really: Nobody has yet written the necessary code. When I put the current implementation together I didn't need Windows support for the problem I was solving. Windows support would be excellent and is definitely an objective, hence this open issue.

[jabdoa2]

I was not asking for a reason. I would opt to implement windows support. However, by looking at the code base i did not find any spot where it might be missing. Is it just untested on windows? What needs to be done?

[rob-smallshire]

For reference on this issue, it seems Windows IOCP (Input/Ouput Completion Ports) can't be used with the asyncio ProactorEventLoop without relying on private implementation details of the event loop. This Stackoverflow issue leads to more details and an alternative asyncio serial implementation which uses the aforementioned private interfaces.

It seems the ultimate solution will require changes to core Python or cloning and fixing the ProactorEventLoop.

[etseidler]

@rob-smallshire i noticed that in python 3.6 (which landed late 2016), the asyncio module received some updates. i am just starting to wade in to this topic of async serial support on Windows. do you think that this issue related to the IOCP and relying on private implementation details was possibly addressed? if you have any suggestions for where i could go look to check for myself, i would be happy to do that. i started looking around in the asyncio module source, but i wasn't sure what i was looking for.

[rob-smallshire]

@etseidler I don't see any changes in 3.6 which make the reliance on private implementation details go away. According to the documentation for 3.6 the Windows proactor event-loop still doesn't support the add_reader() or add_writer() methods used by pyserial-asyncio.

[jabdoa2]

Can we create an issue against asyncio or python to create some awareness about this? They probably do not know that we need this as public method.

[jabdoa2]

Created an issue with python asyncio for this matter: http://bugs.python.org/issue30539

[jabdoa2]

Python issue was been closed. Created a new one as requested: https://bugs.python.org/issue32396

[stlehmann]

I just tried the example in the documentation on my Windows PC and it worked very well. Is it possible that the issue has been taken care of lately?

[jabdoa2]

@stlehmann: Nope. I implemented a workaround using polling on windows (see above). Works but it ain't a great solution

[jabdoa2]

@rob-smallshire i guess that they are right in https://bugs.python.org/issue32396. We could probably just use loop.sock_send()/loop.sock_recv(). Do you think we should implement it this way?

[small-round-object]

@jabdoa2, Sorry to jump into the comments without previously contributing, but I am also interested in trying to get Windows support fully working. Is there a reason that WriteFileEx/ReadFileEx couldn't be used in lieu of WriteFile and ReadFile as the Twisted implimentation has done? Looking over some of the work that the node-serialport team did it seems like they have had success with this method, with a few caveats.

[jabdoa2]

@small-round-object feel free to solve the issue. Afaik this could be done using the methods mentioned in my previous post. If you want to volunteer to do it go ahead!

[mrjohannchang]

Consider using aioserial.

Here's an example:

import aioserial
import asyncio


async def read_and_print(aioserial_instance: aioserial.AioSerial):
    while True:
        raw_data: bytes = await aioserial_instance.read_async()
        print(raw_data.decode(errors='ignore'), end='', flush=True)

asyncio.run(read_and_print(aioserial.AioSerial(port='COM1')))

[sander76]

Consider using aioserial.

Here's an example:

import aioserial
import asyncio


async def read_and_print(aioserial_instance: aioserial.AioSerial):
    while True:
        raw_data: bytes = await aioserial_instance.read_async()
        print(raw_data.decode(errors='ignore'), end='', flush=True)

asyncio.run(read_and_print(aioserial.AioSerial(port='COM1')))

That's running pyserial in a separate thread. That's not what this issue is about.

[mrjohannchang]

Consider using aioserial.
Here's an example:

import aioserial
import asyncio


async def read_and_print(aioserial_instance: aioserial.AioSerial):
    while True:
        raw_data: bytes = await aioserial_instance.read_async()
        print(raw_data.decode(errors='ignore'), end='', flush=True)

asyncio.run(read_and_print(aioserial.AioSerial(port='COM1')))

That's running pyserial in a separate thread. That's not what this issue is about.

That doesn't mean it's not asynchronous unless you insist to use the kernel thread.

[eulores]

I would also prefer an implementation based on WriteFileEx/ReadFileEx, using callbacks.
This should be doable under Python, using ctypes.
This approach also requires threads, but they would be suspended (via SleepEx) until the I/O has finished (based on alertable I/O)

Microsoft documentation:
Asynchronous Procedure Calls
Sample code Named Pipe Server Using Completion Routines

Develop Paper has a nice write-up under section Notification I/O device IO and inter thread communication

[hugh-manning]

Alright, I've had a shot at this problem, and concluded that it sucks. I've got a solution that works well for our use case, and offer it for your consideration, warts and all. See #91.

I used the method suggested by @jabdoa2, working around the issue by using the sock_recv and sock_sendall methods. I'm not convinced this is much better. It doesn't need to access private attributes directly, but it does rely on undocumented private implementation details, which feels like the same thing with extra steps.

I suspect that the ReadFileEx/WriteFileEx approach may provide the best solution, but I certainly don't have the understanding of ctypes, asyncio, and Win32 to implement it.