Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

py: fix read with timeout on windows #10

Open
milahu opened this issue Mar 4, 2024 · 3 comments · May be fixed by #11
Open

py: fix read with timeout on windows #10

milahu opened this issue Mar 4, 2024 · 3 comments · May be fixed by #11

Comments

@milahu
Copy link
Owner

milahu commented Mar 4, 2024

the current signal.SIGALRM solution is unix-only
per Timeout on subprocess readline in Python

similar #6

the javascript version uses child_process.spawnSync to enforce the read timeout

continue #4 (comment)

the messing with signals is needed to get a "read with timeout"

in python we could use subprocess.run

#!/usr/bin/env python3

import sys
import subprocess

fd_read = 0 # stdin # TODO use the jobserver's fd_read

# pipe one byte from fd_read to stdout
py_code = f"""
import sys; sys.stdout.buffer.write(open({fd_read}, "rb").read(1))
"""

args = [
  sys.executable, # python
  "-c", py_code,
]

try:
  input_byte = subprocess.run(
    args,
    capture_output=True,
    timeout=2,
  ).stdout
except subprocess.TimeoutExpired:
  input_byte = None

print("input_byte", input_byte)

sys.executable, # python

... but instead of starting a new python process
first this should try to run dd to save memory

i tried to solve this with asyncio.timeout but no, this hangs at os.read

fail: asyncio.timeout 
#!/usr/bin/env python3

import os
import asyncio

def read_with_timeout():
  loop = asyncio.get_event_loop()
  async def read_with_timeout_inner():
    timeout = 1
    fd_read = 0 # stdin
    if False:
      # no. this hangs at os.read
      try:
        async with asyncio.timeout(timeout):
          buffer = os.read(fd_read, 1)
          return buffer
      except TimeoutError:
        return None
    # no. this hangs at os.read
    await asyncio.wait_for(os.read(fd_read, 1), timeout)
    return "ok"
  return loop.run_until_complete(read_with_timeout_inner())

print("read_with_timeout", read_with_timeout())
@milahu milahu linked a pull request Mar 11, 2024 that will close this issue
Open
@milahu
Copy link
Owner Author

milahu commented Jun 1, 2024

maybe use timeout-decorator

@milahu milahu mentioned this issue Jun 2, 2024
Open
@milahu
Copy link
Owner Author

milahu commented Jun 2, 2024

does this affect us too?

pyca/pyopenssl#168 (comment)

edit: nope. timeout-decorator is limited to the main thread

  File "aia.py", line 212, in get_host_cert_chain
    do_handshake()
  File "/lib/python3.11/site-packages/timeout_decorator/timeout_decorator.py", line 75, in new_function
    old = signal.signal(signal.SIGALRM, handler)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/lib/python3.11/signal.py", line 58, in signal
    handler = _signal.signal(_enum_to_int(signalnum), _enum_to_int(handler))
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: signal only works in main thread of the main interpreter

pnpnpn/timeout-decorator#75

https://github.com/pnpnpn/timeout-decorator#multithreading

Multithreading

By default, timeout-decorator uses signals
to limit the execution time of the given function.

This appoach does not work if your function is executed not in a main thread
(for example if it's a worker thread of the web application).
There is alternative timeout strategy for this case - by using multiprocessing.

To use it, just pass use_signals=False to the timeout decorator function:

import time
import timeout_decorator

@timeout_decorator.timeout(5, use_signals=False)
def mytest():
    print "Start"
    for i in range(1,10):
        time.sleep(1)
        print("{} seconds have passed".format(i))

if __name__ == '__main__':
    mytest()

Warning: Make sure that in case of multiprocessing strategy for timeout,
your function does not return objects which cannot be pickled,
otherwise it will fail at marshalling it between master and child processes.

@milahu
Copy link
Owner Author

milahu commented Jun 2, 2024

maybe use select, see also pyopenssl do_handshake with timeout

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

py: use subprocess to read with timeout milahu/gnumake-tokenpool
1 participant
@milahu