Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Threaded operation #34

Open
wants to merge 19 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
b3349bb
Allow threaded operation, including a handler that places the message…
dmopalmer Dec 1, 2018
6bcf153
Allow threaded operation, including a handler that places the message…
dmopalmer Dec 1, 2018
ed7e8c4
Merge branch 'threaded' of github.com:dmopalmer/pygcn into threaded
dmopalmer Dec 1, 2018
2725775
PEP-8 cleanup.
dmopalmer Dec 1, 2018
fa06b7f
Allow threaded operation, including a handler that places the message…
dmopalmer Dec 1, 2018
836f229
Allow threaded operation, including a handler that places the message…
dmopalmer Dec 1, 2018
19e8a5f
PEP-8 cleanup.
dmopalmer Dec 1, 2018
c4e3615
Merge branch 'threaded' of github.com:dmopalmer/pygcn into threaded
dmopalmer Feb 14, 2019
8b46d45
Merge branch 'master' of github.com:lpsinger/pygcn into threaded
dmopalmer Mar 7, 2019
0b6635b
Merge branch 'master' of github.com:lpsinger/pygcn into threaded
dmopalmer Dec 11, 2019
2fe080b
Removed accidental duplication. Fixed pep-8 (required by Travis).
dmopalmer Jan 8, 2020
9de70e9
Merge branch 'master' of github.com:lpsinger/pygcn into threaded
dmopalmer Oct 20, 2020
3828390
Merge branch 'main' of github.com:lpsinger/pygcn into threaded101
dmopalmer Jul 6, 2021
7b309e7
Added extra and keyword args to the wrappers.
dmopalmer Nov 18, 2021
6b98dfe
Merge branch 'handler_args' of github.com:dmopalmer/pygcn into thread…
dmopalmer Nov 18, 2021
b1f8369
Merge branch 'main' of github.com:lpsinger/pygcn into threaded102
dmopalmer Jan 30, 2023
4b88669
Merge branch 'nasa-gcn:main' into threaded102
dmopalmer Jan 30, 2023
86c4e5c
Removed stopevent capability to simplify threaded operation.
dmopalmer Apr 4, 2023
5db02d4
Readme change
dmopalmer Apr 5, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 39 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,45 @@ def handler(payload, root):
gcn.listen(handler=handler)
```

## Threading

You can run the listener in a separate thread or process and pass the packets back in a `Queue`,
allowing the main program to continue operating while waiting for an event.
Here is an example:

```python
#!/usr/bin/env python
import gcn
import threading
import queue

# Set up communications:
messagequeue = queue.Queue()
# Create a listen handler to enqueue the (payload, root) tuple
handler = gcn.handlers.queuehandlerfor(messagequeue)

# Create and start the thread.
thread = threading.Thread(target=gcn.listen,
kwargs=dict(handler=handler))
thread.start()

# Wait for messages to come in, but do other things if they don't.
nothingcount=0
while True:
try:
# Use block=False if you want to timeout immediately
payload,root = messagequeue.get(timeout=10)
print(root.attrib['ivorn'])
nothingcount = 0
except queue.Empty:
# Do idle stuff here.
print("Nothing...")
nothingcount += 1
if nothingcount > 10:
print("Quitting due to inactivity")
break
```

Comment on lines +91 to +129
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove this. If someone asked how to run the listener in a thread, this isn't how I would suggest to them that they do it. (I would suggest to them that they use an ordinary handler and just launch gcn.listen in a thread or a subprocess.)

Also, this is missing calls to queue.task_done().


[1]: http://gcn.gsfc.nasa.gov
[2]: http://www.ivoa.net/documents/VOEvent
Expand Down
18 changes: 17 additions & 1 deletion gcn/handlers.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
from urllib.parse import quote_plus

__all__ = ('get_notice_type', 'include_notice_types', 'exclude_notice_types',
'archive')
'archive', 'queuehandlerfor')


def get_notice_type(root):
Expand Down Expand Up @@ -85,3 +85,19 @@ def archive(payload, root):
with open(filename, 'wb') as f:
f.write(payload)
logging.getLogger('gcn.handlers.archive').info("archived %s", ivorn)


def _queuehandler(payload, root, *, queue):
""" Place (payload, root) on queue for threaded operation.
This can be used in the following manner:
gcn.listen(handler=functools.partial(partialize_queue, queue=a_queue))
"""
queue.put((payload, root))


def queuehandlerfor(queue):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The other handlers have function names that are verbs or verb phrases. Please rename this one to be consistent. Perhaps 'enqueueorput_queue`?

"""Create a handler that places (payload, root) on the given queue
This can be used in the following manner:
gcn.listen(handler = queuehandlerfor(queue))
"""
return functools.partial(_queuehandler, queue=queue)
23 changes: 22 additions & 1 deletion gcn/tests/test_handlers.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
from . import data
from .. import handlers
from .. import notice_types

import queue

payloads = [resources.read_binary(data, 'gbm_flt_pos.xml'),
resources.read_binary(data, 'kill_socket.xml')]
Expand Down Expand Up @@ -53,3 +53,24 @@ def test_archive(tmpdir):
assert (tmpdir / filename).exists()
finally:
os.chdir(old_dir)


def test_queuehandler():
queue_ = queue.Queue()
queuehandlerfor = handlers.queuehandlerfor(queue_)
assert queue_.empty()

for payload in payloads:
queuehandlerfor(payload, fromstring(payload))
qpayload, qtree = queue_.get()
assert qpayload == payload

assert queue_.empty()

for payload in payloads:
queuehandlerfor(payload, fromstring(payload))
for payload in payloads:
qpayload, qtree = queue_.get()
assert qpayload == payload

assert queue_.empty()
1 change: 1 addition & 0 deletions setup.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ classifiers =
Programming Language :: Python :: 3.8
Programming Language :: Python :: 3.9
Programming Language :: Python :: 3.10
Programming Language :: Python :: 3.11
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a nice addition, but not related to the topic of this PR.

Suggested change
Programming Language :: Python :: 3.11

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know why GitHub is showing this in the diff. It's already on the main branch, as of #35. Would you please rebase?

Topic :: Internet
Topic :: Scientific/Engineering :: Astronomy
project_urls =
Expand Down