Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add zmq function call to Gmoccapy #2803

Draft
wants to merge 2 commits into
base: master
Choose a base branch
from
Draft

Add zmq function call to Gmoccapy #2803

wants to merge 2 commits into from

Conversation

c-morley
Copy link
Collaborator

I am working on a qt version of nativecam.
nativecam wants to reload it's program on each change.
This is currently awkward if the calling program is not part of the original process.

In Qtvcp we have the capability to use zmq to subscribe to external programs that then can call the screen's functions.
This pull request adds this capability to gladevcp and gmoccapy.

In Gmoccapy this could be expanded to allow any authorized function to be called. For instance: axes selection or jogging increment. I think if we could add zmq to halui, then this could be a big jump in user usability.

I would suggest an addition to allow the user to select/deselect zmq in the options page.
This pull request is mostly for brain storming - some additional cleanup and docs are required.

@c-morley c-morley marked this pull request as draft December 23, 2023 03:06
@petterreinholdtsen
Copy link
Collaborator

petterreinholdtsen commented Dec 23, 2023 via email

@c-morley
Copy link
Collaborator Author

Zmq was already used in Qtvcp for quite some time.
It's well documented with examples and has C, C++ and python bindings.
I can't say which is better Zmq or Mqtt
NML is notoriously hard to use - everyone avoids it like the plague.

@c-morley
Copy link
Collaborator Author

@hansu you have any opinion?

@hansu
Copy link
Member

hansu commented Dec 31, 2023

I am working on a qt version of nativecam.

Nice to hear that!

I have no knowledge about mqtt or zmq and what are the strengths of each.

But in general it sounds good to have more capabilities to communicate with the GUI.

For example is that also possible to pass a signal like 'file-reloaded' to an embedded or external (glade) panel?

@c-morley
Copy link
Collaborator Author

As coded the glade panel (or any program that uses zmq) can ask Gmoccapy to reload a fike.
For Gmoccapy to ask is a little bit more code but the basics are there.
There would be decisions to be made about a way to decode who reacts to the message (topics in zmq speak)
I am not an expert either.

@andypugh
Copy link
Collaborator

andypugh commented Jan 7, 2024

Is this likely to move out of draft status?

@c-morley
Copy link
Collaborator Author

c-morley commented Jan 7, 2024

i was hoping for more discussion about the best way to integrate the idea. So whether it comes out of draft or causes another pull request i don't know. I won't let it hang forever if that is what you are asking.

@zz912
Copy link
Contributor

zz912 commented Jan 15, 2024

If I understand correctly, zmq is another way of communication between LCNC modules.

Currently we can communicate between modules using NML and HAL. The Python Interface is the defacto communication through NML. Does zmq belong to these communications? Did I understand correctly?

I understand that zmq is already integrated in LCNC. If that's the case, I think it could be used.

BUT!!! It should be documented in the LCNC documentation. Ideally some examples with usage. As the Python Interface is given. If someone wants to learn how to use LCNC, they must be able to:
C
C++
Python
Bash
HALL
G-code
Qty
Glade
.......
There is quite a lot, so the documentation should be as good as possible so that it does not discourage other developers from joining. It is clear to me that the internal documentation for zmq will be more difficult than its implementation.

Will there be any rules when to use NML, zmq, HAL, header file with extern, glade qt signal?

https://linuxcnc.org/docs/devel/html/code/code-notes.html

@c-morley
Copy link
Collaborator Author

Zmq is used for communication between programs using sockets. it a well documented library with many bindings to other languages.
Qtvcp has some 'hooks' in it that can be turned on to use Zmq.
documented here:
http://linuxcnc.org/docs/stable/html/gui/qtvcp-code-snippets.html#_external_control_with_zmq

NML HAL and Zmq are all appropriate for different things, with strength and weakness on each library.

@zz912
Copy link
Contributor

zz912 commented Jan 17, 2024

Here is documentation too:
http://linuxcnc.org/docs/stable/html/gui/panelui.html#_zmq_messages

One stuff in many places.

Shouldn't it have a main chapter here?
main
Explaining for example what is the difference between ZMQ and NML?
Is ZMQ only for GUI?
Is ZMQ for GUI and realtime component?
Can ZMQ replace NML?
Does it make sense to use ZMQ instead of a header file?

The chapter on NML is an example of how it should not look like:
http://linuxcnc.org/docs/devel/html/code/nml-messages.html
There is information only for person, who knows everything about NML.
But at least it's in the right place.

@rmu75
Copy link
Contributor

rmu75 commented Feb 14, 2024

(off-topic, in case somebody is interested in exploring zmq as replacement for NML: I have a POC running that is using zmq/flatbuffers instead of NML. https://github.com/rmu75/linuxcnc/tree/rs/zmq-experiments / https://github.com/rmu75/cockpit/tree/rs/zmq)

@Sigma1912
Copy link
Contributor

So ZMQ seems to be a promising path forward.

@hydroid7
Copy link

I really appreciate the work done by @rmu75. IMHO Flatbuffers combined with message passing it is the future of LinuxCNC.

However, I'd propose the use of Posix message queues. They are easy to use, built in into Linux, have bindings to literally all languages and are well supported.
In my measurements (which I'm not allowed to publish, but here is a simple comparison) ZMQ has about 5x greater latency than plain PosixMQs.

PosixMQs have an absolute maximum message size of about 16MB, but I think that should not be a limiting factor.

@c-morley
Copy link
Collaborator Author

Id there a preferred python library for posix message queues?

@petterreinholdtsen
Copy link
Collaborator

petterreinholdtsen commented Feb 19, 2024 via email

@andypugh
Copy link
Collaborator

I am not sure what the use case is here, though do recall that Machinekit had an ambition to replace NML with ZMQ.

Is intersystem interopability a desirable feature of the system being studied? If so then being tied into POSIX might not be ideal.

@rmu75
Copy link
Contributor

rmu75 commented Feb 19, 2024

I'm somewhat guilty of derailing this issue, there is probably a more appropriate place or issue to discuss that. I will probably open a draft PR when my code is somewhat more mature and discussion can be continued there. Nonetheless:

@hydroid7 that linked comparison somewhat compares apples and bananas. 0MQ can also use IPC via unix domain sockets (no TCP/UDP or even IP involved), and I doubt that those have to be inherently slower or have higher latency than posix message queues. But I didn't (yet) measure anything. Communications overhead is not that critical, NML is used for GUI stuff, and that doesn't need to be more "current" than what your screen refresh rate allows. Even then more than 30Hz update rate won't gain you much.

IMO the messaging mechanism that replaces NML has to be network transparent. To gain anything over NML, it has to be a well-established and maintained solution that is independent of linuxCNC and in wide enough use so that risk of becoming abandoned is low. Plus points would be cross-platform abilities and availability for a large number or platforms and programming languages, especially javascript/web stuff. Best would be a well-established RPC framework with low overhead, low latency and low complexity (i.e. not CORBA nor SOAP nor derivatives).

That IMO rules out posix message queues and sysv IPC without some interposer that can also talk UDP or TCP like 0MQ.

@c-morley
Copy link
Collaborator Author

This discussion of replacing NML while interesting and important for linuxcnc, it is not really related to the PR.
This PR is for adding a message system to send arbitrary messages between UI programs, something NML does not do.
If linuxcnc had a concept of 'current jog rate/increment/axis' and sent a message for GUIs to update their graphics, then we could avoid the current use case for this ZMQ pr, but I bet in the future there will be more uses.

The nice thing about ZMQ is there are python and C C++ libraries available on the distribution we support.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants