Feature Request? Serial Connection to Pi instead of Wifi

[thomasfjen]

Hello there!
Thank you for the Project! I really the like simple Display option without having to mess klipperscreen and their in relation expensive display.

I'm having frequent wifi dropouts with my screen and i was wondering if a serial connection to the pi would be also possible instead of relying on wifi. A quick google search sadly doesn't gave any results. Do you happen to know if that is possible? And if that's not too much work that would be nice a feature to have.

[suchmememanyskill]

It's not that easy to do so

First i'd have to rewrite all networking code (it's built from the ground up for wifi), second i'd need to write a linux/pi program that actually communicates over serial.

I don't see this to be in scope of the project.

If you want a hardwired connection, use klipperscreen.

[thomasfjen]

Thanks for the answer! That's totally understandable and I already thought that this is too much work and out of scope.

[trueserve]

I disagree with the "use klipperscreen" response. I have a use for this.

While it may not be within your scope of this project, I can see several use cases for this.

• If the CYD is to be located at the printer itself, then why not have hardwire to the printer as an option?
• Displays running KlipperScreen usually cost more than the CYD. When upgrading a particularly low-end printer, every dollar counts.
• There are many Klipper printers on the market with headless boards and limited I/O where a directly connected inexpensive screen would be beneficial. Some boards have no way (or at least no easy way) to support a screen via KlipperScreen. An example would be Elegoo Neptune 4. These printers have a board that has practically every I/O in use, no standard display output, no extra USB output, and use a Nextion touchscreen that communicates to the board over a serial link. If the screen is missing, it would be nice to hook something up to the existing interface that costs less and is more universal than an official replacement.

I think this could be done fairly easily. Would need some software on the printer side that links the Klipper API over UART to the CYD... can probably modify whconsole.py easily enough for this. Modifying CYD-Klipper to abstract API requests which can then be serial or HTTP doesn't look like it would be that difficult.

This is a feature I've wanted, and now I know others want it. I'll try to find some time to hack this together.

[suchmememanyskill]

@trueserve don't get me wrong, i completely understand the use case.

Issue is that http usage runs pretty deep in the app, as i have to account for the low amount of memory quite a bit. Most responses are parsed while being streamed to save on memory usage. The project is built from the ground up for wifi

I don't see this easily being possible in the current codebase. You'd likely have to abstract quite a bit

[trueserve]

What I looked at in the code, it doesn't look difficult. Maybe there's something I missed but it doesn't look difficult to create an abstraction. Once I get some free cycles, likely mid August I'll work on it.

[zefir-o]

Nice suggestion. I want to use this project and connect using serial instead of wifi, as it could be laggy/unstable. Do you know if there's any progress on this topic? Were you able to implement an abstraction @trueserve?

[suchmememanyskill]

Reopening this issue.
Backend is being rewritten and abstracted in #131
Serial implementation is now possible :)

@zefir-o @trueserve @thomasfjen if still interested

[zefir-o]

Thank you @suchmememanyskill great improvement!
Could you point to the documentation how to connect the display to the board? I was not able to find such documentation.
Also, if I see correctly, the latest release doesn't contain this feature and I should compile it and flash manually, right?

[zefir-o]

And also, it would be nice to describe, which changes are required from the klipper side (how to setup it). Thank you

[suchmememanyskill]

@zefir-o it's still in active development, so no release has been made yet.

It uses usb serial for communication. That way a single usb cable can be used for power and communication. You do lose the logs and serial console while using the serial connection for printer communication.

See https://github.com/suchmememanyskill/CYD-Klipper/blob/printer-interface/Manual_Install.md or compile from source from the 'printer-interface' branch if you want to try it in it's current state.

The pc-side server script still needs some work, namely in the way of auto discovery. Clone this repository (again, 'printer-interface' branch), install requests pip3 install requests, open the script serial_server.py, and adjust parameters to your environment. Then python3 serial_server.py to run the server

[thomasfjen]

Reopening this issue. Backend is being rewritten and abstracted in #131 Serial implementation is now possible :)

@zefir-o @trueserve @thomasfjen if still interested

Wow nice! Can't wait to try it out.
Thank you for implementing that!

[zefir-o]

@suchmememanyskill I tested it out. Great work, and thank you again for developing it!

Good news: I was able to connect to the printer by connecting the display via USB to Windows. It works even if I configure the port to 80 instead of Moonraker's port. Is this intended?

However, when I connected the display to Ubuntu, I'm encountering errors almost all the time. The script (serial_server) fails in various places (I think there are 2-3 different errors).

Would you like a detailed report on the errors in Ubuntu?

[suchmememanyskill]

@zefir-o usually on printers port 80, endpoints past /api get re-routed to 7125, so should be fine

I haven not tested this yet on linux. It would be great if you could send me the errors.

I have pushed a new commit recently that should have better auto-detection capabilities, and also does configuration via environment variables rather than in the script. Would you mind testing that too?

[zefir-o]

@suchmememanyskill
The errors, which I meant were caused by using the script on WSL.
Now, I want to report, that I've been using the script under the docker container on orange pi for 2+ days without any issues.
Great improvement!

Also, I wonder, if I can connect the display to the USB on the host machine and use RX-TX on the board directly, to make the connection more compact. Will such a setup work or do I need an adapter?

[suchmememanyskill]

@zefir-o Good to hear!

if I can connect the display to the USB on the host

Can you elaborate what you mean by that? I mean, yes, as long as you can run the script on the host, and connect it over usb, you can use cyd-klipper's serial interface.

For connecting through RX/TX, skipping the usb serial adapter, you can also technically do this. You'd have to hard specify the path to the serial interface though in the script through environment variables.

[thomasfjen]

So while trying to install the host serial server i get this error.
Should a virtual enviroment be installed?

As additional info i have shake&tune installed which does alot of python stuff and might be the reason for that issue?

[suchmememanyskill]

@thomasfjen Oh, i forgot ubuntu devs are completely insane and don't use python's built in package manager.

If you can install packages via pip3 in a venv, then set up a venv. If you do figure that out, could you send me the needed commands?

[thomasfjen]

@thomasfjen Oh, i forgot ubuntu devs are completely insane and don't use python's built in package manager.

If you can install packages via pip3 in a venv, then set up a venv. If you do figure that out, could you send me the needed commands?

#!/bin/bash

KLIPPER_VENV_PATH="${HOME}/klippy-env"

if [ "$EUID" -eq 0 ]; then
    echo "Please do not run as root"
    exit
fi

set -e


# Install dependencies
source "${KLIPPER_VENV_PATH}/bin/activate"
pip3 install -r requirements.txt
deactivate

# Create systemd unit file
mkdir -p ~/.config/systemd/user
echo "[Unit]" > ~/.config/systemd/user/cyd-klipper-serial.service
echo "Description=CYD Klipper serial server" >> ~/.config/systemd/user/cyd-klipper-serial.service
echo "After=network.target" >> ~/.config/systemd/user/cyd-klipper-serial.service
echo "" >> ~/.config/systemd/user/cyd-klipper-serial.service
echo "[Service]" >> ~/.config/systemd/user/cyd-klipper-serial.service
echo "ExecStart=python3 $(pwd)/serial_server.py" >> ~/.config/systemd/user/cyd-klipper-serial.service
echo "WorkingDirectory=$(pwd)" >> ~/.config/systemd/user/cyd-klipper-serial.service
echo "Restart=always" >> ~/.config/systemd/user/cyd-klipper-serial.service
echo "" >> ~/.config/systemd/user/cyd-klipper-serial.service
echo "[Install]" >> ~/.config/systemd/user/cyd-klipper-serial.service
echo "WantedBy=multi-user.target" >> ~/.config/systemd/user/cyd-klipper-serial.service

# Start the service
systemctl --user daemon-reload
systemctl --user enable cyd-klipper-serial
systemctl --user start cyd-klipper-serial

I got it working. TBH i don't know anything regarding python but i looked at the shake&tune install script and copied over three commands from there. If i read that correctly it uses the python venv from it and it did work.

from here

[zefir-o]

@zefir-o Good to hear!

if I can connect the display to the USB on the host

Can you elaborate what you mean by that? I mean, yes, as long as you can run the script on the host, and connect it over usb, you can use cyd-klipper's serial interface.

For connecting through RX/TX, skipping the usb serial adapter, you can also technically do this. You'd have to hard specify the path to the serial interface though in the script through environment variables.

Hi,
The idea is to connect the USB to the host, cut the wire and connect to rx, tx, gnd, 5v pins to jst connector. Do you think it will work?

[zefir-o]

I think, I found a bug. From time to time (quite often) the bed preheat buttons doesn't work. If I press "set" and enter the temp manually it works, but usage of presets 60/70 doesn't work.

[suchmememanyskill]

@zefir-o Cannot reproduce, can you send what it outputs in the serial console log? I expect something along the lines of

>>> HTTP_REQUEST 5000 GET /printer/gcode/script?script=M104%20S70
<<< 200 {"result": "ok"}

[zefir-o]

HTTP_REQUEST 1000 GET /printer/objects/query?extruder&heater_bed&toolhead&gcode_move&virtual_sdcard&print_stats&webhooks&fan&display_status
<<< 200 {"result": {"eventtime": 550.244650762, "st...
[LOG] Deserialization result: Ok
HTTP_REQUEST 5000 GET /printer/gcode/script?script=M140%20S60
<<< 200 {"result": "ok"}
[LOG] Deserialization result: Ok
HTTP_REQUEST 1000 GET /printer/objects/query?extruder&heater_bed&toolhead&gcode_move&virtual_sdcard&print_stats&webhooks&fan&display_status
<<< 200 {"result": {"eventtime": 551.24677497, "sta...
[LOG] Deserialization result: Ok
HTTP_REQUEST 1000 GET /printer/objects/query?extruder&heater_bed&toolhead&gcode_move&virtual_sdcard&print_stats&webhooks&fan&display_status
<<< 200 {"result": {"eventtime": 552.249479679, "st...
[LOG] Deserialization result: Ok
HTTP_REQUEST 1000 GET /printer/objects/query?extruder&heater_bed&toolhead&gcode_move&virtual_sdcard&print_stats&webhooks&fan&display_status
<<< 200 {"result": {"eventtime": 553.252086388, "st...
[LOG] Deserialization result: Ok
HTTP_REQUEST 5000 GET /printer/gcode/script?script=M104%20S70
<<< 200 {"result": "ok"}
[LOG] Deserialization result: Ok
HTTP_REQUEST 1000 GET /printer/objects/query?extruder&heater_bed&toolhead&gcode_move&virtual_sdcard&print_stats&webhooks&fan&display_status
<<< 200 {"result": {"eventtime": 554.25497518, "sta...

70 is set via the "set" bed temp button, whereas 60- using the button "60". Also, I found, that the bug is reproducible when I first set the extruder temp via any button. Otherwise, it works fine.

[suchmememanyskill]

Again, cannot reproduce. Clicking 200c for extruder, then 60c for bed works fine.
Using the mainsail test printer docker image also shows popup messages for set temperatures.
Your printer also accepts it (200 OK). You sure it's not on the printer side? The commands sent look fine

[zefir-o]

🤔 I have no idea what happens. The error code seems to be ok, however setting bed temp works using "set" button and doesn't work from 60/70/0 button.

[suchmememanyskill]

There isn't any difference in the codes sent via serial between setting it using the set button and setting it using the preset buttons, right?

[suchmememanyskill]

Wait, @zefir-o does it by chance set the nozzle temperature? (like the bed temp button mistakenly sets the nozzle temp?)

[zefir-o]

Yes, it does.

[suchmememanyskill]

@zefir-o Fixed in the latest commit

[zefir-o]

Hi @suchmememanyskill ,
I would like to report one more issue for improvement :)
The script running on the host consumes a lot of CPU time. It occupies a single core almost completely which makes it hard to use on single-board computers, like the Raspberry or so.

[suchmememanyskill]

@zefir-o Just pushed a commit, does this lower cpu usage?

[zefir-o]

Thank you @suchmememanyskill ,
It is much better compared to what before! But I would say it's still high for a tiny communication layer between the Klipper and cyd. The running Klipper instance with video streaming, web UI, etc takes less CPU than the script.
It would be nice if you could find some more hotspots for improvements :)

[zefir-o]

Actually, I was confused by the Docker, and the script takes much less than the Klipper. @suchmememanyskill Sorry for confusion. Everyting nice now!
Thank you again! Great project :)

[zefir-o]

Hello @suchmememanyskill
I wanted to use the JST connector on the CYD side and the USB-UART bridge (cp2102) to make the connection more compact on the display side, as the JST is much smaller than the USB. However, this setup doesn't work for some reason.
When I wire the display, the script on the PC side works well, and the LEDs on the USB-UART blink, showing that the data is transferred. However, the script on the PC side reports ''ok,' but the display doesn't show anything.
Do you have any ideas about what should be fixed, or changed? I was trying to look into the sources and could not identify anything.

Thank you.

[suchmememanyskill]

@zefir-o That sounds like an issue on your side (your serial bridge), not mine. As long as it's presented as a serial device, and you can send and receive data, you can use it.

What do you mean the pc script reports 'ok'?

[zefir-o]

I was able to run such a setup by changing:
Serial.begin();
to
Serial.begin(115200, SERIAL_8N1, 1, 3);
I've found some hint:

TTL RX is routed incorrectlly and as it is now, one will never be able to use it.

Its a great device. However, Theres a mistake on the Board. The RXD pin is tied to the CH340 chip TX pin. The result is that the external TTL serial port cannot receive. (without modifying it) This is a problem. There is a resistor in the line but the line that goes to the CPU RX pin should be tied to the other side of the resistor. Do this change in otder for the TTL serial to work.

[dmit2k]

I'm in doubt, how do you all compile from the printer-interface branch?..

I'm getting this error on all my Linux desktops (which still build from the Master branch just fine):

In file included from src/core/bambu/bambu_printer_integration.cpp:1:
src/core/bambu/bambu_printer_integration.hpp:5:10: fatal error: WifiClientSecure.h: No such file or directory

**************************************************************************
* Looking for WifiClientSecure.h dependency? Check our library registry!
*
* CLI  > platformio lib search "header:WifiClientSecure.h"
* Web  > https://registry.platformio.org/search?q=header:WifiClientSecure.h
*
**************************************************************************

 #include <WifiClientSecure.h>
          ^~~~~~~~~~~~~~~~~~~~
compilation terminated.
Compiling .pio/build/esp32-8048S043C-SD/src/core/data_setup.cpp.o
In file included from src/core/bambu/bambu_printer_panels.cpp:1:
src/core/bambu/bambu_printer_integration.hpp:5:10: fatal error: WifiClientSecure.h: No such file or directory

[suchmememanyskill]

Wificlientsecure is a built in library. I spent a while figuring it out but it seems this is a platformio bug. Compilation on windows works fine

[dmit2k]

Wificlientsecure is a built in library. I spent a while figuring it out but it seems this is a platformio bug. Compilation on windows works fine

On MacOS as well, but that's kinda really-really-strange... Then why does it compile on Linux x86-64 without complains for the Master branch?..

By the way, for the Master branch, ArduinoJson 7.3.0 is out, so the Master fails compiling as well.

src/core/data_setup.cpp: In function 'void fetch_printer_data()':
src/core/data_setup.cpp:148:45: error: 'ArduinoJson::V730PB22::detail::MemberProxy< <template-parameter-1-1>, <template-parameter-1-2> >::MemberProxy(const ArduinoJson::V730PB22::detail::MemberProxy< <template-parameter-1-1>, <template-parameter-1-2> >&) [with TUpstream = ArduinoJson::V730PB22::detail::MemberProxy<ArduinoJson::V730PB22::JsonDocument&, ArduinoJson::V730PB22::detail::RamString>; AdaptedString = ArduinoJson::V730PB22::detail::RamString]' is private within this context
         auto status = doc["result"]["status"];
                                             ^
In file included from .pio/libdeps/esp32-8048S043C-SD/ArduinoJson/src/ArduinoJson/Object/JsonObject.hpp:8,
                 from .pio/libdeps/esp32-8048S043C-SD/ArduinoJson/src/ArduinoJson.hpp:30,
                 from .pio/libdeps/esp32-8048S043C-SD/ArduinoJson/src/ArduinoJson.h:9,
                 from src/core/data_setup.cpp:5:
.pio/libdeps/esp32-8048S043C-SD/ArduinoJson/src/ArduinoJson/Object/MemberProxy.hpp:50:3: note: declared private here
   MemberProxy(const MemberProxy& src) // Error here? See https://arduinojson.org/v7/proxy-non-copyable/

Had to replace bblanchon/ArduinoJson@^7.0.0 to bblanchon/[email protected] in the lib_deps.

[mechano]

Its a great device. However, Theres a mistake on the Board. The RXD pin is tied to the CH340 chip TX pin. The result is that the external TTL serial port cannot receive. (without modifying it) This is a problem. There is a resistor in the line but the line that goes to the CPU RX pin should be tied to the other side of the resistor. Do this change in otder for the TTL serial to work.

Can you show pictures of what to do to activate the RX/TX correct transmission?
I've microscope and air soldering machine.