Packaging

[russelltg]

From @russelltg on November 23, 2016 15:50

• windows installer
• flatpak (Enable debugger in flatpak build #18 outstanding)
• appimage
• .app

Copied from original issue: chigraph/chigraph#16

[russelltg]

From @probonopd on December 4, 2016 12:36

For AppImage, have a look at https://github.com/probonopd/linuxdeployqt. You could use it to bundle your application as part of your Travis CI runs. Check out these projects which already use linuxdeployqt on Travis CI to produce AppImages:

• https://github.com/Blinkinlabs/PatternPaint
• https://github.com/fathomssen/redtimer
• https://github.com/coryo/amphetype2
• https://github.com/chkmue/MyQtTravisTemplateProject
• https://github.com/chkmue/qttravisCI_1
• https://github.com/eteran/edb-debugger
• https://github.com/probonopd/linuxdeployqt/ obviously ;-)

[russelltg]

Looks really cool! I almost got it working, just one more thing.

Because I am using KXmlGui, I need to be able to package the .rc file in share/kmlgui5/chiggui/chigguiui.rc

How can I tell it to package it?

[russelltg]

From @probonopd on December 4, 2016 19:19

For now, you can try copy it to the target location manually.
Can you upload the (non-working) AppImage for analysis? I would like to find out where it is searching for the file, since it is not obvious to me where it should go.

References:

• https://mail.kde.org/pipermail/kde-windows/2015-November/009504.html
• https://git.reviewboard.kde.org/r/125595/

[russelltg]

Yeah no problem.
When you run it you get

cannot find .rc file "chigguiui.rc" for component "chiggui"

Here is the image (it was too big for github):
https://www.dropbox.com/s/w7j3knsvj8ie98i/Application-x86_64.AppImage?dl=0

[russelltg]

From @probonopd on December 4, 2016 19:31

When I run this, I get

This application failed to start because it could not find or load the Qt platform plugin "xcb"
in "".

Available platform plugins are: xcb.

Reinstalling the application may fix this problem.
Aborted

patchelf --print-rpath squashfs-root/plugins/platforms/libqxcb.so shows that the rpath is not set and patchelf did not run correctly on this file.
This might be caused by probonopd/linuxdeployqt#25 - simply run the same linuxdeployqt command twice. Does it work then?
Please upload the file again.

[russelltg]

From @probonopd on December 4, 2016 19:33

Back to the .rc file "chigguiui.rc" for component "chiggui" issue: Can you define it in a Qt resource file like in the example on https://git.reviewboard.kde.org/r/128193/diff/3-4/ and then run linuxdeploy twice, does it work then?

[russelltg]

I've never setup a qrc file before, so I had to look up some docs and I'm still not entirely sure I did it right.

It still doesn't work so I fear I did it incorrectly.

here is the commit.

Same error.

I never got the original error you got, I'm testing on a clean VM to see if I can reproduce with the new image.

[russelltg]

From @probonopd on December 4, 2016 20:14

I never got the original error you got

You will only get it if you run the AppImage on a system with a different Qt from what you used at compile time.

[russelltg]

Aah makes sense

[russelltg]

From @probonopd on January 11, 2017 17:9

me@host:~$ /home/me/Downloads/Chigraph-x86_64-a50a7b0.AppImage 
This application failed to start because it could not find or load the Qt platform plugin "xcb"
in "".

Available platform plugins are: xcb.

Reinstalling the application may fix this problem.
Aborted

is still the case with Chigraph-x86_64-a50a7b0.AppImage and xubuntu-16.04-desktop-amd64.iso.

[russelltg]

I'm working on it right now, I've make a few other appimages and not
uploaded them.

I'm getting it to work in CI so I can have a continously running appimage.

I figured out the other issues though :).

On Wed, Jan 11, 2017, 10:09 AM probonopd [email protected] wrote:

me@host:~$ /home/me/Downloads/Chigraph-x86_64-a50a7b0.AppImage
This application failed to start because it could not find or load the Qt platform plugin "xcb"
in "".

Available platform plugins are: xcb.

Reinstalling the application may fix this problem.
Aborted

is still the case with Chigraph-x86_64-a50a7b0.AppImage and
xubuntu-16.04-desktop-amd64.iso.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
chigraph/chigraph#16 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AGxqOqqWZOQf8S_BQDF5UYTfROovxvDbks5rRQzFgaJpZM4K6vxm
.

[russelltg]

Hey so I'm ready to start on this again-finally got all hardcoded paths out. I need to package some files in the lib dir (specificly the basic includes for clang because I have to compile code using libclang), how do I package that as well?

I also need to package two exectuables because my qt exectuable calls another one (that is also compiled by the project)

[russelltg]

From @probonopd on March 2, 2017 7:0

Compile everything you compile, including all your self-compiled binaries and libraries, with PREFIX=/usr but install to an AppDir using make INSTALL_ROOT=appdir install (qmake) or make DESTIDR=appdir install (CMake). Similar to this:

script:
  - qmake PREFIX=/usr
  - make -j4
  - sudo make INSTALL_ROOT=appdir install ; sudo chown -R $USER appdir ; find appdir/
  - wget -c "https://github.com/probonopd/linuxdeployqt/releases/download/continuous/linuxdeployqt-continuous-x86_64.AppImage" 
  - chmod a+x linuxdeployqt*.AppImage
  - unset QTDIR; unset QT_PLUGIN_PATH ; unset LD_LIBRARY_PATH
  - ./linuxdeployqt*.AppImage ./appdir/usr/share/applications/*.desktop -bundle-non-qt-libs
  - ./linuxdeployqt*.AppImage ./appdir/usr/share/applications/*.desktop -appimage
  - find ./appdir -executable -type f -exec ldd {} \; | grep " => /usr" | cut -d " " -f 2-3 | sort | uniq
  - curl --upload-file ./APPNAME*.AppImage https://transfer.sh/APPNAME-git.$(git rev-parse --short HEAD)-x86_64.AppImage

[russelltg]

Hey! So I've done a lot of work getting appimages to work and I have--for the most part. I setup a continous build for it, but there is still one problem: I can't seem to spawn a process from the appimage.

My GUI spawns an executable that is in the same directory, and it finds it like this:

boost::filesystem::path chiPath =
    boost::filesystem::path(QApplication::applicationFilePath().toStdString()).parent_path() / "chi";

Q_ASSERT(boost::filesystem::is_regular_file(chiPath));

The assertion doesn't trigger, but when I try to spawn that using a QProcess, it returns error 127 which is command not found. Any ideas?

[russelltg]

From @probonopd on April 3, 2017 5:40

@russelltg so, let's debug.

First of all, please use a newer version of appimagetool because the --appimage-extract command is broken in your AppImage due to using an old version.

Next, you seem to be missing all libraries.

error while loading shared libraries: libKF5Crash.so.5: cannot open shared object file: No such file or directory

Maybe running something like linuxdeployqt would be a good idea. Alternatively, you can manually bundle all libraries but those that are on the excludelist.

Once this is fixed, I will have another look; please let me know how I can invoke the command in the GUI that launches the QProcess. Need to look at it with strace to see what it is actually doing.

[russelltg]

I am currently running linuxdeployqt (see my script here).

Also I'm pretty sure libKF5Crash.so is being bundled:

$ ./appimagetool-x86_64.AppImage ./chigraph/Chigraph-x86_64.AppImage --list | grep KF5Crash
usr/lib/cmake/KF5Crash
usr/lib/cmake/KF5Crash/KF5CrashConfig.cmake
usr/lib/cmake/KF5Crash/KF5CrashConfigVersion.cmake
usr/lib/cmake/KF5Crash/KF5CrashTargets-release.cmake
usr/lib/cmake/KF5Crash/KF5CrashTargets.cmake
usr/lib/libKF5Crash.so
usr/lib/libKF5Crash.so.5
usr/lib/libKF5Crash.so.5.31.0

As far as launching a process, here's the steps:

1. Load the test workspace in chigraph/test/codegen/worksapce
2. Create a launch configuration by clicking File->Configure launches and press "New Configuration" and in the module field, type looping/main (the GUI is definitely a work in progress there :P)
3. Select that launch config by going to the dropdown in the toolbar and select "New Configuration" (or if you renamed it something else)
4. Press Run
5. A output tab should show up, saying it exited with code 127, which is command not found...

I'll look into strace myself and post the results.

Thank you so so so much for your willingness to help, it's truly appreciated. Appimages are way too cool to pass up 👍.

I'll make sure to test in a fresh VM too.

[russelltg]

You're right, there are some DLL issues. I'm looking into them right now, I think I shouldn't copy them in manually and just let linuxdeployqt do its work.

[russelltg]

Alright so when running it natively (no appimage), strace spits out this relevant output:

stat("/home/russellg/projects/chigraph/build/bin/chi", {st_mode=S_IFREG|0755, st_size=46456432, ...}) = 0
fstat(1, {st_mode=S_IFCHR|0600, st_rdev=makedev(136, 3), ...}) = 0
write(1, "/home/russellg/projects/chigraph"..., 47) = 47
pipe2([12, 13], O_CLOEXEC)              = 0
pipe2([14, 15], O_CLOEXEC)              = 0
pipe2([16, 17], O_CLOEXEC)              = 0
pipe2([18, 19], O_CLOEXEC)              = 0
rt_sigaction(SIGCHLD, {sa_handler=0x7f31fa1c2400, sa_mask=[], sa_flags=SA_RESTORER|SA_NOCLDSTOP, sa_restorer=0x7f31f550afe0}, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=0}, 8) = 0
rt_sigaction(SIGPIPE, {sa_handler=SIG_IGN, sa_mask=[], sa_flags=SA_RESTORER, sa_restorer=0x7f31f550afe0}, NULL, 8) = 0
futex(0x7f31fa67ac84, FUTEX_WAKE_PRIVATE, 2147483647) = 0
pipe2([20, 21], O_CLOEXEC)              = 0
eventfd2(0, EFD_CLOEXEC)                = 22
clone(child_stack=NULL, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7f31ffc7e390) = 21125

/home/russellg/projects/chigraph/build/bin/chi is the process being spawned, and the clone clearly does its job.

But when it's under the appimage, the clone seems ok:

write(1, "/tmp/.mount_kNdEPl/usr/bin/chi\n", 31) = 31 # this prints to stdout the name of the executable for debugging
...
clone(child_stack=NULL, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7f7217246ad0) = 21432

but then there's a SIGCHILD:

--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=21432, si_uid=1000, si_status=127, si_utime=0, si_stime=0} ---

which is where the exit status 127 is coming from.

I'm not that experienced at reading these, any clues?

[russelltg]

From @probonopd on April 4, 2017 5:54

Maybe chi has dependencies that also need to be bundled? Does chi in the AppImage run if you invoke it manually?

[russelltg]

From @probonopd on April 4, 2017 6:25

Let's mount the AppImage and see what is inside:

me@host:~$ sudo mount '/home/me/Downloads/Chigraph-x86_64.AppImage' /mnt -o offset=115696

me@host:~$ file /mnt/usr/bin/chigraphgui 
/mnt/usr/bin/chigraphgui: ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 2.6.24, BuildID[sha1]=a7e2af0ca7b147c54315000245dd807ec395e9a8, not stripped

Please strip files. Actually linuxdeployqt should do that for you.

Do you really, absolutely need your own Docker container to build this in, rather than the environment provided by Travis CI? Here is an example that generates an AppImage using linuxdeploqt on Travis without the need for Docker: https://github.com/lirios/browser/blob/develop/.travis.yml

And where is appimage_recipie.sh so that I can have a look at what you are doing?

Looking at the build log, we see

ERROR: ldd outputLine: "\tlibKF5Crash.so.5 => not found"
ERROR: ldd outputLine: "\tlibKF5TextEditor.so.5 => not found"
ERROR: ldd outputLine: "\tlibKF5XmlGui.so.5 => not found"
ERROR: ldd outputLine: "\tlibKF5Completion.so.5 => not found"
ERROR: ldd outputLine: "\tlibKF5Service.so.5 => not found"
ERROR: ldd outputLine: "\tlibKF5ConfigWidgets.so.5 => not found"
ERROR: ldd outputLine: "\tlibKF5I18n.so.5 => not found"
ERROR: ldd outputLine: "\tlibKF5CoreAddons.so.5 => not found"
ERROR: ldd outputLine: "\tlibKF5WidgetsAddons.so.5 => not found"
ERROR: ldd outputLine: "\tlibKF5ConfigCore.so.5 => not found"

This means that these libraries are not found on the build system, and hence cannot be bundled. Run ldd on your binary to debug this. When ldd says => not found then linuxdeployqt has no chance.

Also, what is /mnt/usr/lib/chigraph/stdlib/ and why is it there? It's not working and should be removed. AppImages use the standard library provided by the target system (glibc and friends).

[russelltg]

Do you really, absolutely need your own Docker container to build this in

Not really, it was nice when I was making them on my local computer but now that I'm using travis, I'll remove it. That'll be a good first step to a sane appimage

And where is appimage_recipie.sh so that I can have a look at what you are doing?

https://github.com/chigraph/chigraph/blob/master/scripts/appimage/appimage_recipie.sh

This means that these libraries are not found on the build system

This is a recent thing. How do I point ldd to those libraries? Is LD_LIBRARY_PATH enough?

what is /mnt/usr/lib/chigraph/stdlib/ and why is it there?

It's not directly used by the chigraph executables. Chigraph itsself is a compiler, and it compiles C code as well, so needs a glibc install. It's only used by chigraph when compiling chigraph code. I should probably delete the libraries in there though, I only really need the headers.

[russelltg]

From @probonopd on April 5, 2017 17:41

Can you make a .travis.yml that compiles Chigraph and its dependencies to usr in the standard Travis Ubuntu 14.04 trusty environment? But when running make install have it install to a directory appdir/, so that the files actually end up in appdir/usr/.... Then I can try to take it from there.

[russelltg]

Yes of course!

Thanks so much :).

💓

[probonopd]

Please point me to copy-and-paste-able build instructions for Ubuntu 14.04 trusty, thanks.

[russelltg]

https://github.com/chigraph/chigraph-gui/blob/master/scripts/appimage/appimage_recipie.sh

That currently works on a trusty travis CI bot. Thanks for following up!

[probonopd]

@russelltg thank you very much. Since people ask from time to time how to use KF5 on trusty on Travis CI, I'd like to add an explanation to the linuxdeployqt project.

Is your https://github.com/chigraph/chigraph/releases/download/dependencies/kf5-5.32.0-release-gcc-linux64.tar.xz so generic that it could be used by other projects too? In this case, would you consider to move it to its own project?

EDIT: It seems it is not, e.g., kf5-release-gcc-linux64.tar.xz seems to be missing knotifications kf5 library.

[russelltg]

Yes it is. I'll move it into it's seperate project when I have the time (probably today)

[probonopd]

Can you include all KF5 parts such as knotifications too? That would be tremendous!

[russelltg]

Yeah, I can. It'd literally be 1 more line of code

[probonopd]

What was the issue that led you to remove AppImage?

[russelltg]

I guess I'm not ruling it out, just I was curious about flatpak and it was incredibly easy to integrate and becoming more and more standard. I wouldn't say issue, just flatpak was easy, and potentially easier for appstore integration.

…

On Tue, Sep 12, 2017, 10:15 AM probonopd ***@***.***> wrote: What was the issue that led you to remove AppImage? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#15 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AGxqOj9OPOZzq4xr5hP0erIBN7GE1cisks5shq4TgaJpZM4M-Lw7> .

[probonopd]

Just be aware that these two solutions are totally different beasts - whereas with AppImage you get one file that runs out of the box on most Linux desktops, the same cannot be said for Flatpak (e.g., last time I tried it did not work on Live systems at all), and you don't get portable apps that you can easily carry around on a USB stick. So why not offer both...

[russelltg]

Yeah that's why I said it's not ruled out. I'll add that checkbox back :)

…

On Tue, Sep 12, 2017, 10:43 AM probonopd ***@***.***> wrote: Just be aware that these two solutions are totally different beasts - whereas with AppImage you get *one file* that runs out of the box on most Linux desktops, the same cannot be said for Flatpak (e.g., last time I tried it did not work on Live systems at all), and you don't get portable apps that you can easily carry around on a USB stick. So why not offer both... — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#15 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AGxqOoTTEhx-hGvxAh0vsXOnX8td8FlKks5shrSvgaJpZM4M-Lw7> .

[ArnCarveris]

Any progress with windows installer?

[russelltg]

A little. I've looked into a few frameworks, but that's about the extent. Sorry for the delay

…

On Fri, Nov 17, 2017 at 4:48 PM Arn ***@***.***> wrote: Any progress with windows installer? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#15 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AGxqOqgja4EVdxcQwwLq2xxaRmmbpz1tks5s3htOgaJpZM4M-Lw7> .

[ArnCarveris]

Take a look, it may will help you

[russelltg]

Alright thanks!

…

On Fri, Nov 17, 2017 at 5:05 PM Arn ***@***.***> wrote: Take a look <https://gitlab.com/osslugaru/lugaru/blob/master/.gitlab-ci.yml>, it may will help you — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#15 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AGxqOibC6YYtI8re6H_F1xHhcODSzxVfks5s3h8zgaJpZM4M-Lw7> .