WIP: Switch to thewtex/centos-build:v1.0.0 base for Slicer compatibility #2

jcfr · 2016-10-04T15:28:35Z

Running slicer using `thewtex/docker-opengl` based of `debian:jessie`

Trying to run the binaries built in slicer/slicer-build image from within the thewtex/docker-opengl fails with the following error:

X Error: BadDrawable (invalid Pixmap or Window parameter) 9
  Major opcode: 62 (X_CopyArea)
  Resource id:  0x800077
X Error: BadAccess (attempt to access private resource denied) 10
  Extension:    130 (MIT-SHM)
  Minor opcode: 1 (X_ShmAttach)
  Resource id:  0x12e

Note: To workaround the error about missing libGLU:

/home/user/work/./bin/SlicerApp-real: error while loading shared libraries: libGLU.so.1: cannot open shared object file: No such file or directory

I apply this patch:

$ git diff
diff --git a/Dockerfile b/Dockerfile
index de109dc..b2af7e4 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -18,7 +18,8 @@ RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y \
   xinit \
   xserver-xorg-video-dummy \
   xserver-xorg-input-void \
-  websockify && \
+  websockify \
+  libglu1-mesa-dev && \
   rm -f /usr/share/applications/x11vnc.desktop && \
   pip install supervisor-stdout && \
   apt-get -y clean

This PR: Update `docker-opengl` to use `centos-build:v1.0.0` as a base

What motivates this experiment is to find way of running test associated with SlicerDocker build without having to rebuild Slicer from scratch just for testing ...

With this set of changes, I was able to interact with Slicer built by https://github.com/thewtex/SlicerDocker

cd /tmp/Slicer-4.5.0-0000-00-00-linux-amd64
~/Projects/docker-opengl/run.sh -r --env="APP=/home/user/work/Slicer"

Note: To work around the missing QtTest lib, in addition of copying the linux package, I add to manually copy the file:

docker cp e82e94fa57a2:/usr/src/Slicer-build/Slicer-build/Slicer-4.5.0-0000-00-00-linux-amd64.tar.gz /tmp/Slicer-4.5.0-0000-00-00-linux-amd64.tar.gz
cd tmp
aunpack Slicer-4.5.0-0000-00-00-linux-amd64.tar.gz
cd Slicer-4.5.0-0000-00-00-linux-amd64
docker cp e82e94fa57a2:/usr/src/qt-everywhere-opensource-release-src-4.8.7/lib/libQtTest.so.4.8.7 ./lib/Slicer-4.5/libQtTest.so.4.8.7
(cd ./lib/Slicer-4.5 && \
ln -s libQtTest.so.4.8.7 libQtTest.so.4.8 && \
ln -s libQtTest.so.4.8.7 libQtTest.so.4 && \
ln -s libQtTest.so.4.8.7 libQtTest.so)

Moving forward:

the version of X used in centos-build:v1.0.0 is ancient, we could build a newer version in docker-opengl, that should help to address performance issue related to x11vnc and may be it could help get rid of http://www.karlrunge.com/x11vnc/Xdummy

jcfr · 2016-10-04T15:29:28Z

Cc: @MayeulChassagnard

thewtex · 2016-10-04T17:36:45Z

👍 Thanks for collaborating on this.

@MayeulChassagnard @jcfr please let me know when further testing / experiments are completed and this is ready for integration.

MayeulChassagnard · 2016-10-05T15:21:52Z

I am currently trying to build the tests inside the slicer/slicer-build docker image because we need them for the slicer/opengl (on centos) docker image.

We can notify that we were not already building test : -DBUILD_TESTING:BOOL=OFF

we made few changes:


--------------------------- slicer-build/Dockerfile ---------------------------
index 777fd50..5ebacb3 100644
@@ -2,7 +2,7 @@ FROM slicer/slicer-build-base:latest
 MAINTAINER 3D Slicer Community <[email protected]>

 RUN cd /usr/src/Slicer-build && \
-  cmake -DBUILD_TESTING:BOOL=OFF .
+  cmake -DBUILD_TESTING:BOOL=ON .

 RUN /usr/src/Slicer-build/BuildSlicer.sh && \
   cd /usr/src/Slicer-build/Slicer-build && \

---------------------------- slicer-build/build.sh ----------------------------
index 5992dba..857d710 100755
 @@ -2,7 +2,7 @@

 script_dir="`cd $(dirname $0); pwd`"

-image=slicer/slicer-build
+image=slicer/slicer-build:BUILD_TESTING_ON

 docker build -t $image \
     --build-arg IMAGE=$image \

And tried to build the image with bash build.sh
But we still have some issues to build the docker image:

Moreover, it's not link with the changes

jcfr · 2016-10-05T15:44:41Z

As discussed, I suspect the error happen when building with the new VTK backend.

To facilitate the debugging, consider mounting source and build directory within the image ... that would speed the development cycle and allow to address error more quickly.

MayeulChassagnard · 2016-10-05T20:45:11Z

It seems that the centos version of thewtex/opengl named slicer/opengl can't run the tests...

To reproduce the error:

docker run -it slicer/slicer-build:latest bash
cmake -DBUILD_TESTING:BOOL=ON .
ninja -j8
tests are now built (to verify it, you can do ctest -N)
Create an archive of the superbuild: in /usr/src# tar -zcvf Slicer-build-with-test-archive.tar.gz Slicer-build/
copy it in your file system: docker cp 1224c1b3d08f:/usr/src/Slicer-build-with-test-archive.tar.gz /home/mayeul/MyProjects/Docker/Slicer-build-with-test-archive.tar.gz
run the centos version of thewtex/opengl: docker run -it slicer/opengl bash
copy back the archive in the centos version: docker cp /home/mayeul/MyProjects/Docker/Slicer-build-with-test-archive.tar.gz dcee9e09a046:/usr/src/Slicer-build-with-test-archive.tar.gz
Unzip the archive: tar -zxvf Slicer-build-with-test-archive.tar.gz
in /usr/src/Slicer-build/Slicer-build# /usr/local/bin/supervisord -c /etc/supervisor/supervisord.conf &
ctest -R qSlicerWidgetTest2 -V

jcfr · 2016-10-05T20:47:24Z

Archiving only the superbuild folder is not enough, the qt libraries need to be available too.

MayeulChassagnard · 2016-10-05T21:51:18Z

After adding the

qt-everywhere-opensource-release-src-4.8.7 (this one is maybe not required because I had to add install-prefix anyway to make it works)
install-prefix folder

to the slicer/opgenl (centos version) image, the test is now indeed passing but with some errors due to:

/usr/src/Slicer-build/Slicer-build/Base/QTGUI/Testing/Cxx/share/MRML/Testing/TestData/fixed.nrrd does not exist.

jcfr · 2016-10-05T22:05:03Z

Two things:

test data doesn't seem avaialble
the BadDrawable/BadAccess errors were the one I got with the debian based docker-opengl, it is strange you also got them with the centos based docker-opengl. do you also see these error if you simply start Slicer using the run.sh script ?

MayeulChassagnard · 2016-10-05T22:12:51Z

There is absolutely no errors on the thewtex/opengl image when I do:

docker run -it thewtex/opengl:already_built bash
in usr/src/Slicer-build/Slicer-build# /usr/bin/supervisord -c /etc/supervisor/supervisord.conf & ctest -R qSlicerWidgetTest2 -V

MayeulChassagnard · 2016-10-05T22:18:40Z

oops my bad, I did a mistake and spoke to fast...

jcfr · 2016-10-05T22:20:15Z

Look like the BadDrawable/BadAccess are not reported here.

That said, I am not sure what is thewtex/opengl:already_built , is it debian or centos based, did you mount Slicer into it or is it built within the image ?

MayeulChassagnard · 2016-10-05T22:25:11Z

The thewtex/opengl:already_built image is a saved image (docker commit) from the thewtex/opengl:v0.1.0 based on debian:jessie where I directly built from scratch Slicer

jcfr · 2016-10-05T22:40:22Z

Ok.

Could you look into the BadDrawable/BadAccess errors you got, do you confirm that there were reported after trying to run the test using docker-opengl based on centos to run tests also built using centos ?

MayeulChassagnard · 2016-10-06T14:22:47Z

I am not sure to have well understood what you are asking but I :

ran the tests (built on centos) in docker-opengl based on centos
ran the tests (built on debian) in docker-opengl based on debian

both are running errors as you have seen in the pictures

But it didn't seem to me that there was errors the first time I tried (one week ago) on docker-opengl based on debian... However, I changed nothing on this image so it should be the same message as the first time

MayeulChassagnard · 2016-10-13T18:48:59Z

WIP here on an image based on centOS 5 (same os than the slicer build images) with a newer version of the X Window System (Xorg x11).

It'll allow us to run the slicer's tests without rebuilding all from scratch.

thewtex · 2016-10-13T18:53:37Z

@MayeulChassagnard looking good! Do you have a Slicer Experimental dashboard build that can be shared?

MayeulChassagnard · 2016-10-17T18:31:03Z

@jcfr @thewtex Here you can find the Experimental Dashboard where I run the Slicer's tests

For now (on the centos build docker image without the full update of x11) we have the following dashboard

thewtex · 2016-10-17T19:14:46Z

@MayeulChassagnard Good job!

It looks like it is picking up an old version of CMake?

CMake Error at /usr/src/Slicer/CMake/ListToString.cmake:21 (cmake_minimum_required):
  CMake 2.8.9 or higher is required.  You are running version 2.6.4

MayeulChassagnard · 2016-10-17T20:04:23Z

Yes, I saw it, it should be fixed now (It was a cmake in the yum update && yum install which was left and was deleting the cmake v3.4.1 previously installed).

Thanks a lot for the review though!

MayeulChassagnard · 2016-10-19T19:36:33Z

Status update:

able to mount volume between slicer/slicer-build (the image which should build all including the tests) & slicer/test-opengl (the image created to run the tests located in slicer based on centOS which include the X window system x11 [named centos/opengl:slicer-testing-image])
Still a problem in centos/opengl:slicer-testing-image with lib/libx11 during the installation of the new version of x11 aimed to replace the Hack. But it doesn't bother the fact that the test are passing though.
Still a problem with the update of the slicer/slicer-build image which can't be built
About the Continuous Integration for Slicer, when do we need to run the tests? Is the circle.yml file should call run.sh to launch the tests? I think so, and then I added in circle.yml the last line:
- bash ./CMake/CircleCI/run.sh $CIRCLE_SHA1 $CIRCLE_BRANCH $SITE_BUILD_TYPE

test:
  override:
    - "echo 'Notice: CircleCI does not build changes to Slicer dependencies' && if git diff --name-only master | grep -q SuperBuild > /dev/null; then false; fi"
    - docker run -e "BUILD_TOOL_FLAGS=-j5" --name slicer -v ~/Slicer:/usr/src/Slicer slicer/slicer-build-deps
    - docker cp slicer:$(docker cp slicer:/usr/src/Slicer-build/Slicer-build/PACKAGE_FILE.txt - | tar xO) $CIRCLE_ARTIFACTS
    - bash ./CMake/CircleCI/run.sh $CIRCLE_SHA1 $CIRCLE_BRANCH $SITE_BUILD_TYPE

which should work if the slicer/slicer-build contains the tests...

thewtex · 2016-10-21T17:51:19Z

build.sh

@@ -2,4 +2,4 @@

 script_dir="`cd $(dirname $0); pwd`"

-docker build -t thewtex/opengl $script_dir
+docker build -t slicer/opengl $script_dir


Please retain the current default. Also, please rebase this branch on master, which has parameterized Makefile build support.

@thewtex If we retain the current default thewtex/opengl there will be 2 same name for 2 different docker image.. I suggest to rename it into thewtex/centos-opengl or centos/opengl

Consider using different tag. E.g thewtex/opengl:jessie-1.0.0, thewtex/opengl:centos-1.0.0, ...

thewtex · 2016-10-21T17:51:48Z

etc/skel/.xinitrc

@@ -1,5 +1,5 @@
 #!/bin/bash

 xsetroot -solid "#333333"
-(sleep 2s && tint2 -c /etc/xdg/tint2/tint2rc) &
+#(sleep 2s && tint2 -c /etc/xdg/tint2/tint2rc) &


Why was this removed?

That was a change made by @jcfr to be able to use X11 on centos.
I think I did some bad manipulation with the commit because the commit he made doesn't appear.. I'm going to fix everything.

MayeulChassagnard · 2016-10-24T17:55:27Z

Updated & Rebased here #11

WIP: Switch to thewtex/centos-build:v1.0.0 base for Slicer compatibility

14949bd

thewtex reviewed Oct 21, 2016

View reviewed changes

thewtex requested changes Oct 21, 2016

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

WIP: Switch to thewtex/centos-build:v1.0.0 base for Slicer compatibility #2

WIP: Switch to thewtex/centos-build:v1.0.0 base for Slicer compatibility #2

jcfr commented Oct 4, 2016 •

edited

Loading

jcfr commented Oct 4, 2016

thewtex commented Oct 4, 2016

MayeulChassagnard commented Oct 5, 2016

jcfr commented Oct 5, 2016

MayeulChassagnard commented Oct 5, 2016

jcfr commented Oct 5, 2016

MayeulChassagnard commented Oct 5, 2016 •

edited

Loading

jcfr commented Oct 5, 2016

MayeulChassagnard commented Oct 5, 2016

MayeulChassagnard commented Oct 5, 2016

jcfr commented Oct 5, 2016

MayeulChassagnard commented Oct 5, 2016 •

edited

Loading

jcfr commented Oct 5, 2016

MayeulChassagnard commented Oct 6, 2016

MayeulChassagnard commented Oct 13, 2016

thewtex commented Oct 13, 2016

MayeulChassagnard commented Oct 17, 2016

thewtex commented Oct 17, 2016

MayeulChassagnard commented Oct 17, 2016

MayeulChassagnard commented Oct 19, 2016

thewtex Oct 21, 2016

MayeulChassagnard Oct 24, 2016

jcfr Oct 24, 2016

thewtex Oct 24, 2016

thewtex Oct 21, 2016

MayeulChassagnard Oct 21, 2016

MayeulChassagnard commented Oct 24, 2016

WIP: Switch to thewtex/centos-build:v1.0.0 base for Slicer compatibility #2

Are you sure you want to change the base?

WIP: Switch to thewtex/centos-build:v1.0.0 base for Slicer compatibility #2

Conversation

jcfr commented Oct 4, 2016 • edited Loading

Running slicer using thewtex/docker-opengl based of debian:jessie

This PR: Update docker-opengl to use centos-build:v1.0.0 as a base

jcfr commented Oct 4, 2016

thewtex commented Oct 4, 2016

MayeulChassagnard commented Oct 5, 2016

jcfr commented Oct 5, 2016

MayeulChassagnard commented Oct 5, 2016

jcfr commented Oct 5, 2016

MayeulChassagnard commented Oct 5, 2016 • edited Loading

jcfr commented Oct 5, 2016

MayeulChassagnard commented Oct 5, 2016

MayeulChassagnard commented Oct 5, 2016

jcfr commented Oct 5, 2016

MayeulChassagnard commented Oct 5, 2016 • edited Loading

jcfr commented Oct 5, 2016

MayeulChassagnard commented Oct 6, 2016

MayeulChassagnard commented Oct 13, 2016

thewtex commented Oct 13, 2016

MayeulChassagnard commented Oct 17, 2016

thewtex commented Oct 17, 2016

MayeulChassagnard commented Oct 17, 2016

MayeulChassagnard commented Oct 19, 2016

thewtex Oct 21, 2016

Choose a reason for hiding this comment

MayeulChassagnard Oct 24, 2016

Choose a reason for hiding this comment

jcfr Oct 24, 2016

Choose a reason for hiding this comment

thewtex Oct 24, 2016

Choose a reason for hiding this comment

thewtex Oct 21, 2016

Choose a reason for hiding this comment

MayeulChassagnard Oct 21, 2016

Choose a reason for hiding this comment

MayeulChassagnard commented Oct 24, 2016

jcfr commented Oct 4, 2016 •

edited

Loading

Running slicer using `thewtex/docker-opengl` based of `debian:jessie`

This PR: Update `docker-opengl` to use `centos-build:v1.0.0` as a base

MayeulChassagnard commented Oct 5, 2016 •

edited

Loading

MayeulChassagnard commented Oct 5, 2016 •

edited

Loading