Gitian unit e #511

Ruteri · 2019-01-30T15:33:13Z

Bumped minimum OS requirements to Debian 8 and Ubuntu 14 (affects builds with --enable-glibc-back-compat)
Reset release name to 0.0.1 (downgrade the client version in separate PR)
Made windows setup honour --with-gui=no
Various improvements to gitian-build.py script:
- Made the script MacOS-friendly
- Made --setup a standalone target (can be ran by itself without passing signer or version)
- Made --setup honour --url option
Renamed repo with signatures to unit-e-sigs
Removed separation for detached signatures (MacOS and Windows)

Still to do:

Verify MacOS build
Automate signing process for Windowss
Unify detached signatures behaviour (they are right now posted on separate branches of the sigs repo)

Addresses #466

Signed-off-by: Mateusz Morusiewicz <[email protected]>

contrib/gitian-build.py

Makefile.am

contrib/gitian-build.py

share/setup.nsi.in

cornelius

This generally looks good. It corresponds to the earlier version I have tested. I didn't test this pull request by running builds, but I think it's good enough to be merged and we can build on it with more changes. Especially as the current version in the repo doesn't work for unit-e it would be good to get to a start of a working version.

So utACK from me.

Signed-off-by: Mateusz Morusiewicz <[email protected]>

scravy · 2019-01-31T09:54:23Z

contrib/devtools/symbol-check.py

@@ -17,30 +17,30 @@
 import sys
 import os

-# Debian 6.0.9 (Squeeze) has:
+# Debian 8.11 (jessie) has:


I am completely fine with the bump of version requirements. ACK this one in particular.

contrib/devtools/symbol-check.py

scravy · 2019-01-31T09:56:01Z

contrib/gitian-build.py

    if args.kvm:
-        programs += ['python-vm-builder', 'qemu-kvm', 'qemu-utils']
+        programs += ['apt-cacher-ng', 'python-vm-builder', 'qemu-kvm', 'qemu-utils']


What does apt-cacher-ng do?

It's required for lxc and kvm, but not for docker (hence the change, notice it was removed from the always-installed stuff).

But what does it do?

It creates a local cache for apt from within the containers (makes it work way faster). I'd leave it be.

contrib/gitian-build.py

scravy · 2019-01-31T09:59:41Z

contrib/gitian-descriptors/gitian-linux.yml

@@ -28,14 +28,14 @@ packages:
 - "ca-certificates"
 - "python"
 remotes:
- "url": "https://github.com/unite/unite.git"
-  "dir": "unite"
+- "url": "[email protected]:dtr-org/unit-e.git"


maybe these should be added as a special case to the clone machine

I think we don't need it as we will be public soon anyway (and then we will use the https), and this url is at the moment overwritten in the script anyway (it's there in case you want to run gitian build manually).

I think we have a way of just retaining some documents verbatim in the clone machine, called "appropriated files" ( https://github.com/dtr-org/clonemachine/blob/master/fork.py#L69 ). Maybe the gitian build script should be part of these?

@cornelius WDYT?

Yes, I think we should "appropriate" the gitian build script. It still might make sense to cover the GitHub URL as special case. I think I have seen this handled wrongly in clonemachine before in other places as well.

scravy · 2019-01-31T09:59:59Z

contrib/gitian-descriptors/gitian-linux.yml

 files: []
 script: |

  WRAP_DIR=$HOME/wrapped
  HOSTS="i686-pc-linux-gnu x86_64-linux-gnu arm-linux-gnueabihf aarch64-linux-gnu"
-  CONFIGFLAGS="--enable-glibc-back-compat --enable-reduce-exports --disable-bench --disable-gui-tests"
+  CONFIGFLAGS="--enable-glibc-back-compat --enable-reduce-exports --disable-bench --disable-gui-tests --with-gui=no"


completely ACK this change :-)

scravy · 2019-01-31T10:00:57Z

contrib/gitian-descriptors/gitian-win.yml

@@ -72,7 +72,7 @@ script: |

  function create_per-host_linker_wrapper {
  # This is only needed for trusty, as the mingw linker leaks a few bytes of
-  # heap, causing non-determinism. See discussion in https://github.com/unite/unite/pull/6900
+  # heap, causing non-determinism. See discussion in https://github.com/bitcoin/bitcoin/pull/6900


also something which might be worthwhile to be added to the clonemachine as an exception

Here I agree.

Yeah, this is the kind of case I was thinking about. I'll create an issue for clonemachine to keep track of this.

castarco · 2019-01-31T10:20:32Z

contrib/gitian-build.py

+import platform
+
+def install_linux_deps():
+    global args


I know that it was like that before, but I think it's a bad idea to use global here, when args could be passed as a param from the function install_deps.

I actually like the args global, as it'd have to be passed to every function.
Also, this PR is mostly about making the script work for Unit-e, we have more refactoring coming up.

I have a patch in the works to get rid of the global variables. Will be in a separate pull request.

Just for the record, there are plenty of reasons why using global is usually a bad idea:

It's more difficult to know where these variables are accessed and modified.

It's easier to introduce subtle bugs because we could have unexpected read/write accesses to some values.

It's more difficult to reason about types

It's more difficult to reason about an object's evolution.

The IDEs have similar problems, so their introspection capabilities become diminished.

It's more difficult to do proper unit testing (I know that this does not apply here because testing is difficult for other reasons too) as everything becomes entangled and side-effects are ubiquitous.

It's more difficult to refactor code.

And there's an extra line that could be avoided by just passing a parameter.

contrib/gitian-build.py

scravy

utACK 5ee0ce7

I like the version bumps, and as we discussed, I do think they are completely appropriate. We might want to add some things to the clone machine, but that's outside the scope of this pull request.

I do lack the time to give that some thorough testing, which I would have liked to do simply to build up the experience with gitian. Some other time :-)

Ruteri added build Build system tools Development tools labels Jan 30, 2019

Ruteri requested review from cornelius, thothd and a team January 30, 2019 15:33

Ruteri added 7 commits January 30, 2019 16:33

WIP: adapted gitian to Unit-e

d29bf84

Signed-off-by: Mateusz Morusiewicz <[email protected]>

Added option to build windows without gui

e503e3e

Signed-off-by: Mateusz Morusiewicz <[email protected]>

Nsis if macro instead of a function

f8a2a75

Signed-off-by: Mateusz Morusiewicz <[email protected]>

Detached signatures rework test

edcc40b

Signed-off-by: Mateusz Morusiewicz <[email protected]>

Set version in gitian descriptors to 0.0.1

02a6a17

Signed-off-by: Mateusz Morusiewicz <[email protected]>

Bumped minimal OS versions

b8cfd23

Signed-off-by: Mateusz Morusiewicz <[email protected]>

Fixed nsi generation without gui

d2a2557

Signed-off-by: Mateusz Morusiewicz <[email protected]>

Ruteri force-pushed the gitian-unit-e branch from 93221e6 to d2a2557 Compare January 30, 2019 15:33

cmihai reviewed Jan 30, 2019

View reviewed changes

contrib/gitian-build.py Outdated Show resolved Hide resolved

cmihai reviewed Jan 30, 2019

View reviewed changes

contrib/gitian-build.py Show resolved Hide resolved

cmihai reviewed Jan 30, 2019

View reviewed changes

contrib/gitian-build.py Show resolved Hide resolved

frolosofsky reviewed Jan 30, 2019

View reviewed changes

Makefile.am Outdated Show resolved Hide resolved

contrib/gitian-build.py Outdated Show resolved Hide resolved

contrib/gitian-build.py Show resolved Hide resolved

share/setup.nsi.in Show resolved Hide resolved

cornelius approved these changes Jan 30, 2019

View reviewed changes

PR fixes, made version and signer regular options

425b968

Signed-off-by: Mateusz Morusiewicz <[email protected]>

Ruteri force-pushed the gitian-unit-e branch from f7d999d to 425b968 Compare January 30, 2019 17:12

Changed order of docker packages on installing

5ee0ce7

Signed-off-by: Mateusz Morusiewicz <[email protected]>

scravy reviewed Jan 31, 2019

View reviewed changes

castarco reviewed Jan 31, 2019

View reviewed changes

contrib/gitian-build.py Show resolved Hide resolved

castarco reviewed Jan 31, 2019

View reviewed changes

contrib/gitian-build.py Show resolved Hide resolved

castarco reviewed Jan 31, 2019

View reviewed changes

contrib/gitian-build.py Show resolved Hide resolved

scravy approved these changes Jan 31, 2019

View reviewed changes

cornelius mentioned this pull request Jan 31, 2019

Handle repository URLs dtr-org/unit-e-clonemachine#23

Closed

Ruteri merged commit f3c9480 into dtr-org:master Jan 31, 2019

Ruteri mentioned this pull request Jan 31, 2019

Reproducible builds of unit-e #466

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Gitian unit e #511

Gitian unit e #511

Ruteri commented Jan 30, 2019 •

edited

Loading

cornelius left a comment

scravy Jan 31, 2019

scravy Jan 31, 2019

Ruteri Jan 31, 2019

scravy Jan 31, 2019

Ruteri Jan 31, 2019

scravy Jan 31, 2019

Ruteri Jan 31, 2019

scravy Jan 31, 2019

cornelius Jan 31, 2019

scravy Jan 31, 2019

scravy Jan 31, 2019

Ruteri Jan 31, 2019 •

edited

Loading

cornelius Jan 31, 2019

castarco Jan 31, 2019

Ruteri Jan 31, 2019

cornelius Jan 31, 2019

castarco Jan 31, 2019

scravy left a comment

Gitian unit e #511

Gitian unit e #511

Conversation

Ruteri commented Jan 30, 2019 • edited Loading

cornelius left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Ruteri Jan 31, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

scravy left a comment

Choose a reason for hiding this comment

Ruteri commented Jan 30, 2019 •

edited

Loading

Ruteri Jan 31, 2019 •

edited

Loading