-
-
Notifications
You must be signed in to change notification settings - Fork 56
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
GH-103 - Refactor build script #105
Conversation
this completes a local development story where 'make' will build all three to ./build/bin for whatever platform you run it on by calling through to 'utils/ci.go install' and 'make test' calls through to 'utils/ci.go test'
the xgo-based build targets appear to build inside an xgo container which can fail a build if go.mod and go.sum are not up-to-date: go: writing go.sum: open /ext-go/1/src/go-repo/go.sum747278511.tmp: read-only file system 2019/09/11 15:45:50 Failed to cross compile package: exit status 1. for more detail see techknowlogick/xgo#36
addresses the 'utils/ci.go xgo' part of darcys22GH-100 and darcys22GH-101
@darcys22 just read your comment on #101
maybe that is why the I had some problems also but seem to have worked through them and now seem to have cross-compiled builds working using the I don't yet understand the purpose of |
perhaps with the github actions matrix approach demonstrated here there is room to simplify the makefile and |
Github Actions may solve cross-compilation through a matrix of parallel docker build environments rather than one |
This is awesome! appreciate the work. Wouldn't worry about the stray logfile, it might be the linter but I'll look into that separately. Getting rid of the 'release' directory and moving everything to 'build/bin' is my preference. The I'm not 100% sold on xgo, it was supposed to be useful for other builds such as arm but it seemed like the devs might not be fully behind it anymore. Would be happy to be guided by your thoughts here considering you actually got it working. If using github actions is easier id be for it. Also planning on coming back to this build script in the not too distant future because I want to get debs working with #108. If this works locally for you im okay merging it |
I got xgo working and it's kind of cool in that it spins up a docker container and spits out binaries, but I'm not sure that it's necessary as where I work we do the same thing locally without a docker container using the go native tools and a shell script. this does work locally for me however so we could leave xgo for now and defer a decision to remove it. removing
the pattern we've settled on where I work is actually that we cross-build using a shell script to
another option might be to keep |
Yeah leave xgo for now if its working. Could we change the build scripts it to be ‘/bin/{GOOS}/appname’ same as your work. To be honest is all an improvement over what was previously and dont have too strong of an opinion on the file structure as long as its consistent. |
as I understand it this is more idomatically go especially in a cross-compile scenario, since 'go install' only makes sense for GOOS-native binaries; cross-compiled binaries can be 'built' but not 'installed' on a local machine due to OS/ARCH mismatches
I haven't dropped this, but it is taking me a while because I want to make sure that I don't break the ability to release the same builds which have been released to-date. turns out the I think I understand now why the etherium project uses I enjoy working through this kind of build/release plumbing so happy to contribute. |
the semantics of 'make' will default to build a native version of all the apps into ./build/bin/native for local development and testing; this places the native binaries at a consistent location across environments. another target 'make release' can be used to kick off cross-compilation and prepare the full suite of release artifacts
- introduces a new release process behind a feature flag - utils/make-release.sh should work as before - release_pattern=xgo utils/make-release.sh will use xgo/docker-based builds - the linting cache dir is moved into ./build/.cache/ - xgo-based build-cross targets now build all apps and dump all build output into ./build/dist/{target}/ where {target} is the xgo cross-compilation target - utils/ci.go's 'xgo' subcommand is updated to: - accept only a single xgo compilation target; - build all three apps (godbledger, ledger_cli, reporter) one at a time (xgo doesn't seem to allow building multiple outputs from a single container, though maybe I just haven't found the correct syntax) - rename the binary output to remove the build target suffix (e.g. rename 'reporter-linux-arm-5' to 'reporter')
if it's not set in the environment, assume it is /home/vagrant/go/
made some progress:
release zipfiles are still written to using the makefile:
using
tomorrow I can update the README with some of these notes and instructions on installing docker to use |
This is gold :). The make-release.sh did work previously and would definitely like to keep the one liner command for creating releases when all is finished. If you have a better method than the script after this however im open to changing the process |
if that the only other option I wondered about was using github actions to simply run in any case this definitely meets my needs to run there's one other simplification I can think of around how the version number is handled but that can wait until a second follow-up PR. will update the README shortly. |
using the xgo pattern to cross-compile requires installing a version of Docker Engine to host the xgo build containers these readme updates point the way towards Docker's platform- specific instructions on how to install Docker Engine on various flavours of linux, as well as the docker-for-desktop builds for MacOS and Windows hopefully this can help others get up and running to build and run godbledger code locally and assist with cross-compiling in the future.
Just tested this all locally and works well for me. Very impressed with how its all come together :) :) Unless you have anything else to add I'm ready to merge this |
merge away. I will offer another small idea for discussion in a subsequent PR around how the version number is passed around. |
From #103:
TL;DR I was unable to locate any stray logfiles from running
utils/ci.go test
, and there are 2 make tasks which don't yet go throughutils/ci.go
.Description of changes
While working on this I added
godbledger-darwin
andgodbledger-windows
make targets following the pattern of thegodbledger-linux-*
targets. This partially addresses #100 and #101Here is a screenshot of running
./build/bin/godbledger-darwing-10.6-amd64
(the output from invokingmake godbledger-darwin
) running on MacOS 10.15.6 Catalina:Here is a screenshot of running
./build/bin/godbledger-windows-4.0-amd64.exe
(the output from invokingmake godbledger-windows
) running on a Windows Server 2016 VM:The root makefile now calls utils/ci.go for almost every target. Notable exceptions are:
linux
./release/godbledger-linux-x64/vlatest/godbledger
along withledger_cli
andreporter
in the same folder) instead of the suffixed names in a standardbin
folder generated by thexgo
-based build (e.g../build/bin/godbledger-linux-arm63
and noledger_cli
norreporter
)cert
utils/ci.go
today, though certainly could beI have updated the
make release
target to depend on thegodbledger-linux-arm godbledger-darwin godbledger-windows
targets, which meansmake release
now uses thexgo
docker container to build linux, mac, and windows binaries.At this point I need some feedback/direction.
Request for Feedback
Given the following:
make release
:make
/make all
:make linux
:Which of these patterns do we want to keep and are there any that we can remove?
For instance, simple
./build/bin
output of all three tools makes sense as an easy way to run local code and tests when developing.Unsuffixed output of all three tools in a versioned folder makes sense (like
make linux
) but doesn't seem to be the norm for the set ofxgo
-based make targets.