Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gha: don't build binary packages #9627

Merged
merged 1 commit into from
Jun 19, 2023
Merged

gha: don't build binary packages #9627

merged 1 commit into from
Jun 19, 2023

Conversation

efiop
Copy link
Contributor

@efiop efiop commented Jun 18, 2023

The build process is much more stable these days and there is no reason to keep building these packages here. Instead every package has itsown building process (dvc-osxpkg, dvc-exe, dvc-s3-repo) and knows best how to build and sign correctly.

pyinstaller hooks need to go into a __pyinstaller later on.

The build process is much more stable these days and there is no reason
to keep building these packages here. Instead every package has itsown
building process (dvc-osxpkg, dvc-exe, dvc-s3-repo) and knows best how
to build and sign correctly.

pyinstaller hooks need to go into a `__pyinstaller` later on.
@efiop efiop added build Issues/features related to building dvc install packages. skip-changelog Skips changelog labels Jun 18, 2023
@efiop efiop merged commit 7e1bb07 into iterative:main Jun 19, 2023
@skshetry
Copy link
Member

Let’s get rid of scripts directory altogether.

@efiop
Copy link
Contributor Author

efiop commented Jun 19, 2023

@skshetry That's the plan, but need to move hooks to __pyinstaller first. Nothing urgent, just didn't want to keep patching timeouts and stuff.

@0xpablo
Copy link

0xpablo commented Jun 24, 2023

Hi @efiop, the packages were super useful to us to reduce CI times. On macOS, installing DVC via brew takes over 4 minutes, whereas installing the package takes less than 10 seconds.
Is there any other alternative so that the install process is fast?

I was actually coming here to open a PR to generate the apple silicon package as well and was surprised to see the packages gone.

Thanks in advance!

@efiop
Copy link
Contributor Author

efiop commented Jun 24, 2023

@0xpablo Oh, sorry for the confusion. Those packages were really only meant for testing. We actually build proper packages that are signed and stuff in other repos and they are all available through https://dvc.org/ (see download button, it will give you links like https://dvc.org/download/linux-deb/dvc-3.2.1 that you can use).

The apple package is created in https://github.com/iterative/dvc-osxpkg . Overall, here are the repos:

https://github.com/iterative/dvc-osxpkg
https://github.com/iterative/dvc-exe
https://github.com/iterative/dvc-s3-repo (for deb and rpm)

The first two are private as they have some sensitive keys for signing stuff and I was a bit overly cautious, but since there is demand and you are keen on contributing, I will make them public in a sec (just need to double check I didn't do anything stupid there).

@efiop
Copy link
Contributor Author

efiop commented Jun 24, 2023

Ugh, need a bit more time than a sec (just missing some things on hand) , will make them public in an hour or so.

@0xpablo
Copy link

0xpablo commented Jun 24, 2023

Thanks for the clarification and support @efiop. Once the macOS one is public I’ll see if I can add support for Apple Silicon. In the meantime I’ll switch to the public downloads and use Rosetta.

Thank you!

@efiop
Copy link
Contributor Author

efiop commented Jun 25, 2023

@0xpablo All are public now. Regarding the build process, note that we still (temporarily) use https://github.com/iterative/dvc/blob/main/scripts/pyinstaller/build.py to build the "binary" (it is a dir really) and we plan on getting rid of it and moving that logic to corresponding projects. You will likely need to run pyinstaller there in dvc-osxpkg directly because you'll need to specify --target-architecture in either cmdline args or the spec file. Just wanted to note that that is totally expected and no one will object that.

Also, please feel free to create an issue for supporting arm64 in https://github.com/iterative/dvc-osxpkg/issues

@efiop
Copy link
Contributor Author

efiop commented Jul 3, 2023

@0xpablo If you are still interested in contributing, note that pyinstaller related code is now in dvc-osxpkg https://github.com/iterative/dvc-osxpkg/blob/main/build_bin.py

Also created iterative/dvc-osxpkg#127

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
build Issues/features related to building dvc install packages. skip-changelog Skips changelog
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants