Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add support for sudo in setup scripts #8943

Closed
wants to merge 2 commits into from
Closed

Add support for sudo in setup scripts #8943

wants to merge 2 commits into from

Conversation

majetideepak
Copy link
Collaborator

@majetideepak majetideepak commented Mar 4, 2024 •
edited
Loading

Ubuntu requires sudo to install at the standard path.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Mar 4, 2024
@netlify Netlify
Copy link

netlify bot commented Mar 4, 2024 •
edited
Loading

✅ Deploy Preview for meta-velox canceled.

Name Link
🔨 Latest commit 5c9abcd
🔍 Latest deploy log https://app.netlify.com/sites/meta-velox/deploys/65e8cb779086c700081603ec

@majetideepak majetideepak force-pushed the fix-ubuntu branch 4 times, most recently from 6c15013 to f3ce37a Compare March 4, 2024 10:05
@majetideepak majetideepak mentioned this pull request Mar 5, 2024
Closed
@majetideepak majetideepak force-pushed the fix-ubuntu branch 2 times, most recently from 640ae8e to f780053 Compare March 5, 2024 07:33
@majetideepak majetideepak marked this pull request as ready for review March 5, 2024 07:33
@majetideepak majetideepak requested review from assignUser and kgpai March 5, 2024 07:34
@majetideepak majetideepak changed the title Fix Ubuntu dependency install in github actions Add support for sudo in setup scripts Mar 5, 2024
kgpai
kgpai reviewed Mar 5, 2024
View reviewed changes
Copy link
Contributor

@kgpai kgpai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, curious why we are having the problem though.

scripts/setup-ubuntu.sh Outdated
@@ -29,6 +29,7 @@ BOOST_VERSION=boost-1.84.0
NPROC=$(getconf _NPROCESSORS_ONLN)
DEPENDENCY_DIR=${DEPENDENCY_DIR:-$(pwd)}
export CMAKE_BUILD_TYPE=Release
SUDO=sudo
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you know why we need sudo now to install ? Any idea why was this not a problem earlier ?

Copy link
Collaborator Author

@majetideepak majetideepak Mar 6, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kgpai We source the ubuntu setup script in github workflows. Sourcing does not install the folly, boost packages.
So we did not see this problem until now.
My internal setup uses root, so I did not see this problem there as well.
But you will see that the system installs in the ubuntu script use sudo. So this change is required.
After this change, we need to change the invocation in the workflows to
run: source ./scripts/setup-ubuntu.sh && install_velox_deps
or
run: ./scripts/setup-ubuntu.sh
CC: @czentgr

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks , makes sense !

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I think we have to export SUDO.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need sudo in the yml file. We must land the sudo change to main first so that the base (main) branch has the sudo fix.

@kgpai
Copy link
Contributor

kgpai commented Mar 5, 2024

@majetideepak I am triggering the nightly workflows with my push to validate that this fixes it. Will revert that change if it works..

@kgpai kgpai force-pushed the fix-ubuntu branch 3 times, most recently from 46cb282 to 48e0692 Compare March 6, 2024 00:16
@majetideepak
Copy link
Collaborator Author

I am triggering the nightly workflows with my push to validate that this fixes it. Will revert that change if it works..

@kgpai We need to fix the workflow command

run: source ./scripts/setup-ubuntu.sh && install_velox_deps
or
run: ./scripts/setup-ubuntu.sh

@majetideepak majetideepak mentioned this pull request Mar 6, 2024
Closed
@kgpai kgpai force-pushed the fix-ubuntu branch 4 times, most recently from ee665ca to 65c1d4e Compare March 6, 2024 06:13
@majetideepak
Copy link
Collaborator Author

@kgpai I don't think we need sudo in the yml files. The sudo changes must land first since the CI benchmarks builds off the base (main) branch first.

@majetideepak
Copy link
Collaborator Author

Torch arrow CI is failing since the setup file has been removed here #8961

@facebook-github-bot
Copy link
Contributor

@kgpai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@kgpai
Copy link
Contributor

kgpai commented Mar 6, 2024

@majetideepak Atleast the sudo for compile step is required..

@facebook-github-bot
Copy link
Contributor

@kgpai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@kgpai
Copy link
Contributor

kgpai commented Mar 6, 2024

@majetideepak Looks like all the sudo's are required : https://github.com/facebookincubator/velox/actions/runs/8177028735/job/22360517904?pr=8943

@facebook-github-bot
Copy link
Contributor

@kgpai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@facebook-github-bot
Copy link
Contributor

@kgpai merged this pull request in 3e5d340.

@PHILO-HE
Copy link
Contributor

PHILO-HE commented Mar 7, 2024

Hi @majetideepak, do we need to also add sudo for boost (see below link)? In our environment, writing data to /usr/local fails during building boost.
https://github.com/majetideepak/velox/blob/5c9abcdd468aa41d0e669b2d4789dbda57027151/scripts/setup-ubuntu.sh#L79

@majetideepak
Copy link
Collaborator Author

@PHILO-HE, yes we do. I am fixing it here https://github.com/facebookincubator/velox/pull/8988/files
But you can add sudo to the script invocation for now.
sudo ./velox/scripts/setup-ubuntu.sh

@PHILO-HE
Copy link
Contributor

PHILO-HE commented Mar 7, 2024

@PHILO-HE, yes we do. I am fixing it here https://github.com/facebookincubator/velox/pull/8988/files But you can add sudo to the script invocation for now. sudo ./velox/scripts/setup-ubuntu.sh

Great! Thanks!

@majetideepak majetideepak mentioned this pull request Mar 18, 2024
Closed
@majetideepak majetideepak deleted the fix-ubuntu branch March 20, 2024 20:48
Joe-Abraham pushed a commit to Joe-Abraham/velox that referenced this pull request Jun 7, 2024
@majetideepak @Joe-Abraham
Add support for sudo in setup scripts (facebookincubator#8943)
8557d4d 
Summary:
Ubuntu requires sudo to install at the standard path.

Pull Request resolved: facebookincubator#8943

Reviewed By: Yuhta

Differential Revision: D54593906

Pulled By: kgpai

fbshipit-source-id: cb7ee939d3fdf2675531ce5852c541b89519e07d
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Yuhta Yuhta Yuhta approved these changes

@kgpai kgpai kgpai approved these changes

@assignUser assignUser Awaiting requested review from assignUser assignUser is a code owner

Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. Merged
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@majetideepak @kgpai @facebook-github-bot @PHILO-HE @Yuhta