Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Apt - Minimum Viable Buildpack - 03 - Installed Packages Layer #7

Merged
merged 1 commit into from
Mar 13, 2024

Conversation

colincasey
Copy link
Contributor

@colincasey colincasey commented Feb 21, 2024

The Debian packages that we download need to be extracted into a target folder. This folder can be cached and several environment variables will refer to this location so that downloaded libraries and tools can be made available to subsequent buildpacks. So, for this, we'll introduce two layers:

  • InstalledPackagesLayer - the cacheable folder where all packages will be installed.
  • EnvironmentLayer - a layer which sets the environment variables pointing to the installed packages which is always recreated.

Caching

Caching for the InstalledPackagesLayer is controlled by two values:

  • the contents of the Aptfile (parsed, not raw)
  • the os and arch of the system the buildpack is executing in

Environment Variables

The EnvironmentLayer prepends the following paths to their respective environment variables:

  • PATH
    • <layer_dir>/bin
    • <layer_dir>/usr/bin
    • <layer_dir>/usr/sbin
  • LD_LIBRARY_PATH / LIBARY_PATH
    • <layer_dir>/usr/lib/x86_64-linux-gnu (if the arch is amd64)
    • <layer_dir>/usr/lib
    • <layer_dir>/lib/x86_64-linux-gnu (if the arch is amd64)
    • <layer_dir>/lib
  • INCLUDE_PATH / CPATH / CPPPATH
    • <layer_dir>/usr/include/x86_64-linux-gnu (if the arch is amd64)
    • <layer_dir>/usr/include
  • PKG_CONFIG_PATH
    • <layer_dir>/usr/lib/x86_64-linux-gnu/pkgconfig (if the arch is amd64)
    • <layer_dir>/usr/lib/pkgconfig

@colincasey colincasey added the skip changelog Skip the check-changelog check label Feb 21, 2024
@colincasey colincasey self-assigned this Feb 21, 2024
@colincasey colincasey force-pushed the apt_mvbp/03-installed-packages-layer-v2 branch from b87ecab to 4563fe7 Compare February 21, 2024 19:53
@edmorley
Copy link
Member

In case it helps you plan - I'm not going to have time to review this today unfortunately. I might be able to tomorrow. If you were otherwise blocked and didn't have something to work on - one of the questions I'll have during the review (either to ask or to look into myself), is whether there are any parts of the deb downloading/extracting strategy from the classic APT buildpack that can be done a more elegant way (ie: not using dpkg) - eg researching if in the last N years apt has added any more install options we could use etc. Have you had a chance to do any research in that area?

@colincasey
Copy link
Contributor Author

@edmorley so the situation with apt and dpkg doesn't seem to have improved at all when it comes to non-admin installation of packages. There are alternative package managers which may be worth investigating (such as Nix) but I considered those alternatives to be outside the scope of this buildpack work.

@colincasey colincasey force-pushed the apt_mvbp/03-installed-packages-layer-v2 branch from c244ebb to 8fb2955 Compare February 26, 2024 20:32
Copy link

@schneems schneems left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a ton for working on this. I made some suggestions, comments and requests. See individual comments for more.

src/layers/installed_packages.rs Show resolved Hide resolved
buildpack.toml Show resolved Hide resolved
src/debian.rs Outdated Show resolved Hide resolved
src/debian.rs Show resolved Hide resolved
src/main.rs Outdated Show resolved Hide resolved
src/main.rs Outdated Show resolved Hide resolved
src/main.rs Show resolved Hide resolved
src/main.rs Outdated Show resolved Hide resolved
src/layers/installed_packages.rs Outdated Show resolved Hide resolved
src/layers/installed_packages.rs Outdated Show resolved Hide resolved
@edmorley edmorley self-requested a review February 27, 2024 22:53
@colincasey colincasey requested a review from schneems February 28, 2024 20:02
Copy link

@schneems schneems left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few real quick suggestions. I'm not done reviewing the new commit

src/aptfile.rs Show resolved Hide resolved
src/debian.rs Show resolved Hide resolved
src/layers/installed_packages.rs Show resolved Hide resolved
@colincasey colincasey force-pushed the apt_mvbp/03-installed-packages-layer-v2 branch from 292e640 to ec0777d Compare March 4, 2024 15:40
Copy link

@schneems schneems left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for incorporating prior changes. I think this looks good. I've moved over to 04

@colincasey colincasey merged commit c440d31 into main Mar 13, 2024
5 checks passed
@colincasey colincasey deleted the apt_mvbp/03-installed-packages-layer-v2 branch March 13, 2024 14:53
@edmorley
Copy link
Member

I hadn't reviewed this :-(

@colincasey
Copy link
Contributor Author

Sorry, I didn't realize I was still waiting on you here @edmorley.

@edmorley edmorley removed their request for review July 15, 2024 21:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
skip changelog Skip the check-changelog check
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants