Skip to content

CLOUDP-324668 - Optimized mongodb-agent docker layers #191

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 1 commit into
base: master
Choose a base branch
from
Draft

CLOUDP-324668 - Optimized mongodb-agent docker layers #191

wants to merge 1 commit into from

Conversation

lsierant
Copy link
Contributor

@lsierant lsierant commented Jun 10, 2025 •
edited
Loading

Summary

This is a PoC of building agent's docker images with a better layer packing to avoid unnecessarily duplicating binaries/archives there.

It works simply by batching all filesystem manipulations into one RUN and moving some .tgz unpacking into the previous build stages.

What's left:

  • Checking agent images built for static architecture
  • Checking agent images built for AppDB and MongoDBCommunity (we don't build them as part of normal patches, only on bumps)

Proof of Work

Before (dive):

$ dive quay.io/mongodb/mongodb-agent-ubi:13.35.0.9498-1_1.1.0
[...]
Image name: quay.io/mongodb/mongodb-agent-ubi:13.35.0.9498-1_1.1.0
Total Image size: 1.1 GB
Potential wasted space: 568 MB
Image efficiency score: 51 %

After (dive):

$ dive 268558157000.dkr.ecr.us-east-1.amazonaws.com/dev/mongodb-agent-ubi:13.35.0.9498-1_684be130a0378600072c885c
Image name: 268558157000.dkr.ecr.us-east-1.amazonaws.com/dev/mongodb-agent-ubi:13.35.0.9498-1_684be130a0378600072c885c  
Total Image size: 581 MB                                                                                                
Potential wasted space: 33 MB                                                                                           
Image efficiency score: 96 %

Before:
For the agents used in static arch we have the following image before the change:

$ docker inspect quay.io/mongodb/mongodb-agent-ubi:13.35.0.9498-1_1.1.0 | jq '.[0].Size' | numfmt --to=si
1.2G

$ docker history quay.io/mongodb/mongodb-agent-ubi:13.35.0.9498-1_1.1.0
IMAGE          CREATED       CREATED BY                                      SIZE      COMMENT
de7b13198c63   8 days ago    HEALTHCHECK &{["CMD-SHELL" "ls /opt/scripts/…   0B        buildkit.dockerfile.v0
<missing>      8 days ago    USER 2000                                       0B        buildkit.dockerfile.v0
<missing>      8 days ago    RUN |1 version=13.35.0.9498-1_1.1.0 /bin/sh …   0B        buildkit.dockerfile.v0
<missing>      8 days ago    RUN |1 version=13.35.0.9498-1_1.1.0 /bin/sh …   0B        buildkit.dockerfile.v0
<missing>      8 days ago    RUN |1 version=13.35.0.9498-1_1.1.0 /bin/sh …   0B        buildkit.dockerfile.v0
<missing>      8 days ago    RUN |1 version=13.35.0.9498-1_1.1.0 /bin/sh …   0B        buildkit.dockerfile.v0
<missing>      8 days ago    RUN |1 version=13.35.0.9498-1_1.1.0 /bin/sh …   0B        buildkit.dockerfile.v0
<missing>      8 days ago    RUN |1 version=13.35.0.9498-1_1.1.0 /bin/sh …   135MB     buildkit.dockerfile.v0
<missing>      8 days ago    RUN |1 version=13.35.0.9498-1_1.1.0 /bin/sh …   139MB     buildkit.dockerfile.v0
<missing>      8 days ago    RUN |1 version=13.35.0.9498-1_1.1.0 /bin/sh …   0B        buildkit.dockerfile.v0
<missing>      8 days ago    RUN |1 version=13.35.0.9498-1_1.1.0 /bin/sh …   0B        buildkit.dockerfile.v0
<missing>      8 days ago    RUN |1 version=13.35.0.9498-1_1.1.0 /bin/sh …   69.7MB    buildkit.dockerfile.v0
<missing>      8 days ago    RUN |1 version=13.35.0.9498-1_1.1.0 /bin/sh …   139MB     buildkit.dockerfile.v0
<missing>      8 days ago    RUN |1 version=13.35.0.9498-1_1.1.0 /bin/sh …   139MB     buildkit.dockerfile.v0
<missing>      8 days ago    COPY /data/mongodb_agent_ubi.tgz /agent/mong…   56.6MB    buildkit.dockerfile.v0
<missing>      8 days ago    COPY /data/mongodb_tools_ubi.tgz /tools/mong…   69.7MB    buildkit.dockerfile.v0
<missing>      8 days ago    RUN |1 version=13.35.0.9498-1_1.1.0 /bin/sh …   20.7MB    buildkit.dockerfile.v0
<missing>      8 days ago    RUN |1 version=13.35.0.9498-1_1.1.0 /bin/sh …   24.6MB    buildkit.dockerfile.v0
<missing>      8 days ago    RUN |1 version=13.35.0.9498-1_1.1.0 /bin/sh …   52.5MB    buildkit.dockerfile.v0
<missing>      8 days ago    RUN |1 version=13.35.0.9498-1_1.1.0 /bin/sh …   46.8MB    buildkit.dockerfile.v0
<missing>      8 days ago    RUN |1 version=13.35.0.9498-1_1.1.0 /bin/sh …   25.2MB    buildkit.dockerfile.v0
<missing>      8 days ago    COPY /data/LICENSE /licenses/LICENSE # build…   160B      buildkit.dockerfile.v0
<missing>      8 days ago    COPY /data/agent-launcher.sh /opt/scripts/ag…   10kB      buildkit.dockerfile.v0
<missing>      8 days ago    COPY /data/agent-launcher-lib.sh /opt/script…   5.31kB    buildkit.dockerfile.v0
<missing>      8 days ago    COPY /data/version-upgrade-hook /opt/scripts…   49.3MB    buildkit.dockerfile.v0
<missing>      8 days ago    COPY /data/readinessprobe /opt/scripts/readi…   54.9MB    buildkit.dockerfile.v0
<missing>      8 days ago    COPY /data/probe.sh /opt/scripts/probe.sh # …   1.02kB    buildkit.dockerfile.v0
<missing>      8 days ago    LABEL name=MongoDB Agent version=13.35.0.949…   0B        buildkit.dockerfile.v0
<missing>      8 days ago    ARG version=13.35.0.9498-1_1.1.0                0B        buildkit.dockerfile.v0
<missing>      4 weeks ago   /bin/sh -c #(nop) LABEL "build-date"="2025-0…   104MB
<missing>      4 weeks ago   /bin/sh -c #(nop) COPY file:58cc94f5b3b2d60d…   0B
<missing>      4 weeks ago   /bin/sh -c #(nop) CMD ["/bin/bash"]             0B
<missing>      4 weeks ago   /bin/sh -c #(nop) COPY file:b37d593713ee21ad…   0B
<missing>      4 weeks ago   /bin/sh -c #(nop) COPY dir:9782e2e1b0ca599e0…   0B
<missing>      4 weeks ago   /bin/sh -c #(nop) ENV container oci             0B
<missing>      4 weeks ago   /bin/sh -c #(nop) LABEL io.openshift.tags="m…   0B
<missing>      4 weeks ago   /bin/sh -c #(nop) LABEL io.openshift.expose-…   0B
<missing>      4 weeks ago   /bin/sh -c #(nop) LABEL io.k8s.display-name=…   0B
<missing>      4 weeks ago   /bin/sh -c #(nop) LABEL io.k8s.description="…   0B
<missing>      4 weeks ago   /bin/sh -c #(nop) LABEL description="The Uni…   0B
<missing>      4 weeks ago   /bin/sh -c #(nop) LABEL summary="Provides th…   0B
<missing>      4 weeks ago   /bin/sh -c #(nop) LABEL com.redhat.license_t…   0B
<missing>      4 weeks ago   /bin/sh -c #(nop) LABEL com.redhat.component…   0B
<missing>      4 weeks ago   /bin/sh -c #(nop) LABEL url="https://www.red…   0B
<missing>      4 weeks ago   /bin/sh -c #(nop) LABEL vendor="Red Hat, Inc…   0B
<missing>      4 weeks ago   /bin/sh -c #(nop) LABEL maintainer="Red Hat,…   0B

After:
Getting the image build in this PR's patch:

$ docker inspect 268558157000.dkr.ecr.us-east-1.amazonaws.com/dev/mongodb-agent-ubi:13.35.0.9498-1_684be130a0378600072c885c | jq '.[0].Size' | numfmt --to=si
581M

$ docker history 268558157000.dkr.ecr.us-east-1.amazonaws.com/dev/mongodb-agent-ubi:13.35.0.9498-1_684be130a0378600072c885c
IMAGE          CREATED       CREATED BY                                      SIZE      COMMENT
1431b4573bf2   3 hours ago   HEALTHCHECK &{["CMD-SHELL" "ls /opt/scripts/…   0B        buildkit.dockerfile.v0
<missing>      3 hours ago   USER 2000                                       0B        buildkit.dockerfile.v0
<missing>      3 hours ago   RUN |1 version=13.35.0.9498-1_684be130a03786…   0B        buildkit.dockerfile.v0
<missing>      3 hours ago   COPY /opt/mongodb-mms-automation-agent-*/mon…   135MB     buildkit.dockerfile.v0
<missing>      3 hours ago   COPY /opt/mongodb-database-tools-*/bin /tool…   139MB     buildkit.dockerfile.v0
<missing>      3 hours ago   COPY /opt/scripts/* /opt/scripts/ # buildkit    104MB     buildkit.dockerfile.v0
<missing>      3 hours ago   COPY /data/LICENSE /licenses/LICENSE # build…   160B      buildkit.dockerfile.v0
<missing>      3 hours ago   RUN |1 version=13.35.0.9498-1_684be130a03786…   98.1MB    buildkit.dockerfile.v0
<missing>      3 hours ago   LABEL name=MongoDB Agent version=13.35.0.949…   0B        buildkit.dockerfile.v0
<missing>      3 hours ago   ARG version=13.35.0.9498-1_684be130a03786000…   0B        buildkit.dockerfile.v0
<missing>      3 days ago    /bin/sh -c #(nop) LABEL "build-date"="2025-0…   104MB
<missing>      3 days ago    /bin/sh -c #(nop) COPY file:58cc94f5b3b2d60d…   0B
<missing>      3 days ago    /bin/sh -c #(nop) CMD ["/bin/bash"]             0B
<missing>      3 days ago    /bin/sh -c #(nop) COPY file:b37d593713ee21ad…   0B
<missing>      3 days ago    /bin/sh -c #(nop) COPY dir:bac616b15ea24c824…   0B
<missing>      3 days ago    /bin/sh -c #(nop) ENV container oci             0B
<missing>      3 days ago    /bin/sh -c #(nop) LABEL io.openshift.tags="m…   0B
<missing>      3 days ago    /bin/sh -c #(nop) LABEL io.openshift.expose-…   0B
<missing>      3 days ago    /bin/sh -c #(nop) LABEL io.k8s.display-name=…   0B
<missing>      3 days ago    /bin/sh -c #(nop) LABEL io.k8s.description="…   0B
<missing>      3 days ago    /bin/sh -c #(nop) LABEL description="The Uni…   0B
<missing>      3 days ago    /bin/sh -c #(nop) LABEL summary="Provides th…   0B
<missing>      3 days ago    /bin/sh -c #(nop) LABEL com.redhat.license_t…   0B
<missing>      3 days ago    /bin/sh -c #(nop) LABEL com.redhat.component…   0B
<missing>      3 days ago    /bin/sh -c #(nop) LABEL url="https://www.red…   0B
<missing>      3 days ago    /bin/sh -c #(nop) LABEL vendor="Red Hat, Inc…   0B
<missing>      3 days ago    /bin/sh -c #(nop) LABEL maintainer="Red Hat,…   0B

Checklist

  • Have you linked a jira ticket and/or is the ticket in the title?
  • Have you checked whether your jira ticket required DOCSP changes?
  • Have you checked for release_note changes?

Reminder (Please remove this when merging)

  • Please try to Approve or Reject Changes the PR, keep PRs in review as short as possible
  • Our Short Guide for PRs: Link
  • Remember the following Communication Standards - use comment prefixes for clarity:
    • blocking: Must be addressed before approval.
    • follow-up: Can be addressed in a later PR or ticket.
    • q: Clarifying question.
    • nit: Non-blocking suggestions.
    • note: Side-note, non-actionable. Example: Praise
    • --> no prefix is considered a question

@lsierant lsierant force-pushed the lsierant/optimize-agent-docker branch 3 times, most recently from 1a6f87a to d07a1dc Compare June 12, 2025 21:55
@lsierant lsierant force-pushed the lsierant/optimize-agent-docker branch from d07a1dc to a467ad0 Compare June 13, 2025 08:28
@lsierant lsierant changed the title Optimized mongodb-agent docker layers CLOUDP-324668 - Optimized mongodb-agent docker layers Jun 13, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@lsierant