Multi-base charm uses same LXC container for different bases

[carlcsaposs-canonical]

Bug Description

When packing a charm with

platforms:
  [email protected]:amd64:
  [email protected]:amd64:

charmcraft uses the same LXC container for both platforms. This means that if you pack the 20.04 charm, then the 22.04 charm, and then the 20.04 charm, the 3rd pack will be slow because the container from the 1st pack was overriden

Expected behavior

charmcraft uses different LXC containers for each platform. (In aforementioned example) the 3rd pack is fast because it reuses the container from the 1st pack

To Reproduce

git clone https://github.com/canonical/mysql-router-operator
cd mysql-router-operator
git checkout f12c6bb4b8b49f238bf9d1874f453fa8950b2cb8
charmcraft pack -v
lxc --project charmcraft ls

$ lxc --project charmcraft ls
+---------------------------------------------------------------+---------+------+------+-----------+-----------+
|                             NAME                              |  STATE  | IPV4 | IPV6 |   TYPE    | SNAPSHOTS |
+---------------------------------------------------------------+---------+------+------+-----------+-----------+
| base-instance-charmcraft-buildd-base-v7--20d64a5acd5c317c9c8d | STOPPED |      |      | CONTAINER | 0         |
+---------------------------------------------------------------+---------+------+------+-----------+-----------+
| base-instance-charmcraft-buildd-base-v7--910533fdd664bd2cdbe9 | STOPPED |      |      | CONTAINER | 0         |
+---------------------------------------------------------------+---------+------+------+-----------+-----------+
| charmcraft-mysql-router-on-amd64-for-amd64-2393879            | STOPPED |      |      | CONTAINER | 0         |
+---------------------------------------------------------------+---------+------+------+-----------+-----------+

Environment

Ubuntu 22.04

$ snap list
Name              Version                  Rev    Tracking            Publisher          Notes
charmcraft        3.2.2.post133+g455880a4  5682   3.x/edge            canonical✓         classic
core18            20240920                 2846   latest/stable       canonical✓         base
core20            20240911                 2434   latest/stable       canonical✓         base
core22            20241119                 1722   latest/stable       canonical✓         base
core24            20240920                 609    latest/stable       canonical✓         base
juju              3.6.1                    29241  3/stable            canonical✓         -
lxd               5.0.4-497fe1e            31333  5.0/stable/…        canonical✓         -
microk8s          v1.29.11                 7435   1.29-strict/stable  canonical✓         -
snapcraft         8.5.1                    13181  latest/stable       canonical✓         classic
snapd             2.66.1                   23258  latest/stable       canonical✓         snapd

charmcraft.yaml

# Copyright 2022 Canonical Ltd.
# See LICENSE file for licensing details.

type: charm
platforms:
  [email protected]:amd64:
  [email protected]:amd64:
  [email protected]:arm64:
# Files implicitly created by charmcraft without a part:
# - dispatch (https://github.com/canonical/charmcraft/pull/1898)
# - manifest.yaml
#   (https://github.com/canonical/charmcraft/blob/9ff19c328e23b50cc06f04e8a5ad4835740badf4/charmcraft/services/package.py#L259)
# Files implicitly copied/"primed" by charmcraft without a part:
# - actions.yaml, config.yaml, metadata.yaml
#   (https://github.com/canonical/charmcraft/blob/9ff19c328e23b50cc06f04e8a5ad4835740badf4/charmcraft/services/package.py#L290-L293
#   https://github.com/canonical/charmcraft/blob/9ff19c328e23b50cc06f04e8a5ad4835740badf4/charmcraft/services/package.py#L156-L157)
parts:
  # "poetry-deps" part name is a magic constant
  # https://github.com/canonical/craft-parts/pull/901
  poetry-deps:
    plugin: nil
    build-packages:
      - curl
    override-build: |
      python3 -m pip install --upgrade pip==24.3.1  # renovate: charmcraft-pip-latest
      curl -sSL https://install.python-poetry.org | python3 - --version 1.8.4  # renovate: charmcraft-poetry-latest
      ln -sf "$HOME/.local/bin/poetry" /usr/local/bin/poetry
  # "charm-poetry" part name is arbitrary; use for consistency
  # Avoid using "charm" part name since that has special meaning to charmcraft
  charm-poetry:
    # By default, the `poetry` plugin creates/primes these directories:
    # - lib, src
    #   (https://github.com/canonical/charmcraft/blob/9ff19c328e23b50cc06f04e8a5ad4835740badf4/charmcraft/parts/plugins/_poetry.py#L76-L78)
    # - venv
    #   (https://github.com/canonical/charmcraft/blob/9ff19c328e23b50cc06f04e8a5ad4835740badf4/charmcraft/parts/plugins/_poetry.py#L95
    #   https://github.com/canonical/craft-parts/blob/afb0d652eb330b6aaad4f40fbd6e5357d358de47/craft_parts/plugins/base.py#L270)
    plugin: poetry
    source: .
    after:
      - poetry-deps
    poetry-export-extra-args: ['--only', 'main,charm-libs']
    build-snaps:
      - rustup
    build-packages:
      - libffi-dev
      - libssl-dev
      - pkg-config
    override-build: |
      rustup default stable
      craftctl default
      # Include requirements.txt in *.charm artifact for easier debugging
      cp requirements.txt "$CRAFT_PART_INSTALL/requirements.txt"
  # "files" part name is arbitrary; use for consistency
  files:
    plugin: dump
    source: .
    build-packages:
      - git
    override-build: |
      # Workaround to add unique identifier (git hash) to charm version while specification
      # DA053 - Charm versioning
      # (https://docs.google.com/document/d/1Jv1jhWLl8ejK3iJn7Q3VbCIM9GIhp8926bgXpdtx-Sg/edit?pli=1)
      # is pending review.
      python3 -c 'import pathlib; import shutil; import subprocess; git_hash=subprocess.run(["git", "describe", "--always", "--dirty"], capture_output=True, check=True, encoding="utf-8").stdout; file = pathlib.Path("charm_version"); shutil.copy(file, pathlib.Path("charm_version.backup")); version = file.read_text().strip(); file.write_text(f"{version}+{git_hash}")'

      craftctl default
    prime:
      - LICENSE
      - charm_version
      - workload_version
      - templates

Relevant log output

Starting charmcraft, version 3.2.2.post133+g455880a4
Logging execution to '/home/ubuntu/.local/state/charmcraft/log/charmcraft-20241220-092903.384319.log'
Launching managed ubuntu 20.04 instance...
Creating new instance from remote
Creating new base instance from remote 
Creating new instance from base instance 
Starting instance 
Starting charmcraft, version 3.2.2.post133+g455880a4
[...]
Packing...
Packing charm [email protected]:amd64.charm
Packed [email protected]:amd64.charm
Launching managed ubuntu 22.04 instance...
Starting instance 
Creating instance from base instance 
Starting instance 
Starting charmcraft, version 3.2.2.post133+g455880a4
[...]
Packing...
Packing charm [email protected]:amd64.charm
Packed [email protected]:amd64.charm

[lengau]

Good catch, thanks! @mr-cal thinks we have a craft-providers bug related to this, and if not this is probably something to address in craft-application, but this is definitely a blocker for 3.3.

[syncronize-issues-to-jira]

Thank you for reporting your feedback to us!

The internal ticket has been created: https://warthogs.atlassian.net/browse/CRAFT-3847.

This message was autogenerated

[syncronize-issues-to-jira]

Thank you for reporting your feedback to us!

The internal ticket has been created: https://warthogs.atlassian.net/browse/CRAFT-3848.

This message was autogenerated

[mr-cal]

To be clear, charmcraft is building each charm in the correct build environment. So the artifacts are correct but this is inconvenient for iterative development.

The problem is that the same container name is re-used, so charmcraft is having to recreate the container every time charmcraft builds on a different base.

This has come up in canonical/craft-providers#544. Now I'm wondering if encoding the platform name in the lxc container is the right move.