Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Need to install pkg-config apt package in all IDAs that use Python mysqlclient #349

Closed
20 of 27 tasks
timmc-edx opened this issue Jul 5, 2023 · 2 comments
Closed
20 of 27 tasks
Assignees
Labels

Comments

@timmc-edx
Copy link
Member

timmc-edx commented Jul 5, 2023

Symptom

Repos that depend on mysqlclient>=2.2.0 will need to install the package pkg-config in their Dockerfile: PyMySQL/mysqlclient#620

If this is missing, then pip install of mysqlclient fails with an error that includes the following:

Exception: Can not find valid pkg-config name.
Specify MYSQLCLIENT_CFLAGS and MYSQLCLIENT_LDFLAGS env vars manually

Fix

  • In the Dockerfile, include pkg-config in the list of packages installed by apt
    • Put the package name close to libssl-dev and libmysqlclient-dev, which are also required by mysqlclient, and likely already present
    • Include a comment (on its own line) to the effect of # mysqlclient>=2.2.0 requires pkg-config to explain the addition
  • Because this may introduce tzdata as a new system dependency, and tzdata requests manual configuration, you may need to include DEBIAN_FRONTEND=noninteractive at the front of the apt install line in order to not have docker image building hang. (It's possible that --no-install-recommends would suffice, if missing, but no harm in using both.)

So, that part of your Dockerfile may end up looking something like:

RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -qy --no-install-recommends \
    python3.8 python3-pip \
    ...
    # mysqlcient dependencies:
    libmysqlclient-dev build-essential pkg-config libssl-dev \
    && rm -rf /var/lib/apt/lists/*

Affected repos

For reference, the following repos have Dockerfiles that reference libmysqlclient-dev, another known apt package dependency for the mysqlclient Python dep (broken out by 2U ownership):

  • aperture:
  • arch-bom:
    • openedx/edx-cookiecutters (cookiecutter-django-ida/{{cookiecutter.repo_name}}/Dockerfile): Issue
    • openedx/edx-platform - thought we'd fixed this, but there are two apt-install blocks in different Docker stages...
    • openedx/edx-repo-health (tests/fake_repos/python_repo/Dockerfile): Issue
  • aurora:
  • Someone else in the Open edX community:
    • openedx/openedx-i18n (docker/Dockerfile)
  • cosmonauts:
    • edx/edx-exams
    • edx/portal-designer
    • edx/program-intent-engagement
    • openedx/edx-analytics-dashboard
    • openedx/edx-analytics-data-api
    • openedx/registrar
  • phoenix:
    • edx/financial-assistance
    • edx/video-encode-manager
    • openedx/course-discovery
  • purchase:
    • edx/commerce-coordinator
    • openedx/ecommerce
  • sre:
    • edx/internal-dockerfiles (gocd-agents/Dockerfile) PR
    • edx/monitoring-scripts PR
  • titans:
    • edx/subscriptions
    • openedx/edx-enterprise
    • openedx/enterprise-access
    • openedx/enterprise-catalog
    • openedx/enterprise-subsidy
    • openedx/license-manager - Pull request
MichaelRoytman added a commit to edx/edx-exams that referenced this issue Jul 6, 2023
Repositiories that depend on mysqlclient>=2.2.0 will need to install the package pkg-config in their Dockerfile: PyMySQL/mysqlclient#620. This commit installs the pkg-config package in the Dockerfile.

If this is missing, then pip install of mysqlclient fails with an error that includes the following:

Exception: Can not find valid pkg-config name.
Specify MYSQLCLIENT_CFLAGS and MYSQLCLIENT_LDFLAGS env vars manually

See edx/edx-arch-experiments#349.
MichaelRoytman added a commit to edx/portal-designer that referenced this issue Jul 6, 2023
Repositiories that depend on mysqlclient>=2.2.0 will need to install the package pkg-config in their Dockerfile: PyMySQL/mysqlclient#620. This commit installs the pkg-config package in the Dockerfile.

If this is missing, then pip install of mysqlclient fails with an error that includes the following:

Exception: Can not find valid pkg-config name.
Specify MYSQLCLIENT_CFLAGS and MYSQLCLIENT_LDFLAGS env vars manually

See edx/edx-arch-experiments#349.
MichaelRoytman added a commit to openedx-unsupported/edx-analytics-dashboard that referenced this issue Jul 6, 2023
Repositiories that depend on mysqlclient>=2.2.0 will need to install the package pkg-config in their Dockerfile: PyMySQL/mysqlclient#620. This commit installs the pkg-config package in the Dockerfile.

If this is missing, then pip install of mysqlclient fails with an error that includes the following:

Exception: Can not find valid pkg-config name.
Specify MYSQLCLIENT_CFLAGS and MYSQLCLIENT_LDFLAGS env vars manually

See edx/edx-arch-experiments#349.
MichaelRoytman added a commit to edx/program-intent-engagement that referenced this issue Jul 6, 2023
Repositiories that depend on mysqlclient>=2.2.0 will need to install the package pkg-config in their Dockerfile: PyMySQL/mysqlclient#620. This commit installs the pkg-config package in the Dockerfile.

If this is missing, then pip install of mysqlclient fails with an error that includes the following:

Exception: Can not find valid pkg-config name.
Specify MYSQLCLIENT_CFLAGS and MYSQLCLIENT_LDFLAGS env vars manually

See edx/edx-arch-experiments#349.
MichaelRoytman added a commit to openedx-unsupported/edx-analytics-data-api that referenced this issue Jul 6, 2023
Repositiories that depend on mysqlclient>=2.2.0 will need to install the package pkg-config in their Dockerfile: PyMySQL/mysqlclient#620. This commit installs the pkg-config package in the Dockerfile.

If this is missing, then pip install of mysqlclient fails with an error that includes the following:

Exception: Can not find valid pkg-config name.
Specify MYSQLCLIENT_CFLAGS and MYSQLCLIENT_LDFLAGS env vars manually

See edx/edx-arch-experiments#349.
grmartin added a commit to edx/commerce-coordinator that referenced this issue Jul 7, 2023
This is to future proof the update of mysql client in future releases. See: edx/edx-arch-experiments#349
grmartin added a commit to openedx/ecommerce that referenced this issue Jul 7, 2023
This is to future proof the update of mysql client in future releases. See: edx/edx-arch-experiments#349
grmartin added a commit to openedx/ecommerce that referenced this issue Jul 7, 2023
This is to future proof the update of mysql client in future releases. See: edx/edx-arch-experiments#349
@timmc-edx timmc-edx added the esre label Jul 7, 2023
@MichaelRoytman
Copy link
Member

MichaelRoytman commented Jul 7, 2023

I've updated the Cosmonauts Dockerfiles and checked off the associated checkboxes above. Thank you!

@rgraber
Copy link
Contributor

rgraber commented Jul 17, 2023

Closing on the grounds that we have told everyone about the necessary changes and merged all the PRs we are responsible for creating.

@rgraber rgraber closed this as completed Jul 17, 2023
grmartin added a commit to edx/commerce-coordinator that referenced this issue Sep 5, 2023
* chore: `pkg-config` added to Docker File + PIP Reqs

This is to future proof the update of mysql client in future releases. See: edx/edx-arch-experiments#349

* fix: Docker Compose issues

* fix: Bringing all pip req.s in line with those used by the code in `2u/project-theseus`

* fix: make upgrade on Py3.8

This is a guess, but since im upgrading the requirements locally, and i rebuilt my venv... It swapped Python versions to 3.10 instead of 3.8 (the target of our other tooling)...

```
brew install [email protected]
python3.8 -m venv .venv
. .venv/bin/activate
make upgrade
```
grmartin added a commit to edx/commerce-coordinator that referenced this issue Sep 5, 2023
* chore: `pkg-config` added to Docker File + PIP Reqs

This is to future proof the update of mysql client in future releases. See: edx/edx-arch-experiments#349

* fix: Docker Compose issues

* fix: Bringing all pip req.s in line with those used by the code in `2u/project-theseus`

* fix: make upgrade on Py3.8

This is a guess, but since im upgrading the requirements locally, and i rebuilt my venv... It swapped Python versions to 3.10 instead of 3.8 (the target of our other tooling)...

```
brew install [email protected]
python3.8 -m venv .venv
. .venv/bin/activate
make upgrade
```

This is based on PR #95
grmartin added a commit to edx/commerce-coordinator that referenced this issue Sep 5, 2023
* chore: `pkg-config` added to Docker File + PIP Reqs

This is to future proof the update of mysql client in future releases. See: edx/edx-arch-experiments#349

* fix: Docker Compose issues

* fix: Bringing all pip req.s in line with those used by the code in `2u/project-theseus`

* fix: make upgrade on Py3.8

This is a guess, but since im upgrading the requirements locally, and i rebuilt my venv... It swapped Python versions to 3.10 instead of 3.8 (the target of our other tooling)...

```
brew install [email protected]
python3.8 -m venv .venv
. .venv/bin/activate
make upgrade
```

This is based on PR #95
grmartin added a commit to edx/commerce-coordinator that referenced this issue Sep 5, 2023
* chore: `pkg-config` added to Docker File + PIP Reqs

This is to future proof the update of mysql client in future releases. See: edx/edx-arch-experiments#349

* fix: Docker Compose issues

* fix: Bringing all pip req.s in line with those used by the code in `2u/project-theseus`

* fix: make upgrade on Py3.8

This is a guess, but since im upgrading the requirements locally, and i rebuilt my venv... It swapped Python versions to 3.10 instead of 3.8 (the target of our other tooling)...

```
brew install [email protected]
python3.8 -m venv .venv
. .venv/bin/activate
make upgrade
```

This is based on PR #95
pshiu pushed a commit to edx/commerce-coordinator that referenced this issue Sep 7, 2023
* chore: `pkg-config` added to Docker File + PIP Reqs

This is to future proof the update of mysql client in future releases. See: edx/edx-arch-experiments#349

* fix: Docker Compose issues

* fix: Bringing all pip req.s in line with those used by the code in `2u/project-theseus`

* fix: make upgrade on Py3.8

This is a guess, but since im upgrading the requirements locally, and i rebuilt my venv... It swapped Python versions to 3.10 instead of 3.8 (the target of our other tooling)...

```
brew install [email protected]
python3.8 -m venv .venv
. .venv/bin/activate
make upgrade
```

This is based on PR #95
pshiu pushed a commit to edx/commerce-coordinator that referenced this issue Sep 7, 2023
* chore: `pkg-config` added to Docker File + PIP Reqs

This is to future proof the update of mysql client in future releases. See: edx/edx-arch-experiments#349

* fix: Docker Compose issues

* fix: Bringing all pip req.s in line with those used by the code in `2u/project-theseus`

* fix: make upgrade on Py3.8

This is a guess, but since im upgrading the requirements locally, and i rebuilt my venv... It swapped Python versions to 3.10 instead of 3.8 (the target of our other tooling)...

```
brew install [email protected]
python3.8 -m venv .venv
. .venv/bin/activate
make upgrade
```

This is based on PR #95
pshiu pushed a commit to edx/commerce-coordinator that referenced this issue Dec 11, 2023
* chore: `pkg-config` added to Docker File + PIP Reqs

This is to future proof the update of mysql client in future releases. See: edx/edx-arch-experiments#349

* fix: Docker Compose issues

* fix: Bringing all pip req.s in line with those used by the code in `2u/project-theseus`

* fix: make upgrade on Py3.8

This is a guess, but since im upgrading the requirements locally, and i rebuilt my venv... It swapped Python versions to 3.10 instead of 3.8 (the target of our other tooling)...

```
brew install [email protected]
python3.8 -m venv .venv
. .venv/bin/activate
make upgrade
```

This is based on PR #95
pshiu pushed a commit to edx/commerce-coordinator that referenced this issue Dec 11, 2023
* chore: `pkg-config` added to Docker File + PIP Reqs

This is to future proof the update of mysql client in future releases. See: edx/edx-arch-experiments#349

* fix: Docker Compose issues

* fix: Bringing all pip req.s in line with those used by the code in `2u/project-theseus`

* fix: make upgrade on Py3.8

This is a guess, but since im upgrading the requirements locally, and i rebuilt my venv... It swapped Python versions to 3.10 instead of 3.8 (the target of our other tooling)...

```
brew install [email protected]
python3.8 -m venv .venv
. .venv/bin/activate
make upgrade
```

This is based on PR #95
pshiu pushed a commit to edx/commerce-coordinator that referenced this issue Dec 11, 2023
* chore: `pkg-config` added to Docker File + PIP Reqs

This is to future proof the update of mysql client in future releases. See: edx/edx-arch-experiments#349

* fix: Docker Compose issues

* fix: Bringing all pip req.s in line with those used by the code in `2u/project-theseus`

* fix: make upgrade on Py3.8

This is a guess, but since im upgrading the requirements locally, and i rebuilt my venv... It swapped Python versions to 3.10 instead of 3.8 (the target of our other tooling)...

```
brew install [email protected]
python3.8 -m venv .venv
. .venv/bin/activate
make upgrade
```

This is based on PR #95
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Status: Done
Archived in project
Development

No branches or pull requests

3 participants