Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Meta-ticket: Distribution as wheels #31251

Open
mkoeppe opened this issue Jan 16, 2021 · 15 comments
Open

Meta-ticket: Distribution as wheels #31251

mkoeppe opened this issue Jan 16, 2021 · 15 comments
Labels
c: build p: major / 3

Comments

@mkoeppe
Copy link
Contributor

mkoeppe commented Jan 16, 2021 •
edited
Loading

Tickets and issues:

Open:

See also:

References:

CC: @tobiasdiez @videlec @isuruf @kliem @culler @dimpase @saraedum

Component: build

Issue created by migration from https://trac.sagemath.org/ticket/31251
@mkoeppe mkoeppe added this to the sage-9.3 milestone Jan 16, 2021
@mkoeppe

This comment has been minimized.

Sign in to view
@mkoeppe

This comment has been minimized.

Sign in to view
@mkoeppe mkoeppe modified the milestones: sage-9.3, sage-9.4 Feb 11, 2021
@videlec
Copy link
Contributor

videlec commented Feb 11, 2021

comment:4

https://github.com/conda-incubator/conda-press?

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Feb 11, 2021

comment:5

Cool, thanks for sharing

@mkoeppe

This comment has been minimized.

Sign in to view
@mkoeppe

This comment has been minimized.

Sign in to view
@mkoeppe

This comment has been minimized.

Sign in to view
@mkoeppe

This comment has been minimized.

Sign in to view
@mkoeppe

This comment has been minimized.

Sign in to view
@mkoeppe

This comment has been minimized.

Sign in to view
@mkoeppe mkoeppe modified the milestones: sage-9.4, sage-9.5 Jul 19, 2021
@mkoeppe

This comment has been minimized.

Sign in to view
@mkoeppe

This comment has been minimized.

Sign in to view
@kliem

This comment has been minimized.

Sign in to view
@mkoeppe mkoeppe modified the milestones: sage-9.5, sage-9.6 Dec 18, 2021
@mkoeppe mkoeppe modified the milestones: sage-9.6, sage-9.7 Apr 2, 2022
@mkoeppe

This comment has been minimized.

Sign in to view
@mkoeppe mkoeppe modified the milestones: sage-9.7, sage-9.8 Aug 31, 2022
@mkoeppe

This comment has been minimized.

Sign in to view
@mkoeppe mkoeppe modified the milestones: sage-9.8, sage-9.9 Jan 7, 2023
@mkoeppe mkoeppe removed this from the sage-10.0 milestone Apr 30, 2023
vbraun pushed a commit to vbraun/sage that referenced this issue Apr 20, 2024
sagemathgh-37503: GH Actions: Build macOS arm64 wheels
81dae84 
    
<!-- ^ Please provide a concise and informative title. -->
<!-- ^ Don't put issue numbers in the title, do this in the PR
description below. -->
<!-- ^ For example, instead of "Fixes sagemath#12345" use "Introduce new method
to calculate 1 + 2". -->
<!-- v Describe your changes below in detail. -->
<!-- v Why is this change required? What problem does it solve? -->
<!-- v If this PR resolves an open issue, please link to it here. For
example, "Fixes sagemath#12345". -->

Using https://cibuildwheel.readthedocs.io/en/stable/changelog/#v2165

Example run: https://github.com/mkoeppe/sage/actions/runs/8088717795/job
/22103468674#step:5:11246

Part of:
- sagemath#31251

### 📝 Checklist

<!-- Put an `x` in all the boxes that apply. -->

- [x] The title is concise and informative.
- [ ] The description explains in detail what this PR is about.
- [ ] I have linked a relevant issue or discussion.
- [ ] I have created tests covering the changes.
- [ ] I have updated the documentation accordingly.

### ⌛ Dependencies

<!-- List all open PRs that this PR logically depends on. For example,
-->
<!-- - sagemath#12345: short description why this is a dependency -->
<!-- - sagemath#34567: ... -->
    
URL: sagemath#37503
Reported by: Matthias Köppe
Reviewer(s): Kwankyu Lee
vbraun pushed a commit to vbraun/sage that referenced this issue Apr 25, 2024
sagemathgh-37503: GH Actions: Build macOS arm64 wheels
0124bab 
    
<!-- ^ Please provide a concise and informative title. -->
<!-- ^ Don't put issue numbers in the title, do this in the PR
description below. -->
<!-- ^ For example, instead of "Fixes sagemath#12345" use "Introduce new method
to calculate 1 + 2". -->
<!-- v Describe your changes below in detail. -->
<!-- v Why is this change required? What problem does it solve? -->
<!-- v If this PR resolves an open issue, please link to it here. For
example, "Fixes sagemath#12345". -->

Using https://cibuildwheel.readthedocs.io/en/stable/changelog/#v2165

Example run: https://github.com/mkoeppe/sage/actions/runs/8088717795/job
/22103468674#step:5:11246

Part of:
- sagemath#31251

### 📝 Checklist

<!-- Put an `x` in all the boxes that apply. -->

- [x] The title is concise and informative.
- [ ] The description explains in detail what this PR is about.
- [ ] I have linked a relevant issue or discussion.
- [ ] I have created tests covering the changes.
- [ ] I have updated the documentation accordingly.

### ⌛ Dependencies

<!-- List all open PRs that this PR logically depends on. For example,
-->
<!-- - sagemath#12345: short description why this is a dependency -->
<!-- - sagemath#34567: ... -->
    
URL: sagemath#37503
Reported by: Matthias Köppe
Reviewer(s): Kwankyu Lee
vbraun pushed a commit to vbraun/sage that referenced this issue Apr 28, 2024
sagemathgh-37099: GH Actions: Build platform-independent wheels of sa…
f048021 
…gemath-environment, sage-setup, sage-sws2rst for PyPI

    
<!-- ^^^^^
Please provide a concise, informative and self-explanatory title.
Don't put issue numbers in there, do this in the PR body below.
For example, instead of "Fixes sagemath#1234" use "Introduce new method to
calculate 1+1"
-->
<!-- Describe your changes here in detail -->

<!-- Why is this change required? What problem does it solve? -->
We already build platform wheels for
**sagemath-{objects,categories,repl}** for PyPI using cibuildwheel.
Here we add building of platform-independent wheels for **sagemath-
environment** and **sage-setup** and **sage-sws2rst** to the GH Actions
workflow. (They are needed, for example, for pyodide / jupyterlite.)

To test locally: `make pypi-noarch-wheels`

We also update actions/upload-artifact, actions/download-artifact to v4.
This requires a restructuring, as we can no longer upload the wheels for
different architectures (built by separate matrix jobs) to the same
artifact: https://github.com/actions/upload-artifact?tab=readme-ov-
file#not-uploading-to-the-same-artifact
Instead we upload them as separate artifacts and deploy them to PyPI
directly from the job that built them.

This is:
- part of sagemath#31251

<!-- If this PR resolves an open issue, please link to it here. For
example "Fixes sagemath#12345". -->
<!-- If your change requires a documentation PR, please link it
appropriately. -->

### 📝 Checklist

<!-- Put an `x` in all the boxes that apply. -->
<!-- If your change requires a documentation PR, please link it
appropriately -->
<!-- If you're unsure about any of these, don't hesitate to ask. We're
here to help! -->
<!-- Feel free to remove irrelevant items. -->

- [x] The title is concise, informative, and self-explanatory.
- [x] The description explains in detail what this PR is about.
- [ ] I have linked a relevant issue or discussion.
- [ ] I have created tests covering the changes.
- [ ] I have updated the documentation accordingly.

### ⌛ Dependencies

<!-- List all open PRs that this PR logically depends on
- sagemath#12345: short description why this is a dependency
- sagemath#34567: ...
-->

<!-- If you're unsure about any of these, don't hesitate to ask. We're
here to help! -->
    
URL: sagemath#37099
Reported by: Matthias Köppe
Reviewer(s): Kwankyu Lee, Matthias Köppe
vbraun pushed a commit to vbraun/sage that referenced this issue May 2, 2024
sagemathgh-37099: GH Actions: Build platform-independent wheels of sa…
248fde3 
…gemath-environment, sage-setup, sage-sws2rst for PyPI

    
<!-- ^^^^^
Please provide a concise, informative and self-explanatory title.
Don't put issue numbers in there, do this in the PR body below.
For example, instead of "Fixes sagemath#1234" use "Introduce new method to
calculate 1+1"
-->
<!-- Describe your changes here in detail -->

<!-- Why is this change required? What problem does it solve? -->
We already build platform wheels for
**sagemath-{objects,categories,repl}** for PyPI using cibuildwheel.
Here we add building of platform-independent wheels for **sagemath-
environment** and **sage-setup** and **sage-sws2rst** to the GH Actions
workflow. (They are needed, for example, for pyodide / jupyterlite.)

To test locally: `make pypi-noarch-wheels`

We also update actions/upload-artifact, actions/download-artifact to v4.
This requires a restructuring, as we can no longer upload the wheels for
different architectures (built by separate matrix jobs) to the same
artifact: https://github.com/actions/upload-artifact?tab=readme-ov-
file#not-uploading-to-the-same-artifact
Instead we upload them as separate artifacts and deploy them to PyPI
directly from the job that built them.

This is:
- part of sagemath#31251

<!-- If this PR resolves an open issue, please link to it here. For
example "Fixes sagemath#12345". -->
<!-- If your change requires a documentation PR, please link it
appropriately. -->

### 📝 Checklist

<!-- Put an `x` in all the boxes that apply. -->
<!-- If your change requires a documentation PR, please link it
appropriately -->
<!-- If you're unsure about any of these, don't hesitate to ask. We're
here to help! -->
<!-- Feel free to remove irrelevant items. -->

- [x] The title is concise, informative, and self-explanatory.
- [x] The description explains in detail what this PR is about.
- [ ] I have linked a relevant issue or discussion.
- [ ] I have created tests covering the changes.
- [ ] I have updated the documentation accordingly.

### ⌛ Dependencies

<!-- List all open PRs that this PR logically depends on
- sagemath#12345: short description why this is a dependency
- sagemath#34567: ...
-->

<!-- If you're unsure about any of these, don't hesitate to ask. We're
here to help! -->
    
URL: sagemath#37099
Reported by: Matthias Köppe
Reviewer(s): Kwankyu Lee, Matthias Köppe
vbraun pushed a commit to vbraun/sage that referenced this issue May 18, 2024
sagemathgh-36525: GH Actions: Fix build of macOS x86_64 wheels, build…
0fcc928 
… wheels for sagemath-{bliss,coxeter3,mcqd,tdlib} for PyPI

    
<!-- ^^^^^
Please provide a concise, informative and self-explanatory title.
Don't put issue numbers in there, do this in the PR body below.
For example, instead of "Fixes sagemath#1234" use "Introduce new method to
calculate 1+1"
-->
<!-- Describe your changes here in detail -->

Because the GitHub runner `macos-latest` is now Apple Silicon, building
x86_64 wheels is broken. We fix this here by using `macos-13` for
building x86_64.

We speed up the build on Linux by bootstrapping only once in each job,
and configuring and building packages of the Sage distribution only once
for each Linux platform. Bootstrapping is now done outside of the
container, and we configure Sage to use a `--prefix` in a directory
mounted from the host, and storing the `config.status`.

We add wheels for the optional-extension packages **sagemath-bliss**,
... that can be built on top of the existing modularized distributions.
(The packages **sagemath-meataxe** and **sagemath-sirocco** have to wait
for **sagemath-modules** to become available.)

Tests at https://github.com/mkoeppe/sage/actions/workflows/dist.yml:
- Successful run at
https://github.com/mkoeppe/sage/actions/runs/9100668826 (the failure in
release_dist is  unrelated)

Follow-up:
- sagemath#37541, refactor as reusable GH Actions workflow -- for use by
cysignals, cypari2, pplpy etc.

Part of:
- sagemath#31251

<!-- Why is this change required? What problem does it solve? -->
<!-- If this PR resolves an open issue, please link to it here. For
example "Fixes sagemath#12345". -->
<!-- If your change requires a documentation PR, please link it
appropriately. -->

### 📝 Checklist

<!-- Put an `x` in all the boxes that apply. -->
<!-- If your change requires a documentation PR, please link it
appropriately -->
<!-- If you're unsure about any of these, don't hesitate to ask. We're
here to help! -->
<!-- Feel free to remove irrelevant items. -->

- [x] The title is concise, informative, and self-explanatory.
- [x] The description explains in detail what this PR is about.
- [ ] I have linked a relevant issue or discussion.
- [ ] I have created tests covering the changes.
- [ ] I have updated the documentation accordingly.

### ⌛ Dependencies

<!-- List all open PRs that this PR logically depends on
- sagemath#12345: short description why this is a dependency
- sagemath#34567: ...
-->
- Depends on sagemath#36521
- Depends on sagemath#37503 (merged here)

<!-- If you're unsure about any of these, don't hesitate to ask. We're
here to help! -->
    
URL: sagemath#36525
Reported by: Matthias Köppe
Reviewer(s):
vbraun pushed a commit to vbraun/sage that referenced this issue May 24, 2024
sagemathgh-36525: GH Actions: Fix build of macOS x86_64 wheels, build…
83a7fac 
… wheels for sagemath-{bliss,coxeter3,mcqd,tdlib} for PyPI

    
<!-- ^^^^^
Please provide a concise, informative and self-explanatory title.
Don't put issue numbers in there, do this in the PR body below.
For example, instead of "Fixes sagemath#1234" use "Introduce new method to
calculate 1+1"
-->
<!-- Describe your changes here in detail -->

Because the GitHub runner `macos-latest` is now Apple Silicon, building
x86_64 wheels is broken. We fix this here by using `macos-13` for
building x86_64.

We speed up the build on Linux by bootstrapping only once in each job,
and configuring and building packages of the Sage distribution only once
for each Linux platform. Bootstrapping is now done outside of the
container, and we configure Sage to use a `--prefix` in a directory
mounted from the host, and storing the `config.status`.

We add wheels for the optional-extension packages **sagemath-bliss**,
... that can be built on top of the existing modularized distributions.
(The packages **sagemath-meataxe** and **sagemath-sirocco** have to wait
for **sagemath-modules** to become available.)

Tests at https://github.com/mkoeppe/sage/actions/workflows/dist.yml:
- Successful run at
https://github.com/mkoeppe/sage/actions/runs/9100668826 (the failure in
release_dist is  unrelated)

Follow-up:
- sagemath#37541, refactor as reusable GH Actions workflow -- for use by
cysignals, cypari2, pplpy etc.

Part of:
- sagemath#31251

<!-- Why is this change required? What problem does it solve? -->
<!-- If this PR resolves an open issue, please link to it here. For
example "Fixes sagemath#12345". -->
<!-- If your change requires a documentation PR, please link it
appropriately. -->

### 📝 Checklist

<!-- Put an `x` in all the boxes that apply. -->
<!-- If your change requires a documentation PR, please link it
appropriately -->
<!-- If you're unsure about any of these, don't hesitate to ask. We're
here to help! -->
<!-- Feel free to remove irrelevant items. -->

- [x] The title is concise, informative, and self-explanatory.
- [x] The description explains in detail what this PR is about.
- [ ] I have linked a relevant issue or discussion.
- [ ] I have created tests covering the changes.
- [ ] I have updated the documentation accordingly.

### ⌛ Dependencies

<!-- List all open PRs that this PR logically depends on
- sagemath#12345: short description why this is a dependency
- sagemath#34567: ...
-->
- Depends on sagemath#36521
- Depends on sagemath#37503 (merged here)

<!-- If you're unsure about any of these, don't hesitate to ask. We're
here to help! -->
    
URL: sagemath#36525
Reported by: Matthias Köppe
Reviewer(s):
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
c: build p: major / 3
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@videlec @mkoeppe @kliem