Skip to content

Commit

Permalink
fix: issues with release to pypi (letsql#228)
Browse files Browse the repository at this point in the history
1. The builds in `ubuntu` failed with the error: 

> The system library `openssl` required by crate `openssl-sys` was not found.

2. The builds for both `windows` and `macos` passed, but the workflows failed  with the error: 

> Failed request: (409) Conflict: an artifact with this name already exists on the workflow run

**Why?**

The error for `ubuntu` was due to the library `hf-hub,` specified as a Rust dependency, but that was not being used.

The errors in the builds of `windows` and `macos` were due to breaking changes in version v4 of the GitHub action upload artifact—more details are in [here](actions/upload-artifact#478). 

**Solution**

The library was removed, and the steps were modified; both changes are included in this PR.

**How can we prevent this from happening in the future?**

To prevent such failures in the future in the middle of the actual release, a pre-release workflow was created (and included in this PR). The idea is that this pre-release will try to upload to TestPyPI, and in case of any failures, we will modify both `.yaml` files (`ci-publish.yaml` and `ci-pre-release.yaml`)

**Additionally**
Refactor out ibis.datafusion/postgres/to_sql/pandas
  • Loading branch information
mesejo authored Aug 22, 2024
1 parent decd997 commit 363492d
Show file tree
Hide file tree
Showing 12 changed files with 163 additions and 413 deletions.
121 changes: 121 additions & 0 deletions .github/workflows/ci-pre-release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
# This file is autogenerated by maturin v1.2.3
# To update, run
#
# maturin generate-ci github
#
name: ci-pre-release

on:
workflow_dispatch:

# we do not want more than one release workflow executing at the same time, ever
concurrency:
group: release
# cancelling in the middle of a release would create incomplete releases
# so cancel-in-progress is false
cancel-in-progress: false

permissions:
contents: read

jobs:
linux:
runs-on: ubuntu-latest
strategy:
matrix:
target: [x86_64, x86, aarch64, armv7, s390x, ppc64le]
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: '3.10'
- name: Build wheels
uses: PyO3/maturin-action@v1
with:
target: ${{ matrix.target }}
args: --release --out dist --find-interpreter
sccache: 'true'
manylinux: auto
- name: Upload wheels
uses: actions/upload-artifact@v4
with:
name: wheel-ubuntu-${{ matrix.target }}
path: dist

windows:
runs-on: windows-latest
strategy:
matrix:
target: [x64, x86]
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: '3.10'
architecture: ${{ matrix.target }}
- name: Build wheels
uses: PyO3/maturin-action@v1
with:
target: ${{ matrix.target }}
args: --release --out dist --find-interpreter
sccache: 'true'
- name: Upload wheels
uses: actions/upload-artifact@v4
with:
name: wheel-windows-${{ matrix.target }}
path: dist

macos:
runs-on: macos-latest
strategy:
matrix:
target: [x86_64, aarch64]
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: '3.10'
- name: Build wheels
uses: PyO3/maturin-action@v1
with:
target: ${{ matrix.target }}
args: --release --out dist --find-interpreter
sccache: 'true'
- name: Upload wheels
uses: actions/upload-artifact@v4
with:
name: wheel-macos-${{ matrix.target }}
path: dist

sdist:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Build sdist
uses: PyO3/maturin-action@v1
with:
command: sdist
args: --out dist
- name: Upload sdist
uses: actions/upload-artifact@v4
with:
name: wheels
path: dist

release:
name: Release
runs-on: ubuntu-latest
needs: [linux, windows, macos, sdist]
steps:
- uses: actions/download-artifact@v4
with:
path: dist
merge-multiple: true
- name: Publish to TestPyPI
uses: PyO3/maturin-action@v1
env:
MATURIN_PYPI_TOKEN: ${{ secrets.TEST_PYPI_API_TOKEN }}
MATURIN_REPOSITORY: "testpypi"
with:
command: upload
args: --non-interactive --skip-existing ./dist/*
12 changes: 7 additions & 5 deletions .github/workflows/ci-publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
name: ci-publish

on:
workflow_dispatch:
release:
types: [ published ]

Expand Down Expand Up @@ -40,7 +41,7 @@ jobs:
- name: Upload wheels
uses: actions/upload-artifact@v4
with:
name: wheels
name: wheel-ubuntu-${{ matrix.target }}
path: dist

windows:
Expand All @@ -63,7 +64,7 @@ jobs:
- name: Upload wheels
uses: actions/upload-artifact@v4
with:
name: wheels
name: wheel-windows-${{ matrix.target }}
path: dist

macos:
Expand All @@ -85,7 +86,7 @@ jobs:
- name: Upload wheels
uses: actions/upload-artifact@v4
with:
name: wheels
name: wheel-macos-${{ matrix.target }}
path: dist

sdist:
Expand All @@ -110,11 +111,12 @@ jobs:
steps:
- uses: actions/download-artifact@v4
with:
name: wheels
path: dist
merge-multiple: true
- name: Publish to PyPI
uses: PyO3/maturin-action@v1
env:
MATURIN_PYPI_TOKEN: ${{ secrets.PYPI_API_TOKEN }}
with:
command: upload
args: --non-interactive --skip-existing *
args: --non-interactive --skip-existing ./dist/*
Loading

0 comments on commit 363492d

Please sign in to comment.