Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Package all natives in a single jar. #9

Merged
merged 41 commits into from
Jun 10, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
29a0539
Adapt dynamic_libs.yaml so that it uploads libs as artifacts.
Zurcusa Jun 6, 2024
c493df0
Add condition when copying native libs to build dir.
Zurcusa Jun 6, 2024
db2deda
Test fix pattern.
Zurcusa Jun 6, 2024
7222d93
Test fix pattern.
Zurcusa Jun 6, 2024
844cb9f
Remove matrix.
Zurcusa Jun 6, 2024
2e75294
Test remove pattern from artifact download.
Zurcusa Jun 6, 2024
f940624
Refactor dynamic_libs.yaml so that it's callable from release.yaml.
Zurcusa Jun 6, 2024
90b86e3
Add needs flag to release.yaml
Zurcusa Jun 6, 2024
a47ec64
Test list structure in native-libs directory.
Zurcusa Jun 6, 2024
7354687
Rework dynamic_libs.yaml so that it's reusable.
Zurcusa Jun 6, 2024
6b62a7d
Test param passs fix.
Zurcusa Jun 6, 2024
c2b3680
test
Zurcusa Jun 6, 2024
8d99d9b
test
Zurcusa Jun 6, 2024
e12ede0
Remove github.event from the reusable workflow.
Zurcusa Jun 6, 2024
67c552f
Use github.ref.type instead of github.ref
Zurcusa Jun 6, 2024
c414ee4
test
Zurcusa Jun 6, 2024
09ed8bd
test
Zurcusa Jun 6, 2024
3736a81
test
Zurcusa Jun 6, 2024
a49e03e
Remove test step from dynamic_libs_reusable.yaml.
Zurcusa Jun 6, 2024
bcf195d
Add docs.
Zurcusa Jun 6, 2024
257a9b5
Test using randomizes artifacts dir.
Zurcusa Jun 7, 2024
bdc6e44
Fix export syntax.
Zurcusa Jun 7, 2024
80e6301
Add section in README.md for env variable.
Zurcusa Jun 7, 2024
c18d01f
Change 'native-libs' to using env var.
Zurcusa Jun 7, 2024
a247e8b
test
Zurcusa Jun 7, 2024
c326cb8
test
Zurcusa Jun 7, 2024
3ccfb91
Push EXT_ARTIFACTS_DIR to gh env.
Zurcusa Jun 7, 2024
618b740
Add mkdir.
Zurcusa Jun 7, 2024
78e7574
Fix mkdir.
Zurcusa Jun 7, 2024
7106719
Fix mkdir.
Zurcusa Jun 7, 2024
4479461
Fix usage of env var in "with" section of step.
Zurcusa Jun 7, 2024
78b3643
Test
Zurcusa Jun 7, 2024
bac7a89
Remove comments.
Zurcusa Jun 7, 2024
01fb7d7
Remove leftovers.
Zurcusa Jun 7, 2024
c8a3b3c
fix: use "uname -m" when retrieving os arch in Native.java. Change wi…
Zurcusa Jun 9, 2024
92de665
fix: translate x86_64 to amd64 to keep uniform naming of artifacts.
Zurcusa Jun 9, 2024
b77263d
Substitute random uuid for mktemp command.
Zurcusa Jun 10, 2024
cc1f341
Test fix env var assign
Zurcusa Jun 10, 2024
bd53621
Test fix env var assign
Zurcusa Jun 10, 2024
2963391
Test remove cache steps.
Zurcusa Jun 10, 2024
59d0a33
Revert cache removal.
Zurcusa Jun 10, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
134 changes: 134 additions & 0 deletions .github/workflows/build_publish.yaml
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left the workflow trigger to "on push" because the Workflow UI would only update when this is merged to master branch. From what I see that doesn't happen often.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Manual invocation wold be better imo.

Original file line number Diff line number Diff line change
@@ -0,0 +1,134 @@
# This workflow compiles the native wasmer_jni lib into various OS and architectures using dynamic_libs_reusable.yaml.
# Then it builds and publishes the JAR based on the GITHUB_REF that calls the workflow. There are 2 options:
# 1. The workflow is executed from a branch. In that case the JAR is available in the GitHub action as an artifact.
# 2. The workflow is executed from a tag. In that case the JAR is published in the corresponding GitHub release.
name: Build and publish

on:
push:
branches:
- '*' # Trigger on push to any branch
tags:
- '*' # Trigger on push to any tag

jobs:
upload_dynamic_libs:
strategy:
matrix:
platform:
- os: 'macos-latest'
target: 'aarch64-apple-darwin'
artifact: 'darwin-arm64'
lib_name: 'libwasmer_jni.dylib'

- os: 'macos-latest'
target: 'x86_64-apple-darwin'
artifact: 'darwin-amd64'
lib_name: 'libwasmer_jni.dylib'

- os: 'ubuntu-latest'
target: 'x86_64-unknown-linux-gnu'
artifact: 'linux-amd64'
lib_name: 'libwasmer_jni.so'

- os: 'windows-latest'
target: 'x86_64-pc-windows-msvc'
artifact: 'windows-amd64'
lib_name: 'wasmer_jni.dll'
uses: ./.github/workflows/dynamic_libs_reusable.yaml
with:
platform_os: ${{ matrix.platform.os }}
platform_target: ${{ matrix.platform.target }}
platform_artifact: ${{ matrix.platform.artifact }}
platform_lib_name: ${{ matrix.platform.lib_name }}

publish_jar:
name: Publish the JARs
runs-on: ubuntu-latest
needs: [upload_dynamic_libs]

steps:
- name: Check out code
uses: actions/checkout@v2

- name: Set up Java 21
uses: actions/setup-java@v1
with:
java-version: 21

- name: Set up Rust
uses: actions-rs/toolchain@v1
with:
toolchain: stable
default: true
override: true

- name: Cache Cargo registry
uses: actions/cache@v1
with:
path: ~/.cargo/registry
key: cargo-registry-${{ hashFiles('**/Cargo.lock') }}

- name: Cache Cargo bin
uses: actions/cache@v1
with:
path: ~/.cargo/bin
key: cargo-bin-${{ hashFiles('**/Cargo.lock') }}

- name: Cache Cargo build
uses: actions/cache@v1
with:
path: target
key: cargo-build-target-${{ hashFiles('**/Cargo.lock') }}

- name: Setup wasmer-jni artifacts dir
shell: bash
run: |
echo "EXT_ARTIFACTS_DIR=$(mktemp -d)" >> $GITHUB_ENV

- name: Download wasmer_jni artifacts
uses: actions/download-artifact@v4
with:
path: ${{ env.EXT_ARTIFACTS_DIR }}

- name: Display structure of downloaded files
run: ls -R $EXT_ARTIFACTS_DIR

- name: Run all the tests
shell: bash
run: |
export PATH="$HOME/.cargo/bin:$PATH"
make test

- name: Create the JAR
id: create_jar
shell: bash
run: |
export PATH="$HOME/.cargo/bin:$PATH"
make package

- name: Upload JAR as workflow artifact
if: startsWith(github.ref, 'refs/heads/')
uses: actions/upload-artifact@v4
with:
name: ${{ steps.create_jar.outputs.name }}
path: ${{ steps.create_jar.outputs.path }}
retention-days: 1

- name: Get release info
id: get_release_info
if: startsWith(github.ref, 'refs/tags/')
uses: bruceadams/[email protected]
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

- name: Upload JAR as Github pre-release asset
if: startsWith(github.ref, 'refs/tags/')
uses: actions/upload-release-asset@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
upload_url: ${{ steps.get_release_info.outputs.upload_url }}
asset_path: ${{ steps.create_jar.outputs.path }}
asset_name: ${{ steps.create_jar.outputs.name }}
asset_content_type: application/java-archive
67 changes: 0 additions & 67 deletions .github/workflows/dynamic_libs.yaml

This file was deleted.

53 changes: 53 additions & 0 deletions .github/workflows/dynamic_libs_reusable.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
# This workflow cross compiles a rust library into a native based on a provided target.
# The compiled library is then uploaded as a GitHub artifact.
name: upload_dynamic_libs

on:
workflow_call:
inputs:
platform_os:
description: Operating system that the runner will use.
type: string
required: true
platform_target:
description: The architecture that rust will compile the libs for.
type: string
required: true
platform_artifact:
description: The name of the artifact that will be uploaded.
type: string
required: true
platform_lib_name:
description: Name of the native library to be uploaded in the artifact.
type: string
required: true


jobs:
publish_dlibs:
name: Publish the dynamic libraries
runs-on: ${{ inputs.platform_os }}
steps:
- name: Check out code
uses: actions/checkout@v3

- name: Set up Rust
uses: actions-rs/toolchain@v1
with:
toolchain: stable
default: true
override: true

- name: Create the dynamic libs
shell: bash
run: |
export PATH="$HOME/.cargo/bin:$PATH"
rustup target add ${{ inputs.platform_target }}
cargo build --release --target=${{ inputs.platform_target }}

- name: Upload the dynamic libs
uses: actions/upload-artifact@v4
with:
name: ${{ inputs.platform_artifact }}
path: ./target/${{ inputs.platform_target }}/release/${{ inputs.platform_lib_name }}
retention-days: 1
85 changes: 0 additions & 85 deletions .github/workflows/mac_release.yaml

This file was deleted.

Loading
Loading