Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Gradle plugin proof-of-concept #21

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
73 changes: 11 additions & 62 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,72 +1,21 @@

config ?= compileClasspath

ifdef module
mm = :${module}:
else
mm =
endif
# Build the plugin
assemble:
./gradlew assemble

clean:
rm -rf .nextflow*
rm -rf work
rm -rf build
rm -rf plugins/*/build
./gradlew clean

compile:
./gradlew :nextflow:exportClasspath compileGroovy
@echo "DONE `date`"


check:
./gradlew check


#
# Show dependencies try `make deps config=runtime`, `make deps config=google`
#
deps:
./gradlew -q ${mm}dependencies --configuration ${config}

deps-all:
./gradlew -q dependencyInsight --configuration ${config} --dependency ${module}

#
# Refresh SNAPSHOTs dependencies
#
refresh:
./gradlew --refresh-dependencies

#
# Run all tests or selected ones
#
# Run plugin unit tests
test:
ifndef class
./gradlew ${mm}test
else
./gradlew ${mm}test --tests ${class}
endif

assemble:
./gradlew assemble

#
# generate build zips under build/plugins
# you can install the plugin copying manually these files to $HOME/.nextflow/plugins
#
buildPlugins:
./gradlew copyPluginZip

#
# Upload JAR artifacts to Maven Central
#
upload:
./gradlew upload

./gradlew test

upload-plugins:
./gradlew plugins:upload
# Install the plugin into local nextflow plugins dir
install:
./gradlew install

publish-index:
./gradlew plugins:publishIndex
# Publish the plugin
release:
./gradlew releasePlugin
109 changes: 37 additions & 72 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,114 +1,79 @@
# nf-hello plugin
# nf-hello plugin

This project contains a simple Nextflow plugin called `nf-hello` which provides examples of different plugin extensions:

- A custom trace observer that prints a message when the workflow starts and when the workflow completes
- A custom channel factory called `reverse`
- A custom operator called `goodbye`
- A custom function called `randomString`

NOTE: If you want to use this project as a starting point for a custom plugin, you must rename the `plugins/nf-hello` folder and update `settings.gradle` with your plugin name.
NOTE: If you want to use this project as a starting point for a custom plugin, you must rename the `plugins/nf-hello`
folder and update `settings.gradle` with your plugin name.

See the [Nextflow documentation](https://nextflow.io/docs/latest/plugins.html) for more information about developing plugins.
See the [Nextflow documentation](https://nextflow.io/docs/latest/plugins.html) for more information about developing
plugins.

## Plugin structure

- `settings.gradle`

Gradle project settings.

- `plugins/nf-hello`

The plugin implementation base directory.

- `plugins/nf-hello/build.gradle`

Plugin Gradle build file. Project dependencies should be added here.

- `plugins/nf-hello/src/resources/META-INF/MANIFEST.MF`

Manifest file defining the plugin attributes e.g. name, version, etc. The attribute `Plugin-Class` declares the plugin main class. This class should extend the base class `nextflow.plugin.BasePlugin` e.g. `nextflow.hello.HelloPlugin`.
- `settings.gradle`

- `plugins/nf-hello/src/resources/META-INF/extensions.idx`

This file declares one or more extension classes provided by the plugin. Each line should contain the fully qualified name of a Java class that implements the `org.pf4j.ExtensionPoint` interface (or a sub-interface).
Gradle project settings.

- `plugins/nf-hello/src/main`
- `src/main`

The plugin implementation sources.
The plugin implementation sources.

- `plugins/nf-hello/src/test`
- `src/test`

The plugin unit tests.
The plugin unit tests.

## Plugin classes

- `HelloConfig`: shows how to handle options from the Nextflow configuration

- `HelloExtension`: shows how to create custom channel factories, operators, and fuctions that can be included into pipeline scripts
- `HelloExtension`: shows how to create custom channel factories, operators, and fuctions that can be included into
pipeline scripts

- `HelloFactory` and `HelloObserver`: shows how to react to workflow events with custom behavior

- `HelloPlugin`: the plugin entry point

## Unit testing

To run your unit tests, run the following command in the project root directory (ie. where the file `settings.gradle` is located):
## Building

To compile and assemble the plugin, run the following command in the project root directory:
```bash
./gradlew check
make assemble
```

## Testing and debugging

To build and test the plugin during development, configure a local Nextflow build with the following steps:
## Unit testing

1. Clone the Nextflow repository in your computer into a sibling directory:
```bash
git clone --depth 1 https://github.com/nextflow-io/nextflow ../nextflow
```

2. Configure the plugin build to use the local Nextflow code:
```bash
echo "includeBuild('../nextflow')" >> settings.gradle
```

(Make sure to not add it more than once!)

3. Compile the plugin alongside the Nextflow code:
```bash
make assemble
```

4. Run Nextflow with the plugin, using `./launch.sh` as a drop-in replacement for the `nextflow` command, and adding the option `-plugins nf-hello` to load the plugin:
```bash
./launch.sh run nextflow-io/hello -plugins nf-hello
```
To run your unit tests, run the following command in the project root directory:
```bash
make test
```

## Testing without Nextflow build
## Testing with Nextflow

The plugin can be tested without using a local Nextflow build using the following steps:
The plugin can be tested without a local Nextflow installation:

1. Build the plugin: `make buildPlugins`
2. Copy `build/plugins/<your-plugin>` to `$HOME/.nextflow/plugins`
3. Create a pipeline that uses your plugin and run it: `nextflow run ./my-pipeline-script.nf`
1. Build and install the plugin to your local Nextflow installation: `make install`
2. Create a pipeline that uses your plugin and run it: `nextflow run ./my-pipeline-script.nf`

## Package, upload, and publish

The project should be hosted in a GitHub repository whose name matches the name of the plugin, that is the name of the directory in the `plugins` folder (e.g. `nf-hello`).

Follow these steps to package, upload and publish the plugin:

1. Create a file named `gradle.properties` in the project root containing the following attributes (this file should not be committed to Git):
1. In `build.gradle` make sure that
* `github.repository` matches the repository of the plugin
* `github.indexUrl` points to your fork of the plugins index repository.

2. Create a file named `$HOME/.gradle/gradle.properties`, where $HOME is your home directory. Add the following properties:

* `github_organization`: the GitHub organisation where the plugin repository is hosted.
* `github_username`: The GitHub username granting access to the plugin repository.
* `github_access_token`: The GitHub access token required to upload and commit changes to the plugin repository.
* `github_commit_email`: The email address associated with your GitHub account.
* `github_username`: The GitHub username granting access to the plugin repository.
* `github_access_token`: The GitHub access token required to upload and commit changes to the plugin repository.
* `github_commit_email`: The email address associated with your GitHub account.

2. Use the following command to package and create a release for your plugin on GitHub:
```bash
./gradlew :plugins:nf-hello:upload
```
3. Use the following command to package and create a release for your plugin on GitHub: `make release`

3. Create a pull request against [nextflow-io/plugins](https://github.com/nextflow-io/plugins/blob/main/plugins.json) to make the plugin accessible to Nextflow.
4. Create a pull request against [nextflow-io/plugins](https://github.com/nextflow-io/plugins/blob/main/plugins.json) to
make the plugin accessible to Nextflow.
30 changes: 30 additions & 0 deletions build.gradle
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
plugins {
id 'io.nextflow.nextflow-plugin' version '0.0.1-alpha'
}

// plugin version
version = '0.6.0'

nextflowPlugin {
// minimum nextflow version
nextflowVersion = '24.11.0-edge'

// plugin metadata
provider = 'nextflow'
className = 'nextflow.hello.HelloPlugin'
extensionPoints = [
'nextflow.hello.HelloFactory',
'nextflow.hello.HelloExtension'
]

publishing {
github {
repository = 'nextflow-io/nf-hello'
userName = project.findProperty('github_username')
authToken = project.findProperty('github_access_token')
email = project.findProperty('github_commit_email')

indexUrl = 'https://github.com/nextflow-io/plugins/blob/main/plugins.json'
}
}
}
14 changes: 0 additions & 14 deletions buildSrc/build.gradle

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

Binary file modified gradle/wrapper/gradle-wrapper.jar
Binary file not shown.
4 changes: 3 additions & 1 deletion gradle/wrapper/gradle-wrapper.properties
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
distributionBase=GRADLE_USER_HOME
distributionPath=wrapper/dists
distributionUrl=https\://services.gradle.org/distributions/gradle-8.4-bin.zip
distributionUrl=https\://services.gradle.org/distributions/gradle-8.10.2-bin.zip
networkTimeout=10000
validateDistributionUrl=true
zipStoreBase=GRADLE_USER_HOME
zipStorePath=wrapper/dists
Loading
Loading