Skip to content

Commit

Permalink
Merge pull request #429 from OpenFn/portability_edit
Browse files Browse the repository at this point in the history 
Portability edit
  • Loading branch information
@aleksa-krolls
aleksa-krolls authored Mar 4, 2024
2 parents f35a87a + c81aba6 commit 2809b91
Show file tree
Hide file tree
Showing 2 changed files with 73 additions and 195 deletions.
107 changes: 1 addition & 106 deletions docs/deploy/portability.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -219,7 +219,7 @@ generate these files and use them for auditing and record keeping, etc. The
OpenFn [CLI](https://github.com/OpenFn/kit/tree/main/packages/cli) comes with
commands that can be used to pull project configurations down from a running
Lightning server, and to deploy or push updates to existing projects on a
Lightning server.
Lightning server. To learn more about automated version control via pull and deploy, head over to our [Version Control](../manage-projects/link-to-gh.md) docs.

:::info Don't have the CLI yet?

Expand Down Expand Up @@ -307,111 +307,6 @@ Project found.
[CLI] ♦ Deployed.
```

### Automated Version Control with Github and Lightning

Representations of projects as code and pull/deploy functionality allows you to
check your whole project into a version control system such as Github.

Lightning comes with a Github App that enables user to sync projects from an
instance to Github using the `openfn pull` command and to do the vice versa
using `openfn deploy`.

To set up version control:

1. Create a project repo connection to a github repository in **Project Settings
-> Sync to Github**.
2. Follow the instructions to install the Lightning Github app in your desired
repository.
3. Once you have created a a connection, set up `pull` and `deploy` workflows
that use openfn github actions below.
4. Add `OPENFN_API_KEY` and `OPENFN_PROJECT_ID` repository secrets to your
Github repo as described below.
5. Add a `.config.json` file to your repository which specifies your endpoint
and paths to project spec and state files.
6. Click the sync to Github button to initiate a sync from Lightning to GitHub.
7. Push a change to your selected branch to push changes from Github to
Lightning.

#### Github Repository Secrets

The workflows that interact with the OpenFn actions will need the repository set
up with two secrets used in the github actions:

- OPENFN_API_KEY: This is your API Key as generated from Lightning and will be
needed for authentication
- OPENFN_PROJECT_ID: This is your Project ID from Lightning this will be used to
pull from the lightning instance

#### Github Repository Structure

Here you can do pretty much what you want, so long as you've got a `config.json`
pointing to your project spec, state, and Lightning endpoint.

#### Example [Deploy Workflow](https://github.com/OpenFn/demo-openhie/blob/main/.github/workflows/deploy.yml) for GitHub

See https://docs.github.com/en/actions/quickstart#creating-your-first-workflow
for more help here.

```yml
on:
push:
branches:
- main

jobs:
deploy-to-lightning:
runs-on: ubuntu-latest
name: A job to deploy to Lightning
steps:
- name: openfn deploy
uses: OpenFn/[email protected]
with:
secret_input: ${{ secrets.OPENFN_API_KEY }}
```
#### Example [Pull Workflow](https://github.com/OpenFn/demo-openhie/blob/main/.github/workflows/pull.yml) for GitHub
See https://docs.github.com/en/actions/quickstart#creating-your-first-workflow
for more help here.
```yml
on: [repository_dispatch]

jobs:
pull-from-lightning:
runs-on: ubuntu-latest
permissions:
contents: write
name: A job to pull changes from Lightning
steps:
- name: openfn pull and commit
uses: OpenFn/[email protected]
with:
secret_input: ${{ secrets.OPENFN_API_KEY }}
project_id_input: ${{ secrets.OPENFN_PROJECT_ID }}
commit_message_input:
'user ${{ github.event.client_payload.message }}'
```
The Lightning [demo instance](https://demo.openfn.org) is currently connected to
[this repo](https://github.com/OpenFn/demo-openhie/). Feel free to play around
with it.
#### Using version control
##### Lightning to GitHub
Once you have configured version control for a project and a related repository
branch, you can sync changes to GitHub by pressing the "Initiate Sync" button on
the version control page and the Lightning GitHub app will run a `openfn pull`
action to update the versioned representation of your project as code.

##### Github to Lightning

Assuming you've configured a deploy action, any time there are changes made to
that branch in your GitHub repo, those changes will be pushed to your Lightning
project via `openfn deploy`.

### Getting Help with the cli

The cli package comes with an inbuilt `help`. Adding `--help` to a command such
Expand Down
161 changes: 72 additions & 89 deletions docs/manage-projects/link-to-gh.md
View file
Original file line number Diff line number Diff line change
@@ -1,30 +1,47 @@
---
title: Linking Your Project to GitHub (Version Control)
sidebar_label: Linking Your Project to GitHub
slug: /link-to-github
title: Linking Your Project to Github (Version Control)
sidebar_label: Linking Your Project to Github
slug: /link-to-Github
---

You can set up version control for your OpenFn projects. This article walks you
through the configuration steps.

### Setting up GitHub integration
### How It Works in a Nutshell

Your whole OpenFn project can be represented as a `project.yaml` file.

The `Sync to Github` feature makes use of Github actions to automatically deploy
(after a commit) or pull (when `Initiate Sync to Branch` button is clicked) to
keep a repository in sync with your OpenFn project.

Using our Command Line Interface, the [@openfn/cli](../deploy/portability.md)
you can pull a project config from OpenFn to a folder or repo on your computer,
and you can deploy a change in your `project.yaml` file from that directory or
repo to OpenFn.

For more detailed information on representing your project as code and using the
@openfn/cli, head over to our documentation on
[Portability](../deploy/portability.md).

### Setting Up Github Integration

You can connect your OpenFn project to a GitHub repository that you have
You can connect your OpenFn project to a Github repository that you have
administrator access to. This enables a 2-way sync - you can sync your project
to GitHub, and you can deploy changes you've made outside of OpenFn to your
to Github, and you can deploy changes you've made outside of OpenFn to your
project.

1. Navigate to `Project Settings > Sync to GitHub` .
1. Navigate to `Project Settings > Sync to Github` .

![Connect to GitHub](/img/lightning_connect_to_gh.png)
![Connect to Github](/img/lightning_connect_to_gh.png)

2. Select `Connect Branch`. You'll be prompted to sign in to GitHub.
2. Select `Connect Branch`. You'll be prompted to sign in to Github.

Click `Configure`...

![Configure](/img/lightning_gh_configure.png)

3. ...then select the GitHub account that owns the repository you want to
3. ...then select the Github account that owns the repository you want to
connect to.

![Install](/img/lightning_gh_install_openfn.png)
Expand All @@ -34,99 +51,65 @@ Click `Configure`...
![Permissions](/img/lightning_gh_permissions.png)

5. You'll be redirected to OpenFn. Select the repository and branch you want to
sync to, and click `Connect Branch`.

6. Once you have created a a connection, set up `pull` and `deploy` workflows
that use openfn github actions below.
7. Add `OPENFN_API_KEY` and `OPENFN_PROJECT_ID` repository secrets to your
sync to.
6. The Github sync process makes use of `pull` and `deploy` Github actions to
sync between your OpenFn project and Github repo (more on this under
`How it works` below.) In order to do this, a `pull.yml` and a `deploy.yml`
file will be committed to your chosen repository when you proceed to connect
your branch. If you have files with these names in your repo, they will be
overwritten.
7. Click `Connect Branch`.
8. Add `OPENFN_API_KEY` and `OPENFN_PROJECT_ID` repository secrets to your
Github repo as described below.
8. Add a `config.json` file to your repository which specifies your endpoint and
paths to project spec and state files.
9. Click the sync to Github button to initiate a sync from Lightning to GitHub.
10. Push a change to your selected branch to push changes from Github to
Lightning.

#### Github Repository Secrets

The workflows that interact with the OpenFn actions will need the repository set
up with two secrets used in the github actions:

- OPENFN_API_KEY: This is your API Key as generated from Lightning and will be
- OPENFN_API_KEY: This is your API Key as generated from OpenFn and will be
needed for authentication
- OPENFN_PROJECT_ID: This is your Project ID from Lightning this will be used to
pull from the lightning instance

#### Github Repository Structure

Here you can do pretty much what you want, so long as you've got a `config.json`
pointing to your project spec, state, and Lightning endpoint.

#### Example [Deploy Workflow](https://github.com/OpenFn/demo-openhie/blob/main/.github/workflows/deploy.yml) for GitHub

See https://docs.github.com/en/actions/quickstart#creating-your-first-workflow
for more help here.

```yml
on:
push:
branches:
- main

jobs:
deploy-to-lightning:
runs-on: ubuntu-latest
name: A job to deploy to Lightning
steps:
- name: openfn deploy
uses: OpenFn/[email protected]
with:
secret_input: ${{ secrets.OPENFN_API_KEY }}
- OPENFN_PROJECT_ID: This is your Project ID from OpenFn this will be used to
pull from the OpenFn instance

9. Add a `config.json` file as below to your repository which specifies your
endpoint and paths to project spec and state files. Both of these files will
be automatically created when you first initiate a sync.

```json
{
"endpoint": "https://app.openfn.org",
"statePath": "./projectState.json",
"specPath": "./project.yaml"
}
```

#### Example [Pull Workflow](https://github.com/OpenFn/demo-openhie/blob/main/.github/workflows/pull.yml) for GitHub
See https://docs.github.com/en/actions/quickstart#creating-your-first-workflow
for more help here.
```yml
on: [repository_dispatch]

jobs:
pull-from-lightning:
runs-on: ubuntu-latest
permissions:
contents: write
name: A job to pull changes from Lightning
steps:
- name: openfn pull and commit
uses: OpenFn/[email protected]
with:
secret_input: ${{ secrets.OPENFN_API_KEY }}
project_id_input: ${{ secrets.OPENFN_PROJECT_ID }}
commit_message_input:
'user ${{ github.event.client_payload.message }}'
```
10. Click the sync to Github button to initiate a sync from OpenFn to GitHub.
11. Push a change to your selected branch to push changes from Github to OpenFn.
12. Your GitHub connection is now set up!

The Lightning [demo instance](https://demo.openfn.org) is currently connected to
[this repo](https://github.com/OpenFn/demo-openhie/). Feel free to play around
with it.
#### Using Version Control

11. Your GitHub connection is now set up! From here, each time you want to sync
between your project and GitHub, click `Initiate Sycn to Branch`.
##### OpenFn to Github

### How It Works in a Nutshell
Each time you want to sync between your project and Github, click the
`Initiate Sync to Branch` button on the version control page and the OpenFn
Github app will run a `openfn pull` action to update the versioned
representation of your project as code.

Your whole OpenFn project can be represented as a project.yaml file.
##### Github to OpenFn

Using our Command Line Interface, the [@openfn/cli](../deploy/portability.md)
you can pull a project config from OpenFn to a folder or repo on your computer,
and you can deploy a change in your project.yaml file from that directory or
repo to OpenFn.
Any time there are changes made to that branch in your Github repo, those
changes will be pushed to your OpenFn project via `openfn deploy`. Note that as
your entire project is represented in your `project.yaml` file, you have to add
all your changes to this file in order for them to be deployed to your OpenFn
project when syncing.

The `Sync to Github` feature makes use of GitHub actions to automatically deploy
(after a commit) or pull (when `Initiate Sync to Branch` button is clicked) to
keep a repository in sync with your OpenFn project.
#### Github Repository Structure

For more detailed information on representing your project as code and using the
@openfn/cli, head over to our documentation on
[Portability](../deploy/portability.md).
Here you can do pretty much what you want, so long as you've got a `config.json`
pointing to your project spec, state, and OpenFn endpoint.

The OpenFn v2 (Lightning) [demo instance](https://demo.openfn.org) is currently
connected to [this repo](https://Github.com/OpenFn/demo-openhie/). Feel free to
play around with it.

0 comments on commit 2809b91

Please sign in to comment.