Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to manage infra changes via otterdog #743

Merged
merged 10 commits into from
Oct 17, 2023
Binary file added docs/assets/bitwarden/ui_get_id.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/github/ui_get_org_id.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/otterdog/dashboard.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/otterdog/otterdog_sample_pr.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/otterdog/playground.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
100 changes: 97 additions & 3 deletions docs/infra/bitwarden.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,107 @@
# Bitwarden

A [password tool](https://bitwarden.com) to help us manage our [organizations secrets](https://vault.bitwarden.com/#/vault?organizationId=e429b264-5b49-4794-be50-b0660125456a).
Another potential tool is the [pass CLI](https://www.passwordstore.org) where `pass` is for password not _our_ Open Access PASS.

Here is an outline of passwords we are tracking
(duplicating here as can better document the integrations with other services
like [Otterdog](/docs/infra/otterdog.md) and Github Actions).

![Bitwarden Secrets User Interface](/docs/assets/bitwarden/ui.png)

| Secret | Description |
| --- | --- |
| HELLO_WORLD | A test secret for demontsrating how secrets are managed |
| Item Name | Item Id | Field | Description |
| --- | --- | --- | --- |
| HELLO_WORLD | 23801ca4-fd27-446c-b5af-b07b0108f443 | quest | A test secret of `holy_grail` for demontsrating how secrets are managed |
| HELLO_WORLD | 23801ca4-fd27-446c-b5af-b07b0108f443 | color | Another test secret of `blue_no_ahh` |

## Installing CLI

Some instructions for installing the [bitwarden cli are here](https://github.com/bitwarden/clients).
The tools command line name is `bw` and you can see which version you are running with

```bash
bw -v
```

To install it via [homebrew](https://brew.sh) you can run

```bash
brew install bitwarden-cli
```

You can log into the account with

```bash
bw login
```

And then you can list the passwords (in plain text, be careful).

```bash
bw list items --search HELLO_WORLD
```

Note these are NOT sensitive passwords so it's OK to be shown

```bash
[
{
"passwordHistory":
[
{
"lastUsedDate": "2023-09-26T17:32:20.947Z",
"password": "HELLO_WORLD"
}
],
"revisionDate": "2023-09-26T17:39:02.833Z",
"creationDate": "2023-09-12T16:04:39.793Z",
"deletedDate": null,
"object": "item",
"id": "23801ca4-fd27-446c-b5af-b07b0108f443",
"organizationId": "e429b264-5b49-4794-be50-b0660125456a",
"folderId": null,
"type": 1,
"reprompt": 0,
"name": "HELLO_WORLD",
"notes": null,
"favorite": false,
"fields":
[
{
"name": "quest",
"value": "holy_grail",
"type": 1,
"linkedId": null
},
{
"name": "color",
"value": "blue_no_ahh",
"type": 1,
"linkedId": null
}
],
"login":
{
"username": null,
"password": "WORLD_HELLO",
"totp": null,
"passwordRevisionDate": "2023-09-26T17:32:20.947Z"
},
"collectionIds":
[
"af5a75d8-8762-4cec-8d96-b06601263cf1"
]
}
]
```


## Troubleshooting

### How do I get the ID of an item from the UI

You will need to **Inspect** the page and watch for the network traffic
after you edit the item.

![Bitwarden Secrets User Interface](/docs/assets/bitwarden/ui_get_id.png)

150 changes: 150 additions & 0 deletions docs/infra/otterdog.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,153 @@ using the [Otterdog tool](https://gitlab.eclipse.org/eclipsefdn/security/otterdo

Our first project will be to manage secrets via [bitwarden](/docs/infra/bitwarden.md).

## Pushing GitHub Infra Changes

To make changes, you must push a [forked PR like this one](https://github.com/eclipse-pass/.eclipsefdn/pull/1).

![Sample PR](/docs/assets/otterdog/otterdog_sample_pr.png)


## Configurations

The base configuration is [eclipse-pass.jsonnet](https://github.com/eclipse-pass/.eclipsefdn/blob/main/otterdog/eclipse-pass.jsonnet),
and can be monitored at [eclipse-pass.org/.eclipsefdn](https://eclipse-pass.org/.eclipsefdn/)

![Otterdog Config Monitoring](/docs/assets/otterdog/dashboard.png)

The base entry for jsonnett configs is via

```javascript
local orgs = import 'otterdog-defaults.libsonnet';
```

### Bitwarden Configs

To integrate [bitwarden into otterdog configs](https://gitlab.eclipse.org/eclipsefdn/security/otterdog#bitwarden)
the request is to add an _"organization"_ directly to the JSON, but instead lets use the jsonnett based
on the outputs from the [otterdog playground](http://eclipse-pass.org/.eclipsefdn/playground/).

```javascript
orgs.newOrg('eclipse-pass') {
credentials+: [{
"provider": "bitwarden",
"item_id" : "23801ca4-fd27-446c-b5af-b07b0108f443"
},
],
}
```

And then we can specify secrets based on the structure of `bitwarden:<item_id>@<field_name>`.

### Organizational Secrets

Here is documentation on managing [organization secrets](https://otterdog.readthedocs.io/en/latest/reference/organization/secret/)

```javascript
orgs.newOrg('eclipse-pass') {
secrets+: [
orgs.newOrgSecret('HELLO_WORLD_QUEST') {
value: "bitwarden:23801ca4-fd27-446c-b5af-b07b0108f443@quest",
},
orgs.newOrgSecret('HELLO_WORLD_COLOR') {
value: "bitwarden:23801ca4-fd27-446c-b5af-b07b0108f443@color",
},
],
}
```

Please refer to [bitwarden for specifics on password management](/docs/infra/bitwarden.md)

## Playground (Online Editor)

From the [monitoring application]((https://eclipse-pass.org/.eclipsefdn/)) you can access a [otterdog playground](http://eclipse-pass.org/.eclipsefdn/playground/)

![Otterdog Playground](/docs/assets/otterdog/playground.png)

## Installing Locally

If you are testing github integrations, you will need to [install Otterdog locally](https://otterdog.readthedocs.io/en/latest/install/)

Below is a summary of the [installation guide above](https://otterdog.readthedocs.io/en/latest/install/).
If you run into problems, then please read the source installation guide and update these docs.

You will need the following tools

```bash
python3 --version # 3.10+, tested on 3.11.4
pip --version # any, tested on 23.2
poetry --version # any, tested on 1.4.2
go version # 1.13+, tested on 1.20.5
jb --version # any, tested on 'dev' (aka 0.5.1 as shown in troubleshooting below)
bw -v # any, tested on 2023.9.0
```

Now you can clone the [Otterdog repo](https://gitlab.eclipse.org/eclipsefdn/security/otterdog/)
and build it locally

```bash
# Somewhere outside of the eclipse-pass repos
git clone [email protected]:eclipsefdn/security/otterdog.git
cd otterdog
make init
```

You now have `otterdog` installed locally.

```bash
otterdog --version
# otterdog.sh, version 0.3.0.dev0
```

### Troubleshooting

#### jb / jsonnet-bundler not found

Here's how you install [jsonnet-bundler](https://github.com/jsonnet-bundler/jsonnet-bundler) (aka `jb`)

```
go install -a github.com/jsonnet-bundler/jsonnet-bundler/cmd/[email protected]
```

Note that [jsonnet-bundler](https://github.com/jsonnet-bundler/jsonnet-bundler) was installed to a `$HOME/go/bin` which was NOT
on our path, so I had to add that to ensure `jb` was accessible.

```bash
# in your ~/.bash_profile or similar
PATH="$HOME/go/bin:$PATH"
```

#### bw / bitwarden not found

Here's how you install [bitwarden](https://github.com/bitwarden/clients) (aka `bw`).

The installation uses `snap`, but (home)`brew` also works

```bash
brew install bitwarden-cli
```

#### .local/bin/otterdog: No such file or directory

If you see something like

```bash
test -f ~/.local/bin/otterdog || ln -s /Users/aforward/sin/projects/eclipse-pass/otterdog/otterdog.sh ~/.local/bin/otterdog
ln: /Users/aforward/.local/bin/otterdog: No such file or directory
make: *** [init] Error 1
```

Then ensure that you have a `~/.local/bin` to receive the file, and that is also on your path

```bash
mkdir -p ~/.local/bin

# in your ~/.bash_profile or similar
PATH="$HOME/.local/bin:$PATH"
```

#### How do I get the github organization ID from the UI

You will need to **Inpsect** the org page and look for `organization:`

![GitHub Organization ID](/docs/assets/github/ui_get_org_id.png)
Loading