Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Restructure configuration to prevent conflicts with other OTA applications #1071

Merged
merged 11 commits into from
May 22, 2024
6 changes: 3 additions & 3 deletions .env.example
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
SENDINBLUE_API_KEY='xkeysib-3f51c…'
SMTP_PASSWORD='password'
GITHUB_TOKEN=ghp_XXXXXXXXX
OTA_ENGINE_SENDINBLUE_API_KEY='xkeysib-3f51c…'
OTA_ENGINE_SMTP_PASSWORD='password'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's best to avoid using the abbreviation OTA.

Suggested change
OTA_ENGINE_SENDINBLUE_API_KEY='xkeysib-3f51c…'
OPENTERMSARCHIVE_ENGINE_SENDINBLUE_API_KEY='xkeysib-3f51c…'

OTA_ENGINE_GITHUB_TOKEN=ghp_XXXXXXXXX
4 changes: 2 additions & 2 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ jobs:
- uses: ankane/setup-mongodb@v1
with:
mongodb-version: 5.0
- uses: actions/setup-node@v3
- uses: actions/setup-node@v4
with:
node-version: 20
- run: npm ci
Expand Down Expand Up @@ -54,7 +54,7 @@ jobs:
with:
repository: OpenTermsArchive/demo-declarations
path: ./demo-declarations
- uses: actions/setup-node@v3
- uses: actions/setup-node@v4
with:
node-version: 20
- run: cd ./demo-declarations && npm install
Expand Down
11 changes: 11 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,17 @@

All changes that impact users of this module are documented in this file, in the [Common Changelog](https://common-changelog.org) format with some additional specifications defined in the CONTRIBUTING file. This codebase adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## Unreleased [major]

> Development of this release was supported by the [French Ministry for Foreign Affairs](https://www.diplomatie.gouv.fr/fr/politique-etrangere-de-la-france/diplomatie-numerique/) through its ministerial [State Startups incubator](https://beta.gouv.fr/startups/open-terms-archive.html) under the aegis of the Ambassador for Digital Affairs.

### Changed

- **Breaking:** Nest all configurations under the `@opentermsarchive/engine` key; wrap the content of `config/production.json` like this: `{ @opentermsarchive/engine: { < your previous configuration >}}`
- **Breaking:** Rename the `api` configuration key to `collection-api`; update this in your `config/production.json`
- **Breaking:** Remove default values for `port` and `basePath` in `collection-api`; specify these configurations explicitly in your `config/production.json`
MattiSG marked this conversation as resolved.
Show resolved Hide resolved
- **Breaking:** Prefix all environment variables with `OTA_ENGINE_`; rename variables such as `SENDINBLUE_API_KEY` to `OTA_ENGINE_SENDINBLUE_API_KEY`, `SMTP_PASSWORD` to `OTA_ENGINE_SMTP_PASSWORD`, and `GITHUB_TOKEN` to `OTA_ENGINE_GITHUB_TOKEN`

## 1.3.0 - 2024-05-22

_Full changeset and discussions: [#1076](https://github.com/OpenTermsArchive/engine/pull/1076)._
Expand Down
2 changes: 2 additions & 0 deletions bin/env.js
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
import 'dotenv/config';

import path from 'path';
import { fileURLToPath } from 'url';

Expand Down
2 changes: 1 addition & 1 deletion bin/ota-dataset.js
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ program
.name('ota dataset')
.description('Export the versions dataset into a ZIP file and optionally publish it to GitHub releases')
.option('-f, --file <filename>', 'file name of the generated dataset')
.option('-p, --publish', 'publish dataset to GitHub releases on versions repository. Mandatory authentication to GitHub is provided through the `GITHUB_TOKEN` environment variable')
.option('-p, --publish', 'publish dataset to GitHub releases on versions repository. Mandatory authentication to GitHub is provided through the `OTA_ENGINE_GITHUB_TOKEN` environment variable')
.option('-r, --remove-local-copy', 'remove local copy of dataset after publishing. Works only in combination with --publish option')
.option('--schedule', 'schedule automatic dataset generation');

Expand Down
2 changes: 1 addition & 1 deletion bin/ota-serve.js
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,4 @@ import { fileURLToPath, pathToFileURL } from 'url';

const __dirname = path.dirname(fileURLToPath(import.meta.url));

await import(pathToFileURL(path.resolve(__dirname, '../src/api/server.js'))); // load asynchronously to ensure env.js is loaded before
await import(pathToFileURL(path.resolve(__dirname, '../src/collection-api/server.js'))); // load asynchronously to ensure env.js is loaded before
6 changes: 4 additions & 2 deletions config/ci.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
{
"services": {
"declarationsPath": "./demo-declarations/declarations"
"@opentermsarchive/engine": {
"services": {
"declarationsPath": "./demo-declarations/declarations"
}
}
}
108 changes: 53 additions & 55 deletions config/default.json
Original file line number Diff line number Diff line change
@@ -1,65 +1,63 @@
{
"services": {
"declarationsPath": "./declarations"
},
"recorder": {
"versions": {
"storage": {
"type": "git",
"git": {
"path": "./data/versions",
"publish": false,
"snapshotIdentiferTemplate": "./data/snapshots/%SNAPSHOT_ID",
"author": {
"name": "Open Terms Archive Bot",
"email": "[email protected]"
"@opentermsarchive/engine": {
"services": {
"declarationsPath": "./declarations"
},
"recorder": {
"versions": {
"storage": {
"type": "git",
"git": {
"path": "./data/versions",
"publish": false,
"snapshotIdentiferTemplate": "./data/snapshots/%SNAPSHOT_ID",
"author": {
"name": "Open Terms Archive Bot",
"email": "[email protected]"
}
}
}
}
},
"snapshots": {
"storage": {
"type": "git",
"git": {
"path": "./data/snapshots",
"publish": false,
"author": {
"name": "Open Terms Archive Bot",
"email": "[email protected]"
},
"snapshots": {
"storage": {
"type": "git",
"git": {
"path": "./data/snapshots",
"publish": false,
"author": {
"name": "Open Terms Archive Bot",
"email": "[email protected]"
}
},
"mongo": {
"connectionURI": "mongodb://127.0.0.1:27017",
"database": "open-terms-archive",
"collection": "snapshots"
}
},
"mongo": {
"connectionURI": "mongodb://127.0.0.1:27017",
"database": "open-terms-archive",
"collection": "snapshots"
}
}
}
},
"fetcher": {
"waitForElementsTimeout": 10000,
"navigationTimeout": 30000,
"language": "en"
},
"logger": {
"smtp": {
"host": "smtp-relay.sendinblue.com",
"username": "[email protected]"
},
"sendMailOnError": false
},
"notifier": {
"sendInBlue": {
"updatesListId": 850,
"updateTemplateId": 7
"fetcher": {
"waitForElementsTimeout": 10000,
"navigationTimeout": 30000,
"language": "en"
},
"logger": {
"smtp": {
"host": "smtp-relay.sendinblue.com",
"username": "[email protected]"
},
"sendMailOnError": false
},
"notifier": {
"sendInBlue": {
"updatesListId": 850,
"updateTemplateId": 7
}
},
"dataset": {
"title": "sandbox",
"versionsRepositoryURL": "https://github.com/OpenTermsArchive/sandbox"
}
},
"dataset": {
"title": "sandbox",
"versionsRepositoryURL": "https://github.com/OpenTermsArchive/sandbox"
},
"api": {
"port": 3000,
"basePath": "/api"
}
}
40 changes: 21 additions & 19 deletions config/production.json
Original file line number Diff line number Diff line change
@@ -1,27 +1,29 @@
{
"recorder": {
"versions": {
"storage": {
"git": {
"path": "../versions",
"publish": true
"@opentermsarchive/engine": {
"recorder": {
"versions": {
"storage": {
"git": {
"path": "../versions",
"publish": true
}
}
},
"snapshots": {
"storage": {
"git": {
"path": "../snapshots",
"publish": true
}
}
}
},
"snapshots": {
"storage": {
"git": {
"path": "../snapshots",
"publish": true
}
"logger": {
"sendMailOnError": {
"to": "[email protected]",
"from": "[email protected]",
"sendWarnings": false
}
}
},
"logger": {
"sendMailOnError": {
"to": "[email protected]",
"from": "[email protected]",
"sendWarnings": false
}
}
}
86 changes: 46 additions & 40 deletions config/test.json
Original file line number Diff line number Diff line change
@@ -1,49 +1,55 @@
{
"services": {
"declarationsPath": "./test/services"
},
"recorder": {
"versions": {
"storage": {
"git": {
"path": "./test/data/versions",
"publish": false,
"snapshotIdentiferTemplate": "https://github.com/OpenTermsArchive/sandbox-snapshots/commit/%SNAPSHOT_ID",
"author": {
"name": "Open Terms Archive Testing Bot",
"email": "[email protected]"
"@opentermsarchive/engine": {
"services": {
"declarationsPath": "./test/services"
},
"recorder": {
"versions": {
"storage": {
"git": {
"path": "./test/data/versions",
"publish": false,
"snapshotIdentiferTemplate": "https://github.com/OpenTermsArchive/sandbox-snapshots/commit/%SNAPSHOT_ID",
"author": {
"name": "Open Terms Archive Testing Bot",
"email": "[email protected]"
}
},
"mongo": {
"connectionURI": "mongodb://127.0.0.1:27017",
"database": "open-terms-archive-test",
"collection": "versions"
}
},
"mongo": {
"connectionURI": "mongodb://127.0.0.1:27017",
"database": "open-terms-archive-test",
"collection": "versions"
}
}
},
"snapshots": {
"storage": {
"git": {
"path": "./test/data/snapshots",
"publish": false,
"author": {
"name": "Open Terms Archive Testing Bot",
"email": "[email protected]"
},
"snapshots": {
"storage": {
"git": {
"path": "./test/data/snapshots",
"publish": false,
"author": {
"name": "Open Terms Archive Testing Bot",
"email": "[email protected]"
}
},
"mongo": {
"connectionURI": "mongodb://127.0.0.1:27017",
"database": "open-terms-archive-test",
"collection": "snapshots"
}
},
"mongo": {
"connectionURI": "mongodb://127.0.0.1:27017",
"database": "open-terms-archive-test",
"collection": "snapshots"
}
}
},
"fetcher": {
"waitForElementsTimeout": 1000
},
"dataset": {
"title": "sandbox",
"versionsRepositoryURL": "https://github.com/OpenTermsArchive/sandbox-versions"
},
"collection-api": {
"port": 3000,
"basePath": "/collection-api"
}
},
"fetcher": {
"waitForElementsTimeout": 1000
},
"dataset": {
"title": "sandbox",
"versionsRepositoryURL": "https://github.com/OpenTermsArchive/sandbox-versions"
}
}
Loading
Loading