Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Corrected env docs; generate env var docs based on opts-specs #346

Merged
merged 4 commits into from
Dec 10, 2024
Merged

Corrected env docs; generate env var docs based on opts-specs #346

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
4318ba2
Fixed docs
mdemare Dec 6, 2024
068b2cc
Generate env var docs from opts-specs (single source of truth)
mdemare Dec 6, 2024
cb3f73c
STORE_HTTP_REQUESTS should use opts-specs
mdemare Dec 6, 2024
c39ef35
Review feedback
mdemare Dec 10, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 6 additions & 5 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -138,9 +138,8 @@ must be set:
```
API_HOSTNAME Hostname for listing web API
API_PORT HTTP port for serving web API
WORKER_HOSTNAME Hostname for listing worker web API
WORKER_PORT HTTP port for serving werker web API
CLIENTS_INFO_PATH CLients info config file
CLIENTS_INFO_PATH Clients info config file
CONNECTION_TIMEOUT_MILLIS HTTP connection timeout in milliseconds
GATEWAY_PASSWORD OOAPI Gateway Password
GATEWAY_ROOT_URL OOAPI Gateway Root URL
GATEWAY_USER OOAPI Gateway Username
Expand All @@ -151,17 +150,19 @@ KEYSTORE_ALIAS Key alias in keystore
KEYSTORE_PASSWORD Keystore password
REDIS_KEY_PREFIX Prefix for redis keys
REDIS_URI URI to redis
RIO_RETRY_ATTEMPTS_SECONDS Comma-separated list of number of seconds to wait after each RIO retry.
RIO_READ_URL RIO Services Read URL
RIO_RECIPIENT_OIN Recipient OIN for RIO SOAP calls
RIO_RETRY_ATTEMPTS_SECONDS Number of seconds to wait for first, second, etc. retry of RIO command, comma separated
RIO_UPDATE_URL RIO Services Update URL
STATUS_TTL_SEC Number of seconds hours to keep job status
STATUS_TTL_SEC Number of seconds to keep job status
STORE_HTTP_REQUESTS Boolean; should all http traffic be logged? Defaults to true.
SURF_CONEXT_CLIENT_ID SurfCONEXT client id for Mapper service
SURF_CONEXT_CLIENT_SECRET SurfCONEXT client secret for Mapper service
SURF_CONEXT_INTROSPECTION_ENDPOINT SurfCONEXT introspection endpoint
TRUSTSTORE Path to trust-store
TRUSTSTORE_PASSWORD Trust-store password
WORKER_API_HOSTNAME Hostname for listing worker web API
WORKER_API_PORT HTTP port for serving worker web API
```

The `CLIENTS_INFO_PATH` should specify a json file with settings for client-id, schac-home and oin:
Expand Down
7 changes: 6 additions & 1 deletion src/nl/surf/eduhub_rio_mapper/cli_commands.clj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,9 @@
(:require [clojure.data.json :as json]
[clojure.java.io :as io]
[clojure.string :as str]
[nl.jomco.envopts :as envopts]
[nl.surf.eduhub-rio-mapper.clients-info :as clients-info]
[nl.surf.eduhub-rio-mapper.config :as config]
[nl.surf.eduhub-rio-mapper.endpoints.api :as api]
[nl.surf.eduhub-rio-mapper.endpoints.worker-api :as worker-api]
[nl.surf.eduhub-rio-mapper.job :as job]
Expand Down Expand Up @@ -119,6 +121,9 @@
(let [[client-info [type id]] (parse-client-info-args args clients)]
(resolver type id (:institution-oin client-info)))

"document-env-vars"
remvee marked this conversation as resolved.
Show resolved Hide resolved
(envopts/specs-description config/opts-spec)

("upsert" "delete" "delete-by-code")
(let [[client-info [type id rest-args]] (parse-client-info-args args clients)
job (merge (assoc client-info
Expand All @@ -132,4 +137,4 @@
name-id id})
{:action command
::ooapi/id id}))]
(job/run! handlers job (= (System/getenv "STORE_HTTP_REQUESTS") "true")))))
(job/run! handlers job (= "true" (:store-http-requests config))))))
17 changes: 10 additions & 7 deletions src/nl/surf/eduhub_rio_mapper/config.clj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,16 +66,16 @@
:api-hostname ["Hostname for listing web API" :str
:default "localhost"
:in [:api-config :host]]
:worker-api-port ["HTTP port for serving web API" :int
:worker-api-port ["HTTP port for serving worker web API" :int
:default 8080
:in [:worker-api-config :port]]
:worker-api-hostname ["Hostname for listing web API" :str
:worker-api-hostname ["Hostname for listing worker web API" :str
:default "localhost"
:in [:worker-api-config :host]]
:job-retry-wait-ms ["Number of ms to wait before retrying job" :int
:job-retry-wait-ms ["Number of milliseconds to wait before retrying a failed job" :int
:default 5000
:in [:worker :retry-wait-ms]]
:job-max-retries ["Max number of retries of a job" :int
:job-max-retries ["Max number of retries of a failed job" :int
:default 3
:in [:worker :max-retries]]
:redis-uri ["URI to redis" :str
Expand All @@ -88,9 +88,12 @@
:default [5,30,120,600]
:parser parse-int-list
:in [:rio-config :rio-retry-attempts-seconds]]
:status-ttl-sec ["Number of seconds hours to keep job status" :int
:status-ttl-sec ["Number of seconds to keep job status" :int
:default (* 60 60 24 7) ;; one week
:in [:status-ttl-sec]]})
:in [:status-ttl-sec]]
:store-http-requests ["Boolean; should all http traffic be logged? Defaults to true." :str
:default "true"
:in [:store-http-requests]]})

(defn help []
(envopts/specs-description opts-spec))
Expand Down Expand Up @@ -164,7 +167,7 @@
config (update cfg :worker merge
{:queues (clients-info/institution-schac-homes clients)
:queue-fn :institution-schac-home
:run-job-fn #(job/run! handlers % (= (System/getenv "STORE_HTTP_REQUESTS") "true"))
:run-job-fn #(job/run! handlers % (= "true" (:store-http-requests cfg)))
:set-status-fn (status/make-set-status-fn cfg)
:retryable-fn status/retryable?
:error-fn status/errors?})]
Expand Down
2 changes: 1 addition & 1 deletion src/nl/surf/eduhub_rio_mapper/endpoints/status.clj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -127,7 +127,7 @@
;; xml response converted to edn.
(assoc :attributes {:opleidingseenheidcode opleidingseenheidcode})

(and (= (System/getenv "STORE_HTTP_REQUESTS") "true")
(and (= "true" (:store-http-requests config))
(#{:done :error :time-out} status)
(-> data :http-messages))
(assoc :http-messages (-> data :http-messages))
Expand Down
10 changes: 4 additions & 6 deletions src/nl/surf/eduhub_rio_mapper/main.clj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,8 @@
(:gen-class))

(def commands
#{"upsert" "delete" "delete-by-code" "get" "show" "resolve" "serve-api" "worker" "help" "dry-run-upsert" "link" "test-rio"})
#{"upsert" "delete" "delete-by-code" "get" "show" "resolve" "serve-api" "worker" "help" "dry-run-upsert" "link"
"document-env-vars" "test-rio"})

(defn -main
[command & args]
Expand All @@ -43,7 +44,7 @@

(let [result (cli-commands/process-command command args (config/make-config-and-handlers))]
(case command
("serve-api" "worker")
("serve-api" "worker" "show" "test-rio")
nil

"get"
Expand All @@ -53,10 +54,7 @@
("dry-run-upsert" "link")
(pprint/pprint result)

("show" "test-rio")
nil

"resolve"
("resolve" "document-env-vars")
(println result)

("upsert" "delete" "delete-by-code")
Expand Down
Loading