Releases: google/exposure-notifications-server
v0.26.0
Changes since v0.25.0
Operations
- Add dashboard for export batches (#1500, @sethvargo)
- Lower data-layer TTLs on background jobs. (#1499, @sethvargo)
Misc
- Add function for clearing in-memory cache. (#1504, @sethvargo)
- Add new interface for secret managers than can manage secret versions. (#1503, @sethvargo)
Dependencies
Added
- github.com/Masterminds/semver/v3: v3.1.1
Changed
- cloud.google.com/go: v0.79.0 → v0.81.0
- github.com/Azure/azure-sdk-for-go: v52.4.0+incompatible → v53.0.0+incompatible
- github.com/aws/aws-sdk-go: v1.37.30 → v1.38.13
- github.com/golang/groupcache: 8c9f03a → 41bb18b
- github.com/golang/protobuf: v1.4.3 → v1.5.2
- github.com/hashicorp/vault/api: 38d91b7 → v1.1.0
- github.com/hashicorp/vault/sdk: 8477cfe → v0.2.0
- github.com/jackc/pgconn: v1.8.0 → v1.8.1
- github.com/jackc/pgtype: v1.6.2 → v1.7.0
- github.com/jackc/pgx/v4: v4.10.1 → v4.11.0
- github.com/prometheus/client_golang: v1.9.0 → v1.10.0
- github.com/prometheus/common: v0.19.0 → v0.20.0
- github.com/prometheus/statsd_exporter: v0.20.0 → v0.20.1
- github.com/sethvargo/go-envconfig: v0.3.2 → v0.3.4
- github.com/ugorji/go/codec: v1.2.4 → v1.2.5
- github.com/ugorji/go: v1.2.4 → v1.2.5
- golang.org/x/crypto: e6e6c4f → 0c34fe9
- golang.org/x/net: e18ecbb → a5a99cb
- golang.org/x/oauth2: cd4f82c → 2e8d934
- golang.org/x/sys: c6e025a → 5e06dd2
- golang.org/x/text: v0.3.5 → v0.3.6
- google.golang.org/api: v0.41.0 → v0.43.0
- google.golang.org/genproto: 8812039 → a39eb2f
- google.golang.org/grpc: v1.36.0 → v1.36.1
- google.golang.org/protobuf: v1.25.0 → v1.26.0
Removed
Nothing has changed.
v0.25.0
Changes since v0.24.0
Upgrade notes
-
This release improves the way metrics and alerts are handled to reduce the number of false alerts in favor of forward-progress alerting. When you run Terraform for the first time, you may see errors that an alert cannot be created due to a missing metric. These errors (and only these errors) can be safely ignored for now. Continue with the deployment steps. After all services are deployed, manually invoke each of the services via Cloud Scheduler. Finally, re-run Terraform to create the alerts based on the new metrics.
-
This release contains new services. Run
terraform taint module.en.null_resource.build
to ensure the new services are built during the Terraform run. This is a one-time operation.
Enhancements
- Add metrics for when a healthAuthorityID exists, but has no public keys (#1488, @mikehelmick)
- Adds support for user-report to SELF_REPORT (#1434, @mikehelmick)
Reliability
- Add forward-progress alerts for mirror service (#1482, @sethvargo)
- Add observability exporter to jwks configuration. (#1486, @sethvargo)
- Add recovery middleware for catching runtime panics in HTTP handlers. (#1479, @sethvargo)
- Add renderer, forward-progress alerts for generate service, and overridable alerts for forward-progress. (#1481, @sethvargo)
- Align points every 1m (#1493, @sethvargo)
- Also always log start and finish in debug mode for background jobs. (#1495, @sethvargo)
- Introduce a new service:
backup
. Be sure to taint the null_resource builder during the Terraform apply to get the new service version. (#1483, @sethvargo) - Refactor background cleanup jobs for forward-progress alerting. This removes unused metrics about the number of exposures/files deleted and simplifies the logic handling. (#1478, @sethvargo)
- Refactor cleanup jobs to use server patterns and middleware. (#1480, @sethvargo)
- Replace CloudSchedulerFailed with ForwardProgressFailed (#1494, @sethvargo)
- Switch export to forward progress monitoring (#1492, @sethvargo)
- Switch export-importer to forward-progress alerting (#1490, @sethvargo)
- Switch to fp alerting for jwks and key-rotation services (#1487, @sethvargo)
Misc
- Add admin console tests for export import keys (#1468, @sethvargo)
- Add more tests for admin routes and template (#1474, @sethvargo)
- Add more tests for database package (#1459, @sethvargo)
- Add special "_all" key to apply to all service environments. The special key
_all
will apply to all services. This is useful for common configuration like log-levels. A service-specific configuration overrides a value in_all
. There are no default values for_all
, so the default behavior is unchanged. (#1496, @sethvargo) - Add tests for admin authorized_apps (#1466, @sethvargo)
- Add tests for admin console exports (#1470, @sethvargo)
- Add tests for admin export importer (#1469, @sethvargo)
- Add tests for admin health authorities and keys (#1471, @sethvargo)
- Add tests for admin index page (#1472, @sethvargo)
- Add tests for admin mirrors (#1473, @sethvargo)
- Add tests for cryptorand package (#1458, @sethvargo)
- Adds placeholder database columns for possible future use of vaccine status (#1463, @mikehelmick)
- Make database package public to share with verification server (#1456, @sethvargo)
- Move admin-console CSS to an external file. (#1465, @sethvargo)
- Remove errorw and upgrade to latest ci lint (#1484, @sethvargo)
Dependencies
Added
- github.com/OneOfOne/xxhash: v1.2.2
- github.com/armon/consul-api: eb2c6b5
- github.com/cespare/xxhash: v1.1.0
- github.com/checkpoint-restore/go-criu/v4: v4.1.0
- github.com/cilium/ebpf: v0.2.0
- github.com/coreos/bbolt: v1.3.2
- github.com/coreos/etcd: v3.3.10+incompatible
- github.com/coreos/go-systemd/v22: v22.1.0
- github.com/cyphar/filepath-securejoin: v0.2.2
- github.com/dgryski/go-sip13: e10d5fe
- github.com/godbus/dbus/v5: v5.0.3
- github.com/magiconair/properties: v1.8.0
- github.com/moby/sys/mountinfo: v0.4.0
- github.com/mrunalp/fileutils: v0.5.0
- github.com/oklog/ulid: v1.3.1
- github.com/opencontainers/selinux: v1.8.0
- github.com/pelletier/go-toml: v1.2.0
- github.com/prometheus/tsdb: v0.7.1
- github.com/seccomp/libseccomp-golang: v0.9.1
- github.com/spaolacci/murmur3: f09979e
- github.com/spf13/afero: v1.1.2
- github.com/spf13/cast: v1.3.0
- github.com/spf13/jwalterweatherman: v1.0.0
- github.com/spf13/viper: v1.4.0
- github.com/syndtr/gocapability: 42c35b4
- github.com/vishvananda/netlink: v1.1.0
- github.com/vishvananda/netns: 0a2b9b5
- github.com/willf/bitset: v1.1.11
- github.com/xordataexchange/crypt: b2862e3
Changed
- cloud.google.com/go/storage: v1.13.0 → v1.14.0
- cloud.google.com/go: v0.76.0 → v0.79.0
- contrib.go.opencensus.io/exporter/prometheus: 6bcf6f8 → v0.3.0
- github.com/Azure/azure-sdk-for-go: v51.1.0+incompatible → v52.4.0+incompatible
- github.com/aws/aws-sdk-go: v1.37.8 → v1.37.30
- github.com/containerd/console: c12b1e7 → v1.0.1
- github.com/containerd/continuity: 50096c9 → 93e1549
- github.com/coreos/pkg: [3ac0863 → 399ea9e](https://github.com/coreos/pkg/c...
v0.24.0
Release notes for v0.24.0
Changelog since v0.23.0
Changes by Kind
Breaking
- Breaking change! This release disables the v1alpha1 API by default. If your clients depend on the v1alpha1 API, you *must- set
ENABLE_V1ALPHA1_API=true
on theexposure
service. (#1427, @sethvargo)
Enhancement
- Add additional metrics for background jobs: cleanup, import-export, mirror (#1451, @mikehelmick)
- Export-importer can set traveller status to true (for the whole import) (#1443, @mikehelmick)
- Add a new metric
export_file_downloaded
, which is emitted when a device downloads an export. The metric extracts the export path (for multi-tenant installations) and the platform via labels for further aggregation. This is on by default but can be disabled by settingcapture_export_file_downloads
to false. This only applies to Google Cloud. (#1447, @sethvargo) - Add import file public keys to admin console and manipulation of traveler status (#1444, @mikehelmick)
- Add uptime checks for known hosts (#1413, @mariliamelo)
- Add utility methods to simplify CSV marshaling. (#1409, @mikehelmick)
- Only log server-side errors when creating a revision token and return a 500 response when making a revision token fails (previously this was undefined behavior). (#1416, @sethvargo)
- Use more structured logging for errors (#1418, @sethvargo)
Docs
- Add documentation for e2e application configuration. (#1430, @mikehelmick)
- Document debugger service (#1415, @sethvargo)
Test-Coverage
- Add test coverage for admin console, show signature infos (#1450, @mikehelmick)
- Increase admin console, signature info unit test coverage (#1453, @mikehelmick)
- Tests for admin console create siginfo (#1452, @mikehelmick)
Infrastructure
- Allow customizing global log retention period for all services in the project. The default value is 14 days. Note: this differs from the unconfigured value of 30 days!. To retain the existing behavior, set
log_retention_period
to30
in the Terraform configuration. However, we strongly recommend using a 14-day retention period instead. (#1419, @sethvargo) - Set Binary Authorization service annotations on Cloud Run services. (#1446, @sethvargo)
Dependencies
Added
Nothing has changed.
Changed
Nothing has changed.
Removed
Nothing has changed.
v0.23.0
Release notes for v0.23.0
Changelog since v0.22.0
Notice The version of golang used in this proejct has been upgraded to Go
1.16.
Changes by Kind
Bug Fixes and Improvements
- Add utility methods to simplify CSV marshaling. (#1409, @mikehelmick)
- Better handling of parallel requests in JWKS service. (#1387, @mikehelmick)
- BREAKING Enforce that STATS_UPLOAD_MINIMUM is set to a value >= 10 (#1396, @mikehelmick)
- Remove timeout on jwks context (it's set on the http client) (#1381, @sethvargo)
- Set blobstore, key manager, and secret manager in migrate (#1377, @sethvargo)
- Set blobstore, key manager, and secret manager on services (#1378, @sethvargo)
- Update accuracy for security alerts for HumanAccessSecret and HumanDecryptedValue (#1384, @sethvargo)
- Upgrade to Go 1.16 (#1402, @sethvargo)
- Use a random prefix instead of hard-coding "parent" in key manager test helper. (#1389, @sethvargo)
Infrastructure
- Additional metrics for export batch creation. (#1398, @mikehelmick)
- Adds database backup cloud scheduler job (#1405, @mariliamelo)
- Change into tmpdir when installing linters (#1408, @sethvargo)
- Enable query insights (#1406, @sethvargo)
- Fixing key revision probability for generate and adding a report type confirmed override (#1397, @bschlaman)
- Get Cloud SQL Proxy from new link in migrate (#1392, @sethvargo)
- Give admin-console signerVerify permissions to sign "hello world" string during new export configuration (#1403, @sethvargo)
- Ignore more annotations in Terraform diff (#1379, @sethvargo)
- Improve locking for background jobs: export-import, jwks, key rotation (#1322, @mikehelmick)
- Migrate admin-console assets to embedded fs. (#1407, @sethvargo)
Dependencies
Added
Nothing has changed.
Changed
- github.com/mikehelmick/go-chaff: v0.4.1 → v0.5.0
- honnef.co/go/tools: v0.0.1-2020.1.4 → v0.1.1
Removed
Nothing has changed.
v0.22.1
Changes since v0.22.0
Operations
- Ignore more annotations in Terraform diff (#1379, @sethvargo)
- Set blobstore, key manager, and secret manager in migrate (#1377, @sethvargo)
- Set blobstore, key manager, and secret manager on services (#1378, @sethvargo)
See also: changes since v0.21.0.
Dependencies
Added
Nothing has changed.
Changed
Nothing has changed.
Removed
Nothing has changed.
v0.22.0
Changes since v0.21.0
Upgrade notes
-
This upgrade includes multiple database migrations that improve performance and indexing. While the migrations run (~5min), the
exposures
table will be locked. We recommend putting the servers into maintenance mode before running the migrations, and scheduling this upgrade off peak hours. -
Cloud-specific dependencies are now a compile-time dependency. When building the binary, you must specify the build tag for your target environments to compile the appropriate Key Manager, Secret Manager, and Blobstore support for that target platform.
-
Introduce paging and non-paging alert types. This requires an update to the alerting channels in the Terraform configuration.
Build and CI/CD
- Switch cloud dependencies to a compile-time dependency. When building the binary, you must specify the build tag for your target environments to compile the appropropriate Key Manager, Secret Manager, and Blobstore support for that target platform. (#1363, @sethvargo)
- Switch linting to GitHub Actions (#1373, @sethvargo)
Observability and reliability
- Add logic to recover from panics in service entrypoints. A panic will still terminate the service with a non-zero exit code, but it will cleanup existing connections and log the panic before doing so. (#1340, @sethvargo)
- Always check if a connection is valid after acquiring it from the pool. (#1345, @sethvargo)
- Check database connection in health check (#1362, @sethvargo)
- Paging and non-paging email channels should be configured in the Terraform files for each individual project. (#1354, @mariliamelo)
- Move DEBUG SERVER message to warning level (#1327, @sethvargo)
- Do not log lock acquisition failure when the lock is already held (#1337, @sethvargo)
- Only log internal errors on publish, move others to debug level (#1330, @sethvargo)
Database
- Improve database indexes to maximize search and delete performance
- DB migrations will have a 15 minute timeout by default (was 15 seconds) (#1332, @mikehelmick)
- Drop usage of serializable transactions (#1325, @sethvargo)
- Switch to ReadCommitted isolation level for locking (#1324, @sethvargo)
- Update database types to their larger values (VARCHAR -> TEXT and INT -> BIGINT) and add indices to common fields to improve performance. (#1326, @sethvargo)
Key Management
- Wait up to 5 seconds for Google Cloud KMS keys to become ready when generating new keys (#1338, @sethvargo)
Service: publish
- Fix lower bound of the accepted key set when validating keys. (#1372, @mikehelmick)
Service: jwks
- Use a custom http client with a separate request timeout for the jwks service. Operators can customize the timeout by setting
REQUEST_TIMEOUT
on the jwks service. The default value is 5 seconds. (#1342, @sethvargo)
Terraform
- Update ignore_changes to avoid recurring diff in Terraform (#1343, @sethvargo)
Misc
- Update to gcloud 324.0.0 in builds (#1339, @sethvargo)
- Upgrade Cloud SQL Proxy to 1.19.1 (#1334, @sethvargo)
- Remove unused performance tests (#1367, @sethvargo)
- Mount middlewares and pull from request context in debugger, e2e, exportimport, keyrotation, and mirror (#1350, @sethvargo)
- Move chaff handling into middleware (#1357, @sethvargo)
- Move e2e into integration, simplify helpers (#1371, @sethvargo)
- Move federationin to middleware/server pattern (#1351, @sethvargo)
- Move maintenance mode handling to middleware (#1355, @sethvargo)
- Move publish to middleware/server pattern (#1353, @sethvargo)
- Pull context from request in export service (#1347, @sethvargo)
- Pull logger and request_id from context in jwks service (#1341, @sethvargo)
- Refactor generate service to be a proper server, use new middlewares (#1344, @sethvargo)
Dependencies
Added
Nothing has changed.
Changed
- cloud.google.com/go/storage: v1.12.0 → v1.13.0
- cloud.google.com/go: v0.74.0 → v0.76.0
- contrib.go.opencensus.io/exporter/ocagent: v0.7.0 → f8c219d
- contrib.go.opencensus.io/exporter/prometheus: v0.2.0 → 6bcf6f8
- github.com/Azure/azure-sdk-for-go: v49.1.0+incompatible → v51.1.0+incompatible
- github.com/Azure/azure-storage-blob-go: v0.12.0 → v0.13.0
- github.com/Azure/go-autorest/autorest/adal: v0.9.10 → v0.9.13
- github.com/Azure/go-autorest/autorest/azure/auth: v0.5.5 → v0.5.7
- github.com/Azure/go-autorest/autorest: v0.11.15 → v0.11.18
- github.com/Azure/go-autorest/logger: v0.2.0 → v0.2.1
- github.com/alecthomas/units: 1786d5e → ff826a3
- github.com/armon/go-metrics: v0.3.4 → v0.3.3
- github.com/aws/aws-sdk-go: v1.36.11 → v1.37.8
- github.com/bitly/go-hostpool: v0.1.0 → a3a6125
- github.com/cncf/udpa/go: efcf912 → 5459f2c
- github.com/containerd/continuity: 1805252 → 50096c9
- github.com/coreos/pkg: 399ea9e → 3ac0863
- github.com/envoyproxy/go-control-plane: v0.9.7 → fd9021f
- github.com/frankban/quicktest: v1.10.0 → v1.11.3
- github.com/ghodss/yaml: 25d852a → v1.0.0
- github.com/gocql/gocql: 34081ed → f6df828
- github.com/google/gofuzz: v1.1.0 → v1.0.0
- github.com/google/pprof: 1bf35d6 → d980be6
- github.com/google/uuid: v1.1.2 → v1.2.0
- github.com/gorilla/handlers: v1.5.1 → v1.4.2
- github.com/gorilla/websocket: v1.4.1 → 4201258
- github.com/gosta...
v0.21.0
Release notes for main
⚠️ Upgrade notes
Deployment notes
This version introduces both Binary Authorization. To help ensure a successful upgrade, operators should taint the build step before running Terraform. This will ensure existing services are attested with the proper signatures for authorization. This impacts deployments on Google Cloud Platform using the provided terraform configuration only.
terraform taint module.en.null_resource.build
Note this will increase the duration of the Terraform run to about 10 minutes. Upon a successful run, continue with the upgrade process as normal.
Changelog since v0.20.0
Changes by Kind
JWKS fixes
- Don't save newlines at the end of public keys when importing via JWKS (#1297, @mikehelmick)
- Thru date on health authority keys will be set correctly when they are synced from JWKS sources. (#1300, @mikehelmick)
- JWKS Service - fix issue where manually added keys containing
\r
characters could prevent upgrading a health authority to use JWKS discovery. (#1315, @mikehelmick) - Admin Console: Fix form issue when adding new health authority public keys (#1314, @mikehelmick)
Reducing database lock contention
- *Behavior Change- All transactions with an isolation level of serializable will automatically retry 3 times with a quick exponential backoff period. (#1320, @mikehelmick)
- Correct database isolation levels on HA key operations (#1304, @mikehelmick)
- Switch to ReadCommitted isolation level for locking (#1324, @sethvargo)
- Update locking procedure to be strongly consistent (#1323, @sethvargo)
Build & deploy changes
- Switch to a faster build process (#1301, @sethvargo)
- Introduce binary authorization for services (#1305, @sethvargo)
Other
- CSV marshaller for statistics (#1307, @whaught)
- Switch default key_cleanup 30d to 720h (#1303, @sethvargo)
Dependencies
Added
Nothing has changed.
Changed
- contrib.go.opencensus.io/exporter/stackdriver: v0.13.4 → v0.13.5
- github.com/gorilla/mux: v1.7.4 → v1.8.0
Removed
Nothing has changed.
v0.20.0
Release notes for exposure-notifications-server v0.20.0
Changelog since v0.19.0
Changes by Kind
Breaking Changes
-
Breaking: [for deployments on Google Cloud Platform using default Terraform] To continue using the Terraform module, the following input variable is needed to avoid introducing a diff (#1260, @yegle):
revision_annotations = { debugger = { "autoscaling.knative.dev/maxScale" : "10" } export = { "autoscaling.knative.dev/maxScale" : "10" } export-importer = { "autoscaling.knative.dev/maxScale" : "10" } exposure = { "autoscaling.knative.dev/maxScale" : "500" } federationin = { "autoscaling.knative.dev/maxScale" : "3" } federationout = { "autoscaling.knative.dev/maxScale" : "5" } mirror = { "autoscaling.knative.dev/maxScale" : "10" } }
Features
- Allow stats API to be enabled/disabled on a per HA basis. (#1272, @mikehelmick)
- Fix race condition in JWKS service (#1270, @mikehelmick)
- Imported keys will be future dated if they haven't expired yet.
Published keys will be moved forward one extra interval when they are adjusted. There is no actual issue here with the default configuration because of the additional 2 hour embargo, but this is more technically correct. (#1274, @mikehelmick) - Publish statistics are written in the background instead of inline with the publish request. (#1263, @mikehelmick)
Infrastructure Changes and Improvements
- Add ability to alert when humans access secrets (#1281, @sethvargo)
- Allow generate service to take up to 2min to complete (#1282, @sethvargo)
- Code coverage displayed as part of presubmit. (#1269, @mikehelmick)
- Document accessing the admin console via the proxy (#1286, @sethvargo)
- Make admin-console deployable (#1275, @sethvargo)
- Monitoring: Add CloudSchedulerJobFailed alert. (#1266, @yegle)
- Output signature in admin-console for new export configs (#1283, @sethvargo)
- Set admin-console templates path in Terraform (#1285, @sethvargo)
Buf fixes and improvements
- Improve unit test coverage on authorizedapp database/model packaged. (#1264, @mikehelmick)
- Increase test coverage on exportimport model and database packages. (#1290, @mikehelmick)
- Increase test coverage on pkg/api/... packages (#1291, @mikehelmick)
- Increased test coverage on internal/publish/model (#1277, @mikehelmick)
- Increased test coverage on publish package. (#1289, @mikehelmick)
Dependencies
Added
Nothing has changed.
Changed
Nothing has changed.
Removed
Nothing has changed.
v0.19.3
Changelog since v0.19.2
Changes by Kind
Bug fixes and improvements
- Improve test coverage in export import path, fix bug on future date calculation (#1276, @mikehelmick)
- Imported keys will be future dated if they haven't expired yet.
Published keys will be moved forward one extra interval when they are adjusted. There is no actual issue here with the default configuration because of the additional 2 hour embargo, but this is more technically correct. (#1274, @mikehelmick)
Dependencies
Added
Nothing has changed.
Changed
Nothing has changed.
Removed
Nothing has changed.
v0.19.2
Release notes for main
Changelog since v0.19.1
Changes by Kind
Bug fixes
- Imported keys will be future dated if they haven't expired yet.
Published keys will be moved forward one extra interval when they are adjusted. There is no actual issue here with the default configuration because of the additional 2 hour embargo, but this is more technically correct. (#1274, @mikehelmick)
Dependencies
Added
Nothing has changed.
Changed
Nothing has changed.
Removed
Nothing has changed.