Monitor A Bunch More Endpoints #4764
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add submission elibility (steps 1-3) and information form endpoints
|
Terraform plan for meta No changes. Your infrastructure matches the configuration.✅ Plan applied in Deploy to Development and Management Environment #961 |
|
Terraform plan for dev Plan: 1 to add, 3 to change, 1 to destroy.Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
!~ update in-place
-/+ destroy and then create replacement
Terraform will perform the following actions:
# module.dev.module.clamav.cloudfoundry_app.clamav_api will be updated in-place
!~ resource "cloudfoundry_app" "clamav_api" {
!~ docker_image = "ghcr.io/gsa-tts/fac/clamav@sha256:40919683d4fbd026ee189783ca5c4c58ca99ec50c35d4defdaf1f88e192e002a" -> "ghcr.io/gsa-tts/fac/clamav@sha256:b74a991de6029f90524ce976f90286263f4e5709fd4fb40f8c1f7055c2478ba4"
id = "779bbc51-f78a-4186-90eb-5acb68d7d746"
name = "fac-av-dev"
# (17 unchanged attributes hidden)
# (1 unchanged block hidden)
}
# module.dev.module.cors.null_resource.cors_header must be replaced
-/+ resource "null_resource" "cors_header" {
!~ id = "*******************" -> (known after apply)
!~ triggers = { # forces replacement
!~ "always_run" = "2025-03-03T19:52:20Z" -> (known after apply)
}
}
# module.dev.module.file_scanner_clamav.cloudfoundry_app.clamav_api will be updated in-place
!~ resource "cloudfoundry_app" "clamav_api" {
!~ docker_image = "ghcr.io/gsa-tts/fac/clamav@sha256:40919683d4fbd026ee189783ca5c4c58ca99ec50c35d4defdaf1f88e192e002a" -> "ghcr.io/gsa-tts/fac/clamav@sha256:b74a991de6029f90524ce976f90286263f4e5709fd4fb40f8c1f7055c2478ba4"
id = "65c83416-4126-4785-99c2-5e1adb810422"
name = "fac-av-dev-fs"
# (17 unchanged attributes hidden)
# (1 unchanged block hidden)
}
# module.dev.module.newrelic.newrelic_one_dashboard_json.fac_monitoring will be updated in-place
!~ resource "newrelic_one_dashboard_json" "fac_monitoring" {
id = "MzkxOTA3NnxWSVp8REFTSEJPQVJEfGRhOjc5NjI5ODA"
!~ json = jsonencode(
!~ {
name = "FAC Monitoring - Dev"
!~ pages = [
# (3 unchanged elements hidden)
{
description = null
name = "File Downloads"
widgets = [
{
layout = {
column = 1
height = 1
row = 1
width = 10
}
linkedEntityGuids = null
rawConfiguration = {
text = "# Single Summary Report Download"
}
title = ""
visualization = {
id = "viz.markdown"
}
},
{
layout = {
column = 1
height = 3
row = 2
width = 2
}
linkedEntityGuids = null
rawConfiguration = {
nrqlQueries = [
{
accountIds = [
3919076,
]
query = "FROM Transaction SELECT count(*) AS 'Transactions (last hour)' WHERE (appName = 'gsa-fac-dev') AND (request.method = 'GET' AND request.uri LIKE '/dissemination/summary-report/xlsx%') SINCE 1 hour ago"
},
]
}
title = "Transactions (last hour)"
visualization = {
id = "viz.billboard"
}
},
{
description = "Average number of transactions per hour over the past week"
layout = {
column = 3
height = 3
row = 2
width = 2
}
linkedEntityGuids = null
rawConfiguration = {
dataFormatters = [
{
name = "Transactions Per Hour"
precision = "0"
type = "decimal"
},
]
linkedEntityGuids = []
nrqlQueries = [
{
accountIds = [
3919076,
]
query = "FROM Transaction SELECT rate(count(*), 1 hour) AS 'Transactions (per hour)' WHERE appName ='gsa-fac-dev' AND request.method = 'GET' AND request.uri LIKE '/dissemination/summary-report/xlsx%' SINCE 7 days ago"
},
]
thresholds = [
{
alertSeverity = "WARNING"
value = 80
},
{
alertSeverity = "CRITICAL"
value = 70
},
]
vizcoConfiguration = {
billboard = {
thresholds = [
{
limit = 80
type = "warning"
},
{
limit = 70
type = "critical"
},
]
}
dataFormatters = [
{
name = "Transactions Per Hour"
precision = "0"
type = "decimal"
},
]
}
}
title = "Transactions Target: 70 (per hour/past week)"
visualization = {
id = "viz.billboard"
}
},
{
layout = {
column = 5
height = 3
row = 2
width = 6
}
linkedEntityGuids = null
rawConfiguration = {
nrqlQueries = [
{
accountId = 3919076
query = "FROM Transaction SELECT count(*) AS 'Transactions' WHERE appName = 'gsa-fac-dev' AND request.method = 'GET' AND request.uri LIKE '/dissemination/summary-report/xlsx%' SINCE 14 days ago TIMESERIES"
},
]
}
title = "Transactions"
visualization = {
id = "viz.line"
}
},
{
layout = {
column = 1
height = 3
row = 5
width = 2
}
linkedEntityGuids = null
rawConfiguration = {
nrqlQueries = [
{
accountIds = [
3919076,
]
query = "FROM TransactionError SELECT count(*) AS 'Errors (last hour)' WHERE (appName = 'gsa-fac-dev') AND (request.method = 'GET' AND request.uri LIKE '/dissemination/summary-report/xlsx%') SINCE 1 hour ago"
},
]
}
title = "Errors (last hour)"
visualization = {
id = "viz.billboard"
}
},
{
layout = {
column = 3
height = 3
row = 5
width = 2
}
linkedEntityGuids = null
rawConfiguration = {
dataFormatters = [
{
name = "Success Rate (%)"
precision = "4"
type = "decimal"
},
]
linkedEntityGuids = []
nrqlQueries = [
{
accountIds = [
3919076,
]
query = "FROM Transaction SELECT 100 - percentage(count(*), WHERE error is true) AS 'Success Rate' WHERE appName ='gsa-fac-dev' AND request.method = 'GET' AND request.uri LIKE '/dissemination/summary-report/xlsx%' SINCE 7 days ago "
},
]
thresholds = [
{
alertSeverity = "WARNING"
value = 0.995
},
{
alertSeverity = "CRITICAL"
value = 0.99
},
]
vizcoConfiguration = {
billboard = {
thresholds = [
{
limit = 0.995
type = "warning"
},
{
limit = 0.99
type = "critical"
},
]
}
dataFormatters = [
{
name = "Success Rate (%)"
precision = "4"
type = "decimal"
},
]
}
}
title = "Success Rate Target: Above 99%"
visualization = {
id = "viz.billboard"
}
},
{
layout = {
column = 5
height = 3
row = 5
width = 6
}
linkedEntityGuids = null
rawConfiguration = {
nrqlQueries = [
{
accountId = 3919076
query = "FROM TransactionError SELECT count(*) AS 'Errors (last hour)' WHERE (appName = 'gsa-fac-dev') AND (request.method = 'GET' AND request.uri LIKE '/dissemination/summary-report/xlsx%') SINCE 14 days ago TIMESERIES"
},
]
}
title = "Errors"
visualization = {
id = "viz.line"
}
},
{
layout = {
column = 1
height = 3
row = 8
width = 2
}
linkedEntityGuids = null
rawConfiguration = {
dataFormatters = [
{
name = "p95"
type = "decimal"
},
{
name = "p50"
type = "decimal"
},
{
name = "Average"
type = "decimal"
},
]
facet = {
showOtherSeries = false
}
nrqlQueries = [
{
accountIds = [
3919076,
]
query = "FROM Transaction SELECT average(convert(duration, 's', 'ms')) AS 'Average', percentile(convert(duration, 's', 'ms'), 50) AS 'p50', percentile(convert(duration, 's', 'ms'), 95) AS 'p95' WHERE (appName = 'gsa-fac-dev') AND (request.method = 'GET' AND request.uri LIKE '/dissemination/summary-report/xlsx%') SINCE 1 hour ago"
},
]
platformOptions = {
ignoreTimeRange = false
}
}
title = "Latency (last hour)"
visualization = {
id = "viz.billboard"
}
},
{
layout = {
column = 3
height = 3
row = 8
width = 2
}
linkedEntityGuids = null
rawConfiguration = {
dataFormatters = [
{
name = "Average Latency (ms)"
precision = "0"
type = "decimal"
},
]
linkedEntityGuids = []
nrqlQueries = [
{
accountIds = [
3919076,
]
query = "FROM Transaction SELECT average(convert(duration, 's', 'ms')) AS 'Average' WHERE (appName = 'gsa-fac-dev') AND (request.method = 'GET' AND request.uri LIKE '/dissemination/summary-report/xlsx%') SINCE 7 days ago "
},
]
thresholds = [
{
alertSeverity = "CRITICAL"
value = 800
},
{
alertSeverity = "WARNING"
value = 700
},
]
vizcoConfiguration = {
billboard = {
thresholds = [
{
limit = 800
type = "critical"
},
{
limit = 700
type = "warning"
},
]
}
dataFormatters = [
{
name = "Average Latency (ms)"
precision = "0"
type = "decimal"
},
]
}
}
title = "Average Latency Target (last week): 800ms"
visualization = {
id = "viz.billboard"
}
},
{
layout = {
column = 5
height = 3
row = 8
width = 6
}
linkedEntityGuids = null
rawConfiguration = {
facet = {
showOtherSeries = false
}
legend = {
enabled = true
}
nrqlQueries = [
{
accountIds = [
3919076,
]
query = "FROM Transaction SELECT average(convert(duration, 's', 'ms')) AS 'Average', percentile(convert(duration, 's', 'ms'), 50) AS 'p50', percentile(convert(duration, 's', 'ms'), 95) AS 'p95' WHERE (appName = 'gsa-fac-dev') AND (request.method = 'GET' AND request.uri LIKE '/dissemination/summary-report/xlsx%') SINCE 14 days ago TIMESERIES"
},
]
platformOptions = {
ignoreTimeRange = false
}
thresholds = {
isLabelVisible = true
}
units = {
unit = "MS"
}
yAxisLeft = {
zero = true
}
yAxisRight = {
zero = true
}
}
title = "Latency"
visualization = {
id = "viz.line"
}
},
{
layout = {
column = 1
height = 1
row = 11
width = 10
}
linkedEntityGuids = null
rawConfiguration = {
text = "# Workbook Downloads"
}
title = ""
visualization = {
id = "viz.markdown"
}
},
{
layout = {
column = 1
height = 3
row = 12
width = 2
}
linkedEntityGuids = null
rawConfiguration = {
nrqlQueries = [
{
accountIds = [
3919076,
]
query = "FROM Transaction SELECT count(*) AS 'Transactions (last hour)' WHERE (appName = 'gsa-fac-dev') AND (request.method = 'GET' AND request.uri LIKE '/audit/excel/%') SINCE 1 hour ago"
},
]
}
title = "Transactions (last hour)"
visualization = {
id = "viz.billboard"
}
},
{
description = "Average number of transactions per hour over the past week"
layout = {
column = 3
height = 3
row = 12
width = 2
}
linkedEntityGuids = null
rawConfiguration = {
dataFormatters = [
{
name = "Transactions Per Hour"
precision = "0"
type = "decimal"
},
]
linkedEntityGuids = []
nrqlQueries = [
{
accountIds = [
3919076,
]
query = "FROM Transaction SELECT rate(count(*), 1 hour) AS 'Transactions (per hour)' WHERE appName ='gsa-fac-dev' AND request.method = 'GET' AND request.uri LIKE '/audit/excel/%' SINCE 7 days ago"
},
]
thresholds = [
{
alertSeverity = "WARNING"
value = 25
},
{
alertSeverity = "CRITICAL"
value = 20
},
]
vizcoConfiguration = {
billboard = {
thresholds = [
{
limit = 25
type = "warning"
},
{
limit = 20
type = "critical"
},
]
}
dataFormatters = [
{
name = "Transactions Per Hour"
precision = "0"
type = "decimal"
},
]
}
}
title = "Transactions Target: 20 (per hour/past week)"
visualization = {
id = "viz.billboard"
}
},
{
layout = {
column = 5
height = 3
row = 12
width = 6
}
linkedEntityGuids = null
rawConfiguration = {
nrqlQueries = [
{
accountId = 3919076
query = "FROM Transaction SELECT count(*) AS 'Transactions' WHERE appName = 'gsa-fac-dev' AND request.method = 'GET' AND request.uri LIKE '/audit/excel/%' SINCE 14 days ago TIMESERIES"
},
]
}
title = "Transactions"
visualization = {
id = "viz.line"
}
},
{
layout = {
column = 1
height = 3
row = 15
width = 2
}
linkedEntityGuids = null
rawConfiguration = {
nrqlQueries = [
{
accountIds = [
3919076,
]
query = "FROM TransactionError SELECT count(*) AS 'Errors (last hour)' WHERE (appName = 'gsa-fac-dev') AND (request.method = 'GET' AND request.uri LIKE '/audit/excel/%') SINCE 1 hour ago"
},
]
}
title = "Errors (last hour)"
visualization = {
id = "viz.billboard"
}
},
{
layout = {
column = 3
height = 3
row = 15
width = 2
}
linkedEntityGuids = null
rawConfiguration = {
dataFormatters = [
{
name = "Success Rate (%)"
precision = "4"
type = "decimal"
},
]
linkedEntityGuids = []
nrqlQueries = [
{
accountIds = [
3919076,
]
query = "FROM Transaction SELECT 100 - percentage(count(*), WHERE error is true) AS 'Success Rate' WHERE appName ='gsa-fac-dev' AND request.method = 'GET' AND request.uri LIKE '/audit/excel/%' SINCE 7 days ago "
},
]
thresholds = [
{
alertSeverity = "WARNING"
value = 0.995
},
{
alertSeverity = "CRITICAL"
value = 0.99
},
]
vizcoConfiguration = {
billboard = {
thresholds = [
{
limit = 0.995
type = "warning"
},
{
limit = 0.99
type = "critical"
},
]
}
dataFormatters = [
{
name = "Success Rate (%)"
precision = "4"
type = "decimal"
},
]
}
}
title = "Success Rate Target: Above 99%"
visualization = {
id = "viz.billboard"
}
},
{
layout = {
column = 5
height = 3
row = 15
width = 6
}
linkedEntityGuids = null
rawConfiguration = {
nrqlQueries = [
{
accountId = 3919076
query = "FROM TransactionError SELECT count(*) AS 'Errors (last hour)' WHERE (appName = 'gsa-fac-dev') AND (request.method = 'GET' AND request.uri LIKE '/audit/excel/%') SINCE 14 days ago TIMESERIES"
},
]
}
title = "Errors"
visualization = {
id = "viz.line"
}
},
{
layout = {
column = 1
height = 3
row = 18
width = 2
}
linkedEntityGuids = null
rawConfiguration = {
dataFormatters = [
{
name = "p95"
type = "decimal"
},
{
name = "p50"
type = "decimal"
},
{
name = "Average"
type = "decimal"
},
]
facet = {
showOtherSeries = false
}
nrqlQueries = [
{
accountIds = [
3919076,
]
query = "FROM Transaction SELECT average(convert(duration, 's', 'ms')) AS 'Average', percentile(convert(duration, 's', 'ms'), 50) AS 'p50', percentile(convert(duration, 's', 'ms'), 95) AS 'p95' WHERE (appName = 'gsa-fac-dev') AND (request.method = 'GET' AND request.uri LIKE '/audit/excel/%') SINCE 1 hour ago"
},
]
platformOptions = {
ignoreTimeRange = false
}
}
title = "Latency (last hour)"
visualization = {
id = "viz.billboard"
}
},
{
layout = {
column = 3
height = 3
row = 18
width = 2
}
linkedEntityGuids = null
rawConfiguration = {
dataFormatters = [
{
name = "Average Latency (ms)"
precision = "0"
type = "decimal"
},
]
linkedEntityGuids = []
nrqlQueries = [
{
accountIds = [
3919076,
]
query = "FROM Transaction SELECT average(convert(duration, 's', 'ms')) AS 'Average' WHERE (appName = 'gsa-fac-dev') AND (request.method = 'GET' AND request.uri LIKE '/audit/excel/%') SINCE 7 days ago "
},
]
thresholds = [
{
alertSeverity = "CRITICAL"
value = 12000
},
{
alertSeverity = "WARNING"
value = 10000
},
]
vizcoConfiguration = {
billboard = {
thresholds = [
{
limit = 12000
type = "critical"
},
{
limit = 10000
type = "warning"
},
]
}
dataFormatters = [
{
name = "Average Latency (ms)"
precision = "0"
type = "decimal"
},
]
}
}
title = "Average Latency Target (last week): 12000ms"
visualization = {
id = "viz.billboard"
}
},
{
layout = {
column = 5
height = 3
row = 18
width = 6
}
linkedEntityGuids = null
rawConfiguration = {
facet = {
showOtherSeries = false
}
legend = {
enabled = true
}
nrqlQueries = [
{
accountIds = [
3919076,
]
query = "FROM Transaction SELECT average(convert(duration, 's', 'ms')) AS 'Average', percentile(convert(duration, 's', 'ms'), 50) AS 'p50', percentile(convert(duration, 's', 'ms'), 95) AS 'p95' WHERE (appName = 'gsa-fac-dev') AND (request.method = 'GET' AND request.uri LIKE '/audit/excel/%') SINCE 14 days ago TIMESERIES"
},
]
platformOptions = {
ignoreTimeRange = false
}
thresholds = {
isLabelVisible = true
}
units = {
unit = "MS"
}
yAxisLeft = {
zero = true
}
yAxisRight = {
zero = true
}
}
title = "Latency"
visualization = {
id = "viz.line"
}
},
]
},
!~ {
!~ name = "Validation and Submission" -> "Presubmission Eligibility"
!~ widgets = [
!~ {
!~ rawConfiguration = {
!~ text = "# Cross Validation" -> "# Eligibility (Step 1)"
}
# (4 unchanged attributes hidden)
},
!~ {
!~ rawConfiguration = {
!~ nrqlQueries = [
!~ {
!~ query = "FROM Transaction SELECT count(*) AS 'Transactions (last hour)' WHERE (appName = 'gsa-fac-dev') AND (request.method = 'POST' AND request.uri LIKE '/audit/cross-validation/%') SINCE 1 hour ago" -> "FROM Transaction SELECT count(*) AS 'Transactions (last hour)' WHERE (appName = 'gsa-fac-dev') AND (request.method = 'POST' AND request.uri LIKE '/report_submission/eligibility/') SINCE 1 hour ago"
# (1 unchanged attribute hidden)
},
]
}
# (4 unchanged attributes hidden)
},
!~ {
!~ rawConfiguration = {
!~ nrqlQueries = [
!~ {
!~ query = "FROM Transaction SELECT rate(count(*), 1 hour) AS 'Transactions (per hour)' WHERE appName ='gsa-fac-dev' AND request.method = 'POST' AND request.uri LIKE '/audit/cross-validation/%' SINCE 7 days ago" -> "FROM Transaction SELECT rate(count(*), 1 hour) AS 'Transactions (per hour)' WHERE appName ='gsa-fac-dev' AND request.method = 'POST' AND request.uri LIKE '/report_submission/eligibility/' SINCE 7 days ago"
# (1 unchanged attribute hidden)
},
]
!~ thresholds = [
!~ {
!~ value = 10 -> 15
# (1 unchanged attribute hidden)
},
!~ {
!~ value = 8 -> 10
# (1 unchanged attribute hidden)
},
]
!~ vizcoConfiguration = {
!~ billboard = {
!~ thresholds = [
!~ {
!~ limit = 10 -> 15
# (1 unchanged attribute hidden)
},
!~ {
!~ limit = 8 -> 10
# (1 unchanged attribute hidden)
},
]
}
# (1 unchanged attribute hidden)
}
# (2 unchanged attributes hidden)
}
!~ title = "Transactions Target: 8 (per hour/past week)" -> "Transactions Target: 10 (per hour/past week)"
# (4 unchanged attributes hidden)
},
!~ {
!~ rawConfiguration = {
!~ nrqlQueries = [
!~ {
!~ query = "FROM Transaction SELECT count(*) AS 'Transactions' WHERE appName = 'gsa-fac-dev' AND request.method = 'POST' AND request.uri LIKE '/audit/cross-validation/%' SINCE 14 days ago TIMESERIES" -> "FROM Transaction SELECT count(*) AS 'Transactions' WHERE appName = 'gsa-fac-dev' AND request.method = 'POST' AND request.uri LIKE '/report_submission/eligibility/' SINCE 14 days ago TIMESERIES"
# (1 unchanged attribute hidden)
},
]
}
# (4 unchanged attributes hidden)
},
!~ {
!~ rawConfiguration = {
!~ nrqlQueries = [
!~ {
Plan is too large to fit in a PR comment. See the full plan in the workflow log.✅ Plan applied in Deploy to Development and Management Environment #961 |
|
gsa-jrothacker
approved these changes
Mar 11, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Monitor A Bunch More Endpoints
Issue: #4741
Changes:
How to test:
The terraform was applied to preview in this workflow run. Visit the preview monitoring dashboard to verify that it works.
Screenshots:
New tabs and part of the eligibility endpoints:
Audit management format:
PR Checklist: Submitter
maininto your branch shortly before creating the PR. (You should also be mergingmaininto your branch regularly during development.)git status | grep migrations. If there are any results, you probably need to add them to the branch for the PR. Your PR should have only one new migration file for each of the component apps, except in rare circumstances; you may need to delete some and re-runpython manage.py makemigrationsto reduce the number to one. (Also, unless in exceptional circumstances, your PR should not delete any migration files.)PR Checklist: Reviewer
make docker-clean; make docker-first-run && docker compose up; then rundocker compose exec web /bin/bash -c "python manage.py test"The larger the PR, the stricter we should be about these points.
Pre Merge Checklist: Merger
-/+ resource "null_resource" "cors_header"should be destroying and recreating its self and~ resource "cloudfoundry_app" "clamav_api"might be updating itssha256for thefac-file-scannerandfac-av-${ENV}by default.main.