Skip to content

Commit

Permalink
Fix repeat flag doesn't terminate monika on fail probe (#1165)
Browse files Browse the repository at this point in the history 
* add maxattempts to handle repeat flag

* seems to fix but weird

* fix probing

* remove previous solution

* refactor and add comments

* fix flaky test

* separate the http and non http probe tests

* fix test description

* fix crash

* update docs

* update readme
  • Loading branch information
@nicnocquee
nicnocquee authored Nov 10, 2023
1 parent a9316c6 commit 1c7c5f9
Show file tree
Hide file tree
Showing 13 changed files with 566 additions and 395 deletions.
27 changes: 14 additions & 13 deletions docs/src/pages/guides/probes.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -58,19 +58,20 @@ probes:
Details of the field are given in the table below.
| Topic | Description |
| :--------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| method (optional) | Http method such as GET, POST, PUT, DELETE. |
| url (required) | This is the url endpoint to dispatch the request to. |
| timeout (optional) | Request timeout in **milliseconds**, Default value is 10000 which corresponds to 10 seconds. If the request takes longer than `timeout`, the request will be aborted. |
| headers (optional) | Http headers you might need for your request. |
| body (optional) | Any http body if your method requires it. |
| interval (optional) | Number of probe's interval (in seconds). Default value is 10 seconds. |
| incidentThreshold (optional) | Number of times an alert should return true before Monika sends notifications. For example, when incidentThreshold is 3, Monika will only send notifications when the probed URL returns non-2xx status 3 times in a row. After sending the notifications, Monika will not send notifications anymore until the alert status changes. Default value is 5. |
| saveBody (optional) | When set to true, the response body of the request is stored in the internal database. The default is off when not defined. This is to keep the log file size small as some responses can be sizable. The setting is for each probe request. |
| alerts (optional) | The condition which will trigger an alert, and the subsequent notification method to send out the alert. See below for further details on alerts and notifications. See [alerts](./alerts) section for detailed information. |
| ping (optional) | (boolean), If set true then send a PING to the specified url instead. |
| allowUnauthorized (optional) | (boolean), If set to true, will make https agent to not check for ssl certificate validity |
| Topic | Description |
| :--------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| method (optional) | Http method such as GET, POST, PUT, DELETE. |
| url (required) | This is the url endpoint to dispatch the request to. |
| timeout (optional) | Request timeout in **milliseconds**, Default value is 10000 which corresponds to 10 seconds. If the request takes longer than `timeout`, the request will be aborted. |
| headers (optional) | Http headers you might need for your request. |
| body (optional) | Any http body if your method requires it. |
| interval (optional) | Number of probe's interval (in seconds). Default value is 10 seconds. |
| incidentThreshold (optional) | Number of times an alert should return true before Monika sends notifications. For example, when incidentThreshold is 3, Monika will only send incident notifications when the probed URL returns non-2xx status 3 times in a row. After sending the notifications, Monika will not send notifications anymore until the alert status changes. Default value is 5. However, the actual number of retries will be the the greatest number between `incidentThreshold` and `recoveryThreshold`. So if you want to have 3 retries, you need to set both `incidentThreshold` and `recoveryThreshold`. |
| recoveryThreshold (optional) | Number of retries before Monika sends recovery notifications. For example, when recoveryThreshold is 3 and when previously a probe is marked as incident, Monika will only send recovery notification when the probing succeeds 3 times in a row. Default value is 5. However, the actual number of retries will be the the greatest number between `incidentThreshold` and `recoveryThreshold`. So if you want to have 3 retries, you need to set both `incidentThreshold` and `recoveryThreshold`. |
| saveBody (optional) | When set to true, the response body of the request is stored in the internal database. The default is off when not defined. This is to keep the log file size small as some responses can be sizable. The setting is for each probe request. |
| alerts (optional) | The condition which will trigger an alert, and the subsequent notification method to send out the alert. See below for further details on alerts and notifications. See [alerts](./alerts) section for detailed information. |
| ping (optional) | (boolean), If set true then send a PING to the specified url instead. |
| allowUnauthorized (optional) | (boolean), If set to true, will make https agent to not check for ssl certificate validity |

## Request Body

Expand Down
49 changes: 16 additions & 33 deletions src/commands/monika.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -284,51 +284,34 @@ export default class Monika extends Command {
}

deprecationHandler(config: Config): Config {
const showDeprecateMsg: Record<
'query' | 'incidentThreshold' | 'recoveryThreshold',
boolean
> = {
const showDeprecateMsg: Record<'query', boolean> = {
query: false,
incidentThreshold: false,
recoveryThreshold: false,
}

const checkedConfig = {
...config,
probes: config.probes?.map((probe) => {
if (probe?.recoveryThreshold) {
showDeprecateMsg.recoveryThreshold = true
}

return {
...probe,
requests: probe.requests?.map((request) => ({
...request,
alert: request.alerts?.map((alert) => {
if (alert.query) {
showDeprecateMsg.query = true
return { ...alert, assertion: alert.query }
}

return alert
}),
})),
alerts: probe.alerts?.map((alert) => {
probes: config.probes?.map((probe) => ({
...probe,
requests: probe.requests?.map((request) => ({
...request,
alert: request.alerts?.map((alert) => {
if (alert.query) {
showDeprecateMsg.query = true
return { ...alert, assertion: alert.query }
}

return alert
}),
}
}),
}

if (showDeprecateMsg.recoveryThreshold) {
log.warn(
'recoveryThreshold is deprecated. It will be managed internally by Monika.'
)
})),
alerts: probe.alerts?.map((alert) => {
if (alert.query) {
showDeprecateMsg.query = true
return { ...alert, assertion: alert.query }
}

return alert
}),
})),
}

if (showDeprecateMsg.query) {
Expand Down
27 changes: 27 additions & 0 deletions src/components/config/validation/validator/default-values.ts
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
/**********************************************************************************
* MIT License *
* *
* Copyright (c) 2023 Hyperjump Technology *
* *
* Permission is hereby granted, free of charge, to any person obtaining a copy *
* of this software and associated documentation files (the "Software"), to deal *
* in the Software without restriction, including without limitation the rights *
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell *
* copies of the Software, and to permit persons to whom the Software is *
* furnished to do so, subject to the following conditions: *
* *
* The above copyright notice and this permission notice shall be included in all *
* copies or substantial portions of the Software. *
* *
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR *
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, *
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE *
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER *
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, *
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE *
* SOFTWARE. *
**********************************************************************************/

export const DEFAULT_INCIDENT_THRESHOLD = 5
export const DEFAULT_RECOVERY_THRESHOLD = 5
export const DEFAULT_INTERVAL = 10
16 changes: 14 additions & 2 deletions src/components/config/validation/validator/probe.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,11 @@ import { getContext } from '../../../../context'
import { FAILED_REQUEST_ASSERTION } from '../../../../looper'
import { compileExpression } from '../../../../utils/expression-parser'
import { isValidURL } from '../../../../utils/is-valid-url'
import {
DEFAULT_INCIDENT_THRESHOLD,
DEFAULT_INTERVAL,
DEFAULT_RECOVERY_THRESHOLD,
} from './default-values'

export async function validateProbes(probes: Probe[]): Promise<Probe[]> {
const alertSchema = joi.alternatives().try(
Expand Down Expand Up @@ -101,8 +106,15 @@ export async function validateProbes(probes: Probe[]): Promise<Probe[]> {
}),
description: joi.string().allow(''),
id: joi.string().required(),
incidentThreshold: joi.number().default(5).min(1),
interval: joi.number().default(10).min(1),
incidentThreshold: joi
.number()
.default(DEFAULT_INCIDENT_THRESHOLD)
.min(1),
recoveryThreshold: joi
.number()
.default(DEFAULT_RECOVERY_THRESHOLD)
.min(1),
interval: joi.number().default(DEFAULT_INTERVAL).min(1),
lastEvent: joi.object({
createdAt: joi.string().allow(''),
recoveredAt: joi.string().allow('', null),
Expand Down
Loading

0 comments on commit 1c7c5f9

Please sign in to comment.