From d95268c9ccb9b1f5c4553a2f3f8c86dbda1aa84b Mon Sep 17 00:00:00 2001 From: David Mytton Date: Thu, 10 Oct 2024 15:50:38 +0100 Subject: [PATCH] chore: Add example of handling missing user-agent header (#133) --- src/content/docs/bot-protection/concepts.mdx | 29 ++++++++++++++----- .../docs/bot-protection/identifying-bots.mdx | 13 +++++---- .../docs/bot-protection/reference/bun.mdx | 12 ++++---- .../docs/bot-protection/reference/nextjs.mdx | 12 ++++---- .../docs/bot-protection/reference/nodejs.mdx | 12 ++++---- .../bot-protection/reference/sveltekit.mdx | 12 ++++---- 6 files changed, 56 insertions(+), 34 deletions(-) diff --git a/src/content/docs/bot-protection/concepts.mdx b/src/content/docs/bot-protection/concepts.mdx index 55237e04..2cb5d2ab 100644 --- a/src/content/docs/bot-protection/concepts.mdx +++ b/src/content/docs/bot-protection/concepts.mdx @@ -36,14 +36,27 @@ Arcjet will help you reduce bot traffic and give you more control over which requests reach your application, but it's important to understand that it's not possible to achieve 100% accuracy. -:::note -Please be aware that requests without `User-Agent` headers can not be identified -as any particular bot and will be marked as an errored decision. Developers -should take caution to check `decision.isErrored()` and decide if you want to -allow or deny the request. See the Error Handling section in the SDK reference -for an example. Our recommendation is to block requests without `User-Agent` -headers because most clients always send this header. -::: +### `User-Agent` header + +Requests without `User-Agent` headers can not be identified as any particular +bot and will be marked as an errored decision. Check `decision.isErrored()` and +decide if you want to allow or deny the request. Our recommendation is to block +requests without `User-Agent` headers because most legitimate clients always +send this header. + +```ts +if (decision.isErrored()) { + if (decision.reason.message.includes("missing User-Agent header")) { + log.warn({ error: decision.reason.message }, "Arcjet user-agent warning"); + // You could return a 400 Bad request error here + // Next.js example: + // return NextResponse.json({ error: "Bad request" }, { status: 400 }); + } else { + // Just log the error and continue + log.error({ error: decision.reason.message }, "Arcjet error"); + } +} +``` ## Blocking based on fingerprint diff --git a/src/content/docs/bot-protection/identifying-bots.mdx b/src/content/docs/bot-protection/identifying-bots.mdx index e028305f..66a5d2b4 100644 --- a/src/content/docs/bot-protection/identifying-bots.mdx +++ b/src/content/docs/bot-protection/identifying-bots.mdx @@ -19,12 +19,13 @@ This list is used to allow developers to choose to allow or deny any or all of these bots. :::note -Please be aware that requests without `User-Agent` headers can not be identified -as any particular bot and will be marked as an errored decision. Developers -should take caution to check `decision.isErrored()` and decide if you want to -allow or deny the request. See the Error Handling section in the SDK reference -for an example. Our recommendation is to block requests without `User-Agent` -headers because most clients always send this header. +Requests without `User-Agent` headers can not be identified as any particular +bot and will be marked as an errored decision. Check `decision.isErrored()` and +decide if you want to allow or deny the request. Our recommendation is to block +requests without `User-Agent` headers because most legitimate clients always +send this header. + +See [an example of how to do this](/bot-protection/concepts#user-agent-header). ::: ## Known bots diff --git a/src/content/docs/bot-protection/reference/bun.mdx b/src/content/docs/bot-protection/reference/bun.mdx index 735e3572..720d6f61 100644 --- a/src/content/docs/bot-protection/reference/bun.mdx +++ b/src/content/docs/bot-protection/reference/bun.mdx @@ -170,11 +170,13 @@ If there is an error condition, Arcjet will return an accessing `decision.reason.message`. :::note -Please be aware that requests without `User-Agent` headers can not be identified -as any particular bot and will be marked as an errored decision. Developers -should take caution to check `decision.isErrored()` and decide if you want to -allow or deny the request. Our recommendation is to block requests without -`User-Agent` headers because most clients always send this header. +Requests without `User-Agent` headers can not be identified as any particular +bot and will be marked as an errored decision. Check `decision.isErrored()` and +decide if you want to allow or deny the request. Our recommendation is to block +requests without `User-Agent` headers because most legitimate clients always +send this header. + +See [an example of how to do this](/bot-protection/concepts#user-agent-header). ::: diff --git a/src/content/docs/bot-protection/reference/nextjs.mdx b/src/content/docs/bot-protection/reference/nextjs.mdx index deae3db4..fd8b5c93 100644 --- a/src/content/docs/bot-protection/reference/nextjs.mdx +++ b/src/content/docs/bot-protection/reference/nextjs.mdx @@ -390,11 +390,13 @@ If there is an error condition, Arcjet will return an accessing `decision.reason.message`. :::note -Please be aware that requests without `User-Agent` headers can not be identified -as any particular bot and will be marked as an errored decision. Developers -should take caution to check `decision.isErrored()` and decide if you want to -allow or deny the request. Our recommendation is to block requests without -`User-Agent` headers because most clients always send this header. +Requests without `User-Agent` headers can not be identified as any particular +bot and will be marked as an errored decision. Check `decision.isErrored()` and +decide if you want to allow or deny the request. Our recommendation is to block +requests without `User-Agent` headers because most legitimate clients always +send this header. + +See [an example of how to do this](/bot-protection/concepts#user-agent-header). ::: diff --git a/src/content/docs/bot-protection/reference/nodejs.mdx b/src/content/docs/bot-protection/reference/nodejs.mdx index cfa5739d..abdedc66 100644 --- a/src/content/docs/bot-protection/reference/nodejs.mdx +++ b/src/content/docs/bot-protection/reference/nodejs.mdx @@ -170,11 +170,13 @@ If there is an error condition, Arcjet will return an accessing `decision.reason.message`. :::note -Please be aware that requests without `User-Agent` headers can not be identified -as any particular bot and will be marked as an errored decision. Developers -should take caution to check `decision.isErrored()` and decide if you want to -allow or deny the request. Our recommendation is to block requests without -`User-Agent` headers because most clients always send this header. +Requests without `User-Agent` headers can not be identified as any particular +bot and will be marked as an errored decision. Check `decision.isErrored()` and +decide if you want to allow or deny the request. Our recommendation is to block +requests without `User-Agent` headers because most legitimate clients always +send this header. + +See [an example of how to do this](/bot-protection/concepts#user-agent-header). ::: diff --git a/src/content/docs/bot-protection/reference/sveltekit.mdx b/src/content/docs/bot-protection/reference/sveltekit.mdx index 5bab0ea9..2dcc7590 100644 --- a/src/content/docs/bot-protection/reference/sveltekit.mdx +++ b/src/content/docs/bot-protection/reference/sveltekit.mdx @@ -246,11 +246,13 @@ If there is an error condition, Arcjet will return an accessing `decision.reason.message`. :::note -Please be aware that requests without `User-Agent` headers can not be identified -as any particular bot and will be marked as an errored decision. Developers -should take caution to check `decision.isErrored()` and decide if you want to -allow or deny the request. Our recommendation is to block requests without -`User-Agent` headers because most clients always send this header. +Requests without `User-Agent` headers can not be identified as any particular +bot and will be marked as an errored decision. Check `decision.isErrored()` and +decide if you want to allow or deny the request. Our recommendation is to block +requests without `User-Agent` headers because most legitimate clients always +send this header. + +See [an example of how to do this](/bot-protection/concepts#user-agent-header). :::