feat: implement Arcjet security bot detection & Shield WAF #330

davidmytton · 2024-11-21T17:48:38Z

This adds Arcjet bot protection and attack detection, with custom rules depending on the route.

It uses the global error handler by default, but this can be customized. Instructions are in the README.

The tsconfig.json type checking was looking in the node_modules directory because of the include match patterns. Changing the match patterns to just consider files in src highlights a lot of errors in the existing files, so I opted to disable the checks for now because fixing them is out of scope of this PR.

vercel · 2024-11-21T17:49:01Z

@davidmytton is attempting to deploy a commit to the Ixartz's projects Team on Vercel.

A member of the Team first needs to authorize it.

ixartz · 2024-11-22T00:16:42Z

Very strange about the TypeScript error, it shouldn't look inside node_modules: https://github.com/ixartz/Next-js-Boilerplate/blob/main/tsconfig.json#L58, node_modules is excluded.

Anyway, it's out of the scope of the PR.

codecov · 2024-11-22T00:19:31Z

Codecov Report

Attention: Patch coverage is 0% with 48 lines in your changes missing coverage. Please review.

Project coverage is 10.23%. Comparing base (2f9646b) to head (ac670a5).
Report is 8 commits behind head on main.

Files with missing lines	Patch %	Lines
src/app/[locale]/layout.tsx	0.00%	18 Missing ⚠️
src/libs/Arcjet.ts	0.00%	10 Missing and 1 partial ⚠️
src/components/Sponsors.tsx	0.00%	10 Missing ⚠️
src/app/[locale]/(marketing)/page.tsx	0.00%	7 Missing ⚠️
src/libs/Env.ts	0.00%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #330      +/-   ##
==========================================
- Coverage   10.67%   10.23%   -0.45%     
==========================================
  Files          39       40       +1     
  Lines        1115     1163      +48     
  Branches       45       46       +1     
==========================================
  Hits          119      119              
- Misses        963     1010      +47     
- Partials       33       34       +1

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

ixartz · 2024-11-22T00:33:28Z

src/app/[locale]/(auth)/layout.tsx

 import { routing } from '@/libs/i18nNavigation';
 import { enUS, frFR } from '@clerk/localizations';
 import { ClerkProvider } from '@clerk/nextjs';
 import { setRequestLocale } from 'next-intl/server';

+const aj = arcjet.withRule(


Is it possible to combine with the one at src/app/[locale]/(marketing)/layout.tsx?

And move to the root layout: https://github.com/ixartz/Next-js-Boilerplate/blob/main/src/app/%5Blocale%5D/layout.tsx

So, we keep only one and everything is inside the root layout.

CATEGORY:SEARCH_ENGINE won't pass the middleware due to authentication.
Most pages under (auth) required authentication. This is why I'm thinking to add directly in the root layout, did I miss something?

I did it that way because often you want more restrictive bot detection on the authenticated routes vs your website, but I've made the change. The Arcjet call is now in the root layout. It can easily be moved by the user later.

ixartz

It looks good to me, I'll merge this weekend.

README.md

ixartz · 2024-11-25T02:39:42Z

@davidmytton, I wanted to merge the PR but I notice the changes in tsconfig.json is related to Arcjet dependency. I thought maybe the issue is related to your environment but actually, I got the same issue.

When I run the type check, npm run check-types and here is the error:

node_modules/@arcjet/protocol/client.ts:165:16 - error TS6133: 'response' is declared but its value is never read.

165         .then((response) => {
                   ~~~~~~~~

node_modules/@arcjet/protocol/convert.ts:63:13 - error TS6133: '_exhaustive' is declared but its value is never read.

63       const _exhaustive: never = mode;
               ~~~~~~~~~~~

node_modules/@arcjet/protocol/convert.ts:84:13 - error TS6133: '_exhaustive' is declared but its value is never read.

84       const _exhaustive: never = emailType;
               ~~~~~~~~~~~

node_modules/@arcjet/protocol/convert.ts:107:13 - error TS6133: '_exhaustive' is declared but its value is never read.

107       const _exhaustive: never = emailType;
                ~~~~~~~~~~~

node_modules/@arcjet/protocol/convert.ts:130:13 - error TS6133: '_exhaustive' is declared but its value is never read.

130       const _exhaustive: never = stack;
                ~~~~~~~~~~~

node_modules/@arcjet/protocol/convert.ts:147:13 - error TS6133: '_exhaustive' is declared but its value is never read.

147       const _exhaustive: never = stack;
                ~~~~~~~~~~~

node_modules/@arcjet/protocol/convert.ts:168:13 - error TS6133: '_exhaustive' is declared but its value is never read.

168       const _exhaustive: never = ruleState;
                ~~~~~~~~~~~

node_modules/@arcjet/protocol/convert.ts:187:13 - error TS6133: '_exhaustive' is declared but its value is never read.

187       const _exhaustive: never = conclusion;
                ~~~~~~~~~~~

node_modules/@arcjet/protocol/convert.ts:208:13 - error TS6133: '_exhaustive' is declared but its value is never read.

208       const _exhaustive: never = conclusion;
                ~~~~~~~~~~~

node_modules/@arcjet/protocol/convert.ts:274:13 - error TS6133: '_exhaustive' is declared but its value is never read.

274       const _exhaustive: never = proto.reason;
                ~~~~~~~~~~~

node_modules/@arcjet/protocol/convert.ts:511:13 - error TS6133: '_exhaustive' is declared but its value is never read.

511       const _exhaustive: never = decision.conclusion;
                ~~~~~~~~~~~

node_modules/@arcjet/protocol/index.ts:694:23 - error TS6133: 'Props' is declared but its value is never read.

694 export type ArcjetRule<Props extends {} = {}> = {

So, it's only related to Arcjet.

I would be great to know the root cause before merging it, in the tsconfig:

node_modules/ is defined in exclude

The type check shouldn't be run inside node_modules, it don't happen with other libraries.

On my side, I'll also try to investigate.

mbtools · 2024-11-25T18:41:41Z

Aj looks like a nice feature, but do you want to introduce a dependency on a solution in alpha release?
As a potential customer, this is a turn-off.

davidmytton · 2024-11-25T19:39:11Z

The tsconfig.json is configured to compile all .ts files which include those in the node_modules directory. The line at

Next-js-Boilerplate/tsconfig.json

Line 64 in a08638b

"**/*.ts",

is causing this and it overrides the exclude. That's causing the node_modules directory to be compiled.

This is what I was referring to in the PR description. Changing it to scope only files in src causes a lot of TS errors because they weren't previously being checked, so I disabled the check for this PR.

To avoid this, we're going to stop publishing the TS files in a new SDK release shortly (arcjet/arcjet-js#2326). Ideally you'd be able to change the tsconfig.json to exclude node_modules, but we've seen this a few times with other configs. There are some benefits to including the TS files in our package, but avoiding these issues outweighs them.

Once the release is out I'll update this PR and it should resolve the problem.

That's why we still keep the SDK in alpha @mbtools - we're operating the service as if it were stable and fully supported, but there may still be some API changes we want to make like the fix needed here. arcjet/arcjet-js#928 is tracking pre-beta changes, but everything so far has been backwards compatible. Our approach is to avoid breaking anything even though the label is technically alpha.

davidmytton · 2024-11-25T19:47:00Z

I'd also add there are other dependencies that are technically alpha (i.e pre v1.0) already in this boilerplate e.g. @electric-sql/pglite, @logtail/pino, drizzle-orm, etc. 😅

davidmytton · 2024-11-26T18:58:07Z

@ixartz I just pushed the version update and reverted the tsconfig.json changes, which resolves the type check errors. This should be good to merge now.

github-actions · 2024-11-26T22:33:23Z

🎉 This PR is included in version 3.60.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

feat: implement Arcjet security bot detection & Shield WAF

96d95f4

ixartz reviewed Nov 22, 2024

View reviewed changes

ixartz approved these changes Nov 22, 2024

View reviewed changes

chore: move Arcjet to root layout

7cf815f

davidmytton requested a review from ixartz November 22, 2024 13:30

ixartz approved these changes Nov 22, 2024

View reviewed changes

ixartz reviewed Nov 25, 2024

View reviewed changes

README.md Outdated Show resolved Hide resolved

chore: update to Arcjet .32

83a41a8

davidmytton added 2 commits November 26, 2024 18:58

chore: revert tsconfig changes

e728ea7

fix: change Arcjet URL

ac670a5

ixartz merged commit 909d0df into ixartz:main Nov 26, 2024
3 of 6 checks passed

github-actions bot added the released label Nov 26, 2024

davidmytton deleted the david/arcjet branch November 27, 2024 09:34

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: implement Arcjet security bot detection & Shield WAF #330

feat: implement Arcjet security bot detection & Shield WAF #330

davidmytton commented Nov 21, 2024

vercel bot commented Nov 21, 2024

ixartz commented Nov 22, 2024

codecov bot commented Nov 22, 2024 •

edited

Loading

ixartz Nov 22, 2024 •

edited

Loading

davidmytton Nov 22, 2024

ixartz left a comment

ixartz commented Nov 25, 2024 •

edited

Loading

mbtools commented Nov 25, 2024

davidmytton commented Nov 25, 2024

davidmytton commented Nov 25, 2024

davidmytton commented Nov 26, 2024

github-actions bot commented Nov 26, 2024

feat: implement Arcjet security bot detection & Shield WAF #330

feat: implement Arcjet security bot detection & Shield WAF #330

Conversation

davidmytton commented Nov 21, 2024

vercel bot commented Nov 21, 2024

ixartz commented Nov 22, 2024

codecov bot commented Nov 22, 2024 • edited Loading

Codecov Report

ixartz Nov 22, 2024 • edited Loading

Choose a reason for hiding this comment

davidmytton Nov 22, 2024

Choose a reason for hiding this comment

ixartz left a comment

Choose a reason for hiding this comment

ixartz commented Nov 25, 2024 • edited Loading

mbtools commented Nov 25, 2024

davidmytton commented Nov 25, 2024

davidmytton commented Nov 25, 2024

davidmytton commented Nov 26, 2024

github-actions bot commented Nov 26, 2024

codecov bot commented Nov 22, 2024 •

edited

Loading

ixartz Nov 22, 2024 •

edited

Loading

ixartz commented Nov 25, 2024 •

edited

Loading