Lint 404s (#10814)

* 404 linter script

* 404 linter github workflow

* temporarily run linter it's own PR branch

* fix file path

* skip run if both docs (user facing and dev docs) skip build

* fix wording

* run a a prod server on localhost before 404 linting

* fix commands

* fix job name

* change

* dedupe job name

* add progress bar

* [getsentry/action-github-commit] Auto commit

* use cli-progress for the prgoress bar

* add --progress flag to avoid stdout spam

* fix: update remaining 404s (#10853)

* docs: add section for custom otel sampler (#10843)

* fix: remaining 404s

* fix 404

* chagne report working

* remove unused dependency

* remove phony change

* switch target branch to master

---------

Co-authored-by: getsantry[bot] <66042841+getsantry[bot]@users.noreply.github.com>
Co-authored-by: Charly Gomez <[email protected]>

Author: 3 people

.github/workflows/lint-404s.yml

@@ -0,0 +1,59 @@
+name: Lint Docs for 404s
+
+on:
+  push:
+    branches: [master]
+  pull_request:
+    branches: [master]
+
+jobs:
+  index:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v2
+      - uses: getsentry/action-setup-volta@c52be2ea13cfdc084edb806e81958c13e445941e # v1.2.0
+      - uses: dorny/paths-filter@v3
+        id: filter
+        with:
+          filters: |
+            docs:
+              - 'docs/**'
+              - 'includes/**'
+              - 'platform-includes/**'
+            dev-docs:
+              - 'develop-docs/**'
+      - uses: oven-sh/setup-bun@v1
+        with:
+          bun-version: latest
+
+      - uses: actions/cache@v4
+        id: cache
+        with:
+          path: ${{ github.workspace }}/node_modules
+          key: node-${{ runner.os }}-${{ hashFiles('**/yarn.lock') }}
+
+      - run: yarn install --frozen-lockfile
+        if: steps.cache.outputs.cache-hit != 'true'
+
+      # Remove the changelog directory to avoid a build error due to missing `DATABASE_URL`
+      # and save some build time.
+      - run: rm -r app/changelog
+
+      - run: yarn build
+        if: steps.filter.outputs.docs == 'true'
+
+      - run: yarn build:developer-docs
+        if: steps.filter.outputs.dev-docs == 'true'
+
+      - name: Start Http Server
+        run: yarn start &
+        if: steps.filter.outputs.docs == 'true' || steps.filter.outputs.dev-docs == 'true'
+
+      - name: Lint 404s
+        run: bun ./scripts/lint-404s/main.ts
+        if: steps.filter.outputs.docs == 'true' || steps.filter.outputs.dev-docs == 'true'
+
+      - name: Kill Http Server
+        run: kill $(lsof -t -i:3000) || true
+        if: steps.filter.outputs.docs == 'true' || steps.filter.outputs.dev-docs == 'true'
+        continue-on-error: true

docs/platforms/apple/common/features/experimental-features.mdx

@@ -7,12 +7,14 @@ description: "Learn about the experimental features available for Sentry's Apple
 Do you want to try some new experimental features? On the latest version of the Apple SDK, you can:
 
 - Enable <PlatformLink to="/tracing/instrumentation/automatic-instrumentation/#time-to-full-display">Time to Full Display (TTFD)</PlatformLink> to gain insight into how long it takes your view controller to launch and load all of its content.
+<PlatformSection notSupported={["apple.tvos", "apple.watchos", "apple.visionos"]}>
 - Enable <PlatformLink to="/profiling/#enable-launch-profiling">App Launch Profiling</PlatformLink> to get detailed profiles for your app launches.
+</PlatformSection>
 - If you use Swift concurrency, stitch together stack traces of your async code with the `swiftAsyncStacktraces` option. Note that you can enable this in your Objective-C project, but only async code written in Swift will be stitched together.
 
-
 <Note>
-Experimental features are still a work-in-progress and may have bugs. We recognize the irony.
+  Experimental features are still a work-in-progress and may have bugs. We
+  recognize the irony.
 </Note>
 
 ```swift {tabTitle:Swift}

docs/platforms/javascript/guides/aws-lambda/index.mdx

@@ -22,7 +22,7 @@ Depending on your setup, there are different ways to install and use Sentry in y
 - [Install the Sentry AWS Lambda Layer](./install/cjs-layer) if your Lambda functions are written in CommonJS (CJS) using `require` syntax.
 - [Install the Sentry AWS NPM package](./install/esm-npm) if your Lambda functions are running in EcmaScript Modules (ESM) using `import` syntax.
 
-If you're not sure which installation method to use or want an overview of all available options to use Sentry in your Lambda functions, read the [installation methods overview](/guides/aws-lambda/install).
+If you're not sure which installation method to use or want an overview of all available options to use Sentry in your Lambda functions, read the [installation methods overview](./install).
 
 ## Configuration
 

docs/platforms/javascript/guides/aws-lambda/install/cjs-layer.mdx

@@ -4,7 +4,7 @@ description: "Learn how to add the Sentry Node Lambda Layer to use Sentry in you
 sidebar_order: 1
 ---
 
-The easiest way to get started with Sentry is to use the Sentry [Lambda Layer](https://docs.aws.amazon.com/Lambda/latest/dg/configuration-layers.html) instead of adding `@sentry/aws-serverless` with `npm` or `yarn` [manually](../cjs-manual).
+The easiest way to get started with Sentry is to use the Sentry [Lambda Layer](https://docs.aws.amazon.com/Lambda/latest/dg/configuration-layers.html) instead of adding `@sentry/aws-serverless` with `npm` or `yarn` [manually](../cjs-npm).
 If you follow this guide, you don't have to worry about deploying Sentry dependencies alongside your function code.
 To actually start the SDK, you can decide between setting up the SDK using environment variables or in your Lambda function code. We recommend using environment variables as it's the easiest way to get started. [Initializing the SDK in code](#alternative-initialize-the-sdk-in-code) instead of setting environment variables gives you more control over the SDK setup if you need it.
 

docs/platforms/php/common/tracing/instrumentation/caches-module.mdx

@@ -15,7 +15,7 @@ For detailed information about which data can be set, see the [Cache Module deve
 
 ## Custom Instrumentation
 
-If you're using anything other than our <PlatformLink to="/guides/laravel/">Laravel SDK</PlatformLink>, you'll need to manually instrument the [Cache Module](https://sentry.io/orgredirect/organizations/:orgslug/performance/caches/) by following the steps below.
+If you're using anything other than our [Laravel SDK](/platforms/php/guides/laravel/), you'll need to manually instrument the [Cache Module](https://sentry.io/orgredirect/organizations/:orgslug/performance/caches/) by following the steps below.
 
 ### Add Span When Putting Data Into the Cache
 

docs/product/issues/issue-details/index.mdx

@@ -105,7 +105,7 @@ It's the most important piece of information that the Sentry grouping algorithm
 
 You can set your own <PlatformLink to="/enriching-events/breadcrumbs/">breadcrumbs</PlatformLink> to make them more useful for debugging.
 
-If you’ve enabled [Session Replay](/product/explore/session-replay/), you’ll see a replay preview under Breadcrumbs if there’s one associated with the event you’re viewing. Replays can be associated with both frontend and [backend errors](/product/explore/session-replay/getting-started#replays-for-backend-errors) (as long as distrubted tracing is set up). Clicking on the replay preview will lead you to the [Replay Details](/product/explore/session-replay/replay-details/) page.
+If you’ve enabled [Session Replay](/product/explore/session-replay/), you’ll see a replay preview under Breadcrumbs if there’s one associated with the event you’re viewing. Replays can be associated with both frontend and [backend errors](/product/explore/session-replay/getting-started#replays-for-backend-errors) (as long as distrubted tracing is set up). Clicking on the replay preview will lead you to the [Replay Details](/product/explore/session-replay/web/replay-details/) page.
 
 ## Tags
 

docs/product/performance/transaction-summary.mdx

@@ -157,7 +157,7 @@ Spans with the same operation and description are grouped together into a single
 
 ### Replays
 
-The Replays tab displays a list of replays where the transaction you’re viewing had occurred. Go directly to [Replay Details](/product/explore/session-replay/replay-details/) for any replay and see how a slow transaction impacted the user experience. Note: you must have [Session Replay](/product/explore/session-replay/) enabled to see this tab.
+The Replays tab displays a list of replays where the transaction you’re viewing had occurred. Go directly to [Replay Details](/product/explore/session-replay/web/replay-details/) for any replay and see how a slow transaction impacted the user experience. Note: you must have [Session Replay](/product/explore/session-replay/) enabled to see this tab.
 
 ## Additional Actions
 

scripts/lint-404s/ignore-list.txt

@@ -0,0 +1,2 @@
+/
+/changelog/

scripts/lint-404s/main.ts

@@ -0,0 +1,155 @@
+/* eslint-disable no-console */
+
+import {readFileSync} from 'fs';
+import path, {dirname} from 'path';
+import {fileURLToPath} from 'url';
+
+const baseURL = 'http://localhost:3000/';
+type Link = {href: string; innerText: string};
+
+const trimSlashes = (s: string) => s.replace(/(^\/|\/$)/g, '');
+
+// @ts-ignore
+const ignoreListFile = path.join(dirname(import.meta.url), './ignore-list.txt');
+
+const showProgress = process.argv.includes('--progress');
+
+// Paths to skip
+const ignoreList: string[] = readFileSync(fileURLToPath(ignoreListFile), 'utf8')
+  .split('\n')
+  .map(trimSlashes)
+  .filter(Boolean);
+
+async function fetchWithFollow(url: URL | string): Promise<Response> {
+  const r = await fetch(url);
+  if (r.status >= 300 && r.status < 400 && r.headers.has('location')) {
+    return fetchWithFollow(r.headers.get('location')!);
+  }
+  return r;
+}
+
+async function main() {
+  const sitemap = await fetch(`${baseURL}sitemap.xml`).then(r => r.text());
+
+  const slugs = [...sitemap.matchAll(/<loc>([^<]*)<\/loc>/g)]
+    .map(l => l[1])
+    .map(url => trimSlashes(new URL(url).pathname))
+    .filter(Boolean);
+  const allSlugsSet = new Set(slugs);
+
+  console.log('Checking 404s on %d pages', slugs.length);
+
+  const all404s: {page404s: Link[]; slug: string}[] = [];
+
+  // check if the slug equivalent of the href is in the sitemap
+  const isInSitemap = (href: string) => {
+    // remove hash
+    const pathnameSlug = trimSlashes(href.replace(/#.*$/, ''));
+
+    // some #hash links result in empty slugs when stripped
+    return pathnameSlug === '' || allSlugsSet.has(pathnameSlug);
+  };
+
+  function shoudlSkipLink(href: string) {
+    const isExternal = (href_: string) =>
+      href_.startsWith('http') || href_.startsWith('mailto:');
+    const isLocalhost = (href_: string) =>
+      href_.startsWith('http') && new URL(href_).hostname === 'localhost';
+    const isIp = (href_: string) => /(\d{1,3}\.){3}\d{1,3}/.test(href_);
+    const isImage = (href_: string) => /\.(png|jpg|jpeg|gif|svg|webp)$/.test(href_);
+
+    return [
+      isExternal,
+      (s = '') => ignoreList.includes(trimSlashes(s)),
+      isImage,
+      isLocalhost,
+      isIp,
+    ].some(fn => fn(href));
+  }
+
+  async function is404(link: Link, pageUrl: URL): Promise<boolean> {
+    if (shoudlSkipLink(link.href)) {
+      return false;
+    }
+
+    const fullPath = link.href.startsWith('/')
+      ? trimSlashes(link.href)
+      : // relative path
+        trimSlashes(new URL(pageUrl.pathname + '/' + link.href, baseURL).pathname);
+
+    if (isInSitemap(fullPath)) {
+      return false;
+    }
+    const fullUrl = new URL(fullPath, baseURL);
+    const resp = await fetchWithFollow(fullUrl);
+    if (resp.status === 404) {
+      return true;
+    }
+    return false;
+  }
+
+  for (const slug of slugs) {
+    const pageUrl = new URL(slug, baseURL);
+    const now = performance.now();
+    const html = await fetchWithFollow(pageUrl.href).then(r => r.text());
+
+    const linkRegex = /<a[^>]*href="([^"]*)"[^>]*>([^<]*)<\/a>/g;
+    const links = Array.from(html.matchAll(linkRegex)).map(m => {
+      const [, href, innerText] = m;
+      return {href, innerText};
+    });
+    const page404s = (
+      await Promise.all(
+        links.map(async link => {
+          const is404_ = await is404(link, pageUrl);
+          return [link, is404_] as [Link, boolean];
+        })
+      )
+    )
+      .filter(([_, is404_]) => is404_)
+      .map(([link]) => link);
+
+    if (page404s.length) {
+      all404s.push({slug, page404s});
+    }
+
+    if (showProgress) {
+      console.log(
+        page404s.length ? '❌' : '✅',
+        `in ${(performance.now() - now).toFixed(1).padStart(4, '0')} ms | ${slug}`
+      );
+    }
+  }
+
+  if (all404s.length === 0) {
+    console.log('\n\n🎉 No 404s found');
+    return false;
+  }
+  const numberOf404s = all404s.map(x => x.page404s.length).reduce((a, b) => a + b, 0);
+  console.log(
+    '\n❌ Found %d %s across %d %s',
+    numberOf404s,
+    numberOf404s === 1 ? '404' : '404s',
+    all404s.length,
+    all404s.length === 1 ? 'page' : 'pages'
+  );
+  for (const {slug, page404s} of all404s) {
+    console.log('\n🌐', baseURL + slug);
+    for (const link of page404s) {
+      console.log(`    - [${link.innerText}](${link.href})`);
+    }
+  }
+
+  console.log(
+    '\n👉 Note: the markdown syntax is not necessarily present on the source files, but the links do exist on the final pages'
+  );
+  // signal error
+  return true;
+}
+const now = performance.now();
+main().then(has404s => {
+  console.log(`\n Done in ${(performance.now() - now).toFixed(1)} ms`);
+  process.exit(has404s ? 1 : 0);
+});
+
+export {};

yarn.lock

@@ -10741,7 +10741,16 @@ string-length@^4.0.1:
     char-regex "^1.0.2"
     strip-ansi "^6.0.0"
 
-"string-width-cjs@npm:string-width@^4.2.0", string-width@^4.1.0, string-width@^4.2.0, string-width@^4.2.3:
+"string-width-cjs@npm:string-width@^4.2.0":
+  version "4.2.3"
+  resolved "https://registry.yarnpkg.com/string-width/-/string-width-4.2.3.tgz#269c7117d27b05ad2e536830a8ec895ef9c6d010"
+  integrity sha512-wKyQRQpjJ0sIp62ErSZdGsjMJWsap5oRNihHhu6G7JVO/9jIB6UyevL+tXuOqrng8j/cxKTWyWUwvSTriiZz/g==
+  dependencies:
+    emoji-regex "^8.0.0"
+    is-fullwidth-code-point "^3.0.0"
+    strip-ansi "^6.0.1"
+
+string-width@^4.1.0, string-width@^4.2.0, string-width@^4.2.3:
   version "4.2.3"
   resolved "https://registry.yarnpkg.com/string-width/-/string-width-4.2.3.tgz#269c7117d27b05ad2e536830a8ec895ef9c6d010"
   integrity sha512-wKyQRQpjJ0sIp62ErSZdGsjMJWsap5oRNihHhu6G7JVO/9jIB6UyevL+tXuOqrng8j/cxKTWyWUwvSTriiZz/g==
@@ -10827,7 +10836,14 @@ stringify-entities@^4.0.0:
     character-entities-html4 "^2.0.0"
     character-entities-legacy "^3.0.0"
 
-"strip-ansi-cjs@npm:strip-ansi@^6.0.1", strip-ansi@^6.0.0, strip-ansi@^6.0.1:
+"strip-ansi-cjs@npm:strip-ansi@^6.0.1":
+  version "6.0.1"
+  resolved "https://registry.yarnpkg.com/strip-ansi/-/strip-ansi-6.0.1.tgz#9e26c63d30f53443e9489495b2105d37b67a85d9"
+  integrity sha512-Y38VPSHcqkFrCpFnQ9vuSXmquuv5oXOKpGeT6aGrr3o3Gc9AlVa6JBfUSOCnbxGGZF+/0ooI7KrPuUSztUdU5A==
+  dependencies:
+    ansi-regex "^5.0.1"
+
+strip-ansi@^6.0.0, strip-ansi@^6.0.1:
   version "6.0.1"
   resolved "https://registry.yarnpkg.com/strip-ansi/-/strip-ansi-6.0.1.tgz#9e26c63d30f53443e9489495b2105d37b67a85d9"
   integrity sha512-Y38VPSHcqkFrCpFnQ9vuSXmquuv5oXOKpGeT6aGrr3o3Gc9AlVa6JBfUSOCnbxGGZF+/0ooI7KrPuUSztUdU5A==
@@ -11840,7 +11856,16 @@ wordwrap@^1.0.0:
   resolved "https://registry.yarnpkg.com/wordwrap/-/wordwrap-1.0.0.tgz#27584810891456a4171c8d0226441ade90cbcaeb"
   integrity sha512-gvVzJFlPycKc5dZN4yPkP8w7Dc37BtP1yczEneOb4uq34pXZcvrtRTmWV8W+Ume+XCxKgbjM+nevkyFPMybd4Q==
 
-"wrap-ansi-cjs@npm:wrap-ansi@^7.0.0", wrap-ansi@^7.0.0:
+"wrap-ansi-cjs@npm:wrap-ansi@^7.0.0":
+  version "7.0.0"
+  resolved "https://registry.yarnpkg.com/wrap-ansi/-/wrap-ansi-7.0.0.tgz#67e145cff510a6a6984bdf1152911d69d2eb9e43"
+  integrity sha512-YVGIj2kamLSTxw6NsZjoBxfSwsn0ycdesmc4p+Q21c5zPuZ1pl+NfxVdxPtdHvmNVOQ6XSYG4AUtyt/Fi7D16Q==
+  dependencies:
+    ansi-styles "^4.0.0"
+    string-width "^4.1.0"
+    strip-ansi "^6.0.0"
+
+wrap-ansi@^7.0.0:
   version "7.0.0"
   resolved "https://registry.yarnpkg.com/wrap-ansi/-/wrap-ansi-7.0.0.tgz#67e145cff510a6a6984bdf1152911d69d2eb9e43"
   integrity sha512-YVGIj2kamLSTxw6NsZjoBxfSwsn0ycdesmc4p+Q21c5zPuZ1pl+NfxVdxPtdHvmNVOQ6XSYG4AUtyt/Fi7D16Q==