[EDU-2101] - Update llmstxt.ts to structure llms.txt a bit cleaner

[GregHolmes]

Description

Restructured llms.txt generation with product categorisation This update reorganises the generated llms.txt file from a flat list into a structured hierarchy with categories (Platform, Pub/Sub, Chat, Spaces, LiveObjects, LiveSync, Asset Tracking) and subcategories based on URL patterns. Added a more informative preamble describing Ably's core features, support for Kotlin and React languages, and logic to skip redundant language-specific URLs when the language already matches the page slug (e.g., prevents /getting-started/go?lang=go). The structured format improves LLM navigation and discovery of relevant documentation.

Attached an example of the llms.txt generated:

llms.txt

Build output:

info onPostBuild: Found 205 pages to place into llms.txt (26 textile, 179 MDX)
info onPostBuild: Successfully wrote llms.txt with 591 pages

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

✨ Finishing touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch EDU-2101-Update-llms.txt-to-be-better-structured

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[kennethkalmer]

@GregHolmes should we even have asset tracking in here since it is being deprecated?

[m-hulbert]

This is a really great idea, Greg.

There's a few tweaks I think we need to make to Platform and Pub/Sub and I also wonder if we should look at breaking down other products too (at least maybe Chat and LiveObjects) into further categories. WDYT?

[GregHolmes]

@GregHolmes should we even have asset tracking in here since it is being deprecated?

Yes and no, I think, until we actually deprecate it we should keep it. Then remove when the big changes are made.

[m-hulbert]

This looks a lot better, just a few things:

• All the Pub/Sub API refs (+ REST API) are mixed in together (and REST and realtime have pages named the same thing, so it's quite confusing).
• Control API reference is missing.
• Some pages aren't generating in multiple langs (e.g. basic auth)