[Documentation]: Suggestion: Adding llms.txt to llamaindex documentation website

[BetterAndBetterII]

Documentation Issue Description

Ref:
llms.txt
Cursor @Docs

Large language models increasingly rely on website information, but face a critical limitation: context windows are too small to handle most websites in their entirety. Converting complex HTML pages with navigation, ads, and JavaScript into LLM-friendly plain text is both difficult and imprecise.

While websites serve both human readers and LLMs, the latter benefit from more concise, expert-level information gathered in a single, accessible location. This is particularly important for use cases like development environments, where LLMs need quick access to programming documentation and APIs.

It would be quite convenience for cursor-like tools to retrieve in llama-index's doc if we add llms.txt to doc website.

Using llms.txt will also make it simple for ChatGPT to search information about llama-index.

Documentation Link

https://docs.llamaindex.ai/llms.txt