Add robots.txt and update docfx.json to include it (#52)

Updated `docfx.json` to include `robots.txt` in the list of resource files. Added a new `robots.txt` file that: - Allows all user agents. - Disallows crawling of URLs with query strings. - Specifies the sitemap location at `https://netcord.dev/sitemap.xml`.
NetCordDev · Sep 30, 2024 · 9a04794 · 9a04794
1 parent b546a0a
commit 9a04794
Show file tree

Hide file tree

Showing 2 changed files with 5 additions and 1 deletion.
diff --git a/Documentation/docfx.json b/Documentation/docfx.json
@@ -30,7 +30,7 @@
     ],
     "resource": [
       {
-        "files": ["images/**", "favicon.ico", "logo.svg"]
+        "files": ["images/**", "favicon.ico", "logo.svg", "robots.txt"]
       }
     ],
     "overwrite": [

diff --git a/Documentation/robots.txt b/Documentation/robots.txt
@@ -0,0 +1,4 @@
+User-agent: *
+Disallow: /*?
+
+Sitemap: https://netcord.dev/sitemap.xml