Add robots.txt

k3jph · May 1, 2024 · b7caa83 · b7caa83
1 parent 653b341
commit b7caa83
Showing 1 changed file with 5 additions and 0 deletions.
diff --git a/robots.txt b/robots.txt
@@ -0,0 +1,5 @@
+User-agent: *
+Disallow: 
+Crawl-delay: 5
+Disallow: /cgi-bin/
+Sitemap: https://jameshoward.us/sitemap.xml