Added robots.txt

amooma · Jan 23, 2015 · 284dfde · 284dfde
1 parent dcc8ec5
commit 284dfde
Showing 1 changed file with 8 additions and 0 deletions.
diff --git a/robots.txt b/robots.txt
@@ -0,0 +1,8 @@
+# See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file
+#
+# To ban all spiders from the entire site uncomment the next two lines:
+# User-Agent: *
+# Disallow: /
+
+User-Agent: *
+Disallow: /downloads