# See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file User-Agent: * Allow: / Disallow: /js/ Disallow: /css/ Disallow: /includes/ Disallow: /news-thanks.php Disallow: /404.php