################################################################################ # This is the robots.txt file for staging only! # For pushing to production, use "robots_production.txt", rename to robots.txt, # push, then return this file to robots.txt. ################################################################################ # 2/22/07 GC - Since staging is no longer accessible outside of the network, this # file could probably be just for production User-agent: * Crawl-delay: 5 Disallow: /scripts/ Disallow: /cgi-bin/ Disallow: /places of interests/ Disallow: /404/ Disallow: /search/ Disallow: /throp/ Disallow: /ext/ Disallow: /error/