# robots.txt - http://www.aisneco.com User-agent: msnbot Crawl-delay: 15 User-agent: slurp Crawl-delay: 10 User-agent: * Disallow: /document/ Disallow: /stats/