User-agent: * Crawl-Delay: 10 Disallow: /archive/ Disallow: /aggregator/ Disallow: /tracker/ Disallow: /comment/reply/ Disallow: /node/add/ Disallow: /user/ Disallow: /files/ Disallow: /search/ Disallow: /book/print/ Disallow: /forward/ Disallow: /spam/report/ Disallow: /classified/by-category Disallow: /classified/by-state Disallow: /classified/ads-by-breed Disallow: /event User-agent: IRLbot Disallow: / Crawl-delay: 100