User-agent: * Crawl-delay: 30 Disallow: /cvs/ Disallow: /usage/ User-agent: tailrank Disallow: / User-agent: Tailrank Disallow: / User-agent: Spinn3r Disallow: /