User-agent: * Crawl-delay: 2 Disallow: /*.torrent$ Disallow: /download.php$ Disallow: /include/ User-agent: Alexabot Disallow: User-agent: MarkMonitor Disallow:/ User-agent: MarkWatch Disallow:/