# Exclude Files From All Robots: User-agent: * Crawl-Delay: 10 Disallow: /admin Disallow: /user Disallow: /taxonomy # End robots.txt file