# /robots.txt file for http://webcrawler.com/ User-agent: webcrawler Disallow:/gallery2/