# User-agent: Disallow: /anzeigen/ Disallow: /cgi-bin/ Disallow: /main/ Disallow: /openweb/ Disallow: /Sign/ Disallow: /sign/ Disallow: /sources/ Disallow: /sources/applets/ Disallow: /sources/download/ Disallow: /sources/images/ Disallow: /sources/scripts/ Disallow: /sources/styles/ User-agent: Crawler V 0.2.1 admin@crawler.de User-agent: Crawler User-agent: Googlebot User-agent: Scooter User-agent: KIT-Fireball User-agent: wget User-agent: webzip User-agent: webmirror User-agent: webcopy Disallow: /cgi-bin/ Disallow: /main/ Disallow: /openweb/ Disallow: /Sign/ Disallow: /sign/ Disallow: /sources/ User-agent: MegaSheep User-agent: Slurp User-agent: A-Online Search User-agent: EmailCollector Disallow: /