# robots.txt for http://www.endliche-automaten.de User-agent: * Disallow: /daten/ # but allow htdig to index our doc-tree