User-agent: * # directed to all spiders # Disallow: * Disallow: dev Disallow: db1 Disallow: db2 Disallow: fmp Disallow: tmp Disallow: stats Disallow: dbmi Disallow: admin Disallow: te Disallow: /marie/gb/ Disallow: /gb/ # this info was removed Disallow: /trish/ Disallow: /marie/ User-agent: freefind Disallow: * User-Agent: htdig # NetMind Search-It's indexer Disallow: * User-Agent: OmniExplorer_Bot Disallow: / #