# See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file User-agent: Twiceler-0.9 Disallow: / User-agent: Twiceler Disallow: / User-agent: twiceler Disallow: / User-agent: * Disallow: /search/* Disallow: /search?* Disallow: /features/* Disallow: /EEDaily/* Disallow: /Greenwire/* Disallow: /eenewspm/* Disallow: /Landletter/* Disallow: /climatewire/* Disallow: /-1/* Disallow: /0/* Disallow: /1/* Disallow: /2/* Disallow: /3/* Disallow: /4/* Disallow: /5/* Disallow: /6/* User-agent: * Allow: /public/Greenwire/* Allow: /public/EEDaily/* Allow: /public/eenewspm/* Allow: /public/Landletter/* Allow: /public/climatewire/*