# robots.txt file # Version 2.2 Edward-3 - 1/7/2008 # BASIC robots.txt file to reduce crawler load. # Must be named "robots.txt" and be in the root folder. # Only Google, MSN, Yahoo, and Google-Image are allowed. # Certain files excluded. # Yahoo-Slurp held back to 1 download every 45 seconds # Throttle Yahoo Slurp User-agent: Slurp Crawl-delay: 45 # For google mini User-Agent: gsa-crawler Disallow: # Googlebot Yahoo MSN User-agent: Googlebot User-agent: Slurp User-agent: msnbot Disallow: Disallow: /blank Disallow: /BLANK Disallow: /print Disallow: /PRINT Disallow: /secure Disallow: /images Disallow: /?src= Disallow: /Advantage.asp Disallow: /Advantage.cfm Disallow: /Body.cfm Disallow: /Documents/ Disallow: /Default.cfm?src= Disallow: /Dynamicurllist.cfm Disallow: /Images/ Disallow: /Mpe Disallow: /Mpeg/ Disallow: /Patientreg/ Disallow: /Scripts/ Disallow: /Worfiles/ Disallow: /Workfiles/ # Exclude all other robots User-agent: * Disallow: / # Google Images User-agent: Googlebot-Image Allow: /