# See http://www.robotstxt.org/robotstxt.html for documentation on how to use the robots.txt file # # Event, place and story pages stay crawlable. The calendar's date-picker URLs # (?start_date= month views, ?from= day views) are the same listings re-sorted # by date and used to run without end, so crawlers stop there. Filtered calendar # views (?category[]= and ?town[]= in any combination) were each a fresh heavy query # (CVL-346), so crawlers skip those too. User-agent: * Disallow: /*?*start_date= Disallow: /*?*from= Disallow: /*?*category[ Disallow: /*?*category%5B Disallow: /*?*town[ Disallow: /*?*town%5B