User-agent: Scrapy Allow: / User-agent: * Disallow: /calendar/action~posterboard/ Disallow: /calendar/action~agenda/ Disallow: /calendar/action~oneday/ Disallow: /calendar/action~month/ Disallow: /calendar/action~week/ Disallow: /calendar/action~stream/ Disallow: /calendar/action~undefined/ Disallow: /calendar/action~http:/ Disallow: /calendar/action~default/ Disallow: /calendar/action~poster/ Disallow: /calendar/action~*/ Disallow: /*controller=ai1ec_exporter_controller* Disallow: /*/action~*/ Disallow: /events/action~posterboard/ Disallow: /events/action~agenda/ Disallow: /events/action~oneday/ Disallow: /events/action~month/ Disallow: /events/action~week/ Disallow: /events/action~stream/ Disallow: /events/action~undefined/ Disallow: /events/action~http:/ Disallow: /events/action~default/ Disallow: /events/action~poster/ Disallow: /events/action~*/