User-agent: Scrapy
Allow: /
User-agent: 
*
Disallow: /privacy-policy/
Disallow: /term-and-condition/

Disallow: /author/admin/