User-agent: * Disallow: /admin/ Disallow: /csv-files/ Disallow: /downloads/ Disallow: /images/ Disallow: /scripts/