User-agent: * Allow: /uploads/ Disallow: /api Disallow: /logs Disallow: /pdf Disallow: /web_api