# Bober AI Systems - open for search engines and AI assistants. # /uz/* and /kz/* (except /kz, /kz/pricing) -> 301 to RU. /order -> noindex. # Do NOT Disallow those paths: crawlers must see the 301 / meta robots. # # Google Search indexing (AI Overviews / AI Mode) → Googlebot # Gemini Apps grounding + Gemini training eligibility → Google-Extended # Allowing Google-Extended does not boost ranking; blocking it removes Gemini grounding. User-agent: * Allow: / Disallow: /api/ Disallow: /partners/ru.html User-agent: Googlebot Allow: / Disallow: /api/ Disallow: /partners/ru.html User-agent: Google-Extended Allow: / # Yandex-specific rules (this block replaces User-agent: * for Yandex). User-agent: Yandex Allow: / Disallow: /api/ Disallow: /partners/ru.html Clean-param: utm_source&utm_medium&utm_campaign&utm_content&utm_term&yclid&fbclid&gclid&_openstat User-agent: GPTBot Allow: / User-agent: ChatGPT-User Allow: / # ChatGPT Search citations (OpenAI also uses Bing + own OAI-SearchBot). # Training crawler GPTBot is a separate agent above. User-agent: OAI-SearchBot Allow: / User-agent: ClaudeBot Allow: / User-agent: Claude-User Allow: / User-agent: Claude-SearchBot Allow: / User-agent: anthropic-ai Allow: / User-agent: PerplexityBot Allow: / User-agent: Perplexity-User Allow: / User-agent: Applebot Allow: / User-agent: Applebot-Extended Allow: / User-agent: meta-externalagent Allow: / User-agent: FacebookBot Allow: / User-agent: Amazonbot Allow: / User-agent: Bytespider Allow: / User-agent: CCBot Allow: / User-agent: Diffbot Allow: / User-agent: cohere-ai Allow: / User-agent: AI2Bot Allow: / User-agent: YouBot Allow: / User-agent: DuckAssistBot Allow: / User-agent: MistralAI-User Allow: / User-agent: YandexAdditional Allow: / User-agent: YandexAdditionalBot Allow: / User-agent: PetalBot Allow: / User-agent: Claude-Web Allow: / User-agent: GoogleOther Allow: / User-agent: Google-CloudVertexBot Allow: / # Bing index feeds ChatGPT Search / Copilot — keep open + IndexNow (npm run webmaster:bing). User-agent: Bingbot Allow: / User-agent: SemrushBot-OCOB Allow: / User-agent: Timpibot Allow: / User-agent: Webzio-Extended Allow: / User-agent: Omgilibot Allow: / User-agent: ImagesiftBot Allow: / User-agent: ProRataInc Allow: / User-agent: TikTokSpider Allow: / Host: www.bober-systems.ru Sitemap: https://www.bober-systems.ru/sitemap.xml # LLM: /llms.txt (alias /llm.txt), /llms-full.txt, /info.md, /info.txt