# Verxus.gg Robots.txt # Last updated: 2025-11-30 # =========================================== # DEFAULT RULE - Allow all crawlers # =========================================== User-agent: * Allow: / # =========================================== # SEARCH ENGINE CRAWLERS # =========================================== # Google User-agent: Googlebot Allow: / # Bing (also powers DuckDuckGo, Yahoo, ChatGPT browsing) User-agent: Bingbot Allow: / # Yandex (Russia) User-agent: Yandex Allow: / # Baidu (China) User-agent: Baiduspider Allow: / # =========================================== # AI ASSISTANT CRAWLERS # Explicitly allowing AI tools to index VERXUS # =========================================== # OpenAI / ChatGPT User-agent: GPTBot Allow: / # OpenAI ChatGPT Plugins User-agent: ChatGPT-User Allow: / # OpenAI Search (not training, shows links in ChatGPT) User-agent: OAI-SearchBot Allow: / # Anthropic / Claude User-agent: Claude-Web Allow: / User-agent: ClaudeBot Allow: / User-agent: anthropic-ai Allow: / # Google AI (Gemini, Bard) User-agent: Google-Extended Allow: / # Perplexity AI User-agent: PerplexityBot Allow: / # Perplexity user-triggered fetches User-agent: Perplexity-User Allow: / # Cohere AI User-agent: cohere-ai Allow: / # Meta AI User-agent: Meta-ExternalAgent Allow: / User-agent: Meta-ExternalFetcher Allow: / User-agent: FacebookBot Allow: / # Amazon (Alexa AI) User-agent: Amazonbot Allow: / # DuckDuckGo AI User-agent: DuckAssistBot Allow: / # You.com AI Search User-agent: YouBot Allow: / # Phind AI (Developer Search) User-agent: PhindBot Allow: / # Apple (Siri, Spotlight) User-agent: Applebot Allow: / User-agent: Applebot-Extended Allow: / # Common Crawl (used by many AI training datasets) User-agent: CCBot Allow: / # =========================================== # SITEMAP & LLMS.TXT LOCATIONS # =========================================== Sitemap: https://verxus.gg/sitemap.xml # AI-readable site description (emerging standard) # See: https://llmstxt.org/ # Location: https://verxus.gg/llms.txt # =========================================== # DISALLOWED PATHS # =========================================== # None currently - VERXUS is fully public # Example: Disallow: /admin/