# As a condition of accessing this website, you agree to abide by the following # content signals: # (a) If a Content-Signal = yes, you may collect content for the corresponding # use. # (b) If a Content-Signal = no, you may not collect content for the # corresponding use. # (c) If the website operator does not include a Content-Signal for a # corresponding use, the website operator neither grants nor restricts # permission via Content-Signal with respect to the corresponding use. # The content signals and their meanings are: # search: building a search index and providing search results (e.g., returning # hyperlinks and short excerpts from your website's contents). Search does not # include providing AI-generated search summaries. # ai-input: inputting content into one or more AI models (e.g., retrieval # augmented generation, grounding, or other real-time taking of content for # generative AI search answers). # ai-train: training or fine-tuning AI models. # use: how AI systems may consume the content (immediate, reference, or full). # ANY RESTRICTIONS EXPRESSED VIA CONTENT SIGNALS ARE EXPRESS RESERVATIONS OF # RIGHTS UNDER ARTICLE 4 OF THE EUROPEAN UNION DIRECTIVE 2019/790 ON COPYRIGHT # AND RELATED RIGHTS IN THE DIGITAL SINGLE MARKET. # BEGIN Bloque anti-IA de FemGol # La lista sale de servidor/bots.json y va A LA PAR del muro de # servidor/Caddyfile: esto es el cartel y aquello la puerta. Pasa quien pueda # traer visitas: por eso NO estan aqui los buscadores, los indexadores que # citan con enlace (OAI-SearchBot, Claude-SearchBot, Amzn-SearchBot, YouBot, # PerplexityBot, AionBot), la visita con humano detras (ChatGPT-User, # Claude-User, Perplexity-User) ni las tarjetas al compartir. User-agent: * Content-Signal: search=yes,ai-train=no,use=reference Allow: / User-agent: AhrefsBot Disallow: / User-agent: AI2Bot Disallow: / User-agent: amazon-kendra Disallow: / User-agent: Amazonbot Disallow: / User-agent: anthropic-ai Disallow: / User-agent: Applebot-Extended Disallow: / User-agent: Awario Disallow: / User-agent: Barkrowler Disallow: / User-agent: bedrockbot Disallow: / User-agent: BLEXBot Disallow: / User-agent: Brightbot Disallow: / User-agent: Bytespider Disallow: / User-agent: CCBot Disallow: / User-agent: ChatGLM-Spider Disallow: / User-agent: Claude-Web Disallow: / User-agent: ClaudeBot Disallow: / User-agent: CloudflareBrowserRenderingCrawler Disallow: / User-agent: Cocolyzebot Disallow: / User-agent: cohere-training-data-crawler Disallow: / User-agent: Cotoyogi Disallow: / User-agent: Crawlspace Disallow: / User-agent: DataForSeoBot Disallow: / User-agent: DeepSeekBot Disallow: / User-agent: Diffbot Disallow: / User-agent: DomainStatsBot Disallow: / User-agent: DotBot Disallow: / User-agent: ExaSearchBot Disallow: / User-agent: FacebookBot Disallow: / User-agent: FirecrawlAgent Disallow: / User-agent: Google-CloudVertexBot Disallow: / User-agent: Google-Extended Disallow: / User-agent: GPTBot Disallow: / User-agent: iaskspider Disallow: / User-agent: ICC-Crawler Disallow: / User-agent: ImagesiftBot Disallow: / User-agent: img2dataset Disallow: / User-agent: Kangaroo Bot Disallow: / User-agent: Lightpanda Disallow: / User-agent: MegaIndex Disallow: / User-agent: meta-externalads Disallow: / User-agent: meta-externalagent Disallow: / User-agent: MistralAI-Training Disallow: / User-agent: MJ12bot Disallow: / User-agent: omgili Disallow: / User-agent: PanguBot Disallow: / User-agent: Poseidon Research Disallow: / User-agent: SBIntuitionsBot Disallow: / User-agent: Scrapy Disallow: / User-agent: SearchmetricsBot Disallow: / User-agent: SemrushBot Disallow: / User-agent: SeobilityBot Disallow: / User-agent: SEOkicks Disallow: / User-agent: SERankingBacklinksBot Disallow: / User-agent: serpstatbot Disallow: / User-agent: Sidetrade indexer bot Disallow: / User-agent: SISTRIX Disallow: / User-agent: SiteCheckerBotCrawler Disallow: / User-agent: TavilyBot Disallow: / User-agent: TerraCotta Disallow: / User-agent: Timpibot Disallow: / User-agent: VelenPublicWebCrawler Disallow: / User-agent: Webzio-Extended Disallow: / User-agent: XoviOnpageCrawler Disallow: / User-agent: YandexAdditionalBot Disallow: / User-agent: ZoominfoBot Disallow: / # END Bloque anti-IA de FemGol # Los cuatro endpoints de /api/ son funciones POST: alta y baja del boletín, # webhook de Brevo y un export protegido con 401. A un GET responden 4xx, que # es lo correcto, y por eso Search Console reportaba /api/unsubscribe como # «Bloqueada debido a otro problema de tipo 4xx» (2026-08-06). Google no llegó # por un enlace: no existe ninguno. Lo sacó leyendo el fetch() dentro del JS de # /desuscribir/. No se indexan, no deben rastrearse y rastrearlas gasta cuota # que hace falta en otra parte. # # OJO CON /stats/, QUE NO SE BLOQUEA ENTERO. Es el proxy de Umami y lo único que # sirve es /stats/script.js. Google necesita cargar el JS para renderizar las páginas, # y bloquear scripts en robots.txt es justo lo que su documentación desaconseja. # # Lo que sí se bloquea es /stats/api/, el contador: al renderizar, Googlebot # ejecuta el script y POSTea /stats/api/send, y el muro se lo niega con un 403 # (la medición es de los lectores). Medido en el registro del 16 al 17-sep-2026: # 1.777 de las 5.599 peticiones de Google a la casa eran ese POST. Es la misma # cuota que hace falta en otra parte. # # Y ojo con lo que se lee aquí: producción NO sirve este fichero tal cual. # Cloudflare le antepone su bloque gestionado (los Content-Signal y los # Disallow a los bots de IA). Para ver lo servido, curl, no este fichero. User-agent: * Allow: / Disallow: /api/ Disallow: /stats/api/ Sitemap: https://femgol.com/sitemap-index.xml