NeovaAdvanced SEO Platform

Non-Canonical Host Variant Serves Its Own robots.txt That Blocks or Differs From the Canonical Rules

In Neova Runs from the crawl itself: nothing to set up.

Rule last reviewed October 17, 2026

The rule, the method and the ticket for this check are kept for Neova users and not published here.

Why It Matters

A crawler does not fetch a disallowed URL, so a Disallow on an old host hides its 301s and slows consolidation after a migration. Bing, Applebot and AI crawlers read robots.txt per host too (Anthropic asks for the file on every subdomain), so a stale variant file can give each crawler different rules.

Written by Neova Labs · Back to the Check Library