No official source defines a site as "too open" or "appropriately closed" to AI bots: every source only provides mechanisms (robots.txt, Content Signals, aipref, IP feeds, reverse DNS, Web Bot Auth, WebMCP) and leaves the choice to the site owner; no tool (Japanese or international) scores the balance.

The question I could not answer

"Is my site too open to AI?" is a fair question once AI crawlers show up in your logs. My scanner does not invent its own scoring: every verdict has to rest on an official spec or tool. So before answering, I needed an official yardstick.

Where I looked

On 22 October 2026 I read:

• RFC 9309, the robots.txt standard.
• The IETF drafts for AI preferences (aipref, version 08, which itself says it does not reflect consensus) and for Web Bot Auth.
• Content Signals.
• Cloudflare's verified bots policy.
• The crawler pages of OpenAI, Anthropic, Google, Perplexity and Apple.
• From Japan: the Agency for Cultural Affairs paper 「AIと著作権に関する考え方について」 (March 2024) and its checklist and guidance (July 2024), and the AI事業者ガイドライン version 1.2 (March 2026).
• The public pages of several Japanese and international LLMO and agent-readiness tools.

What I found

Mechanisms, and only mechanisms. robots.txt, Content Signals, aipref, published IP lists, reverse DNS, Web Bot Auth, WebMCP. Each one tells you how to open or close. None tells you how much is right.

The closest thing to a judgement came from the Agency for Cultural Affairs. Blocking the major training crawlers with robots.txt is one factor it names when Article 30-4 of the Copyright Act is weighed. That is a legal point about one choice, not a measure of balance. The AI business guidelines do not mention robots.txt or crawlers at all.

The tools I checked publish nothing on how closed a site should be. One agent-readiness checker scores whether things exist, not whether the choices make sense. Cloudflare's AI Crawl Control comes closest: it shows sites on Cloudflare, bot by bot, how much each one crawls compared with the visitors it sends back. Even there, the decision stays with you.

What this changes for my scanner

My scanner measures both sides, open and closed. It sets each one next to an official baseline, and gives a verdict only where a spec exists. "Too open" stays your call. It depends on what you sell and who you want to reach.

What you can do

Decide bot by bot, not all at once. OpenAI, Anthropic and Perplexity, for example, run separate crawlers for training, for search and for fetching a page on a user's request, and their own crawler pages name each one. You can block training and still allow search, and robots.txt can say so. Which mix is right for you, no official source will tell you.

What this does not prove

• "Not found" is not "does not exist". I may have missed a source.
• I have not yet read 知的財産推進計画2026 (Japan's Intellectual Property Strategic Program 2026), or the ACP and AP2 specifications, directly.

Both sides of openness, measured on my lab site: Open and closed to AI.