Norg Edge Mirror Routing: Solution Architecture

Where a CDN the site owner controls already proxies the site, the edge worker is the smallest change and the only one that keeps agent requests off the origin entirely. Where the CDN is bundled with the host and not configurable, the in-application pattern avoids introducing a new network layer for this alone. A CDN can also be added in front of one hostname without moving DNS; what that involves and costs on Cloudflare and Fastly is on CNAME setup. Where nothing is to be installed, the NORG-hosted mirror serves the same pages from a hostname NORG runs.

NORG’s crawler reads the site’s public sitemap and pages over HTTPS, as a search engine does, and renders each into a mirror. The integration in the site owner’s estate never uploads page content to NORG: it fetches mirrors, it does not produce them. A page that is not in the sitemap is not mirrored, and commerce or account endpoints are never rendered.

Browser, DNS, the site’s CDN or host cache, the origin, exactly as today. The search crawlers Googlebot, Bingbot, Applebot, DuckDuckBot, Baiduspider, Yandex and Slurp are matched by a fixed list and passed to the origin before any classification runs. Serving them anything different would be cloaking, so no setting exists for it.

1. The user agent is matched against the policy feed. Today the diverted set is GPTBot, OAI-SearchBot and ChatGPT-User (OpenAI); ClaudeBot, Claude-User and Claude-SearchBot (Anthropic); PerplexityBot and Perplexity-User (Perplexity); each operator is a per-site switch.
2. The source address is checked against the ranges that operator publishes. The check is address-range only and fails closed: no match, no mirror. A person who fakes a GPTBot user agent gets the normal page.
3. For a verified agent the integration fetches the mirror for that path from the content service, authenticated by Site ID and Site Key, and returns it at the site’s own URL with Cache-Control: private, no-store, so no cache ever stores it or shows it to a person.
4. If the content service answers 404, the integration asks NORG to render the page and serves the live page instead. On WordPress the live page is first stripped to its text; if fewer than 120 words remain, the untouched page is served.
5. Any other outcome (a timeout, an error, NORG unreachable) serves the origin page. The mirror fetch is bounded: 4 s on WordPress; each CDN adapter’s own bound is on its page.

The crawler fetches the public sitemap and the pages it lists over HTTPS, using public URLs only. Sites on the default interval schedule are re-crawled every 7 days, adjustable per site; a schedule saved in the portal defaults to monthly. NORG has no CMS login, no database access and no host access for this. A single page can also be re-rendered on request.

Everything the integration sends, and how often. Nothing is sent for an ordinary human visitor in serving mode. The cadences and budgets below are the WordPress plugin’s; the feed interval and heartbeat are common to every adapter, and each CDN adapter’s own bounds are on its page.

In record-only mode the integration serves nothing different and instead posts visit events, and, while citation tracking is on for the site (the default), page-views for people who arrived from an AI answer or a search result. Those page-views carry a pseudonymous, daily-rotating visitor id rather than an address or a cookie. This is the flow a privacy assessment covering human visitors reviews, and the one that is switched off on request.

When the site owner requests uplift for a page, NORG sends that page’s text (from NORG’s own mirror, capped at 24,000 characters) and the relevant knowledge-base context to a large language model and receives additional content for the page: a question-and-answer block and, where the knowledge base supplies the context for it, an in-depth write-up grounded in that context. The output is appended to the page’s mirror and copied into the public scope of the knowledge base. The page text is instructed to win over the knowledge base where they conflict. Nothing runs on a schedule, and the live site is not touched. The generated content can be edited or cleared by the site owner through MCP.

A document is uploaded through the portal or the private MCP, stored in the workspace’s private blob container, parsed inside NORG’s worker (PDF and Office parsing runs locally; images are read by a vision model), and indexed into the workspace’s private graph: the text is chunked, embedded and mined for entities and relations by hosted models. Uploaded documents are never placed in the public scope directly. They can reach public output in one way: uplift grounds its questions and answers in the private graph, and the resulting block is published on the mirror and copied to the public scope. The decision of what to supply therefore governs what can surface.

Public. An AI agent that speaks MCP to the site has its request relayed by the integration to NORG’s public server with the site’s identity attached, so it can only read that site: search the public knowledge-base scope, read business details, list and fetch published pages and related items. Every tool is read-only. Searches are limited to 600 a minute per site and the query text is not stored. Private. Workspace members and their agents, holding a scoped API key, use the full read and write surface (content, publishing, knowledge base, edge routing, uplift); every call is written to an audit trail. Connector. The ChatGPT and Claude connectors authenticate the site owner by OAuth; their tools include installs and content edits, and an install acts in the site owner’s CDN account or WordPress only with the credential the owner lends for it.

1. NORG creates the site record in the workspace and issues the Site ID, a one-time Site Key and the integration package for the chosen pattern.
2. The site owner’s team, or NORG with access granted for the purpose, installs the integration and gives it the two values, using the secret store the pattern offers.
3. Where the host’s cache runs before the integration, the one rule from Hosting-specific changes is added so agent requests reach it.
4. Verification: the integration’s own check confirms that a simulated agent reaches it and that a browser does not, and the first heartbeat moves the site from pending to active in NORG.

A Cloudflare Workers, Fastly or Bunny account for the edge pattern; WordPress 6.0 and PHP 7.4 or later for the plugin; a Next.js or Vercel middleware for the npm package. None of the integrations bundles other runtime libraries. The site does not depend on NORG to stay up: with NORG unreachable, every visitor receives the origin.

No DNS change for the in-application patterns or an existing CDN. No CMS login, database access or host access for NORG. No change to mail routing. No inbound firewall rule: the only traffic NORG originates towards the site owner’s estate is the crawler reading public pages, and, where remote connect is chosen, an administrator REST call the site owner can revoke.

It can fetch the site’s own mirrors and post the site’s own telemetry. It cannot read or change anything in the site owner’s estate, and it cannot reach another workspace: the tenant is always resolved from the authenticated site record, never from a request parameter. The integration sends it only to the two NORG hosts, over TLS, with redirects disabled.

• Fails safe. Every error, timeout and unknown state ends at the origin response, in every adapter.
• No cloaking. Search engines always receive what people receive. The list is fixed in code; there is no setting.
• Verified agents only. A user agent is never trusted on its own; the source address must be inside the operator’s published ranges.
• No cache contamination. Every mirror response is private, no-store, with the CDN and cache-plugin equivalents set alongside, so no cache stores it.
• Checksum-verified supply chain. Each adapter’s package is pinned or verified by SHA-256 before it runs.
• Same content, re-presented. Mirrors are built from the site’s own public pages. NORG adds no claims and changes no wording.

Privacy. AI-agent visits are bot traffic, not personal data; the analytics store has no IP-address column and receives no cookies or form data. The one flow that touches people, citation page-views in record-only mode, uses a pseudonymous daily-rotating id and can be switched off per site.

Data residency. NORG’s database is Azure Database for PostgreSQL in Australia East. Mirrors are stored in Cloudflare R2 and served through Cloudflare’s global edge network, which caches each mirror close to the requesting agent for performance.

Retention. Visit analytics are not expired on a schedule, because they are not tied to individuals. Mirrors, the site record and the analytics are deleted on request, and deleting the integration removes its settings from the site owner’s estate.

Change control. The change is one or two items (an integration and, where needed, one host rule), each of which can go through the site owner’s normal process and be scheduled. A trial can start in record-only mode, or on a single page.

The WordPress plugin hooks in at the earliest plugin stage and handles only GET requests for pages (plus HEAD on its own artefacts and POST to the MCP path). It skips admin, AJAX, cron, CLI and REST requests. It reads the path, user agent and source address, and never cookies, authorisation headers or request bodies. The Vercel middleware behaves the same way inside the framework’s request pipeline.

Settings sit in one CMS option or in environment variables. On WordPress the Site Key may be defined in wp-config.php, in which case it is never written to the database; entered on the settings screen, it is encrypted with a key derived from the site’s own salts. Each constant and its default is in the technical reference.

Updates come from NORG’s manifest and are SHA-256 verified before install; automatic updates can be turned off. The plugin registers two REST endpoints, both restricted to administrators, so NORG’s dashboard can connect it where the site owner chooses. Deactivating removes its cron jobs and cache rules; uninstalling also removes every option and transient it created.

• Cloudflare, Fastly. Remove the route or the service: immediate.
• WordPress. Tick Disable, define NORG_EDGE_DISABLED as true, or deactivate the plugin: effective on the next request. Removing the host rule instead means agents receive the cached page.
• Vercel. Set the disable variable and redeploy, or remove the middleware.
• From NORG’s side. Revoking a site’s entitlement reaches the integration at its next policy refresh, within an hour. There is no instant remote off-switch; the immediate ones are the site owner’s.
• Uplift and knowledge base. Uplift is cleared per page through MCP. Knowledge-base documents are deleted per document or per snapshot; full removal of the workspace’s documents, graph and storage container is performed by NORG on instruction. Disabling public MCP for the site stops anonymous MCP reads.

• Which integration pattern, and who performs the install: the site owner’s team, or NORG with access granted for the duration.
• Where the Site Key is held on the site owner’s side: a worker secret, Secrets Manager, wp-config.php, or a sensitive environment variable.
• Update policy: automatic from NORG’s checksum-verified channel, or approved through the site owner’s change control.
• Whether to start in record-only mode, or in a single-page scope, before serving every mirrored page.
• Citation page-view tracking: left on (the default) or switched off for the site.
• Which AI operators to enable. Each is a per-site switch; search engines are never on the list.
• A staging environment to trial on first, if the platform has one.
• For an in-application install: that the origin is scaled for AI-crawler traffic in addition to human traffic, since those requests bypass the host cache.
• Whether to supply documents to the knowledge base, and which: anything supplied can inform uplift output that is published.
• Whether uplift is used at all, and who may request it per page.
• Whether public MCP is enabled for the site, so AI agents can query it as well as read mirrored pages.
• The retention terms required, confirmed with NORG in writing for the engagement.

• Edge routing white paper: The product argument, cloaking, and the comparison across CDNs.
• WordPress plugin technical reference: Every constant, header, timeout and surface the plugin has.
• WordPress plugin on WP Engine: The one rewrite rule that gets agents past WP Engine’s page cache.
• Cloudflare Worker install: The two-pass install and the token permissions it needs.
• CNAME setup: A CDN in front of one hostname without moving DNS: Cloudflare and Fastly, costs, and the comparison with a full zone move.
• Shopify (Orange-to-Orange): The DNS arrangement Shopify stores need.
• Vercel middleware: The npm package and its two environment variables.
• Hosting options: NORG-hosted versus self-hosted, and what leaves the site owner’s account.