# Onto-Reader/1.0 — crawler policy and contact > Onto-Reader/1.0 is an on-demand fetcher triggered by Onto Read API customers. Identity, behavior, robots.txt rules, and how to allow or block it. **Source:** /about-bot **Extracted:** 2026-05-20T20:59:17.547Z --- Crawler policy ## Onto-Reader/1.0. _An on-demand fetcher, not a crawler._ This page exists because our user-agent string points to it. If you found Onto-Reader in your server logs and want to understand what it is, what it does, and how to control it — that's the whole purpose of this page. Plain English, no surprises. One request per API call. Honors robots.txt. Identifies itself. _Contact us at founder@buildonto.dev if anything looks wrong._ 01 — How to identify it ### The fields you'll see in _your access logs._ Onto-Reader sends a single, consistent user-agent on every request. There is no rotation, no spoofing, no alternate identifier. If a request claims to be Onto-Reader but doesn't originate from Vercel's edge network, it isn't us. User-Agent `Onto-Reader/1.0 (+https://buildonto.dev/about-bot)` Origin network `Vercel edge (api.buildonto.dev)` Operator `Onto — buildonto.dev` Contact `founder@buildonto.dev` 02 — How it behaves ### Four things Onto-Reader _always_ does. These aren't aspirational. They're what the fetch code does on every single request, with no toggle to disable them. Behavior · 01 #### On-demand, not a crawler. Onto-Reader never spiders. It fetches a URL only when an Onto Read API customer makes an authenticated request for that exact URL. No queues, no link-following, no background indexing. Behavior · 02 #### robots.txt is honored. Before every fetch, Onto-Reader requests /robots.txt and checks for a User-agent: GPTBot or User-agent: \* block with Disallow: /. If your site disallows AI crawlers globally, Onto-Reader treats that as a block and the API surface returns the policy state to the caller. Behavior · 03 #### Lightweight by design. One GET request per call. 10-second hard timeout. Follows redirects. Sends Accept: text/html,application/xhtml+xml. No JavaScript execution. No cookies. No form submission. No re-fetching the same URL inside the 1-hour cache window. Behavior · 04 #### Errors are surfaced clearly. If your origin returns 403 or 401, the API returns WAF\_BLOCKED to the developer. If your origin times out, the API returns TIMEOUT. Customers see why their request failed and can choose another source — there are no silent retries against your site. 03 — Allow or block ### Control Onto-Reader from your _robots.txt._ Onto-Reader honors directives addressed to GPTBot or to the wildcard agent (\*). Add one of the rules below to allow or disallow it at the path level you care about. #### Allow Onto-Reader everywhere ``` User-agent: * Allow: / ``` Default behavior on most sites. Onto-Reader will fetch any URL a customer asks for. #### Block Onto-Reader entirely ``` User-agent: GPTBot Disallow: / ``` Onto-Reader checks the GPTBot section as the canonical AI-crawler directive. A site-wide Disallow blocks it from fetching anything. #### Allow only public pages ``` User-agent: GPTBot Disallow: /admin/ Disallow: /account/ Disallow: /api/ ``` Keep authenticated and internal paths off-limits while letting agents read your public content. Recommended for most sites. #### Block at the network layer ``` # In your WAF or edge rules: # Match User-Agent starts with "Onto-Reader/" # Return 403 ``` If you need a hard block independent of robots.txt, deny by user-agent prefix in your CDN or WAF. The API will surface WAF\_BLOCKED to the developer and stop trying. 04 — Why it exists ### One fetch, identified, on behalf of a _known developer._ AI developers building agents need to read the public web. The alternative to a transparent shared fetcher is every team operating its own rotating-IP scraper, with no consistent identity and no way for sites to opt out cleanly. Onto-Reader is the honest version of that traffic: one user-agent, one policy page, one inbox you can write to. > If you don't want agents reading your site through us, block us in robots.txt and we'll stop. That's the whole policy. Onto crawler policy Report abuse #### See Onto-Reader doing something it shouldn't? Email us with the request log line and a sample timestamp. We respond within one business day. [founder@buildonto.dev](mailto:founder@buildonto.dev?subject=Onto-Reader%20abuse%20report) Learn more #### Curious what we actually do with the page? Onto-Reader feeds the Read API: HTML in, clean Markdown plus an accuracy score out. Nothing is stored beyond a 1-hour edge cache. [How it works](/how-it-works)