# Legal, cookies and crawlers
The legal pages and what each covers, exactly what this site stores in your browser, the AI disclosure, and the crawler policy — including where it differs by bot.
**What this covers:** The legal pages; What this site stores in your browser; The AI disclosure; The crawler policy.
**Assumed knowledge:** Nothing.
**Canonical HTML:** https://jobopportunitiesapi.org/docs/website/legal
**Machine-readable index:** https://jobopportunitiesapi.org/docs/ai/index.md
**Last verified:** 2026-08-22
**Superseded by:** the live API at https://api.jobopportunitiesapi.org and its spec at https://jobopportunitiesapi.org/openapi.json — where this file and the API disagree, the API is right.
---
## 1. The legal pages
Four pages, each answering a different question, all linked from the footer of every page on the site.
### 1.1 In depth
- **[/privacy](https://jobopportunitiesapi.org/privacy)** — What the website collects about visitors and customers, what the analytics receive, and what is never collected. There is no personal data in the ledger itself — a job row names a company, not a person.
- **[/cookies](https://jobopportunitiesapi.org/cookies)** — Exactly what is stored in your browser and why. See [below](#site-cookies).
- **[/terms](https://jobopportunitiesapi.org/terms)** — The terms of use for the site and the licence for API data.
- **[/ai](https://jobopportunitiesapi.org/ai)** — The AI disclosure: which fields are machine-generated, by which models, and what is never sent to one. See [below](#site-ai-policy).
- **[/contact?topic=data](https://jobopportunitiesapi.org/contact?topic=data)** — Corrections and takedowns for a single listing.
- **[/optout](https://jobopportunitiesapi.org/optout)** — Self-service removal of a whole domain. [Detail](./ledger-optout.md).
- **`support@jobopportunitiesapi.org`** — A real, monitored mailbox, published in the footer of every page.
### 1.2 Exact contract
Two structural points that matter more than the wording. First, the ledger holds no personal data: there are no candidates, no applications, no CVs and no jobseeker accounts, so most of what a privacy policy usually has to cover simply does not arise. Second, removal is self-service and verified — a site owner does not have to email anybody or wait for a human to act.
**See also**
- [Removing your site](./ledger-optout.md#optout-how)
- [Corrections and takedowns for a single listing](./ledger-optout.md#takedowns)
- [The AI disclosure](./website-legal.md#site-ai-policy)
## 2. What this site stores in your browser
A signed session cookie when you are signed in, a record of your consent choice, and analytics only if you allow them.
### 2.1 In depth
| Stored | Purpose | Set when |
| --- | --- | --- |
| `joa_session` | Signed sign-in session, 30 days, HTTP-only. Not a database row — the server verifies it cryptographically. | You follow a sign-in link. |
| Consent record | Remembers your cookie choice so the notice does not reappear. | You answer the notice. |
| Analytics | Product analytics, only with consent. | You opt in. |
| Documentation feedback | Which documentation pages this browser has voted on, so the tally is not a click counter. No identifier, first-party, never sent to the API. | You vote on a docs page. |
The consent manager is reachable at any time from **Cookie preferences** in the footer of every page. The essential category is always on and says so — it covers signing in and remembering the notice itself, which cannot be declined without breaking both.
### 2.2 Exact contract
Consent decisions are recorded, so there is an auditable record of what was chosen and when, rather than only a cookie in your browser. What the analytics provider does and does not receive is stated explicitly on [/privacy](https://jobopportunitiesapi.org/privacy) rather than left to the provider's own documentation.
> **The API sets no cookies at all** — `api.jobopportunitiesapi.org` is stateless and cookie-free. Do not send `credentials: "include"` to it. Everything on this page is about the website.
**See also**
- [CORS and calling from a browser](./api-overview.md#cors-and-browsers)
- [Telling us a page is wrong](./conventions.md#feedback-loop)
- [Navigation — the header and the footer](./website-pages.md#site-nav)
## 3. The AI disclosure
Two fields are always machine-generated, three are sometimes, and every row says which. No candidate or customer data is ever sent to a model.
### 3.1 In depth
| Field | Where it comes from | Provenance |
| --- | --- | --- |
| `title` | The posting, unchanged. | always published |
| `company` | The posting. The display name is tidied, never invented. | always published |
| `location` | The posting, or derived by us from it. | `field_sources.location` says which: `published` when the employer stated it, `inferred` when we resolved it against a gazetteer |
| `apply_url` | The employer's own link. Never rewritten, never proxied. | always published |
| `employment_type` | The posting, when the employer states it. | published or absent |
| `posted_at` | The posting or the source feed. | published or absent |
| `description` | The employer's own text, unchanged. | published or absent |
| `category` | **Always ours.** A model reads the title and assigns a family. | always inferred |
| `seniority` | **Always ours.** A model reads the title and assigns a level. | always inferred |
| `remote` | Published when the employer says so; inferred when we conclude it from the text. | either — check the row |
| `salary` | Published when the posting carries structured pay; inferred when parsed out of prose. | either — check the row |
| `source_type` | Published when the source declares it; inferred otherwise. | either — check the row |
Nothing else in the ledger is model-generated. There is no generated description text, no generated salary figure, no generated employer and no generated apply link.
> **No decisions are made about people** — This service classifies job postings. It does not score, rank, filter, match or evaluate any person, and it never sees one — there are no candidates in the system. If you build something that does evaluate people, that obligation is yours, and the per-field markers exist precisely so you can avoid building it on inferred fields without knowing they are inferred.
### 3.2 Exact contract
What is sent to a model is the text of a public job posting and a company name — the same words the employer published on their own careers page. **No customer data**: your account, your API key, your queries and your usage are never used as model input, never used to train anything, and never shared with a model vendor. **No candidate data**, because none exists.
The disclosure also states where the classification is weakest, which is worth reading if you are building on `category` or `seniority`: on a clear job title they are dependable, on a vague or invented one they are a guess, and the marker says `inferred` either way because the row cannot tell you which case you have. The full page, including which models are used for which pass, is at [/ai](https://jobopportunitiesapi.org/ai).
**See also**
- [field_sources — per-field provenance](./ledger-provenance.md#field-sources)
- [Salary, specifically](./ledger-provenance.md#salary-provenance)
- [Why require_fields=category is an error](./ledger-provenance.md#the-422)
## 4. The crawler policy
Search engines get the whole ledger. Answer engines get the pages that describe the product but not the two million rows that are the product.
### 4.1 In depth
| Who | Allowed | Refused |
| --- | --- | --- |
| Search engines (Googlebot, Bingbot, Applebot, DuckDuckBot, Yandex, Baidu, …) | Everything, including `/job/*` and `/company/*`. | `/api/`, `/dashboard`, `/admin`, and the faceted query space `/*?`. |
| Answer engines (GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, PerplexityBot, CCBot, …) | The home page, `/docs`, `/api`, `/about`, `/coverage`, `/facts`, `/compare`, `/pricing`, `/provenance`, and the job and company **indexes**. | The per-row corpus: `/job/*` and `/company/*`. Plus the same private paths. |
| Link unfurlers (Twitterbot, facebookexternalhit, LinkedInBot, Slackbot, Discordbot) | Everything except the private paths — an unfurler fetches the one URL a human just pasted, which is a citation, not a harvest. | `/api/`, `/dashboard`, `/admin`, `/*?`. |
| Everything else | Nothing. | The ledger is free to read in a browser and licensed for bulk use through the API; scraping is the one route that is neither. |
**This documentation is allowed for everyone**, answer engines included. It describes the product; it is not the corpus.
### 4.2 Exact contract
The reasoning behind the asymmetry, since it is a deliberate choice rather than a reflex: a job page ranking in a search engine sends a human here, and that human can see a price. An answer engine that ingests the same page answers the question inside its own product and the visit never happens. Handing two and a half million rows to a model vendor for free, while selling the same rows through the API, is selling something you are also giving away — to the buyer least likely to come back. What answer engines keep is deliberately generous: enough to describe the dataset accurately, cite it, and tell somebody where to get it.
`/*?` — the faceted query space — is refused to every crawler. The home page exposes country, city, category, remote and employment as query parameters, so the number of distinct URLs is combinatorial. One crawler enumerating them generated 1,562 requests in three minutes, 97% of them a query string never seen before, so every one missed the page cache and forced a fresh render and a fresh query; that single pattern held the server at 96% CPU for as long as it continued. Nothing of value is lost — a filtered view is duplicate content whose canonical is the bare page, and every individual job and company already has its own indexable URL in the sitemap.
> **robots.txt is a request; the edge is the enforcement** — Naming a user agent in `robots.txt` grants nothing on its own. Verified-bot identity is checked at the edge by IP and reverse DNS, not by the user-agent string — so declaring yourself Googlebot achieves nothing.
**See also**
- [Linking to one section](./conventions.md#deep-links)
- [The machine-readable mirror](./for-agents.md#agents-start-here)
- [Two hosts, and which one to send requests to](./api-overview.md#hosts)
---
## Where to go next
This file is part of **The website**. Others in the same group:
- [The ledger browser](./website-ledger.md) — Every control on the home page, the job page and the company pages — what each one sends to the API, and what each label on a row actually asserts.
- [Every page and every form](./website-pages.md) — The complete route list, what the header and footer link to, and every form on the site with the fields it collects and where they go.
Always useful:
- [index.md](./index.md) — the map of every file here
- [BUILD-A-SITE.md](./BUILD-A-SITE.md) — the paste-whole brief for building against this API
- [quickstart.md](./quickstart.md) — zero to a first authenticated response
- [api-errors.md](./api-errors.md) — every status code and whether to retry it