# Keeping a local copy

For products that store jobs in their own database: how to key rows, load once, stay current for the fewest records, handle closures, re-openings and outages, and how large a copy each plan can keep fresh.

**What this covers:** Identity: key every row on id; First load; Staying current: which feed, on which plan; Replace the whole row on every update; Remote roles open to US candidates; Closures and re-openings; Outages, retries, and why there is no idempotency key; How large a copy each plan can keep fresh; Vectors for local semantic search; Display rules: storing, crediting, linking.

**Assumed knowledge:** [The record meter](./account-record-meter.md) (one record per row delivered) and [pagination](./api-pagination.md).

**Canonical HTML:** https://jobopportunitiesapi.org/docs/recipes/local-copy  
**Machine-readable index:** https://jobopportunitiesapi.org/docs/ai/index.md  
**Last verified:** 2026-11-08  
**Superseded by:** the live API at https://api.jobopportunitiesapi.org and its spec at https://jobopportunitiesapi.org/openapi.json. Where this file and the API disagree, the API is right.

---

<a id="local-copy-identity"></a>

## 1. Identity: key every row on id

Use `id` as the primary key. It survives apply link changes, edits, closure and re-opening. Do not key on apply_url.

### 1.1 In depth

JOA assigns `id` once, when it first harvests a posting, and never changes or reuses it. Internally the posting is matched on its source and the source's own posting identifier, so a new apply URL, a corrected title, a new salary or a rewritten description arrives as an update to the same id. A closed job that re-opens keeps its id too.

- **When a new id appears for what looks like the same job:** the source published it as a new posting (a re-post under a new requisition or posting number), or the employer moved to another applicant tracking system. JOA cannot prove those are one vacancy. When a re-crawl under a new id has the same company, title, city and the exact same apply URL, JOA withholds the duplicate, so you usually see one of the two.
- **Why not apply_url:** it changes when an employer edits a posting or migrates its ATS, and the id does not. A copy keyed on the URL turns one job into two the day the URL moves.
- **source_requisition_id** is the reference the source page itself states, copied verbatim when the page has one (a minority of rows). It is not validated and can be odd (occasionally a URL the page used as an identifier). Use it as a hint when you link re-posts, never as a key. JOA does not publish its internal per-source posting key, and there is no other employer requisition field.
- **canonical_url** exists only on roles an employer pushed to JOA directly; with it comes `attribution`. Crawled rows carry neither: their link is `apply_url`.
- **slug and company_slug** are readable aliases. The slug follows title corrections and company slugs can change; store them, do not key on them.

**See also**

- [Identity: ids, slugs and domains](./ledger-data-model.md#ids-and-slugs)
- [The job object](./api-fields-and-values.md#fv-job)

<a id="local-copy-first-load"></a>

## 2. First load

Choose your slice with filters, page with limit=200, persist next_cursor after each committed page, and record when you started.

### 2.1 In depth

1. **Size the slice first.** `include_total=true` adds a count that is not charged, so `/v1/jobs?<your filters>&include_total=true&limit=1` costs at most one record and tells you how many rows the load will cost (exact up to 10,000).
2. **Record the start time** before the first page. It is the `since` of your first `/v1/changes` poll, or the first `listed_after` of your update loop.
3. **Page with `limit=200`**, the largest page on every plan (`GET /v1/me` shows yours as `limits.max_page_size`). Records are counted per row, not per request, so small pages cost the same records and more requests. With `include_description=true` the cap is 50; with `include=embedding` it is 100.
4. **Write each page, then persist its `next_cursor`,** in the same transaction if you can. A crash then repeats at most one page. The cursor is opaque: store the string, never build one.
5. **Upsert by id.** A row can appear twice if your load overlaps a change (it moved while you paged). The second copy is newer; replace.

Signal and Scale include [/v1/export](./endpoints-export.md), a stream of the whole slice in one request. Its records are charged per row written, the same as paging.

### 2.3 Worked examples

How many rows a slice holds, and how many entered it since a date: at most one record

```bash
curl -s -H "Authorization: Bearer $JOA_KEY" \
  'https://api.jobopportunitiesapi.org/v1/jobs?country=US&listed_after=2026-11-01T00:00:00Z&include_total=true&limit=1' \
  | jq '{total_estimate, total_estimate_status}'
```

**See also**

- [How paging works](./api-pagination.md#cursor-basics)
- [How many rows match: include_total](./api-pagination.md#include-total)
- [Resuming an interrupted pull](./api-pagination.md#resumable-pulls)

<a id="local-copy-current"></a>

## 3. Staying current: which feed, on which plan

Growth and above: /v1/changes, one record per changed row per poll. Cheaper on records, and on every plan: listed_after for new rows plus the free /v1/jobs/expired and /v1/jobs/withdrawn for rows that left, at the price of not seeing edits.

### 3.1 In depth

| Approach | Plans | What you receive | Records |
| --- | --- | --- | --- |
| `/v1/changes?since=<next_since>` | Growth, Signal, Scale | Every row of your countries that was created, updated, closed or withdrawn since your cursor, as the complete job object. | One per event. Events caused only by JOA correcting its own data carry `correction: true` and are free. |
| `/v1/jobs?<filters>&listed_after=<last run>` plus `/v1/jobs/expired` and `/v1/jobs/withdrawn` | Every plan (the two id feeds too, since 9 Nov 2026) | New rows, re-admitted rows and re-opened rows; plus the ids of rows that closed, and of rows that stopped being served without closing (duplicates, holds, opt-outs). | One per new row. Both id feeds are free. |
| `/v1/jobs/{id}` | Every plan | One row, with its description, on demand. | One per call. A closed row is 404 unless you add `include_closed=true`; a withdrawn row answers `status: "withdrawn"` with `withdrawn_at` and `withdrawn_reason`, free. |

The change feed keeps one entry per job, at its latest change. A row that changes three times between two of your polls costs one record, not three, so polling less often is cheaper for rows that change often. Polling every few hours is enough: the ledger refreshes in batches.

`listed_after` selects rows that **entered** the served set after the instant you pass: new, re-admitted after a withdrawal, or re-opened after a closure (each counts from its latest entry). It does not return rows you already hold that were edited (a changed salary, say). If you use it, refresh rows you care about with `/v1/jobs/{id}` or re-load the slice on a schedule. Subtract a few minutes of overlap from your last run time; duplicates are harmless when you upsert by id.

`posted_after` is not an update filter: `posted_at` is the employer's own date, typically hours to weeks behind the moment the row reaches the API.

**See also**

- [The sync loop, written correctly](./endpoints-changes.md#changes-loop)
- [GET /v1/jobs/expired](./endpoints-jobs.md#endpoint-jobs-expired)
- [GET /v1/jobs/withdrawn](./endpoints-jobs.md#endpoint-jobs-withdrawn)
- [What counts as a record](./account-record-meter.md#meter-what-counts)

<a id="local-copy-replace"></a>

## 4. Replace the whole row on every update

Every row and every created or updated event carries the complete job object. Replace your stored row by id; never merge field by field.

### 4.1 In depth

`/v1/jobs`, `/v1/jobs/{id}`, `/v1/jobs/closed` and `/v1/changes` all build the job object from one column list and one serialiser, so an event's job is exactly the row `/v1/jobs` would return. Optional fields are **omitted** when they have no value, so a missing key means "no value now": if a salary disappears at the source, the key disappears from the object, and a merge would keep the old figure. A few keys are always present and may be `null` instead (`closed_at`, `closed_reason`, `category_confidence`, `language`, `language_confidence`, `language_basis`). [Fields and values](./api-fields-and-values.md#fv-job) lists the presence of every key.

- **description is the exception.** It is never on `/v1/changes` events, on `/v1/jobs` only with `include_description=true`, and always on `/v1/jobs/{id}`. Keep it in its own column and do not blank it when an event arrives without it. `description_length` is on every object; when it differs from the length of the text you hold, the text changed: fetch `/v1/jobs/{id}` (one record).
- **embedding** is present only when you ask for it with `include=embedding`; keep your stored vector when an object arrives without one.
- **Branch on the event kind, not on job.status.** A `withdrawn` event can carry the complete job with `status: "live"` (the vacancy is still open, JOA stopped serving it). Stop showing it anyway.

**See also**

- [The four change kinds, and the one that breaks integrations](./endpoints-changes.md#change-kinds)
- [Nullability, in one table](./api-fields.md#nullability)
- [The job object](./api-fields-and-values.md#fv-job)

<a id="local-copy-remote"></a>

## 5. Remote roles open to US candidates

A remote US role carries country US and remote remote. Single-country remote roles get their country since 8 Nov 2026 (older rows are backfilled during the day); a role open in two or more countries still carries only one.

### 5.1 In depth

- **country is one ISO code.** There is no multi-country field. A remote role open to US candidates normally carries `country: "US"`: on 8 Nov 2026, 110,345 live remote rows did.
- **remote** is `remote`, `hybrid` or `on_site`. `remote_inferred` and `field_sources.remote` say whether the source stated it (`published`) or we read it from the location or title (`inferred`). Of those 110,345 US rows 79,822 were stated and the rest inferred, so keeping only `published` values drops about a quarter of US remote roles. `remote_confirmed=true` filters to stated values on the API side.
- **Single-country remote roles carry their country.** Until 8 Nov 2026, 3,668 live remote rows that name the US in their location text ("Remote US", "US Remote", "Anywhere in the U.S.") had no country, so `country=US` missed them. Since 06:10 UTC that day new rows get their country, and older rows are being backfilled during the day as free corrections (`updated` events with `correction: true`, not charged; `updated` events are billed only when the employer changed the job, changes from our own processing are free corrections).
- **A role open in two or more countries carries one of them.** About 440 served remote roles name two or more countries (for example the United States and Canada) and are stored under one, so a single-country filter misses those stored under the other. How to represent them is not decided yet.
- **Anywhere.** 1,255 live remote rows say anywhere, worldwide or global. They carry a country when the text names one ("Anywhere in the U.S." is US) and none otherwise.
- **city and region describe the stated location,** often the employer's office, not where the candidate must live. A remote US role can carry `region: "UT"` and a city.

Measured on 8 Nov 2026 on the database replica over live, not opted out rows, before the serving gates (about three quarters of live remote US rows were served in a sample), so treat the counts as proportions rather than as what `/v1/jobs` returns.

**See also**

- [The job object](./api-fields-and-values.md#fv-job)
- [Place: country, city, state](./api-parameters.md#params-place)
- [Provenance and serving: source, provider, quality, poster type, status](./api-parameters.md#params-provenance-filters)

<a id="local-copy-closures"></a>

## 6. Closures and re-openings

Mark closed rows closed rather than deleting them. A closed job can come back with the same id: on /v1/changes as updated, in listed_after as a new entry, and it then leaves /v1/jobs/expired.

### 6.1 In depth

| Where | A job closes | The same job re-opens |
| --- | --- | --- |
| `/v1/changes` | `delisted` event; the job carries `status: "closed"`, `closed_at`, `closed_reason`. | `updated` event (never `created`), `status: "live"`, `closed_at` and `closed_reason` null. |
| `/v1/jobs?listed_after=` | Not returned (it lists served rows only). | Returned if it re-opened after your `listed_after`: each row counts from its latest entry. |
| `/v1/jobs/expired` | Listed with `closed_at` and `closed_reason`. | No longer listed: the feed lists rows that are closed now. If it closes again it reappears with a newer `closed_at`. |

**With listed_after and the two id feeds:** each cycle, first read `/v1/jobs/expired` and `/v1/jobs/withdrawn` from their cursors (mark expired ids closed, stop showing withdrawn ids), then read `/v1/jobs?listed_after=` and upsert every row it returns as live, **even when your copy holds that id as closed or withdrawn**: a row `listed_after` returns is served now, so it wins. In that order a job that left and came back between two runs ends live, and one that came back and left again is caught by the next page of the id feeds. Without `/v1/jobs/withdrawn` a copy keeps rows JOA stopped serving without closing (duplicates, holds, opt-outs) for ever. Never treat a closure as permanent and never refuse to re-open an id.

`closed_reason` is an open set; every current value and its meaning is in [Fields and values](./api-fields-and-values.md#fv-values). Treat an unknown reason as closed.

**See also**

- [GET /v1/jobs/expired](./endpoints-jobs.md#endpoint-jobs-expired)
- [GET /v1/jobs/withdrawn](./endpoints-jobs.md#endpoint-jobs-withdrawn)
- [The four change kinds, and the one that breaks integrations](./endpoints-changes.md#change-kinds)
- [Closed roles](./ledger-data-model.md#closed-roles)

<a id="local-copy-outage"></a>

## 7. Outages, retries, and why there is no idempotency key

There is no idempotency key and you do not need one: the same page asked for again within 15 minutes of the charged call costs 0 records when the answer is the same. Persist cursors after each committed page and retry only the request that failed. On `GET /v1/jobs`, a job you were already charged for in the last 24 hours is not charged again, so overlapping `since` windows and retried pages cost nothing extra for jobs you already hold.

### 7.1 In depth

Records are charged when a page is served. If your client gave up, or the connection failed, before the first byte of the response left the server, nothing is charged. If it broke while the body was being sent, the page is charged once, and asking for **the same page again within 15 minutes** of that charge (same key, endpoint and parameters; a different `cursor`, `next_since` or `limit` is a different page) costs 0 records when the response is byte for byte the same. So after an ambiguous failure, retry promptly with exactly the same request instead of holding the page for review. [The record meter](./account-record-meter.md#meter-repeat) has the full rule.

| Header | Use it to |
| --- | --- |
| `X-JOA-Records-Charged` | Record what each response cost: the rows minus any free repeats, or 0 for a free repeat. |
| `X-JOA-Repeat` | `true` when this exact answer was already charged to your key in the last 15 minutes. |
| `X-JOA-Records-Repeated` | On `GET /v1/jobs`, the rows of this response that were free because you had already been charged for them in the last 24 hours. |
| `X-Request-Id` | Log it with every page you store; it identifies the response if you ever ask us about a charge. |

- **Retry only the failed request,** with its own cursor, after a short back-off and well inside the 15 minutes. A 503 is a query timeout and is safe to retry; a 429 says when to retry; a 402 means the monthly allowance is spent.
- **Persist the cursor right after storing the page.** A crash then repeats at most one page, and that repeat is free inside the window. After 15 minutes, or when rows in the page changed, a re-read is charged again, except on `GET /v1/jobs`, where a job already charged to your key in the last 24 hours stays free.
- **The change feed and the id feeds replay.** A `since` cursor can be sent again, and `since` can go back to any earlier time (the expired feed holds closures back to August 2026, the withdrawn feed starts on 23 Oct 2026). After an outage of any length, resume from the last cursor you stored; nothing is lost while you are away.
- **Keep the page size large.** Small pages do not make a failure cheaper in total, they only make more requests.

> **If a charge still looks wrong**: Email hello@jobopportunitiesapi.org with the X-Request-Id and we will correct it.

**See also**

- [Paying once for the same rows](./account-record-meter.md#meter-repeat)
- [What counts as a record](./account-record-meter.md#meter-what-counts)
- [Resuming an interrupted pull](./api-pagination.md#resumable-pulls)
- [next_cursor is opaque: this is the rule that bites](./api-pagination.md#cursor-opacity)

<a id="local-copy-sizing"></a>

## 8. How large a copy each plan can keep fresh

Divide the plan's monthly records by 30 for a daily budget, and divide that by the share of your slice that changes each day. Measure your own slice; the figures below are one week's.

### 8.1 In depth

| plan | display_name | price | records_per_month | req_per_day | req_per_min | delta_feed | bulk_export | single_country | continent_lock |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| explore | Explore | Free | 1,000 | 5,000 | 30 | false | false | false | false |
| starter | Starter | €29/month | 10,000 | 20,000 | 60 | false | false | false | false |
| growth | Growth | €80/month | 60,000 | 100,000 | 120 | true | false | false | false |
| signal | Signal | €299/month | 400,000 | 400,000 | 300 | true | true | true | false |
| scale | Scale | €899/month | 2,000,000 | 1,000,000 | 600 | true | true | false | true |
| starter_annual | Starter (annual) | €261 | 10,000 | 20,000 | 60 | false | false | false | false |
| growth_annual | Growth (annual) | €720 | 60,000 | 100,000 | 120 | true | false | false | false |
| signal_annual | Signal (annual) | €2,691 | 400,000 | 400,000 | 300 | true | true | true | false |
| scale_annual | Scale (annual) | €8,091 | 2,000,000 | 1,000,000 | 600 | true | true | false | true |

_Generated from `GET https://api.jobopportunitiesapi.org/public/plans` (no key required). Only purchasable plans appear there._

_Measured 8 Nov 2026. Live rows: `/v1/meta/facets` at 05:09 UTC. Changed rows: the change feed ledger on the database replica, counting the billable events (correction events excluded) whose latest position fell in the 24 hours, or the 7 days, before 03:56 UTC, which is what a poller reading once a day, or once a week, would have paid. That week included large re-admissions after new sources were added, so treat the upper end as a busy week. Since then `updated` events are billed only when the employer changed the job; changes from our own processing are delivered free and marked as corrections, so a poller pays less than these counts._
| Country | Live rows | Changed rows in 24 h | Per day, 7-day average | Daily change share |
| --- | --- | --- | --- | --- |
| US | 1,578,733 | 261,719 | 159,124 | 10 to 17% |
| DE | 200,243 | 14,391 | 10,080 | 5 to 7% |
| GB | 96,860 | 12,892 | 8,641 | 9 to 13% |
| IN | 88,980 | 11,609 | 10,012 | 11 to 13% |
| FR | 74,903 | 6,806 | 5,247 | 7 to 9% |
| CA | 73,499 | 15,561 | 7,472 | 10 to 21% |
| NL | 46,664 | 2,363 | 2,375 | 5% |
| AU | 27,950 | 2,792 | 2,347 | 8 to 10% |

**The arithmetic.** Growth's 60,000 records a month is about 2,000 a day. Polling `/v1/changes` daily for a slice that changes 10 to 17% a day, that keeps roughly 12,000 to 20,000 rows fresh, and nothing is left for loading. All of the US changes about 160,000 to 260,000 rows a day, far beyond Growth; it fits a Signal key locked to the US or a Scale key locked to North America, where job rows are not metered. On Growth, either narrow the slice (category, region, remote, seniority) until its daily changes fit, or use `listed_after` plus the free expired and withdrawn feed, which costs only the rows that enter your slice and drops edits to rows you hold.

**Measure your own slice** before you choose. The total of `/v1/jobs?<filters>&listed_after=<24 hours ago>&include_total=true&limit=1` is how many rows entered it in a day (at most one record). For the change rate, poll `/v1/changes` with your countries for one day and read `count`.

**See also**

- [The plans](./account-plans.md#plans-table)
- [The country-locked plan](./account-record-meter.md#country-locked)
- [The continent-locked plan](./account-record-meter.md#continent-locked)
- [What counts as a record](./account-record-meter.md#meter-what-counts)

<a id="local-copy-vectors"></a>

## 9. Vectors for local semantic search

Add include=embedding to /v1/jobs or /v1/changes and store the 1024-dimension vector next to the row; the records charged do not change.

### 9.1 In depth

- Each job then carries `embedding`: `model`, `dim` (1024), `normalized` (true) and `vector`, or `vector: null` with a `reason` (`no_vector`: not embedded yet; `vectors_unavailable`: repeat the call). `limit` is capped at 100.
- Store it in a vector column (for example pgvector `vector(1024)` with an HNSW cosine index over live rows). Vectors are unit length, so cosine similarity and the inner product are the same.
- Ask for it on the change feed too, so an `updated` event refreshes the vector with the row. When the event has no vector, keep the stored one.
- Embed your users' queries with the same model and the query prefix described under [embeddings](./endpoints-jobs.md#job-embeddings). Search then runs entirely in your database.

**See also**

- [Embeddings: include=embedding](./endpoints-jobs.md#job-embeddings)

<a id="local-copy-display"></a>

## 10. Display rules: storing, crediting, linking

Storing and caching are allowed, also after a job closes and after you cancel. Show the credit where listings are public, and use apply_url as the apply button.

### 10.1 In depth

From the [terms](https://jobopportunitiesapi.org/terms), section "What you may do with the data", verbatim:

- "Every commercial use is allowed, on any plan, including the free Explore plan. You may store, cache and display the records (including to your own users and customers), summarise or abstract the descriptions, extract data from them, use them in your own products and analyses, and re-package and resell them, including after a listing closes and after you cancel."
- "All of this is within your account's plan limits; those limits are shared by all the keys of your account and cover all downstream use, including what you resell."
- "When you display listings publicly, show a visible credit, "Data: Job Opportunities API", linking to https://jobopportunitiesapi.org."
- "Under these terms you do not need separate permission from each employer. The adverts remain the employers' own content."
- "Employers have the right to ask for their data to be removed. If an employer asks us, we remove it and we will email you, and you must then remove that employer's data from your product too. The ids to remove are also listed, free and on every plan, at /v1/jobs/withdrawn (and closed jobs at /v1/jobs/expired)."
- "If an employer asks you to remove its data, remove it and tell us at support@jobopportunitiesapi.org, so we can remove it here too. This rarely happens, but it protects everyone."

- **Apply button:** use `apply_url`, the employer's own application page or ATS link. JOA never sits between the candidate and the employer.
- **attribution:** when a job carries it, show it with the listing; when it also carries `canonical_url`, link to it. Some sources require their own credit (for example NAV and USAJOBS); the terms list their conditions.
- **Removals:** an employer opt-out reaches you as a `withdrawn` event on `/v1/changes`, or as an id with `withdrawn_reason: "employer"` on `/v1/jobs/withdrawn`. Removing the row then is how you honour it.

**See also**

- [How it propagates](./ledger-optout.md#optout-propagation)
- [Replace the whole row on every update](./recipes-local-copy.md#local-copy-replace)

---

## Where to go next

This file is part of **Recipes**. Others in the same group:

- [Client code](./recipes-languages.md): A working client in curl, Python, JavaScript and Go. No SDK, no dependencies, and every request re-sent against the live API before this page ships.
- [Keeping data current](./recipes-sync.md): Four jobs you will end up writing: a daily country pull, an incremental sync, closure detection, and a bulk export, with the record cost of each.
- [Things to build](./recipes-build.md): Three complete builds (a job board, an employer watcher, a CRM enrichment job), a semantic search recipe over job embeddings, and an honest list of which integrations exist today.

Always useful:

- [index.md](./index.md): the map of every file here
- [BUILD-A-SITE.md](./BUILD-A-SITE.md): the paste-whole brief for building against this API
- [quickstart.md](./quickstart.md): zero to a first authenticated response
- [api-errors.md](./api-errors.md): every status code and whether to retry it
