> ## Documentation Index
> Fetch the complete documentation index at: https://docs.chift.eu/llms.txt
> Use this file to discover all available pages before exploring further.
# Fetching data
> Keep your own copy of the data layer in sync efficiently: an initial full load, incremental updated_after reads, deletes surfaced by webhook, and a periodic full fetch as a safety net.
If you mirror Chift data into your own store — a warehouse, a reporting database, an AI context store — you don't want to re-read everything on every cycle. The data layer is built for exactly this: reads are served from Chift's store rather than the live source, so you can fetch often and cheaply, and you can ask for **only what changed** since last time.
The recommended strategy combines two complementary patterns:
On every cycle, fetch only the records created or changed since your last fetch, using the `updated_after` parameter.
On a slower cadence (for example weekly), fetch everything to realign your store — a safety net that also recovers any missed deletes.
Everything below is about **reading** from the data layer. Make sure the data layer is enabled on the connection and that you send the `x-chift-datalayer: true` header on these reads — see [Activating the data layer](/developer-guides/datalayer/activation). All examples send that header.
## The sync model
Run a full load once to seed your store, then keep it fresh incrementally and reconcile periodically:
```mermaid theme={null}
sequenceDiagram
participant C as Chift data layer
participant Y as Your system
participant T as Timer
rect rgb(214, 245, 235)
note over C,Y: Data-changed — on each refresh
C->>Y: Webhook: account.datalayer.refresh_executed
Y->>C: Read with updated_after = previous fetch start
C-->>Y: Created / changed records
end
rect rgb(255, 241, 214)
note over Y,T: Periodic full fetch — weekly
T->>Y: Weekly trigger
Y->>C: Full read (no updated_after)
C-->>Y: Full data set → reconcile deletes
end
```
The first time, page through each resource with no `updated_after` filter and upsert every record into your store. This seeds your dataset.
On each subsequent cycle, request the same resources with `updated_after` set to the start of your previous fetch. You get back only the records that were created or changed since, and you upsert them.
On a slower cadence (for example once a week), do a full fetch and reconcile as a safety net for **deletes** — the `account.datalayer.data_deleted` webhook reports removals in real time, and the full pass recovers any you might have missed.
## Incremental reads with `updated_after`
The accounting resources that grow over time accept an `updated_after` query parameter. When set, the data layer returns only the records whose last-updated timestamp is **after** the value you pass — created or modified records — so each incremental cycle stays small and fast regardless of how much history exists.
`updated_after` is available on:
* Clients (`/accounting/clients`)
* Suppliers (`/accounting/suppliers`)
* Invoices (`/accounting/invoices`)
* Journal entries (`/accounting/journal-entries`)
```bash cURL theme={null}
curl -G "https://api.chift.eu/consumers/{consumerId}/accounting/invoices" \
-H "Authorization: Bearer " \
-H "x-chift-datalayer: true" \
--data-urlencode "updated_after=2024-01-31T15:00:00Z"
```
A few rules to get right:
* **Always pass UTC.** Use an ISO 8601 timestamp in UTC (for example `2024-01-31T15:00:00Z`). UTC is the only format supported consistently across connectors.
* **Use the start of your previous fetch, not its end.** Record when each cycle *began*, and pass that value as `updated_after` next time. This overlaps the window slightly and avoids missing records that changed while the previous fetch was running.
* **Reads still paginate.** A large incremental window can still span several pages — keep following pagination until the page is empty. See [Pagination limits](/developer-guides/pagination).
* **Upsert, don't append.** A record can come back because it was updated, so key your writes on the record id and upsert.
Reference data — folders, chart of accounts, journals, VAT codes and book years — is small and changes rarely, so there's no incremental mode for it. Just re-fetch each set in full on your cycle; it's cheap.
## React to refresh webhooks
The data layer is only as fresh as its last sync, so there's no point fetching more often than it refreshes. Instead of polling on a timer, subscribe to the datalayer webhooks and react when a refresh happens:
* `account.datalayer.refresh_initiated` — a refresh started for a consumer.
* `account.datalayer.refresh_executed` — the refresh finished for that consumer, with a `status` of `success` or `error`.
* `account.datalayer.data_changed` — which resources changed in that refresh.
* `account.datalayer.data_deleted` — which records were **removed** from the data layer in that refresh.
When you receive `account.datalayer.refresh_executed` (or `data_changed`), trigger an incremental read with `updated_after` set to the start of your previous fetch and upsert the results — this keeps your store aligned without guessing the cadence. When you receive `account.datalayer.data_deleted`, delete the listed records from your store (see [Handling deletes](#handling-deletes)). See [Webhooks](/developer-guides/webhooks) for the payload format and how to verify the signature.
## Handling deletes
`updated_after` returns created and changed records — it can **never** return a record that was deleted in the source, because a deleted record simply stops appearing in reads. Incremental reads alone will therefore let deleted records linger in your store.
The data layer surfaces deletes in two ways:
* **The `account.datalayer.data_deleted` webhook (recommended).** Each refresh emits this event with the records it removed, so you can drop exactly those from your store as soon as the refresh completes — no extra reads, and deletes stay as fresh as your incremental updates.
* **A periodic full fetch, as a safety net.** On a slower cadence (weekly is a good default), fetch each resource in full and reconcile against your store: any record you hold that is absent from the full result set has been removed at the source and should be deleted (or soft-deleted) on your side. This also recovers any delete notification you might have missed.
Don't reconcile deletes from an incremental result set. A response filtered by `updated_after` is only a partial slice of the data — treating absence from it as a delete would wipe every record that simply didn't change. Only reconcile deletes against a **full**, unfiltered fetch.
## Putting it together
A typical loop:
Full load once. Record the timestamp at which the load started.
On each `account.datalayer.refresh_executed` webhook (or your chosen cadence), read each incremental resource with `updated_after` = the start of your previous fetch, upsert the results, then store the new fetch-start timestamp. Apply any `account.datalayer.data_deleted` events to drop removed records.
Once a week, full-fetch every resource and reconcile against your store as a safety net — this recovers any delete you might have missed.
This keeps day-to-day traffic tiny while guaranteeing your store fully realigns with the source on every full pass.
## Related
What the data layer is and how it stays fresh.
Enable it on a connection and use the header.