> ## Documentation Index
> Fetch the complete documentation index at: https://docs.chift.eu/llms.txt
> Use this file to discover all available pages before exploring further.

# Incremental synchronization

When you synchronize your customers' financial data with the Unified API, you want to fetch only what's new or changed since your last synchronization. Incremental synchronization is the most efficient way to keep data up-to-date without fetching the entire dataset on every run.

## Why incremental synchronization matters

Fetching all records on every synchronization has real costs:

* **Slower synchronizations** — more data to transfer and process
* **Higher load** on the connected systems
* **Unnecessary processing** — comparing all records to detect changes is expensive
* **Rate limiting risk** — repeated full synchronizations can hit connector rate limits

With incremental synchronization, you only fetch what's changed since your last successful synchronization.

## How it works

Most Chift Unified API collection endpoints support the `updated_after` parameter, which returns only records created or updated on or after a specified timestamp.

1. Store the timestamp of your last successful synchronization
2. On the next synchronization run, pass that timestamp as `updated_after`
3. Process only the new and updated records
4. Store the new timestamp for the next run

### Example request

```http theme={null}
GET /accounting/clients?updated_after=2024-10-15T14:30:00Z
Authorization: Bearer <token>
```

**Response:** Only clients created or updated after `2024-10-15T14:30:00Z`.

***

## Implementing incremental synchronization

### 1. Check if `updated_after` is supported

The API reference for each endpoint documents whether `updated_after` is available. Not all connectors support it for every endpoint, and some newer connectors may have limited support.

### 2. Store your last synchronization timestamp

Keep the timestamp of your last successful synchronization for each consumer and each endpoint. This is the value you'll pass as `updated_after` on the next run.

```javascript theme={null}
// Example: storing the last synchronization time in your database
const lastSyncTime = new Date('2024-10-15T14:30:00Z');

const response = await fetch(
  `https://api.chift.eu/consumers/{consumer_id}/accounting/clients?updated_after=${lastSyncTime.toISOString()}`,
  {
    headers: {
      'Authorization': `Bearer ${token}`,
    }
  }
);

// After successful processing:
await db.updateLastSync(consumerId, 'customers', new Date());
```

***

## First synchronization: full initial load

On your first synchronization for a consumer, you don't have a previous timestamp to use. Instead, use the `date_from` and `date_to` parameters (if supported) to load historical data, or simply fetch all records without `updated_after`.

```http theme={null}
GET /accounting/clients
Authorization: Bearer <token>
```

After this initial load completes successfully, store the timestamp and switch to incremental synchronizations with `updated_after`.

```javascript theme={null}
// First synchronization: full load
if (!hasPreviousSyncTime) {
  // Fetch all records in the desired date range
  const firstSyncResponse = await fetch(
    `https://api.chift.eu/consumers/{consumer_id}/accounting/clients`,
    { headers: /* ... */ }
  );
  
  // Process all records...
  
  // Store the synchronization time for next run
  lastSyncTime = new Date();
} else {
  // Subsequent synchronizations: incremental only
  const incrementalResponse = await fetch(
    `https://api.chift.eu/consumers/{consumer_id}/accounting/clients?updated_after=${lastSyncTime.toISOString()}`,
    { headers: /* ... */ }
  );
  
  // Process only changed records...
}
```

***

## When `updated_after` is not supported

Some connectors don't support incremental synchronization for certain endpoints. When this happens, you'll need to manually track changes by comparing the current response with your stored records.

### Strategy: Compare and identify changes

1. **Fetch all records** from the endpoint (you'll need to paginate if there are many)
2. **Compare** each record with your stored version
3. **Identify** three types:
   * **New records** — in the API response but not in your system
   * **Updated records** — in your system but with changed values (check `updated_at` if available)
   * **Deleted records** — in your system but missing from the API response (optional: implement soft deletes or mark as archived)

### Performance considerations

Manually comparing all records is less efficient than true incremental synchronization. We recommend:

* Requesting higher [pagination limits](/developer-guides/unified-api/pagination) to reduce API calls
* Running these synchronizations less frequently (e.g., once per day instead of every hour)
* Consider using caching headers to avoid unnecessary fetches — see [Caching in Chift API](/developer-guides/unified-api/caching)

***

## Best practices

### 1. Always store timestamps in UTC

Always work with UTC timestamps. Many timezone bugs come from mixing timezones.

```javascript theme={null}
// ✅ Good
const syncTime = new Date().toISOString(); // "2024-10-15T14:30:00.000Z"

// ❌ Avoid
const syncTime = new Date().toString(); // Contains local timezone info
```

### 2. Be precise with timing

Store the exact timestamp of when the synchronization completed, not rounded times. This ensures you don't miss edge-case records created at exact second boundaries.

### 3. Handle synchronization failures gracefully

If a synchronization fails partway through, don't update your stored timestamp. Only update it after all records have been successfully processed and stored.

### 4. Monitor connector-specific behavior

Some connectors have quirks:

* Records might be updated within a brief window after creation
* Deleted records might still appear for a short time with a `deleted_at` field
* Timestamps might have millisecond precision on some systems and second precision on others

Test your synchronization logic with your integrated connectors to understand their behavior.

### 5. Plan for the first synchronization carefully

Initial synchronizations can be large. Consider:

* Running them at off-peak times
* Breaking them into smaller date ranges if the dataset is huge
* Using `date_from` and `date_to` to limit the initial load period

***

## Troubleshooting

### No records returned, but I know data was updated

Check:

* Is `updated_after` supported for this endpoint? (Check the API reference)
* Is your timestamp in UTC?
* Are you using the correct format? (ISO 8601: `YYYY-MM-DDTHH:MM:SSZ`)
* Is your stored timestamp actually *before* the data you expect? (Clocks can drift)

### Different results between connectors

Connector behavior varies:

* Some update `updated_at` when related objects change (e.g., marking an invoice as paid)
* Others only update `updated_at` when the record itself changes
* Some don't support `updated_at` at all

Always test incremental synchronization with each connector you plan to support.

### Timestamps are drifting

If your synchronizations keep missing recent data, you may have a clock drift issue between your system and Chift's servers. Monitor the `server_time` in API responses and adjust your next synchronization time accordingly.

### Memory issues with full synchronizations

If you're doing manual comparison (for connectors without `updated_after` support) and hitting memory limits with large datasets:

* Fetch and process records in smaller batches rather than loading everything at once
* Use streaming if your API client supports it

***

## Related documentation

* [Pagination limits](/developer-guides/unified-api/pagination) — Understanding page sizes and throughput
* [Caching in Chift API](/developer-guides/unified-api/caching) — How cached responses work and when to bypass them
* [Good practices for create and update requests](/developer-guides/unified-api/good-practice-create-and-update-requests) — Handling mutations reliably
* [API reference](/api-reference) — See which endpoints support `updated_after`