# Async SaaS integrations ## Overview Fides offers support for async integrations where an access or erasure request cannot be fulfilled immediately. Two async strategies are supported: **callback** and **polling**. - **Callback**: The vendor calls back to Fides when the job is complete. Fides waits passively. - **Polling**: Fides submits the request and then actively polls the vendor's status endpoint on a scheduled interval until the job is complete. ## Prerequisite: Enable DSR 3.0 DSR 3.0 task scheduling added the ability to run tasks in parallel and introduced task persistence to the application database to support async integrations. To enable DSR 3.0, you must be running a celery worker as part of your Fides deployment, in addition to the environment variables below: ```plaintext FIDES__CELERY__TASK_ALWAYS_EAGER=false FIDES__EXECUTION__USE_DSR_3_0=true ``` Further, you can optionally update these configuration variables which specify how long data lives in your system and the time to wait between status updates. See [Configuration variables](/content/docs/platform/configuration/configuration#execution/index.html) for more information. ```plaintext FIDES__REDIS__DEFAULT_TTL_SECONDS=604800 FIDES__EXECUTION__REQUEST_TASK_TTL=604800 FIDES__EXECUTION__STATE_POLLING_INTERVAL=30 ``` ## Callback strategy In the callback strategy, the third-party service is responsible for POSTing results back to Fides when the async job completes. The integration config must denote `strategy: callback` and include `reply-to` and `reply-to-token` headers so Fides can pass callback instructions to the vendor. ### Example Async SaaS Config Endpoint In the example below, we describe how a third-party custom integration can denote its `callback` async strategy, as well as add `reply-to` and `reply-to-token` values to request headers to tell the third party service how to send the results asynchronously back to Fides. ```yaml saas_config: fides_key: saas_async_config name: Async Callback Example Custom Connector type: async_callback_example description: Test Async Config version: 0.0.1 connector_params: - name: domain - name: api_token label: API token client_config: protocol: http host: authentication: strategy: bearer configuration: token: test_request: method: GET path: / endpoints: - name: user requests: read: method: GET path: /api/v1/user query_params: - name: query value: param_values: - name: email identity: email async_config: strategy: callback headers: - name: reply-to value: - name: reply-to-token value: ``` ### Asynchronous Access Results Asynchronous access results can be posted back to the Fides `reply-to` endpoint as a list of rows, using the `reply-to-token` as the Bearer Token. Note that these results may be filtered and returned to the end user and/or used to make downstream dependent requests. ```plaintext curl -X 'POST' \ 'http://localhost:8080/api/v1/request-task/callback' \ -H 'accept: application/json' \ -H 'Authorization: Bearer ' \ -H 'Content-Type: application/json' \ -d '{\ "access_results": [\ {"id": "user_id", "system_id": "test_system_id", "state": "test_state"}\ ] }' ``` ### Polling strategy In the polling strategy, Fides submits the initial request to the vendor and then actively checks a vendor-provided status endpoint on a recurring schedule until the job is confirmed complete. Use this strategy when the vendor exposes a status check endpoint but cannot — or will not — POST results back to Fides. ### How it works 1. Fides sends the initial access or erasure request to the vendor endpoint. 2. Fides extracts a **correlation ID** from the response body using the `correlation_id_path` field. This ID uniquely identifies the async job at the vendor. 3. Fides stores the job as a sub-request and enters a waiting state. 4. On each scheduled check (controlled by `FIDES_EXECUTION_ASYNC_POLLING_INTERVAL_HOURS`, default every 1 hour), Fides calls the `status_request` endpoint, substituting `` in the path or parameters. 5. Fides reads the status field at `status_path` in the response. When it matches `status_completed_value`, the job is considered done. 6. For access requests, Fides then calls the `result_request` endpoint to retrieve the user's data. For erasure requests, no result fetch is needed — completion of the status check is sufficient. 7. If the job does not complete within `FIDES__EXECUTION__ASYNC_POLLING_REQUEST_TIMEOUT_DAYS` (default: 30 days), the task is marked as errored. ### Example Async Polling SaaS Config The example below shows a connector with both an access (`read`) and an erasure (`delete`) endpoint configured for polling. ```yaml saas_config: fides_key: saas_async_polling_config name: Async Polling Example Custom Connector type: async_polling_example description: Example SaaS config using the polling async strategy version: 0.0.1 connector_params: - name: domain - name: api_token label: API token sensitive: true client_config: protocol: https host: authentication: strategy: bearer configuration: token: test_request: method: GET path: / endpoints: - name: user requests: read: method: GET path: /api/access-package param_values: - name: email identity: email correlation_id_path: request_id # JSON path in the response to extract the job ID async_config: strategy: polling configuration: status_request: method: GET path: /api/access-package/status status_path: status # JSON path in the status response to evaluate status_completed_value: completed # value that signals the job is done result_request: # required for access requests; omit for erasure method: GET path: /api/access-package/result delete: method: DELETE path: /api/anonymize-user/ param_values: - name: email identity: email correlation_id_path: correlation_id async_config: strategy: polling configuration: status_request: method: GET path: /api/anonymize-user//status status_path: status status_completed_value: completed # result_request is omitted for erasure operations ``` ### Polling configuration field reference | Field | Required | Description | | :-- | :-- | :-- | | `correlation_id_path` | Yes | Dot-notation path into the initial response body to extract the job/correlation ID (e.g. `request_id`, `data.job.id`). Can also reference a `param_value` in scope if the response body is empty. | | `async_config.strategy` | Yes | Must be `polling`. | | `status_request.method` / `path` | Yes | HTTP method and path for the status check endpoint. The `` placeholder is available for use in the path or query parameters. | | `status_request.status_path` | Yes* | Dot-notation path within the status response JSON to the field indicating job state (e.g. `status`, `data.state`). | | `status_request.status_completed_value` | Yes* | The value at `status_path` that indicates the job has completed successfully (e.g. `completed`, `done`, `true`). | | `result_request.method` / `path` | No | HTTP method and path to retrieve the final results. Required for access requests; omit for erasure requests. | | `result_request.result_path` | No | Dot-notation path within the result response to the array of result rows (e.g. `data.users`). Omit if results are at the response root. | \* Required unless using `request_override` for custom status-check logic.