# Fides Workers Reference

Fides uses background workers to process various tasks asynchronously. This guide provides detailed information about worker types, configuration, and resource requirements for all deployment methods.

## Worker Configuration by Deployment Method

Worker setup varies significantly based on your deployment method. Be sure to follow the instructions for your specific deployment type.

### Automatic Worker Deployment

The Fides Helm chart **automatically includes** specialized workers by default:

**Standard Fides Deployments:**

- **DSR worker** (1 instance) - Handles data subject requests
- **Other worker** (1 instance) - Handles general tasks (excludes DSR and privacy preference queues)

**Fidesplus Deployments:**  
Includes all standard workers plus:

- **Classification worker** (1 instance) - For data discovery classification tasks
- **Helios worker** (1 instance) - For detection and promotion tasks
- **Consent worker** (1 instance) - For privacy preferences and consent webhooks

Production deployments should customize these defaults based on your specific product usage. See [Customizing Workers](/content/docs/platform/workers#customizing-workers/index.html) below.

## Production Configuration

### Product-Specific Requirements

**Lethe Users:**

- DSR worker

**Janus Users:**

- Consent worker
- Consent Webhook worker

**Helios Users:**

- Detection worker
- Classification worker
- Promotion worker

### General Recommendations

- Always configure an "Other" worker to handle messaging tasks and new task types released by Ethyca
- Use `exclude-queues` to prevent overlap with your specific workers:

```
    fides worker --exclude-queues=fides.dsr,fides.privacy_preferences
    ```

## Resource Requirements

| Worker Type                         | vCPU  | Memory             | Fides | Fidesplus |
|------------------------------------|-------|-------------------|-------|-----------|
| DSR                                | 0.5-1 | scalable up to 16Gi** | ✅     | ✅         |
| Consent                            | 0.5-1 | 0.75-2Gi         | ✅     | ✅         |
| Messaging                          | 0.5-1 | 1-2Gi            | ✅     | ✅         |
| Consent Webhook                    | 0.5-1 | 0.75-2Gi         | ❌     | ✅         |
| Detection*                         | 0.5-1 | 1-2Gi            | ❌     | ✅         |
| Classification (Legacy)*           | 2-4   | scalable up to 16Gi** | ❌     | ✅         |
| Classification (LLM-enabled)***    | 0.5-1 | 1-2Gi            | ❌     | ✅         |
| Promotion*                         | 0.5-1 | 1-2Gi            | ❌     | ✅         |

**
* These tasks are only available when the following environment variable is set: `FIDESPLUS__DETECTION_DISCOVERY__MONITOR_CELERY_TASKS_ENABLED=true` (enabled in Helm by default).

**
** Memory usage is heavily dependent on workload. Workers scale up during processing of large tables or DSRs but don't maintain peak memory usage continuously.

**\*\*\* To configure your environment for LLM-enabled classification, see the [LLM Classification setup guide](/content/docs/data-mapping/guides/llm-classification/index.html).

## Customizing Workers

### Overriding Default Workers

By default, the Helm chart includes DSR and Other workers (plus additional workers for Fidesplus). To customize or disable these defaults, you must **explicitly define your worker configuration**.

**To disable a default worker**, set its count to 0:

```yaml
workers:
  - name: dsr
    count: 0  # Disables the default DSR worker
  - name: custom-dsr
    count: 2
    queues:
      - fides.dsr
    image:
      repository: ethyca/fides
      tag: 2.51.0-slim
```

**To modify default workers**, explicitly define them with your desired configuration:

```yaml
workers:
  - name: dsr
    count: 3  # Override default count
    queues:
      - fides.dsr
    resources:
      requests:
        memory: "2Gi"
        cpu: "1000m"
```

## Worker Types Reference

### DSR (Data Subject Requests)

Processes data subject requests and collects data from systems integrated with Lethe.

**Image:** Any `fides` or `fidesplus` image. For `fidesplus`, use `-slim` suffixed tags to reduce memory usage.

**Startup command:**

```
fides worker --queues=fides.dsr
```

**Use Case:** Lethe privacy request execution - retrieving and deleting data across integrated systems

### Consent

Processes consent preference saves from users on properties that serve `fides.js`.

**Image:** Any `fides` or `fidesplus` image. For `fidesplus`, use `-slim` suffixed tags to reduce memory usage.

**Startup command:**

```
fides worker --queues=fides.privacy_preferences
```

**Use Case:** Janus consent management - propagating user consent preferences to integrated systems

### Messaging

Processes email tasks and notifications.

**Image:** Any `fides` or `fidesplus` image. For `fidesplus`, use `-slim` suffixed tags to reduce memory usage.

**Startup command:**

```
fides worker --queues=fidesops.messaging
```

**Use Case:** Email notifications for privacy requests and system alerts

### Consent Webhook (Fidesplus Only)

Processes consent preference updates from third-party systems via [Bidirectional consent](/content/docs/consent/guides/bidirectional-consent/index.html).

**Image:** Any `fidesplus` image. Recommend using `-slim` suffixed tags to reduce memory usage.

**Startup command:**

```
fides worker --queues=fidesplus.consent_webhooks
```

**Use Case:** Receiving consent preference updates from external systems

### Detection (Fidesplus Only)

Monitors data sources and detects changes for data discovery workflows.

**Image:** Any `fidesplus` image. Recommend using `-slim` suffixed tags to reduce memory usage.

**Startup command:**

```
fides worker --queues=fidesplus.discovery_monitors_detection
```

**Use Case:** Helios data discovery - detecting schema and data changes in connected systems

### Classification (Fidesplus Only)

Classifies data detected in data detection tasks for data discovery workflows.

**Image:** Any `fidesplus` image. Use `-slim` suffixed tags **only** when performing LLM-enabled classification. For legacy classification tasks, do not use `-slim` suffixed tags as classification requires full dependencies.

**Startup command:**

```
fides worker --queues=fidesplus.discovery_monitors_classification
```

**Use Case:** Helios data discovery - automatically classifying detected data fields

### Promotion (Fidesplus Only)

Promotes detection and discovery results to the Fides data inventory (triggered when clicking Confirm in the "Data discovery" view).

**Image:** Any `fidesplus` image. Recommend using `-slim` suffixed tags to reduce memory usage.

**Startup command:**

```
fides worker --queues=fidesplus.discovery_monitors_promotion
```

**Use Case:** Helios data discovery - promoting discovered data into the data map

## Deployment Examples

Configure workers in your `values.yaml`:

```yaml
workers:
  # DSR worker
  - name: dsr
    count: 2
    queues:
      - fides.dsr
    resources:
      requests:
        memory: "1Gi"
        cpu: "500m"
      limits:
        memory: "16Gi"
        cpu: "1000m"

# Consent worker
  - name: consent
    count: 1
    queues:
      - fides.privacy_preferences
    resources:
      requests:
        memory: "0.75Gi"
        cpu: "500m"
      limits:
        memory: "2Gi"
        cpu: "1000m"

# Other worker (excluding specific queues)
  - name: other
    count: 1
    excludeQueues:
      - fides.dsr
      - fides.privacy_preferences
    resources:
      requests:
        memory: "256Mi"
        cpu: "100m"
      limits:
        memory: "512Mi"
        cpu: "500m"
```

## Scaling Considerations

- **DSR Workers:** Scale based on privacy request volume and number of integrated systems
- **Consent Workers:** Scale based on consent preference update volume (fides.js traffic)
- **Classification Workers:** Scale based on data source size and discovery frequency
- **Messaging Workers:** Typically 1-2 workers are sufficient unless sending high volumes of emails

## Monitoring

Monitor worker health and performance through:

- Health endpoint: `/health/workers`
- Redis queue depth monitoring
- Worker logs for task processing times
- Resource utilization (CPU, memory)

For production monitoring setup, see the [Production Kubernetes Deployment](/content/docs/platform/installation/production/kubernetes/index.html) guide.