Fides Documentation – Ethyca

CLI Reference

Annotation Coverage Reports

scan

Hands-on: Try the Fides Quick Start. For more in-depth details on the scan command, check out the Annotation Coverage tutorial.

The fides scan evaluates all resources against their source systems and reports on any differences or gaps in annotation. This is extremely helpful for:

  1. Identifying changes in a source system's schema that may require annotation.
  2. Reporting on the percentage of annotation coverage across one, or multiple systems.

Usage

Usage: fides scan <commands> [options] working/directory/

This command accepts several subcommands and options depending on the resource being scanned. For example:

Scanning a dataset accepts db (database) as the type, followed by connection information and the working directory.

fides scan dataset db {connection_information} working/directory/

Scanning systems accepts either aws (Amazon Web Services) or okta (Okta Sign-on) as the type, followed by connection information and the working directory.

fides scan system [aws || okta] {connection_information} working/directory/

Here are the full list of accepted commands, subcommands and arguments:

Running this command should result in output that resembles the examples below.

Examples: Scanning Dataset Coverage

Scanning datasets provides a valuable way to quickly know annotation coverage and identify gaps in dataset annotation.

Example: Scan a database for annotation coverage using a connection config file

This example uses connection details stored in the fides.toml to connect to a database and compare the schema for annotation coverage against resources in the .fides working directory.

In this example, pg-credentials is the key that represents the credentials in the configuration file. For more in-depth details on using configuration files, check out the Fides Configuration tutorial.

$ fides scan dataset db --credentials-id "pg-credentials" .fides/
Loaded config from: .fides/fides.toml
Loading resource manifests from: .fides/
Taxonomy successfully created.
Loaded the following dataset manifests:
        public_c94f8a9702
Successfully scanned the following datasets:
        public

Annotation coverage: 100%

Example: Scan a database for annotation coverage using a connection string

This example connects to the database and compares the schema for annotation coverage against resources in the .fides working directory.

$ fides scan dataset db \
--connection-string postgresql://username:password@localhost:5432/database .fides/
Loaded config from: .fides/fides.toml
Loading resource manifests from: .fides/
Taxonomy successfully created.
Loaded the following dataset manifests:
        public_c94f8a9702
Successfully scanned the following datasets:
        public

Annotation coverage: 100%

Examples: Scanning System Coverage

Scanning systems provides a valuable way to quickly check annotation coverage for systems across a tech stack.

Example: Scan AWS system annotation coverage with connection config file

This example uses connection details stored in the fides.toml to connect to AWS and scan for annotation coverage against resources in the .fides working directory.

In this example, aws-credentials is the key that represents the credentials in the configuration file. For more in-depth details on using configuration files, check out the Fides Configuration tutorial.

$ fides scan system aws \
--credentials-id "aws-credentials" .fides
Loaded config from: .fides/fides.toml
Loading resource manifests from: .fides
Taxonomy successfully created.
Successfully scanned the following systems:
        redshift-cluster-1
        database-1

Resource coverage: 100%

Example: Scan AWS system annotation coverage with credential arguments

This example passes connection details as arguments to connect to an AWS account and scan for annotation coverage against resources in the .fides working directory.

$ fides scan system aws \
--access_key_id "AWS-ACCESS-KEY" \
--secret_access_key "AWS-SECRET-KEY" \
--region "AWS-REGION" .fides
Loaded config from: .fides/fides.toml
Loading resource manifests from: .fides
Taxonomy successfully created.
Successfully scanned the following systems:
        redshift-cluster-1
        database-1

Resource coverage: 100%

Example: Scan Okta system annotation coverage with connection config

This example uses connection details stored in the fides.toml to connect to an Okta server and scan for annotation coverage against resources in the .fides working directory. As can be seen in the results, the scan identified new systems that were not annotated as existing system resources.

In this example, okta-credentials is the key that represents the credentials in the configuration file. To learn more about configuration files, check out the Fides Configuration tutorial.

$ fides scan system okta  --credentials-id "okta" .fides
Loaded config from: .fides/fides.toml
Loading resource manifests from: .fides
Taxonomy successfully created.
Successfully scanned the following systems:
        okta_browser_plugin
        okta_enduser
        salesforce
        workday
        zendesk

The following system were not found in existing manifest
Missing Okta Application:
        okta_browser_plugin
        okta_enduser
        salesforce
        workday
        zendesk

Resource coverage: 0%

Example: Scan Okta system annotation coverage with credential arguments

This example passes connection details as arguments to connect to an Okta server and scan for annotation coverage against resources in the .fides working directory. As can be seen in the results, the scan identified new systems that were not annotated as existing system resources.

$ fides generate system okta \
--org-url "OKTA-URL" \
--token "OKTA-TOKEN"  .fides
Loaded config from: .fides/fides.toml
Loading resource manifests from: .fides
Taxonomy successfully created.
Successfully scanned the following systems:
        okta_browser_plugin
        okta_enduser
        salesforce
        workday
        zendesk

The following system were not found in existing manifest
Missing Okta Application:
        okta_browser_plugin
        okta_enduser
        salesforce
        workday
        zendesk

Resource coverage: 0%