Fides Documentation – Ethyca
Creating datasets from the CLI
Upload a dataset YAML using fides push
The fides push command allows you to quickly add YAML datasets from your working project directory to the Fides server.
$ fides push --help
> Loaded config from: .fides/fides.toml
Usage: fides push [OPTIONS] [MANIFESTS_DIR]
Parse local manifest files and upload them to the server.
╭─ Arguments ──────────────────────────────────────────────────────────────────────╮
│ MANIFESTS_DIR PATH │
╰──────────────────────────────────────────────────────────────────────────────────╯
╭─ Options ────────────────────────────────────────────────────────────────────────╮
│ --dry Do not upload results to the Fides webserver. │
│ --diff Print any diffs between the local & server objects │
│ --help -h Show this message and exit. │
╰──────────────────────────────────────────────────────────────────────────────────╯
The command will upload dataset YAML documents from the .fides/ working directory and then persist any changes to the Fides Server. Visit the push CLI guide for an in-depth review of the command.
Example
$ fides push
Loaded config from: .fides/fides.toml
Loading resource manifests from: .fides/
Taxonomy successfully created.
----------
Processing system resource(s)...
PUSHED 4 system resource(s).
----------
Processing dataset resource(s)...
PUSHED 1 dataset resource(s).
----------
After execution, navigate to the Manage datasets tab in the Fides Admin UI to review your dataset.
Generate from a database connection
The fides generate dataset command allows you to generate a dataset by connecting Fides to a datastore.
fides generate dataset --help
> Loaded config from: .fides/fides.toml
Usage: fides generate dataset [OPTIONS] COMMAND [ARGS]...
Generate Fides datasets.
╭─ Options ─────────────────────────────────────────────────────────────────────────────────────────────────╮
│ --help -h Show this message and exit. │
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Commands ────────────────────────────────────────────────────────────────────────────────────────────────╮
│ aws Generate Fides datasets from specific Amazon Web Services. │
│ db Generate a Fides dataset by walking a database and recording every schema/table/field. │
│ gcp Generate Fides datasets from Google Cloud Platform. │
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────╯
Visit the generate CLI guide for an in-depth review of the command.
Example
This example generates a dataset from a database using connection details stored in the fides.toml. It will also write a dataset YAML at the location .fides/database.yml.
In this example, pg-credentials is the key that represents the credentials in the configuration file. For more in-depth details on using configuration files, check out the Fides Configuration tutorial.
fides.toml
[pg-credentials]
user = "[DB-USERNAME]"
password = "[PASSWORD]"
server = "127.0.0.1"
port = "5432"
db = "[DATABASE]"
$ fides generate dataset db \
--credentials-id "pg-credentials" .fides/database.yml
Loaded config from: .fides/fides.toml
Generated dataset manifest written to .fides/database.yml