babi4 / santa_dbt_dwh_dev

Installing dbt

1. Activate your venv and run pip3 install dbt
2. Copy airbyte-normalization/sample_files/profiles.yml over to ~/.dbt/profiles.yml
3. Edit to configure your profiles accordingly

Running dbt

1. cd airbyte-normalization
2. You can now run dbt commands, to check the setup is fine: dbt debug
3. To build the dbt tables in your warehouse: dbt run

Running dbt from Airbyte generated config

1. You can also change directory (cd /tmp/dev_root/workspace/1/0/normalize for example) to one of the workspace generated by Airbyte within one of the normalize folder.
2. You should find profiles.yml and a bunch of other dbt files/folders created there.
3. To check everything is setup properly: dbt debug --profiles-dir=$(pwd) --project-dir=$(pwd)
4. You can modify the .sql files and run dbt run --profiles-dir=$(pwd) --project-dir=$(pwd) too
5. You can inspect compiled dbt .sql files before they are run in the destination engine in normalize/build/compiled or normalize/build/run folders