fivetran / dbt_facebook_ads_creative_history

Fivetran backwards compatibility data models for the Facebook Ads connector, using dbt.

Facebook Ads Backwards Compatibility

⚠️ Warning! ⚠️

This package is no longer maintained by the Fivetran team and is not recommended for use. This package was required for an older version of the Facebook Ads dbt package. This compatibility is no longer needed and is not listed as a dependency in our other Facebook Ads packages. We recommend removing this package from your packages.yml and using the latest version of the dbt_facebook_ads package instead.

To view the current release of the dbt_facebook_ads package, please refer to the release list.

This package models Facebook Ads data from Fivetran's connector. It uses data in the format described by this ERD.

The main focus of the package is to transform the core ad object tables into models with schemas that match the old Fivetran Facebook connector.

Models

This package contains transformation models, designed to work simultaneously with our Facebook Ads source package and our multi-platform Ad Reporting package. A dependency on the source package is declared in this package's packages.yml file, so it will automatically download when you run dbt deps. The primary outputs of this package are described below.

model	description
stg_facebook_ads__app_link	Maps to the app_link table in the old connector
stg_facebook_ads__carousel_media_url_tags	Maps to the carousel_media_url_tag table in the old connector
stg_facebook_ads__carousel_media	Maps to the carousel_media table in the old connector
stg_facebook_ads__creative_history_asset_feed_spec_link_url	Maps to the creative_history_asset_feed_spec_link_url table in the old connector
stg_facebook_ads__asset_feed_spec_link_url	(Postgres Only) Maps to the creative_history_asset_feed_spec_link_url table in the old connector
stg_facebook_ads__url_tag	Maps to the url_tag table in the old connector

Installation Instructions

Check dbt Hub for the latest installation instructions, or read the dbt docs for more information on installing packages.

Include in your packages.yml

packages:
  - package: fivetran/facebook_ads_creative_history
    version: [">=0.4.0", "<0.5.0"]

Configuration

By default, this package will look for your Facebook Ads data in the facebook_ads schema of your target database. If this is not where your Facebook Ads data is, please add the following configuration to your dbt_project.yml file:

# dbt_project.yml

...
config-version: 2

vars:
    facebook_ads_schema: your_schema_name
    facebook_ads_database: your_database_name 

For additional configurations for the source models, visit the Facebook Ads source package.

Changing the Build Schema

By default this package will build the Facebook Ads staging models within a schema titled (<target_schema> + _stg_facebook_ads) and the Facebook Creative History final models with a schema titled (<target_schema> + _facebook_ads_creative_history) in your target database. If this is not where you would like your modeled Facebook data to be written to, add the following configuration to your dbt_project.yml file:

# dbt_project.yml

...
models:
  facebook_ads_creative_history:
    +schema: my_new_schema_name # leave blank for just the target_schema
  facebook_ads_source:
    +schema: my_new_schema_name # leave blank for just the target_schema

Database Support

This package has been tested on BigQuery, Snowflake, Redshift, Postgres, and Databricks.

Databricks Dispatch Configuration

dbt v0.20.0 introduced a new project-level dispatch configuration that enables an "override" setting for all dispatched macros. If you are using a Databricks destination with this package you will need to add the below (or a variation of the below) dispatch configuration within your dbt_project.yml. This is required in order for the package to accurately search for macros within the dbt-labs/spark_utils then the dbt-labs/dbt_utils packages respectively.

# dbt_project.yml

dispatch:
  - macro_namespace: dbt_utils
    search_order: ['spark_utils', 'dbt_utils']

Contributions

Additional contributions to this package are very welcome! Please create issues or open PRs against main. Check out this post on the best workflow for contributing to a package.

Resources:

• Provide feedback on our existing dbt packages or what you'd like to see next
• Have questions or feedback, or need help? Book a time during our office hours here or email us at [email protected].
• Find all of Fivetran's pre-built dbt packages in our dbt hub
• Learn how to orchestrate your models with Fivetran Transformations for dbt Core™
• Learn more about Fivetran overall in our docs
• Check out Fivetran's blog
• Learn more about dbt in the dbt docs
• Check out Discourse for commonly asked questions and answers
• Join the chat on Slack for live discussions and support
• Find dbt events near you
• Check out the dbt blog for the latest news on dbt's development and best practices

dbt_facebook_ads_creative_history's Issues

spark/databricks compatibility

currently no models will run if the target warehouse is not bq/redshift/snowflake. how much of a lift would incorporating databricks be?

BUG - Package Fails to Run on Spark Databricks

Are you a current Fivetran customer?
Vincent, Head of Data Science at Oroton

Describe the bug

Currently using the DBT package in a new vanilla DBT build on version 0.20.0

(env) (base) ~ dbt --version
installed version: 0.20.0
   latest version: 0.20.1

Your version of dbt is out of date! You can find instructions for upgrading here:
https://docs.getdbt.com/docs/installation

Plugins:
  - bigquery: 0.20.0
  - snowflake: 0.20.0
  - redshift: 0.20.0
  - postgres: 0.20.0
  - spark: 0.20.0

I'm running this on Azure Databricks using the spark sql setup and have also installed the Spark SQL module and add the dispatcher to my config.

  dbt will be unable to find the correct resource when ref("stg_facebook_ads__carousel_media") is used. To fix this,
  change the name of one of these resources:
  - model.facebook_ads_creative_history.stg_facebook_ads__carousel_media (models/snowflake/stg_facebook_ads__carousel_media.sql)
  - model.facebook_ads_creative_history.stg_facebook_ads__carousel_media (models/bigquery/stg_facebook_ads__carousel_media.sql)

Steps to reproduce
Add package and run on DBT 0.20.0 on Azure Databricks

Expected behavior
No error to

Project variables configuration

# Name your project! Project names should contain only lowercase characters
# and underscores. A good package name should reflect your organization's
# name or the intended use of these models
name: 'oroton_dbt_databricks'
version: '1.0.0'
config-version: 2

# This setting configures which "profile" dbt uses for this project.
profile: 'databricks_cluster'

# These configurations specify where dbt should look for different types of files.
# The `source-paths` config, for example, states that models in this project can be
# found in the "models/" directory. You probably won't need to change these!
source-paths: ["models"]
analysis-paths: ["analysis"]
test-paths: ["tests"]
data-paths: ["data"]
macro-paths: ["macros"]
snapshot-paths: ["snapshots"]

target-path: "target"  # directory which will store compiled SQL files
clean-targets:         # directories to be removed by `dbt clean`
    - "target"
    - "dbt_modules"

# Configuring models
# Full documentation: https://docs.getdbt.com/docs/configuring-models

vars:
    ad_reporting__pinterest_enabled: False
    ad_reporting__microsoft_ads_enabled: False
    ad_reporting__linkedin_ads_enabled: False
    ad_reporting__google_ads_enabled: True
    ad_reporting__twitter_ads_enabled: False
    ad_reporting__facebook_ads_enabled: True
    ad_reporting__snapchat_ads_enabled: False
    facebook_ads_schema: silver_fivetran_facebookads
    facebook_ads_database: silver_fivetran_facebookads
    google_ads_schema: silver_fivetran_adwordsspark
    google_ads_database: silver_fivetran_adwordsspark

dispatch:
  - macro_namespace: dbt_utils
    search_order: ['spark_utils', 'dbt_utils']

# In this example config, we tell dbt to build all models in the example/ directory
# as tables. These settings can be overridden in the individual model files
# using the `{{ config(...) }}` macro.
models:
    # disable both pinterest models if not using pinterest ads
    pinterest:
        enabled: false
    pinterest_source:
        enabled: false

    # disable both microsoft ads models if not using microsoft ads
    microsoft_ads:
        enabled: false
    microsoft_ads_source:
        enabled: false

    # disable both linkedin ads models if not using linkedin ads
    linkedin:
        enabled: false
    linkedin_source:
        enabled: false

    # disable both twitter ads models if not using twitter ads
    twitter_ads:
        enabled: false
    twitter_ads_source:
        enabled: false

    # disable all three facebook ads models if not using facebook ads
    facebook_ads:
        +schema: fivetran_general
        enabled: true #IF YOU ARE USING FACEBOOK, DELETE THIS CONFIG, DO NOT SIMPLY SET TO TRUE
    facebook_ads_source:
        +schema: fivetran_general
        enabled: true #IF YOU ARE USING FACEBOOK, DELETE THIS CONFIG, DO NOT SIMPLY SET TO TRUE
    facebook_ads_creative_history:
        +schema: fivetran_general
        enabled: true #IF YOU ARE USING FACEBOOK, DELETE THIS CONFIG, DO NOT SIMPLY SET TO TRUE

    # disable both google ads models if not using google ads
    google_ads:
        +schema: fivetran_general
        enabled: true
    google_ads_source:
        +schema: fivetran_general
        enabled: true

    # disable both snapchat ads models if not using snapchat ads
    snapchat_ads:
        enabled: false
    snapchat_ads_source:
        enabled: false%

Package Version

packages:
  - package: fivetran/ad_reporting
    version: 0.4.0
  - package: dbt-labs/dbt_utils
    version: 0.7.1
  - package: dbt-labs/spark_utils
    version: 0.2.2
  - package: fivetran/facebook_ads
    version: 0.3.0

Warehouse

• BigQuery
• Redshift
• Snowflake
• Postgres
• Databricks
• Other (provide details below)

Additional context

I patched just by removing all the folders and keeping the Spark SQL modules and seems to work. But this will break as soon as i update or run on a CI/CD workflow

Screenshots

Please indicate the level of urgency

Unable to go-live with my purchase of Fivetran

Are you interested in contributing to this package?

• Yes, I can do this and open a PR for your review.
• Possibly, but I'm not quite sure how to do this. I'd be happy to do a live coding session with someone to get this fixed.
• No, I'd prefer if someone else fixed this. I don't have the time and/or don't know what the root cause of the problem is.