fivetran / dbt_intercom

Data models for Fivetran's Intercom connector built using dbt.

Intercom Transformation dbt Package (Docs)

📣 What does this dbt package do?

• Produces modeled tables that leverage Intercom data from Fivetran's connector in the format described by this ERD and builds off the output of our Intercom source package.

• Enables you to better understand the performance, responsiveness, and effectiveness of your team's conversations with customers via Intercom. It achieves this by:

  • Creating an enhanced conversations table to enable large-scale reporting on all current and closed conversations
  • Enriching conversation data with relevant contacts data
  • Aggregating your team's performance data across all conversations
  • Providing aggregate rating and timeliness metrics for customer conversations to enable company-level conversation performance reporting

The following table provides a detailed list of all models materialized within this package by default.

Model	Description
intercom__admin_metrics	Each record represents an individual admin (employee) and a unique team they are assigned on, enriched with admin-specific conversation data like total conversations, average rating, and median response times by specific team.
intercom__company_enhanced	Each record represents a single company, enriched with data related to the company industry, monthly spend, and user count.
intercom__company_metrics	Each record represents a single row from intercom__company_enhanced, enriched with data like total conversation count, average satisfaction rating, median time to first response, and median time to last close with contacts associated to a single company.
intercom__contact_enhanced	Each record represents a single contact, enriched with data like the contact's role, company, last contacted information, and email list subscription status.
intercom__conversation_enhanced	Each record represents a single conversation, enriched with conversation part data like who was assigned to the conversation, which contact the conversation was with, the current conversation state, who closed the conversation, and the final conversation ratings from the contact.
intercom__conversation_metrics	Each record represents a single row from intercom__conversation_enhanced, enriched with data like time to first response, time to first close, and time to last close.

🎯 How do I use the dbt package?

Step 1: Prerequisites

To use this dbt package, you must have the following:

• At least one Fivetran Intercom connector syncing data into your destination.
• A BigQuery, Snowflake, Redshift or PostgreSQL destination.

Step 2: Install the package

Include the following intercom package version in your packages.yml file:

TIP: Check dbt Hub for the latest installation instructions or read the dbt docs for more information on installing packages.

packages:
  - package: fivetran/intercom
    version: [">=0.9.0", "<0.10.0"]

Step 3: Define database and schema variables

By default, this package runs using your destination and the intercom schema. If this is not where your Intercom data is (for example, if your Intercom schema is named intercom_fivetran), add the following configuration to your root dbt_project.yml file:

vars:
    intercom_database: your_database_name
    intercom_schema: your_schema_name

(Optional) Step 4: Additional configurations

Expand for configurations

Adding passthrough metrics

You can add additional columns to the intercom__company_enhanced, intercom__contact_enhanced, and intercom__conversation_enhanced tables using our pass-through column variables. These variables allow for the pass-through fields to be aliased (alias) and casted (transform_sql) if desired, but not required. Datatype casting is configured via a sql snippet within the transform_sql key. You may add the desired sql while omitting the as field_name at the end and your custom pass-though fields will be casted accordingly. Use the below format for declaring the respective pass-through variables in your root dbt_project.yml.

vars:
  intercom__company_history_pass_through_columns: 
    - name: company_history_custom_field
      alias: new_name_for_this_field
      transform_sql:  "cast(new_name_for_this_field as int64)"
    - name:           "this_other_field"
      transform_sql:  "cast(this_other_field as string)"
    - name: custom_monthly_spend
    - name: custom_paid_subscriber
  # a similar pattern can be applied to the rest of the following variables.
  intercom__contact_history_pass_through_columns:
  intercom__conversation_history_pass_through_columns:

Disabling Models

This package assumes that you use Intercom's company tag, contact tag, contact company, and conversation tag, team and team admin mapping tables. If you do not use these tables, add the configuration below to your dbt_project.yml. By default, these variables are set to True:

# dbt_project.yml

...
vars:
  intercom__using_contact_company: False
  intercom__using_company_tags: False
  intercom__using_contact_tags: False
  intercom__using_conversation_tags: False
  intercom__using_team: False

Changing the build schema

By default this package will build the Intercom staging models within a schema titled (<target_schema> + _stg_intercom) and the Intercom final models with a schema titled (<target_schema> + _intercom) in your target database. If this is not where you would like your modeled Intercom data to be written to, add the following configuration to your dbt_project.yml file:

models:
  intercom:
    +schema: my_new_schema_name # leave blank for just the target_schema
  intercom_source:
    +schema: my_new_schema_name # leave blank for just the target_schema

Change the source table references

If an individual source table has a different name than the package expects, add the table name as it appears in your destination to the respective variable:

IMPORTANT: See this project's dbt_project.yml variable declarations to see the expected names.

vars:
    intercom_<default_source_table_name>_identifier: your_table_name 

Limitations

Intercom V2.0 does not support API exposure to company-defined business hours. We therefore calculate all time_to metrics in their entirety without subtracting business hours.

🔍 Does this package have dependencies?

This dbt package is dependent on the following dbt packages. Please be aware that these dependencies are installed by default within this package. For more information on the following packages, refer to the dbt hub site.

IMPORTANT: If you have any of these dependent packages in your own packages.yml file, we highly recommend that you remove them from your root packages.yml to avoid package version conflicts.

packages:
    - package: fivetran/intercom_source
      version: [">=0.8.0", "<0.9.0"]

    - package: fivetran/fivetran_utils
      version: [">=0.4.0", "<0.5.0"]

    - package: dbt-labs/dbt_utils
      version: [">=1.0.0", "<2.0.0"]

🙌 How is this package maintained and can I contribute?

Package Maintenance

The Fivetran team maintaining this package only maintains the latest version of the package. We highly recommend you stay consistent with the latest version of the package and refer to the CHANGELOG and release notes for more information on changes across versions.

Contributions

A small team of analytics engineers at Fivetran develops these dbt packages. However, the packages are made better by community contributions!

We highly encourage and welcome contributions to this package. Check out this dbt Discourse article on the best workflow for contributing to a package!

🏪 Are there any resources available?

• If you have questions or want to reach out for help, please refer to the GitHub Issue section to find the right avenue of support for you.
• If you would like to provide feedback to the dbt package team at Fivetran or would like to request a new dbt package, fill out our Feedback Form.
• Have questions or want to be part of the community discourse? Create a post in the Fivetran community and our team along with the community can join in on the discussion!

dbt_intercom's Issues

Intercom Version Compatibility

Be sure to include within the ReadMe documentation that this package will only be compatible with the connector version 1 for Intercom. This means popular version 0 tables such as user_history will bot be included in the transform package.

[Bug] first_admin_response_at can be grouped under "part_type= 'assignment'"

Is there an existing issue for this?

• I have searched the existing issues

Describe the issue

Hi team, I'm opening an issue on behalf of the customer.

This is related to the issue posted here, where we are grabbing the 2nd admin comment as the "first admin response":
https://community.fivetran.com/t5/working-with-data/inconsistency-with-intercom-dbt-data/td-p/642

Intercom support has confirmed that the first admin comment can be grouped under part_type=assignment:

On the issue of how the Fivetran integration analyzes Intercom data from admin replies in conversations:

It looks like the issue is around an admin reply being coupled under "assignment" when they occur at the same time, is that correct? And therefore the conversation part type is "assignment", rather than "comment".

Looking at this conversation: https://app.intercom.com/a/apps/urub4jwk/conversations/7758

The admin Jesus's first reply is grouped with an "assignment" at 2023-01-26 17:10:48 UTC. Then 2nd reply from Jesus is at 2023-01-26 17:23:15 UTC and it is a part type "comment".

This is because, when conversations are self assigned from an admin reply, the first reply will be grouped under "assignment"

In the Conversation Model / Object it would look like this

Then subsequent admin replies would be a "comment"

So if Fivetran is ignoring a reply in the "body" section of the "assignment" part type, they would need to address this on their end, as this is the expected behavior in Intercom of how this data is grouped

Relevant error log or model output

No response

Expected behavior

For the first_admin_response_at, we also need to check if the first admin response is coming through as part_type=assignment.

dbt Project configurations

n/a

Package versions

n/a

What database are you using dbt with?

snowflake

dbt Version

n/a

Additional Context

This is related to the issue posted here, where we are grabbing the 2nd admin comment as the "first admin response":
https://community.fivetran.com/t5/working-with-data/inconsistency-with-intercom-dbt-data/td-p/642

Are you willing to open a PR to help address this issue?

• Yes.
• Yes, but I will need assistance and will schedule time during our office hours for guidance
• No.

[Feature] Adding passthrough Custom Attributes also for Conversations

Is there an existing feature request for this?

• I have searched the existing issues

Describe the Feature

The source table conversation_history like contact and company now contains custom_ attributes from conversations.

While it is currently possible to pull the custom attributes through for contact and company it is not possible for conversations.

Similiar to how pass through attributes can be defined in the dbt_project.yml under vars I would expect the same to work for conversations.

Describe alternatives you've considered

Joining the source table conversation_history back to the intercom__conversation_metrics.

Are you interested in contributing this feature?

• Yes.
• Yes, but I will need assistance and will schedule time during your office hours for guidance.
• No.

Anything else?

No response

dbt run fails

Struggling to get this working. I think we have a pretty vanilla Intercom setup and we created our connector in Dec.

Here is the output from running dbt run locally. We use FiveTran connected to RedShift.

I've tried playing around with disabling some of the vars, but no luck.

Completed with 3 errors and 0 warnings:

Database Error in model stg_intercom__contact_history_tmp (models/tmp/stg_intercom__contact_history_tmp.sql)
  relation "stg_intercom__contact_history_tmp__dbt_backup" already exists
  compiled SQL at target/run/intercom_source/models/tmp/stg_intercom__contact_history_tmp.sql

Database Error in model stg_intercom__conversation_tag_history_tmp (models/tmp/stg_intercom__conversation_tag_history_tmp.sql)
  relation "intercom.conversation_tag_history" does not exist
  compiled SQL at target/run/intercom_source/models/tmp/stg_intercom__conversation_tag_history_tmp.sql

Database Error in model stg_intercom__company_tag_history_tmp (models/tmp/stg_intercom__company_tag_history_tmp.sql)
  relation "intercom.company_tag_history" does not exist
  compiled SQL at target/run/intercom_source/models/tmp/stg_intercom__company_tag_history_tmp.sql

BUG - [Duplicates in intercom__admin_metrics]

Are you a current Fivetran customer?
Yes, I am a Data Analyst at Aircall.

Describe the bug
Unicity test on admin_id fails when refreshing the intercom__admin_metrics model due to duplicates caused by admins part of more than 1 team.

Steps to reproduce

1. Run the model intercom__admin_metrics
2. See error: Failure in test unique_intercom__admin_metrics_admin_id (models/intercom.yml)

Expected behavior
No fail.

Project variables configuration

# A good package name should reflect your organization's name or the intended use of these models
name: 'aircall'
version: '1.0.0'
config-version: 2

# This setting configures which "profile" dbt uses for this project.
profile: 'aircall'

source-paths: ["models"]
analysis-paths: ["analysis"]
test-paths: ["tests"]
data-paths: ["data"]
macro-paths: ["macros"]

target-path: "target"  # directory which will store compiled SQL files
clean-targets:         # directories to be removed by `dbt clean`
  - "target"

models:
  aircall:
    +materialized: view_if_not_exists
    intermediate_marts:
      +schema: intermediate
    sources:
      +schema: sources

vars:
  dbt_date:time_zone: 'UTC'
  phone_schema: "client_events_phone_staging"
  android_schema: "client_events_android_staging"
  ios_schema: "client_events_ios_staging"
  browser_extension_schema: "client_events_browser_extension_staging"
  dashboard_schema: "client_events_dashboard_staging"
  dry_run : False #adding a flag variable to the compile step in order to avoid failure during compiling, when the current code depends on the data of a model which is not yet materialized
  jira:
    jira_include_comments: false
    # ----- IF YOU WANT TO ADD A FIELD TO jira_changelog ----- #
    field_columns: # lower case with _ if more than 1 word
      - name: status
        table: stg_jira__status
        joining_id: status_id
        source_field: status_name
      - name: assignee
        table: stg_jira__user
        joining_id: user_id
        source_field: user_display_name
      - name: assignment_group
        table: clean_jira_field_option
        joining_id: field_id
        source_field: field_name

    # ----- IF YOU WANT TO ADD A FIELD TO jira_issues_d (and jira_issues) ----- #
    issue_field_history_columns: ['customfield_10161', 'assignee', 'priority', 'customfield_10170'] # variable used in jira__daily_issue_field_history

    # ----- IF YOU WANT TO ADD A CUSTOM FIELD TO jira_issues ONLY (not to jira_issues_d)----- #
    custom_fields_columns: ['product_category','severity','labels','blast_radius','region','service_impact','symptoms']

    # ----- IF YOU WANT TO ADD A SPECIAL CUSTOM FIELD TO jira_issues ONLY (not to jira_issues_d)----- #
    # ----- special custom field: field for which the value is not represented by an id but its value in int_jira__combine_field_histories----- #
    special_custom_fields_columns: ['story_points', 'email_verification']

  intercom:
    using_contact_tags: false
    using_team: false

Package Version

packages:
  - package: dbt-labs/redshift
    version: [">=0.5.0", "<0.6.0"]
  - package: fivetran/jira
    version: [">=0.5.0", "<0.6.0"]
  - package: fivetran/intercom
    version: [">=0.3.0", "<0.4.0"]
  - package: calogica/dbt_expectations
    version: [">=0.4.0", "<0.5.0"]

Warehouse

• BigQuery
• Redshift
• Snowflake
• Postgres
• Databricks
• Other (provide details below)

Additional context
No

Screenshots

Please indicate the level of urgency
Impacting us as we cannot refresh any of our Intercom models due to this error.

Are you interested in contributing to this package?

• Yes, I can do this and open a PR for your review.
• Possibly, but I'm not quite sure how to do this. I'd be happy to do a live coding session with someone to get this fixed.
• No, I'd prefer if someone else fixed this. I don't have the time and/or don't know what the root cause of the problem is.

[Bug] Package version range in latest README is incorrect

Is there an existing issue for this?

• I have searched the existing issues

Describe the issue

Currently the version range of our latest version (v0.5.0) instructs users to install [">=0.4.0", "<0.5.0"]. This means if users follow our README then they will not get the latest version of the package.

We need to update our README to have the correct range, highlighted below:

packages:
  - package: fivetran/intercom
    version: [">=0.5.0", "<0.6.0"]

Relevant error log or model output

N/A

Expected behavior

Customers are instructed to install the latest version range of our package and not an older/out of date range.

dbt Project configurations

N/A

Package versions

v0.5.0

What database are you using dbt with?

postgres

dbt Version

N/A

Additional Context

N/A

Are you willing to open a PR to help address this issue?

• Yes.
• Yes, but I will need assistance and will schedule time during our office hours for guidance
• No.

Ignore Nulls - Postgres

Is there an existing issue for this?

• I have searched the existing issues

Describe the issue

It seems the Ignore Nulls feature has been remedied in this update:
https://github.com/fivetran/dbt_intercom/pull/25/files

However, it has not been propagated across all scripts in the package. This is causing my transformation to fail in the intercom__conversation_enhanced.sql script

Relevant error log or model output

Database Error in model intercom__conversation_enhanced (models/intercom__conversation_enhanced.sql)
14:57:31    syntax error at or near "ignore"
14:57:31    LINE 120:     first_value(author_id ignore nulls) over (partition by c...
14:57:31                                        ^
14:57:31    compiled Code at target/run/intercom/models/intercom__conversation_enhanced.sql

Expected behavior

Error in the intercom__conversation_enhanced.sql script is preventing the table intercom.intercom__conversation_enhanced table from being created. This is also preventing other tables from being created.

dbt Project configurations

name: 'Intercom_transform'
version: '1.0.0'
config-version: 2

# This setting configures which "profile" dbt uses for this project.
profile: 'Intercom_transform'

# These configurations specify where dbt should look for different types of files.
# The `model-paths` config, for example, states that models in this project can be
# found in the "models/" directory. You probably won't need to change these!
model-paths: ["models"]
analysis-paths: ["analyses"]
test-paths: ["tests"]
seed-paths: ["seeds"]
macro-paths: ["macros"]
snapshot-paths: ["snapshots"]

target-path: "target"  # directory which will store compiled SQL files
clean-targets:         # directories to be removed by `dbt clean`
  - "target"
  - "dbt_packages"


# Configuring models
# Full documentation: https://docs.getdbt.com/docs/configuring-models

# In this example config, we tell dbt to build all models in the example/ directory
# as tables. These settings can be overridden in the individual model files
# using the `{{ config(...) }}` macro.
models:
  Intercom_transform:
    # Config indicated by + and applies to all files under models/example/
    example:
      +materialized: view

Package versions

packages:

• package: fivetran/intercom
  version: [">=0.4.0", "<0.5.0"]

What database are you using dbt with?

postgres

dbt Version

1.3.0

Additional Context

No response

Are you willing to open a PR to help address this issue?

• Yes.
• Yes, but I will need assistance and will schedule time during our office hours for guidance
• No.

test issue

[Feature] Intercom API Upgrade and History Changes

Is there an existing feature request for this?

• I have searched the existing issues

Describe the Feature

In October 2023, the Intercom API upgraded from 2.9 to 2.10. As a result, there were a number of changes to the connector schema which will impact these dbt data models. This feature request is to take into consideration the changes and apply them in an update to the dbt package so users may be able to seamlessly migrate following the upgrade.

For more details on these changes you may refer to:

• The connector release notes
• Intercom history mode documentation
• The Intercom connector migration comms
  • 

Describe alternatives you've considered

The upgrade likely will not result in any failed dbt runs. However, the subsequent runs following the upgrade will see significant null records and possible non active records being used in downstream transformations. We will want to upgrade the package as there are no real other alternatives.

Are you interested in contributing this feature?

• Yes.
• Yes, but I will need assistance and will schedule time during your office hours for guidance.
• No.

Anything else?

These changes will likely be rolling out to Intercom connector customers over the next two weeks. This will be a breaking change and will request users to upgrade to the latest version.

This package should include metrics from the statistical model outlined in github link below

https://github.com/fivetran/engineering/issues/78658

[Feature] Add new Intercom fields for real-time performance tracking based on conversation type and rating

Is there an existing feature request for this?

• I have searched the existing issues

Describe the Feature

Several metrics are useful for assessing performance more rapidly in Intercom and can be mostly identified via case statements and aggregations.

Live conversation metrics

Intercom has four states of a conversation: Open, idle, snoozed, closed. Making it easier to track the former three conversation types will allow for better live tracking of Intercom data for end customers.

• Open and snoozed tickets can be derived in conversations_metrics model by conversation_state based on Intercom API, but aggregates need to be calculated upstream for total_conversations_open, total_conversations_snoozed in admin_metrics and company_metrics.
• For idle, is_idle boolean can be created in conversation_metrics based on lack of first_admin_response_at value. Idle conversations can be derived by null values in first_admin_response_at. total_conversations_idle can then be calculated upstream in admin_metrics and company_metrics.
• Stretch:total_conversations_participated gives us a potential true denominator for each of these metrics--we could examine the conversation parts model. This should be fairly doable for company_metrics, but might need some creativity for admin_metrics as we'll need to break down which teammates participated in which conversations. We might need to dis-aggregate all_conversation_admins to associate conversations with admins.

Conversation ratings

Intercom has the satisfaction rating to assess CSAT for most customers, looking at the % of conversations that have received a 4 or 5 star rating (a 😀 or 🤩 review from a customer after an Intercom conversation has concluded).

An is_satisfied boolean in conversation_metrics would be the easiest solve, and then upstream satisfaction_ratings to count those totals in company_metrics and admin_metrics`.

Describe alternatives you've considered

Most of these metrics can probably be derived in the current models, but would require a ton of filtering on the dashboards themselves to generate. This provides simpler model metrics from which to perform aggregate calculations.

Are you interested in contributing this feature?

• Yes.
• Yes, but I will need assistance and will schedule time during your office hours for guidance.
• No.

Anything else?

No response

When two events occur at the same time, they should be separated.

Is there an existing issue for this?

• I have searched the existing issues

Describe the issue

I have an issue with conversation_part_history table, sometimes when two events occur at the same time, they appear only in one row, for example, on the screenshot attached, when part_type is open, body should be null and the text in body column should be in a new row as a comment, it also occurs when part_type is assignment

Relevant error log or model output

No response

Expected behavior

If these two events occur at the same time, I want them to be separated in different rows.

dbt Project configurations

 intercom_database: fivetran
  intercom_schema: intercom
  intercom__using_team: False
  intercom__using_company_tags: False

Package versions

packages:
  - package: fivetran/intercom
    version: [">=0.6.0", "<=0.7.0"]

What database are you using dbt with?

snowflake

dbt Version

Additional Context

No response

Are you willing to open a PR to help address this issue?

• Yes.
• Yes, but I will need assistance and will schedule time during our office hours for guidance
• No.

[Feature] Add SLA metrics to Intercom reports

Is there an existing feature request for this?

• I have searched the existing issues

Describe the Feature

SLA metrics are becoming a more crucial reporting mechanism for Intercom customers, particularly with how customer success teams evaluate the effectiveness of their teammates.

This feature would add the following data:

• Add median_next_response_time metric to intercom_conversations model, would need to be calculated upstream in intercom_conversation_parts model. (This could be the heaviest lift, so open to discussing).
• Configurable variables for target_first_response_time_minutes and target_next_response_time_minutes for the customer and set, to allow us to define whether SLAs get hit or not in our models.
• Add is_sla_hit or is_sla_miss boolean fields based on target_first_response_time_minutes and target_next_response_time_minutes configurable variables in `conversation_metrics.
• Add sla_hits and sla_misses to eventually calculate SLA hits and misses by admin and company, to admin_metrics and company_metrics.

Describe alternatives you've considered

The SLA metrics we get directly from Intercom do have 'hit', 'miss', 'cancelled' values according to documentation. So we could technically get aggregate SLA hit and miss rate over a particular time period if we want to scope down.

However, we do not know what the actual SLA targets are, customer-by-customer, as they are configurable goals that could very much differ. It'd be better if we set SLA goal metrics on our end based on targets.

If median_next_response_time is too complicated to calculate, it might be worth discussing whether we parse down this to request to only the first_response SLA targets.

Are you interested in contributing this feature?

• Yes.
• Yes, but I will need assistance and will schedule time during your office hours for guidance.
• No.

Anything else?

Read more on Intercom SLA performance here.

Failure on intercom__conversation_metrics model build

Is there an existing issue for this?

• I have searched the existing issues

Describe the issue

Running dbt run on --select +intercom+ returns an error for the intercom__conversation_metrics model. We are using a Redshift database.

As per the error below, the issue occurs within the date_diff functions. The model runs if these five lines are removed from the code.

Relevant error log or model output

Postgres adapter: Postgres error: function pg_catalog.date_diff("unknown", timestamp with time zone, timestamp with time zone) does not exist
HINT:  No function matches the given name and argument types. You may need to add explicit type casts.

Expected behavior

Models should build without error.

dbt Project configurations

# Name your project! Project names should contain only lowercase characters
# and underscores. A good package name should reflect your organization's
# name or the intended use of these models
name: 'infogrid_bi'
version: '1.0.0'
config-version: 2

# This setting configures which "profile" dbt uses for this project.
profile: 'default'

# These configurations specify where dbt should look for different types of files.
# The `model-paths` config, for example, states that models in this project can be
# found in the "models/" directory. You probably won't need to change these!
model-paths: ["models"]
analysis-paths: ["analysis"]
test-paths: ["tests"]
seed-paths: ["data"]
macro-paths: ["macros"]
snapshot-paths: ["snapshots"]

target-path: "target"  # directory which will store compiled SQL files
clean-targets:         # directories to be removed by `dbt clean`
    - "target"
    - "dbt_modules"

  # Materialize all views as late-binding to reduce Looker downtime. See https://docs.getdbt.com/reference/resource-configs/redshift-configs#late-binding-views 
  +bind: false 

# Tables output as a result of package runs go into separate schemas.
# These outputs are then transferred to the analytics schema as needed. This helps reduce model clutter.
  hubspot:
    +schema: dbt_packages_output
  hubspot_source:
    +schema: dbt_packages_source
  xero:
    +schema: dbt_packages_output
  xero_source:
    +schema: dbt_packages_source
  lever:
    +schema: dbt_packages_output
  lever_source:
    +schema: dbt_packages_source
  intercom:
    +schema: dbt_packages_output
  intercom_source:
    +schema: dbt_packages_source

# Tables that we've built for final analysis go in the analytics schema as tables
# Staging models should only contain light transformation so these are materialized as views
# Intermediate models often contain heavy transformations. These are materialized as tables for query performance reasons. They are stored in a separate schema to avoid clutter.
  infogrid_bi:
    athena_sensor_events:
      01_staging:
        schema: 01_staging
        materialized: view
      02_intermediate:
        schema: 02_intermediate
        materialized: table
      03_core:
        materialized: table

    aws_cost_report:
      01_staging:
        schema: 01_staging
        materialized: view

    reporting:
      materialized: table
    
    hubspot:
      01_staging:
        schema: 01_staging
        materialized: view
      02_intermediate:
        schema: 02_intermediate
        materialized: table
      03_core:
        materialized: table

    lever:
      01_staging:
        schema: 01_staging
        materialized: view
      02_intermediate:
        schema: 02_intermediate
        materialized: table
      03_core:
        materialized: table

    hibob:
      01_staging:
        materialized: view
      02_intermediate:
        schema: 02_intermediate
        materialized: table
      03_core:
        materialized: table

    xero:
      01_staging:
        schema: 01_staging
        materialized: view
      # 02_intermediate:
      #   schema: 02_intermediate
      #   materialized: table
      03_core:
        materialized: table

    company_targets:
      01_staging:
        materialized: view
      # 02_intermediate:
      #   schema: 02_intermediate
      #   materialized: table
      03_core:
        materialized: table

    webapp_rds:
      01_staging:
        schema: 01_staging
        materialized: table
      02_intermediate:
        schema: 02_intermediate
        materialized: table
      03_core:
        materialized: table

seeds:
  infogrid_bi:
    hibob_people_daily_audit_historic_seed:
      +column_types:
        employee_id: varchar
        manager_s_id: varchar

vars:
  segment:
    segment_page_views_table: "{{ source('segment', 'pages') }}"

# Setting custom options for Fivetran Hubspot dbt package
  hubspot__pass_through_all_columns: true 
  hubspot_email_event_spam_report_enabled: false
  hubspot_email_event_print_enabled: false
  hubspot_email_event_forward_enabled: false
  hubspot_contact_merge_audit_enabled: true 

# Options for Fivetran's Intercom dbt package
  intercom__using_company_tags: false
  intercom__using_conversation_tags: false
  
# Setting the source data location for Fivetran dbt packages
  hubspot_database: bi_database
  hubspot_schema: hubspot_fivetran 
  lever_database: bi_database
  lever_schema: lever_fivetran 
  xero_database: bi_database
  xero_schema: xero_fivetran 
  segment_database: bi_database
  segment_schema: webapp
  intercom_database: bi_database
  intercom_schema: intercom

  

Package versions

packages:
  - package: emilyriederer/dbtplyr
    version: [">=0.3.1"]
  - package: fivetran/hubspot
    version: ["=0.5.1"]
  - package: fivetran/lever
    version: [">=0.4.0"]
  - git: "https://github.com/scott-infogrid/dbt_xero.git" # Temporary fork of the dbt_xero package to remove testing error.
    revision: "v1.0.0"
  - package: dbt-labs/redshift
    version: [">=0.8.0"]
  - package: fivetran/intercom
    version: [">=0.4.0", "<0.5.0"]

What database are you using dbt with?

redshift

dbt Version

1.2 (latest)

Additional Context

No response

Are you willing to open a PR to help address this issue?

• Yes.
• Yes, but I will need assistance and will schedule time during our office hours for guidance
• No.

[Feature] Configure business hour variables

Is there an existing feature request for this?

• I have searched the existing issues

Describe the Feature

Currently business hours are only known to the end customer, making it difficult to perform end calculations between start and end time. Creating variables that allow the customer to configure business hours in dbt could result in more accurate calculations of a host of response time metrics.

Creating variables of business_start_time_by_hour, business_end_time_by_hour that end users could configure could provide more accurate data to our customers.

This will also probably require us to refactor our data models to account for these models, so we can more accurate measurements for median first response time and other valuable time-based calculations.

Note: Complexity increases significantly if business hours have to be configured by day and would have to be scoped out.

Describe alternatives you've considered

Currently our data models do not account for business hours in a satisfactory manner, which likely prevents higher adoption of our Intercom data.

Are you interested in contributing this feature?

• Yes.
• Yes, but I will need assistance and will schedule time during your office hours for guidance.
• No.

Anything else?

No response

[Feature] Databricks compatibility

Is there an existing feature request for this?

• I have searched the existing issues

Describe the Feature

Internal - Add databricks compatibility

Describe alternatives you've considered

No response

Are you interested in contributing this feature?

• Yes.
• Yes, but I will need assistance and will schedule time during your office hours for guidance.
• No.

Anything else?

No response

LISTAGG limit error on Redshift in intercom__contact_enhanced

Hi!

I've already opened a quite similar ticket for dbt_jira. As I'm using Redshift, I'm encountering this error while running intercom__contact_enhanced model:

Result size exceeds LISTAGG limit DETAIL: ----------------------------------------------- error: Result size exceeds LISTAGG limit code: 8001 context: LISTAGG limit: 65535 query: 14150509 location: 0.cpp:839 process: query1_411_14150509 [pid=19351] ----------------------------------------------- compiled SQL at target/run/intercom/models/intercom__contact_enhanced.sql

As stated in the Jira issue, fivetran_utils.string_agg is responsible for this, because with Redshift, through LISTAGG, 65535 bytes is the longest data type that Redshift is capable of storing.

Contrary to the Jira repo, there are here some variables that can disable the different LISTAGG in the repo to be run (which is imo the best thing to do to make sure LISTAGG cannot trigger this kind of error):
using_contact_company: False using_company_tags: False using_contact_tags: False using_conversation_tags: False

Assigning False to the last 3 variables is fine. However, if using_contact_company = False, intercom__company_enhanced and intercom__company_metrics models will be disabled, so I won't have any final models regarding companies, Therefore, I don't really understand the use of {{ config(enabled=var('using_contact_company', True)) }} condition at the beginning of both models. In some other models such as intercom__contact_enhanced, instead of this,{% if var('using_contact_tags', True) %}is used, which enables you to get a model that can be run, even though some metrics are disabled.

I hope i'm clear enough.
Thank you in advance!

[Feature] README Updates

Is there an existing feature request for this?

• I have searched the existing issues

Describe the Feature

This packages README is not currently inline with our existing standards. It would be ideal for this repo to see README updates to be in line with our current documentation guidelines.

Describe alternatives you've considered

No response

Are you interested in contributing this feature?

• Yes.
• Yes, but I will need assistance and will schedule time during your office hours for guidance.
• No.

Anything else?

During these updates, we should also add the identifier variables and ensure the integration_tests/dbt_project.yml references the identifiers as well.

Add support for Intercom Events

Within our Intercom setup we send custom events to Intercom. Any plans to update FiveTran and this package to support Intercom Events?

Reference:
https://developers.intercom.com/intercom-api-reference/reference#list-user-events

Incorrect FIRST_CLOSE_AT on intercom__conversation_metrics

Is there an existing issue for this?

• I have searched the existing issues

Describe the issue

Please refer to ticket #125497 as it was the first reported issue we found with Fivetran's Intercom packages. After months of troubleshooting we finally found the issue with the package and Fivetran fixed.

From Intercom data we can clearly see that time of first close for the conversation with ID 38756 was 19:23 UTC but Fivetran's package is showing the next day at 22:08 UTC (the last close instead of the first). Could we please fix.

Relevant error log or model output

No response

Expected behavior

The expected behavior is to grab the first timestamp the conversations was closed

dbt Project configurations

Intercom

intercom_database: fivetran
intercom_schema: intercom
intercom__using_team: False
intercom__using_company_tags: False

Package versions

packages:
# dbt_date, dbt_expectations, codegen can all be upgraded
# but are blocked by github which has a dependency on
# dbt_utils [">=0.8.0", "<0.9.0"].

• package: fivetran/github
  version: [">=0.7.0", "<0.8.0"]
• package: dbt-labs/metrics
  version: [">=1.3.0", "<1.4.0"]
• package: fivetran/app_reporting
  version: [">=0.2.0", "<0.3.0"]
• package: fivetran/intercom
  version: [">=0.6.0", "<0.7.0"]
• package: fivetran/iterable
  version: [">=0.6.0", "<0.7.0"]

What database are you using dbt with?

snowflake

dbt Version

Additional Context

No response

Are you willing to open a PR to help address this issue?

• Yes.
• Yes, but I will need assistance and will schedule time during our office hours for guidance
• No.

[Bug] Update or Remove Conversation Contact History from Conversation End Models

Is there an existing issue for this?

• I have searched the existing issues

Describe the issue

Currently the intercom__conversation_enhanced model joins in the conversation_contact_history staging model by joining on the conversation_id. However, upon inspecting the ERD, this join should be performed on both the conversation_ids and source_author_id to contact_id.

As of right now, the join is not properly taking into account the true uniqueness of the conversation_contact_history model when joining to the conversation_history table.

Relevant error log or model output

No errors reported just yet, but this should be taken into consideration proactively

Expected behavior

The joins are taken into consideration to properly join on unique values. Otherwise, I realized the conversation_contact components are not even used in the final model. Maybe we can remove this join altogether.

dbt Project configurations

n/a

Package versions

packages:

• package: fivetran/intercom
  version: [">=0.9.0", "<0.10.0"]

What database are you using dbt with?

bigquery

dbt Version

v1.7.1

Additional Context

We should consider just removing this join. However, first we should understand why this was included in the first place and if we still need to leverage this join in the end model.

Are you willing to open a PR to help address this issue?

• Yes.
• Yes, but I will need assistance and will schedule time during our office hours for guidance
• No.