dbt-labs / docs.getdbt.com

The code behind docs.getdbt.com

We use docusaurus to power docs.getdbt.com.

Table of Contents

• Code of Conduct
• Contributing
• Writing content
• Running the docs site locally

Code of conduct

Please review the dbt docs contributors code of conduct. Creating an inclusive and equitable environment for our documents is more important than any other aspect. Syntax errors can be corrected, but trust, once lost, is difficult to gain back.

Contributing

We welcome contributions from community members to this repo:

• Fixes: When you notice an error, you can use the Edit this page button at the bottom of each page to suggest a change.
• New documentation: If you contributed code in dbt-core, we encourage you to also write the docs here! Please reach out in the dbt community if you need help finding a place for these docs.
• Major rewrites: You can file an issue to propose ideas for a content area that requires attention.

You can use components documented in the docusaurus library.

Writing content

The dbt Labs docs are written in Markdown and sometimes HTML. When writing content, refer to the style guide and content types to help you understand our writing standards and how we break down information in the product documentation.

SME and editorial reviews

All PRs that are submitted will be reviewed by the dbt Labs Docs team for editorial review.

Content that is submitted by our users and the open-source community are also reviewed by our dbt Labs subject matter experts (SMEs) to help ensure technical accuracy.

Versioning and single-sourcing content

We now enable you to reuse content between different docs pages, version pages, and establish product variables in the dbt Labs product documentation. To learn more about how to single source content between versions, product variables, and other content, see Single-sourcing content.

Adding tabbed components to a page

You can add code snippets and other content in a tabbed view. To learn more about adding tabbed components, see Adding page components.

Running the Docs site locally

You can click a link available in a Vercel bot PR comment to see and review your changes rendered on a staging server. You are also able to see and review your proposed modifications locally on your computer. Our setup instructions use homebrew:

Prerequisites

• (Mac Terminal) Install Xcode Command Line Tools
  • Open a terminal window, run xcode-select --install, and follow the on-screen prompts in the pop-up window.
• (Mac and Linux) Install homebrew
  • Copy and paste /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)" in to a terminal window and follow the prompts. Once the installation has completed, follow the Next Steps instructions listed in terminal.
• (Windows) Install Node.js

1. (Mac and Linux only) Install node: brew install node
2. Clone this repo: git clone https://github.com/dbt-labs/docs.getdbt.com.git
3. cd into the repo: cd docs.getdbt.com
4. cd into the website subdirectory: cd website
5. Install the required node packages: make install or npm install (optional — install any updates)
6. Build the website: make run or npm start
7. Before pushing your changes to a branch, run make build or npm run build and check that all links work

Advisory:

• If you run into an fatal error: 'vips/vips8' file not found error when you run npm install, you may need to run brew install vips. Warning: this one will take a while -- go ahead and grab some coffee!

Running the Cypress tests locally

Method 1: Utilizing the Cypress GUI

1. cd into the repo: cd docs.getdbt.com
2. cd into the website subdirectory: cd website
3. Install the required node packages: npm install
4. Run npx cypress open to open the Cypress GUI, and choose E2E Testing as the Testing Type, before finally selecting your browser and clicking Start E2E testing in {broswer}
5. Click on a test and watch it run!

Method 2: Running the Cypress E2E tests headlessly

1. cd into the repo: cd docs.getdbt.com
2. cd into the website subdirectory: cd website
3. Install the required node packages: npm install
4. Run npx cypress run

docs.getdbt.com's Issues

FAQ on the tutorial to hint at incremental models

FAQ: Can I use different schemas in my sources.yml for dev and prod

Steps to reproduce:

Expected results:

Actual results

Extra details

https://www.notion.so/dbtlabs/IDE-v1-1-Documentation-f8a3b1afec1b41e9a47359804933cb74

Use a different colored favicon to differentiate between docs and dbt Cloud tabs

When you have the docs and dbt Cloud open at the same time, it becomes confusing which is which.

We should consider using a different color favicon for the docs — grey? navy? Over to you @amy-byrum

Link broken in docs?

Describe the bug

On this page: https://docs.getdbt.com/reference/resource-configs/schema/

There's a link to this page under Using Custom Schemas: https://docs.getdbt.com/reference/resource-configs/schema/

...which gives me a Page Not Found

Thanks!

Bug: markdown relative links break when you go "back" to the page and re-click them

Steps to reproduce:

1. Go to dbt_project.yml docs
2. Click the data-paths hyperlink ([data-paths](project-configs/data-paths))
3. Success, it works! It goes to:

https://docs.getdbt.com/reference/project-configs/data-paths/

4. Go back using the back button
5. Click the same hyperlink
6. Now it doesn't work. It goes to:

https://docs.getdbt.com/reference/dbt_project.yml/project-configs/data-paths

Extra details

• This cannot be reproduced when using localhost.
• We can workaround it by using unique slugs and linking to those. For example, a hyperlink to [data-paths](data-paths) always works (I've tested this further on #97, here)
• This issue is not related to linky markdown — it can be reproduced by clicking the link outside of yml on this deploy preview. I think this is why I had opted to use the .md links previously (which do not work in linky markdown)

Bug: Markdown links do not render in Alert components

(note to self — this may be because I didn't put enough new lines in! But my node environment is messed up right now)

Bug: No page preview for docs.getdbt.com home page

When I share the docs.getdbt.com URL with no page path (e.g in Slack), no page preview gets rendered 😞

Fix home page button alignment

The buttons are misaligned on docs.getdbt.com

I don't think it always looked like this, but I'm not sure when it broke!

Create pages for each warehouse-specific config

Each of these pages should be turned into a category instead.

The config listed in each page should be made into separate pages.

Each config should:

• have a page for it, with description, default, usage examples etc.
• be listed/linked to on the relevant "model configurations", "snapshot configs" etc pages, to demonstrate the full gamut of configs a user can use
• be listed as a page in the in the "-specific configs"

Bug: Link broken in doc

Steps to reproduce:

Link to Configuring Quoting is broken in Class Reference - Relation section

Expected results:

Jump to the right page

Actual results

Page Not Found

Extra details

I think this should be the landing page.

Upgrade to dbt_project config-version: 2 syntax

This is going to be a ton of tiny changes. E.g. one that tripped me up today:

This

seeds:
  jaffle_shop:
    country_codes:
      column_types:
        country_code: varchar(2)
        country_name: varchar(32)

needs to become

seeds:
  jaffle_shop:
    country_codes:
      +column_types:
        country_code: varchar(2)
        country_name: varchar(32)

Add file headers to Tutorial

We didn't have this component when I wrote the tutorial, now we do!

Add "dynamically choose a schema" example to the schema reference docs

This comes up often enough that it would be good to have an example in the docs. Potentially with an FAQ to link between the guide and the reference doc.

sources:
  - name: core
    schema: "{{ 'core_production' if target.name == 'prod' else 'core_staging' }}"
    tables:
      - name: orders

Group Attributes for Setting up Okta is a tad confusing

We ran into an issue where group was configured as the name of the group attributes but it actually should have been groups. The image of Okta could also be clearer.

FAQ: dbt Cloud vs dbt Core

FAQ: How do I run tests on sources only?

Update dbt Cloud docs V1

Here is the list of the things that we can quickly update to make the dbt Cloud docs easier to understand:

• Overview page --> should actually have a burb at the top with a quick overview of what dbt cloud is (versus dbt Core)
• Update the screenshots for the dbt Cloud Quickstart page to take into account the new create a project flow
• update the links in the dbt Cloud Quickstart page
• Redo the connect a repo section with links to the different ways to set up your repo connection (link to this link)
• Update screenshot for Using a dbt Cloud managed repository page
• Update Connecting your GitHub Account so users fully understand they need to setup the dbt cloud app on their github organization

[BUG] "dbt Cloud" header sometimes splits over two lines

Steps to reproduce:

At a certain window size, the "dbt Cloud" item in the header bar splits over two lines.

Also the dbt logo gets stretched?

Finish off "unfinished" docs

So, uh, there were some pages where I was like "yeah, I'll come back to those", and I have not yet.

They are:

• materialized — this one is probably the most urgent
• alias
• schema
• database
• sql_header

Add link to the bottom of every page on the docs that links to the page in the repo

Expected results:

When I see a problem on the docs page, I want to be able to scroll all the way to the bottom to take me to the page in the repo for me to create a PR to fix it.

Actual results

I have to go digging through the repo

Extra details

Example implementation (I am totally spoiled): https://about.gitlab.com/handbook/ceo/chief-of-staff-team/

Docs reference kebab_case while using snake_case

Steps to reproduce:

Expected results:

Actual results

Extra details

Improvements to "Reference > dbt Jinja Functions" section

Leaving this as a place to come back to.

We could do a ton to improve this section. Things that come to mind:

• Properly documenting each function — some are well documented, others aren't. Each function should have headings like:

# var

{{ var(variable_name, default_value=None) }}

## Description
Returns a project variable, as defined in your `dbt_project.yml`, or via the command line

## Related documentation
- Using variables

## Arguments
(ok probably not needed here)
### variable_name (required)

### default_value
An optional default value. Note that if this variable is not present, a compilation error will be thrown.

## Examples
### Limit data based on a variable
...
### Other more complicated example
...

• Generate these definitions by documenting the macros in dbt-core
• Group the functions better. This gets grey, but I think there's a broad categorization e.g.:
  • blocks: doc, snapshot etc.
  • macros that do things: run_query, tojson, toyaml — in my mind, things that make sense inside of {% do ... %} blocks
  • variables that return a thing (generally don't have any arguments): target, var (arguably a function), project_name, invocation_id, this
  • filters: as_text
  • ??? statements (what is a statement anyway, is it a macro? why do you have to call it?)
• document the context each object is available in — at the moment we have some "floating" articles about contexts, some of which contain extra Jinja objects, but I'd prefer to have this information listed in the "function" pages, like "oh schema is only available in an on-run-end context"
• Generate the list of functions (i.e. just iterate over the sub-directory), and also use yaml front-matter for a short description

Improvements to "References > Profiles.yml" section

Again, placeholder in the hopes that one day we'll come back to it (🤞)
Though, hopefully one day we rebuild the profiles.yml into a more sane structure than the current one (see this dbt issue)

Improvements:

• Create articles for each (shared) key in profiles.yml, rather than putting them as subheadings on the "profiles.yml" section (this follows the structure of other sections). Add undocumented keys.
• Do a better job of rationalizing what's in "connecting to your data warehouse" versus what's not — it's all a bit of a mess right now.

Feature: Better yaml highlighting

The current YAML highlighting doesn't differentiate between keys and values.

In our docs:

In my code editor:

Import tabs component into the global context / add to styles sheet

Sometimes I want to use the Tab component with Snowflake/Redshift/BQ headers.

Currently I have to import it every time

Code example has 'from app_data.payments' vs. 'from {{ ref('raw_payments') }}'

One of the code examples on the using Jinga page (last one before the "Write modular macros" section) uses 'from app_data.payments' vs. 'from {{ ref('raw_payments') }}' which throws an error when used with the prior part of the tutorial.

Runtime Error in model order_payment_method_amounts (models/order_payment_method_amounts.sql)
404 Not found: Dataset dbt-tutorial-274612:app_data was not found in location US

Create reference section for packages.yml

This got skipped in the re-structure. We should make a (very short) reference section for packages.yml — probably overkill to have sub-pages for it tbh.

Better rendering for code blocks in reference section

Ideas:

• Use bold to indicate required keys (e.g. name: in dbt_project.yml)
• Rather than using < lt and gt signs > to indicate that a value should be replaced (which is currently inconsistently used), render it with grey italics. Potentially with a more subtle hyperlink to a list of datatypes (?)

This will required changes to the "linky code block" we've been using so far

Random to-dos

Jotting these down here so I don't forget:

• Move the "related documentation" section higher
• update the database config docs to mention the changelog
• use the changelog component
• ensure that tabs aren't imported in places they don't need to be
• create a reference home page that links to the dbt cloud api docs
• ~ update the style of the changelog component

Feature: docs home page

Random musings:
The docs.getdbt.com gives you a very clean landing page (but with little info), and if you click the "docs" link you get pushed straight into the Introduction.

I kind of like how in the Django docs there's a page which is like "here's where everything is". It makes it easier to get oriented quickly.

I'm not sure if this should live on the docs.getdbt.com landing page, or on docs.getdbt.com/docs

Anyway, just a thought 🤔

Add a step to tutorial to merge changes

Since we made the master branch protected, we added an instruction to create a branch, but forgot to add an instruction to open a pull request!

Though, how does one do that if they are using a managed repo? 🤔

Improvements to "Reference > commands" section

We can do a lot to improve this section:

• Use a (yet-to-be-made) CLI react component (see #171)
• Ensure each command has robust docs, like so:

# clean
## description
The `clean` is a utility function that deletes all folders specified in the [clean-targets](clean-targets) list specified in your `dbt_project.yml` file. This is useful for deleting installed packages in the `dbt_modules` directory, and compiled files in the `target` directory.

## Options (flags?)
This flag has no options

## Examples
(use the examples from the `clean-targets` docs

• List each flag under a command flags section. Each article should list the commands that the flag applies to (this is especially important now that people run commands from dbt Cloud), and whether it is CLI only, or also works in Cloud
  • --models (-m), --select (-s) and --exclude
  • --vars
  • --target (-t) (CLI only)
  • --fail-fast (-x)
  • --no-version-check
  • etc

# --target (-t) (CLI only)
``
$ dbt <command> --target [<target_name>](target_name)
$ dbt <command> -t [<target_name>](target_name)
``
## Description
Switch the target you're using for this command

## Commands
- `run`
- `test`
- `seed`
- etc

## Examples
### Perform a prod run using the production credentials, from your laptop
(You wouldn't really do this, would you?)
``
$ dbt run --target prod
``

• Create separate articles for each global command option (the ones that go here: $ dbt <option> <command>
• Have better model selection syntax docs, probably move this to the --models, --select, --exclude article
• At some point, it might be worth pulling out some concepts into a separate section (a glossary? something else?). For example "threads" — this is part of profiles.yml, is a command line flag, and is part of a dbt Cloud "connection" (?). But where should the actual definition of "threads" live? Not sure! ¯_(ツ)_/¯

Rewrite "Configure your profile"

Related to #245
Now that we have better homes for things, it makes sense to make this article much lighter

Bug: Previews show markdown instead of styled content

Steps to reproduce:

When I paste this URL in Slack:
https://docs.getdbt.com/docs/building-a-dbt-project/building-models/using-custom-schemas/

I get:

Expected results:

Markdown links and headers should not show as raw markdown

Actual results

Raw markdown is shown

FAQ: How do I declare a column type?

Bug: FAQ previews don't show the Title

e.g. https://tutorial.getdbt.com/tutorial/faqs/unique-model-names/
renders as:

Instead of: dbt - Documentation · Your entire analytics engineering workflow
I'd like: dbt - Documentation · Do model names need to be unique?

Condense "Using the dbt IDE", "Using the CLI" and "Using the python API" into one article

We're saying less in each of these articles. We should condense these into one article, and just have a quick pros/cons of each approach :)

Tutorial: Make it clear that you need to save things in the IDE before running

A common stumbling block for new users. Would be good to understand if the product is doing something on this front soon though!

Feature: Add language type to file component

Instead of this:
(lol couldn't get the formatting right)

I'd love to write:

<File name='dbt_project.yml', language='yml'>

data-paths: ["data", "seeds"]

</File>

Improve CLI installation docs

Rather than have separate pages for OS specific guides, we should just have:

• A guide for pip, brew (recommended for MacOS) and Install from source
• OS-specific notes on the overview page

Feature: Code blocks should have a horizontal scroll, rather than wrap over two lines

When code blocks have long lines of code, they warp over two lines, rather than having a horizontal scroll.

This can be confusing as to whether a line break exists. I'd prefer a horizontal scroll bar to appear.

Note, this PR indicates that this behavior should already be implemented — is this a case of needing to bump a version number, or are we on our own since we have our own implementation of code blocks?

Example: this config block gets wrapped:

Refactor: Rationalize use of alerts and callouts

These two components are very similar. Can we just choose to use one over the other?

Docs: docker image

via https://hub.docker.com/r/fishtownanalytics/dbt

Let's:

• update the dockerhub page
• add info about the existence of these images, and how you can use them

Feature: glossary component

Would be cool to have a hover-over component that shows a definition for some of the dbt-specific language we use.

This was a feature in Readme, but we didn't use it a ton.

FAQ: Test before running the next model

We get asked questions like this really often

Bug: 404s on relative link references

References on the build your first models page to Setting Up404.

The code in question is both:

[Setting up](setting-up)
resolves to:
https://tutorial.getdbt.com/tutorial/build-your-first-models/tutorial/setting-up.md

[Setting up](tutorial/setting-up.md)
resolves to:
https://tutorial.getdbt.com/tutorial/build-your-first-models/tutorial/setting-up.md

FAQ: Can I connect to two databases?

Add `warn` macro to the Jinja function reference docs

Logging here since we're in the middle of a migration 🤐

FAQ: "Can you call a macro within another macro"

I mean, technically not a dbt question, but we get it often enough that it might be worth writing it up

FAQ: What does the number in the CLI output mean?

Add an FAQ for this output:

15:22:14 | 1 of 1 OK hook: logging.on-run-end.0................................. [SUCCESS 1 in 1.80s]

(especially when there's two numbers for incremental models!)

We get this questions every so often, and I never remember the answer, so would be good to know it