GithubHelp home page GithubHelp logo

dbt-labs / dbt-docs Goto Github PK

View Code? Open in Web Editor NEW
132.0 17.0 70.0 2.05 MB

Auto-generated data documentation site for dbt projects

License: Apache License 2.0

Makefile 0.15% HTML 31.53% JavaScript 67.42% CSS 0.71% Ruby 0.05% Python 0.14%

dbt-docs's Introduction

Understanding dbt-docs

dbt helps analysts write reliable, modular code using a workflow that closely mirrors software development.

This repository contains code for generating a documentation site for dbt projects. Check out the dbt documentation for more information.


Getting Started

In your dbt project, run dbt docs generate then dbt docs serve. Learn more about docs here.

Join the dbt Community

Reporting bugs and contributing code

  • Want to report a bug or request a feature? Let us know and open an issue
  • Want to help us build dbt-docs? Check out the Contributing Guide

Code of Conduct

Everyone interacting in the dbt project's codebases, issue trackers, chat rooms, and mailing lists is expected to follow the dbt Code of Conduct.

dbt-docs's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

dbt-docs's Issues

cache bust the json files

These files should never be cached.

Use a url parameter with the current timestamp when fetching the json files to bust the browser cache.

How to build custom dbt-docs into a dbt build

Sorry if I've missed the instructions for this. We are looking again at the lineage styling/colouring piece. However, with the seperation of the dbt and dbt-docs repos, we cannot see how the later builds into the former i.e. how we'd get a version of dbt build from source (or even not built from source, since we don't really need any core dbt changes) to use a custom dbt-docs - could anyone point us in the right direction for this? Thanks!

[CT-960] Configurable Color-Coding of Schemas/Nodes in Docs

Similar to here.
Essentially, we are interested in being able to color code either schemas or individual models in the dbt docs in a similar fashion to the way you can configure materialization of individual models.

e.g.

models:
  rad_warehouse:
    materialized: view
    staging:
      schema: staging
      materialized: view
      color: #453453
    analytics:
      schema: analytics
      materialized: table
      color: #589593

We really like the difference in coloration between source and models, and it would be awesome to carry this further to make it easier to distinguish between different parts of the dag.

A colleague (Taylor M.) and I are interested in working on this. We can imagine implementing this in a couple of different ways, either through the dbt_project.yml, or through configurable tags in models or schemas.

Do you have any suggestions, tips, warnings, or words of caution?

Thanks!

White labelling

This could take the form of:

  • Optional org logo in top-left corner (in lieu of or addition to dbt logo)
  • Customizable color scheme for docs skin

DBT Docs Column Disappears When Window Is Not Wide Enough

Issue

Issue description

When the dbt docs screen is brought to some width threshold the sidebar disappears.
BEFORE:
screen shot 2018-10-12 at 1 10 00 pm
AFTER:
screen shot 2018-10-12 at 1 10 11 pm

Results

I would expect the sidebar to collapse and be expandable with a toggle. There currently is not expand option.

System information

The output of dbt --version:

0.11.1

The operating system you're running on: MacOS Mojave, 10.14

The python version you're using (probably the output of python --version): Python 3.6.6

Steps to reproduce

Open dbt docs in a browser (I happen to be in Chrome) or use the example project here. Make the screen less-wide.

Lineage Graph does not filter source:*

In the Lineage Graph view in dbt docs, it's currently not possible to filter out all sources from the graph without specifying each one separately.

This does not work:
-- exclude source:*

Also, assuming source yml files live in models._sources, this also does not work:
-- exclude _sources.*
-- exclude _sources

(Btw, this also does not work for --models.)

image

Show dbt project.yml in dbt docs

Sometimes I want to check a configuration for a model and realize it might be in my dbt_project.yml file, which means I have to leave my dbt docs to check it.

Consolidating this info in one place would be useful.

Documentation website should (optionally) lowercase automatically generated column and type names

On the documentation website, columns that were not explicitly defined in a schema.yml file are shown in uppercase (on Snowflake) since that's the casing returned by the database.
If the columns defined in schema.yml are lower-cased, the resulting documentation has mixed casing, which looks ugly. Many practitioners use lower case for their model names and, thus, also use lower case for column names.

For example (of an admittedly poorly documented model):
image

Ideally, users should be able to define a casing option for column and type names, or all column and type names should be lower-cased.

Document macros

Macros should appear in the docs site and be documentable

Errors building from source

When trying to build from source on a stock Ubuntu 18.04 system, a few dependencies are missing .e.g. ruby-dev. Also, several of the commands, as written, need to be run with sudo. Happy to prepare a PR for these simple changes. However, when running jekyll build

I get

Configuration file: /home/ubuntu/dbt-docs/styles/_config.yml
            Source: /home/ubuntu/dbt-docs/styles
       Destination: /home/ubuntu/dbt-docs/styles/_site
 Incremental build: disabled. Enable with --incremental
      Generating...
  Conversion error: Jekyll::Converters::Scss encountered an error while converting 'ui/css/styles.scss':
                    Error: unclosed parenthesis in media query expression on line 1:33 of _includes/sass/mixins/_mixin-queries.scss from line 2:1 of _includes/sass/mixins/mixins.scss from line 4:1 of styles.scss >> @mixin small(){ @media ( "max-width: " ($tablet) ) { @content; } } --------------------------------^
                    ------------------------------------------------
      Jekyll 4.0.0   Please append `--trace` to the `build` command
                     for any additional information or backtrace.
                    ------------------------------------------------

With --trace

Configuration file: /home/ubuntu/dbt-docs/styles/_config.yml
            Source: /home/ubuntu/dbt-docs/styles
       Destination: /home/ubuntu/dbt-docs/styles/_site
 Incremental build: disabled. Enable with --incremental
      Generating...
  Conversion error: Jekyll::Converters::Scss encountered an error while converting 'ui/css/styles.scss':
                    Error: unclosed parenthesis in media query expression on line 1:33 of _includes/sass/mixins/_mixin-queries.scss from line 2:1 of _includes/sass/mixins/mixins.scss from line 4:1 of styles.scss >> @mixin small(){ @media ( "max-width: " ($tablet) ) { @content; } } --------------------------------^
Traceback (most recent call last):
        28: from /usr/local/bin/jekyll:23:in `<main>'
        27: from /usr/local/bin/jekyll:23:in `load'
        26: from /var/lib/gems/2.5.0/gems/jekyll-4.0.0/exe/jekyll:15:in `<top (required)>'
        25: from /var/lib/gems/2.5.0/gems/mercenary-0.3.6/lib/mercenary.rb:19:in `program'
        24: from /var/lib/gems/2.5.0/gems/mercenary-0.3.6/lib/mercenary/program.rb:42:in `go'
        23: from /var/lib/gems/2.5.0/gems/mercenary-0.3.6/lib/mercenary/command.rb:220:in `execute'
        22: from /var/lib/gems/2.5.0/gems/mercenary-0.3.6/lib/mercenary/command.rb:220:in `each'
        21: from /var/lib/gems/2.5.0/gems/mercenary-0.3.6/lib/mercenary/command.rb:220:in `block in execute'
        20: from /var/lib/gems/2.5.0/gems/jekyll-4.0.0/lib/jekyll/commands/build.rb:18:in `block (2 levels) in init_with_program'
        19: from /var/lib/gems/2.5.0/gems/jekyll-4.0.0/lib/jekyll/command.rb:89:in `process_with_graceful_fail'
        18: from /var/lib/gems/2.5.0/gems/jekyll-4.0.0/lib/jekyll/command.rb:89:in `each'
        17: from /var/lib/gems/2.5.0/gems/jekyll-4.0.0/lib/jekyll/command.rb:89:in `block in process_with_graceful_fail'
        16: from /var/lib/gems/2.5.0/gems/jekyll-4.0.0/lib/jekyll/commands/build.rb:36:in `process'
        15: from /var/lib/gems/2.5.0/gems/jekyll-4.0.0/lib/jekyll/commands/build.rb:65:in `build'
        14: from /var/lib/gems/2.5.0/gems/jekyll-4.0.0/lib/jekyll/command.rb:28:in `process_site'
        13: from /var/lib/gems/2.5.0/gems/jekyll-4.0.0/lib/jekyll/site.rb:76:in `process'
        12: from /var/lib/gems/2.5.0/gems/jekyll-4.0.0/lib/jekyll/site.rb:202:in `render'
        11: from /var/lib/gems/2.5.0/gems/jekyll-4.0.0/lib/jekyll/site.rb:515:in `render_pages'
        10: from /var/lib/gems/2.5.0/gems/jekyll-4.0.0/lib/jekyll/site.rb:515:in `each'
         9: from /var/lib/gems/2.5.0/gems/jekyll-4.0.0/lib/jekyll/site.rb:516:in `block in render_pages'
         8: from /var/lib/gems/2.5.0/gems/jekyll-4.0.0/lib/jekyll/site.rb:523:in `render_regenerated'
         7: from /var/lib/gems/2.5.0/gems/jekyll-4.0.0/lib/jekyll/renderer.rb:63:in `run'
         6: from /var/lib/gems/2.5.0/gems/jekyll-4.0.0/lib/jekyll/renderer.rb:84:in `render_document'
         5: from /var/lib/gems/2.5.0/gems/jekyll-4.0.0/lib/jekyll/renderer.rb:100:in `convert'
         4: from /var/lib/gems/2.5.0/gems/jekyll-4.0.0/lib/jekyll/renderer.rb:100:in `reduce'
         3: from /var/lib/gems/2.5.0/gems/jekyll-4.0.0/lib/jekyll/renderer.rb:100:in `each'
         2: from /var/lib/gems/2.5.0/gems/jekyll-4.0.0/lib/jekyll/renderer.rb:102:in `block in convert'
         1: from /var/lib/gems/2.5.0/gems/jekyll-sass-converter-2.1.0/lib/jekyll/converters/scss.rb:182:in `convert'
/var/lib/gems/2.5.0/gems/jekyll-sass-converter-2.1.0/lib/jekyll/converters/scss.rb:190:in `rescue in convert': Error: unclosed parenthesis in media query expression (Jekyll::Converters::Scss::SyntaxError)
        on line 1:33 of _includes/sass/mixins/_mixin-queries.scss
        from line 2:1 of _includes/sass/mixins/mixins.scss
        from line 4:1 of styles.scss
>> @mixin small(){        @media ( "max-width: " ($tablet) ) { @content; } }

   --------------------------------^

Any ideas?

White-labeling dbt docs

Describe the feature

I'd like to be able to easily customize our dbt docs site https://gitlab-data.gitlab.io/analytics/dbt/snowflake/#!/model/model.gitlab_dw

Specifically:

  • Replace the dbt logo with our logo
    • I'd still love the dbt logo in there though - powered by dbt docs maybe?
  • Adjust main colors so they can be more on-brand

Describe alternatives you've considered

"The cheapo version of this is probably just a stylesheet in the target/ dir that the docs website will try to load"

Who will this benefit?

Anybody who wants a more white-label version of docs

feature request: discover data by column

The idea in brief:

To improve data discovery users can search by column name to find related tables.

As an analogy, this would be similar to querying information_schema.columns, where column_name ilike '%your_string%'

Add last updated date to dbt docs

Feature

Feature description

Running dbt and deploying updated dbt docs may happen at the same time, but they also might not. One might fail, but the other might night.

It would be great if it were explicit on the dbt docs site when they were last updated (for example the time that the last dbt docs serve was run.

Current dbt home page
Screen Shot 2019-04-12 at 10 40 26 AM

Proposed
docs last updated_ 2019-04-12

Who will this benefit?

Anyone consuming docs.

Export model graph to file

When running dbt docs generate is it possible to add an "Export to file..." button with options for PNG and PDF?

It would be extremely useful when sharing the model overview with other people, and general documentation outside of dbt.

[Bug] Incorrect code highlighting

Steps to reproduce:

  1. Include the following model in your project:
{{
    config({
        "materialized" : 'incremental',
        "sql_where" : 'TRUE',
        "post-hook" : [
          after_commit("{{ vacuum( var('maintenance', false) ) }}"),
          after_commit("{{ analyze( var('maintenance', false) ) }}")
         ]
    })
}}

select
    current_timestamp as run_at,
    {{var('maintenance', false)}} as maintenance_jobs_run
  1. dbt docs generate && dbt docs serve

Expected results:
Normal code highlighting

Actual results:
screen shot 2018-12-14 at 10 06 21 am

System information:
Output of dbt debug (omitting connection details>

dbt version: 0.12.2-rc1
python version: 3.7.1
python path: /usr/local/Cellar/[email protected]/0.12.2rc1/libexec/bin/python3.7
os info: Darwin-18.2.0-x86_64-i386-64bit
Using profiles.yml file at /Users/claire/.dbt/profiles.yml

Configuration:
  profiles.yml file [✓ found and valid]
  dbt_project.yml file [✓ found and valid]
  profile: <profile-name> [✓ found]
  target: dev [✓ found]

Required dependencies:
 - git [✓ found]

Show custom tests

Tests defined for columns show up in the generated documentation.

However, tests defined at the model level do not appear anywhere in the generated documentation.

Tags break sources in the lineage view

From Slack

Given an example source configured with a tag:

version: 2
sources:
  - name: myproject
    tags:
      - mytag
    tables:
      - name: mytable

The source is not rendered in the lineage DAG. Removing the mytag tag fixes the issue.

Add analyses to dependency graph

Currently, analyses that are defined in the analysis folder of a DBT project do not appear in the dependency graph that gets generated. It would be fantastic to have a way to link an analysis to its dependent model. Any change that gets made regarding this should ensure that analyses that don't depend on DBT models should be able to be represented well as people will version control their SQL queries as analysis during a migration into using DBT more.

It would be fantastic if it were possible to use some visual differentiator between models and analysis.

Non-alphabetical sort of Source Tables in source overview page

Currently, the documentation site Table of Contents shows all Source Tables in alphabetical order, while the Source Schema page will show the same list of tables in the order they appear in catalog.json, which appears to be arbitrary.

Would love to see either the option to sort, or an alphabetical sort be applied by default.

Screenshot Example

Select multiple nodes on data lineage graph

Describe the feature

Firstly - congratulations on on the series A funding!

I have a bunch of suggestions for the Data Lineage tool. If this is useful, I'll keep them coming.

Problem: I could only remove nodes I selected one at a time to perform an action.
Screen Shot 2020-04-30 at 16 51 56

Whereas it would speed up the workflow by allowing multi select to perform the same action on a group of nodes.
multi-select

Describe alternatives you've considered

You could try and be smart with the model selection syntax, but this in itself can slow newer users.

Who will this benefit?

General workflow improvement when utilising dbt docs.

"Edit this definition" link

Describe the feature

Viewers of a given resource page in the auto-generated documentation site should be able to click a link that directs them to the appropriate configuration file in the dbt project git repository, such that they could edit descriptions for that resource.

Challenges

  • Each manifest node would need to store the path to the YML file which configures it. As I understand it, the manifest includes configuration file paths for sources but not for models.
  • The dbt project / docs needs to somehow know the URL of its repository. This makes more sense in dbt Cloud than it does in dbt core.

Alternatives

  • A more involved implementation would direct users to the specific line with the description of a given model/column. I think directing users to the right YML file is good enough for the V1.
  • This is much easier to build vs. an interface that would allow viewers to live-edit docs!

Who will this benefit?

Organizations with more active/engaged docs viewers

Engagement tracking

Analytics teams may wish to embed their own tracking snippets (Snowplow, GA, etc) on the docs site. This would enable them to:

  • Track adoption of data documentation across the wider organization (and potentially prove their value addition)
  • Identify which models receive the greatest interest

I'm wondering if there's an approach that can work across all deployment types:

  • Self-hosted docs
  • Hosted dbt Cloud
  • On-prem dbt Cloud

Questions:

  • If we allow embedding any custom JS, what are the risks? How to prevent malicious behavior?
  • Should usage tracking be anonymous only? Or should there be a mechanism to support identifying users by their dbt Cloud account / other authentication if self-hosted?

Allow customisation of base path

Current behaviour

When trying to host DBT documentation on a non-top level path the docs site looks for a few files at the root path

e.g.

Hosting the index.html file at https://my-internal-domain.com/dbt
The docs fail to load due to requesting https://my-internal-domain.com/catalog.json

Expected behaviour

Allow customisation of the base url so JSON file requests resolve from the given url, e.g https://my-internal-domain.com/dbt/catalog.json.

Workaround

For now we're fixing this by explicitly proxying .json files generated by DBT to the path we expose the dbt documentation on:

location ~ ^/(catalog|run_results|manifest)\.json$ {
  rewrite ^/(.*)$ /dbt/$1;
}

Create ERD from relationships in schema.yml files

For complicated table relationships, it's nice to have an ERD (i.e. schema diagram) to help understand how everything fits together.

Schema.yml files already have relationship tests built into them that show the relationship between tables, so those could be parsed to get the data needed for the "keys," which may not actually be keys in most warehouse systems.

A first implementation of this functionality might just look at the schema.yml file and get any columns listed there so that only one data source is needed. A fuller implementation would get all columns from the tables.

Other thoughts:

  • Integrating the descriptions that are in the schema.yml files would also be great, perhaps with a tooltip next to the table or column name
  • If there are multiple schema.yml files in a project, it might be nice to color or box the tables that are known to be related to each other because they're in the same yml. Also would be nice to be able to filter for things in the same yml file. I think this is the nicest way to deal with being able to look at individual layers of a project, but would like someone else to think about that too
  • Being able to zoom in/out and pan around like in the graph view would be a nice feature here

DBT docs are not evaluating variables in the documentation output.

DBT docs are not evaluating variables in the documentation output. For example, I have

- name:  var('company_name')
  description: "The company name after being passed through a cleansing process."
  tests:
    - not_null

And while it displays and runs tests on the column, it displays the column name in docs as
var('company_name')
I would rather want it to point to the underlying value defined in the dbt_project.yml and the value is cleaned_name that should show in the doc.

Dbt docs "search" includes field matches

Feature

Feature description

In the "search for models" search bar of the DBT Docs, it would be great to be able to search for fields, as currently it seems only to be possible to search by models.

Who will this benefit?

This would benefit business users (and data engineers) in that they would be able to determine if a field is available in the data warehouse and if it is, to see in which models it lives.

Multiple snapshots, same file, missing docs

Snapshots are missing from the docs if multiple snapshot blocks are defined in the same snapshots/**.sql file.

Steps to replicate

  • Define two or more snapshot blocks within a snapshots/my_snapshot.sql file
  • Generate + view docs. Only the last snapshot block from each file will appear in the docs.
  • Move each snapshot blocks into a different snapshots/**.sql file
  • Generate + view docs. Each snapshot block now appears in the docs.
dbt --version
installed version: 0.17.0-rc1
   latest version: 0.16.1

Your version of dbt is ahead of the latest release!

Plugins:
  - bigquery: 0.17.0rc1
  - snowflake: 0.17.0rc1
  - redshift: 0.17.0rc1
  - postgres: 0.17.0rc1

Show a list of all Tags on the documentation site

There is high probability that different members in a big team may be using similar tags. Like some maybe using marketing and some mktg.
It will be neat to get the list of all the tags being used within a team to double check, before someone creates a new one.

This can be on the documentation site along with other information. As Drew suggested, it would be neat to show a list of tags, then click one to see all of the associated models.

==

Search also in SQL

It would be great if the search searches also in SQL statements and not only in tables and columns, for example when I want to search a particular hardcoded filter in the SQL and I know the keyword. Thanks

Standardize use of `--models` syntax across CLI + docs site

The --models syntax in the dbt docs lineage graph viewer produces different results from the command line.

From the command line, --models zendesk.base recognizes 4 models.

09:12:44 ~/repos/analytics/transform/snowflake-dbt/models/zendesk (1689-fix-zendesk-models) $ dbt run --models zendesk.base
Running with dbt=0.13.1
Found 221 models, 877 tests, 4 archives, 6 analyses, 253 macros, 8 operations, 4 seed files, 113 sources

09:26:11 | Concurrency: 8 threads (target='dev')
09:26:11 |
09:26:11 | 1 of 4 START view model emilie_scratch_staging.zendesk_organizations. [RUN]
09:26:11 | 2 of 4 START view model emilie_scratch_staging.zendesk_ticket_metrics [RUN]
09:26:11 | 3 of 4 START view model emilie_scratch_staging.zendesk_tickets....... [RUN]
09:26:11 | 4 of 4 START view model emilie_scratch_staging.zendesk_users......... [RUN]
09:26:16 | 3 of 4 OK created view model emilie_scratch_staging.zendesk_tickets.. [SUCCESS 1 in 4.62s]
09:26:16 | 1 of 4 OK created view model emilie_scratch_staging.zendesk_organizations [SUCCESS 1 in 4.84s]
09:26:16 | 2 of 4 OK created view model emilie_scratch_staging.zendesk_ticket_metrics [SUCCESS 1 in 4.97s]
09:26:17 | 4 of 4 OK created view model emilie_scratch_staging.zendesk_users.... [SUCCESS 1 in 5.41s]
09:26:22 |
09:26:22 | Finished running 4 view models in 43.02s.

Completed successfully

Done. PASS=4 ERROR=0 SKIP=0 TOTAL=4

In our dbt docs site, we get sad results https://gitlab-data.gitlab.io/analytics/dbt/snowflake/#!/overview

Screen Shot 2019-07-11 at 9 26 20 AM

My expected behavior would be to see the graph of each of those models and the + syntax to be how they flow/evolve/transform.

For my case, the solution was to use zendesk.base.*+ on the docs site. Thanks @cclaus for the tip!

Versions

09:26:26 ~/repos/analytics/transform/snowflake-dbt/models/zendesk (1689-fix-zendesk-models *) $ dbt --version
installed version: 0.13.1
   latest version: 0.14.0

Your version of dbt is out of date! You can find instructions for upgrading here:
https://docs.getdbt.com/docs/installation
09:29:29 ~/repos/analytics/transform/snowflake-dbt/models/zendesk (1689-fix-zendesk-models *) $ python --version
Python 3.7.1

Export documentation site as a set of static pages.

@philippe-lavoie commented on Tue Nov 12 2019

Describe the feature

Shipping a zip of static HTML pages makes it easy to ship internal documentation to others. I'd use something like Gatsby to create the static pages. But to stick to a Python pipeline, something like Pelican could be used. I have never tried it though.

Describe alternatives you've considered

An alternative would be to generate a PDF file corresponding to the documentation, but would have to include the node graph and in practice, this might be a true alternative in the sense that it improves portability.

Additional context

I wanted to ship what I had to a client but I can't expose an internal site nor can easily create a locked-down public site for them. Shipping a site.zip would make this trivial.

Who will this benefit?

Anyone that wants to more easily share what's going on inside all those DBT models. Also, with a static site, you can add the documentation has a sub-folder of a larger site which makes exposing documentation even easier.


@philippe-lavoie commented on Tue Nov 12 2019

The documentation states that the site is static. However, when loading directly from the index.html page, I get

dbt Docs was unable to load the mantifest file at path: manifest.json?cb=1573569439241

Perhaps, just the fix the fetch to use the above, but when it fails defaults to manifest.json instead ?


@drewbanin commented on Tue Nov 12 2019

hey @philippe-lavoie - the site is "static" in that you don't need to run a database or an application webserver to use it. The site does however load a few .json files that contain information about your project code and the state of the database.

You're seeing this problem because your web browser is disallowing the docs site to request other local files (eg. file://Users/drew/my_project/target/manifest.json). This is a security feature intended to prevent sites from reading arbitrary files on a user's hard drive.

I think that if we were to do something here, it would be to embed the json data directly into the index.html file. That way, you wouldn't need to send along a .zip file with a bunch of html files inside of it - you'd just have a single index.html file that showed all of the docs.

I'm going to transfer this issue over to the docs repo for further discussion. Let me know if you have any questions or thoughts!

Support object types in document site

I would like to generate the document site for tables modelled in BigQuery. Some of the columns are stored as RECORDs and it would be nice if this could be modelled in the schema.yml in such a way

Here is an example of the data I am trying document:

**contexts_com_snowplowanalytics_snowplow_ua_parser_context_1_0_0**              RECORD REPEATED    
contexts_com_snowplowanalytics_snowplow_ua_parser_context_1_0_0.device_family    STRING REQUIRED    
contexts_com_snowplowanalytics_snowplow_ua_parser_context_1_0_0.os_family        STRING 
REQUIRED    
contexts_com_snowplowanalytics_snowplow_ua_parser_context_1_0_0.useragent_family STRING 
REQUIRED

Maybe is could be modelled like this:

-name: contexts_com_snowplowanalytics_snowplow_ua_parser_context_1_0_0
 description: UA parser context
  -name: device_family
   description: Device family
  -name: os_family
   description: Operating system family

Render documentation for seed files

Seed files do not show in the side panel.
You can get to their docs by right clicking on the node in the DAG, and selecting "View Documentation". However, if you have documentation/tests for the seed files, it won't be rendered here.
Related to #15

dbt snapshot meta-fields' column descriptions could be automatically filled out

Describe the feature

After creating a snapshot, the description of the meta-fields (i.e. dbt_valid_from, dbt_valid_to, etc.) on the documentation website is filled out automatically.

Who will this benefit?

Anyone who reads the documentation. Especially those who want to recover data or build on the snapshot but aren't familiar with dbt's meta field vocab.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.