sql-bi / contoso-data-generator-v2

Custom Contoso database generator V2

Contoso Data Generator V2

DataGenerator is a tool for generating sample data, ready to be imported into PowerBI or Fabric OneLake for analysis. This is the V2 version, evolution of the older one.

If you are just interested in ready to use sets of data , download them here.

Supported output formats:

• Parquet
• Delta Table (files)
• CSV
• CSV multi file
• CSV multi file - gz compressed
• Sql Server, via bulk insert script of the generated CSV files

Delta Table output can be directly used in Fabric LakeHouse without any conversion:

Data schema:

Usage overview

FULL DOCUMENTATION available here: ⇒ ⇒ https://docs.sqlbi.com/contoso-data-generator/ ⇐ ⇐

DataGenerator requires four mandatory elements to run:

• a configuration file (json)
• a data file (excel)
• an output folder
• a cache folder
• [optional parameters]

databasegenerator.exe  configfile  datafile  outputfolder  cachefolder   [param:AAAAA=nnnn] [param:BBBBB=mmmm]

Example:

databasegenerator.exe  c:\temp\config.json  c:\temp\data.xlsx  c:\temp\OUT\  c:\temp\CACHE\

Note: the tool needs some files containing static data: fake customers, exchange rates, postal codes, etc. The files are cached after been downloaded over the Internet from a specific SQLBI repository.

contoso-data-generator-v2's Issues

Sales view not available when 'Orders' is specified

I generated the CSV files using Orders parameters and it worked fine, but when I loaded the CSV into SQL using Sql_ImportData it didn't create the sales view, is this expected behaviour and we have to create the sales view? I think following sales view for Orders option can be useful when sales table isn't needed and we don't have to generate the view every time for each database.

SELECT 
        Orders.OrderKey AS [Order Number],
        OrderRows.[LineNumber] AS [Line Number],
        Orders.[OrderDate],
        Orders.[DeliveryDate],
        Orders.CustomerKey,
        Orders.StoreKey,
        OrderRows.ProductKey,
        OrderRows.Quantity,
        OrderRows.[UnitPrice],
        OrderRows.[NetPrice],
        OrderRows.[UnitCost],
        Orders.[CurrencyCode],
        [CurrencyExchange].Exchange AS [Exchange Rate]
    FROM
        [Data].Orders  
            LEFT OUTER JOIN [Data].OrderRows
                ON Orders.OrderKey = OrderRows.OrderKey
            LEFT OUTER JOIN [Data].[CurrencyExchange]
                ON 
                    [CurrencyExchange].Date = Orders.[OrderDate] AND
                    [CurrencyExchange].[ToCurrency] = Orders.[CurrencyCode] AND
                    [CurrencyExchange].[FromCurrency] = 'USD'

DatabaseGenerator.winx64 doesn't have Script/Dataset folder

In docs you have mentioned Scripts/Dataset but when I downloaded the DatabaseGenerator.winx64 file it only has 4 files - Config, data, DatabaseGenerator, readme. There is no Script/Dataset folder

And the DatabaseGenerator is application file but it doesn't executes?