microsoft / azuresynapsescriptsandaccelerators Goto Github PK

Windows version: windows server 2022
Power shell version: 7.3.9
SQL Server version: SQL Server 2019 Standard

I'm trying to export parquet from MSSQL server using Export-ToParquet.ps1 with Parquet.Net dependency.

I got theses message when I export decimal type from MSSQL
MethodInvocationException: Exception calling ".ctor" with "3" argument(s): "scale cannot be less than 1 (Parameter 'scale')

but the job still complete after this exception.

Here's my table schema and config file
schema:

CREATE TABLE [dbo].[DM_BedsClothes_2017](

        [Site] [varchar](4) NOT NULL,

        [Emplid] [nvarchar](10) NOT NULL,

        [BedClothes] [char](1) NOT NULL,

        [ReceiveDate] [varchar](10) NOT NULL,

        [Prices] [decimal](18, 0) NOT NULL,

        [CreateBy] [varchar](10) NOT NULL,

        [CreateOn] [smalldatetime] NOT NULL,

        [ModifyBy] [varchar](10) NOT NULL,

        [ModifyOn] [smalldatetime] NULL

) ON [PRIMARY]

config file:

Enabled,Database,JobName,Query,FilePath
1,"quickstart_db","select_test_job","select * from DM_BedsClothes_2017",".\Output\test2.parquet"

Should I convert the data type or other solution to solve this issue?

I am looking for a conversion from bit to smallint instaed of boolean and nvarchar to varchar

I was following the instructions for running translatetableddls, table statements get created but they are missing the column information. gettting error with invoke sql-cmd.

It would be nice to have the ability to address a .sql query file as JobName in config file.

Trying to use this utility and works great on table and column that have a standard naming convention. The tables and column I am working with contain a '$'.

I have tried to ESC in front of the $.

• put [] around the table name
• use %36 instead of the $

No error is generated. The shell of the file is created. No data.

Any suggestions?

First of all, thanks so much for putting this project together. I am using it for a critical migration from On Prem SQL into Synapse dedicated SQL pool. One issue I have found is having the same table name under more than one schema results in duplicate column names when generating the CREATE TABLE statements. The issue is in file GetTableMetaData.sql under the folder "1_TranslateTableDDLs" as it does not take the schema into account in the Column_List CTE. Also need to account for passing in the schema name into this script. I will attach a screenshot of what I had to run manually to get around this but ideally I wanted to bring this use case to your attention as it would be nice to include into the tool