GithubHelp home page GithubHelp logo

martinjw / dbschemareader Goto Github PK

View Code? Open in Web Editor NEW
293.0 38.0 128.0 55.38 MB

Read database metadata (from SqlServer/Oracle/MySql/SQLite/PostgreSql/DB2 etc) into one simple model

License: Microsoft Public License

C# 97.26% PLpgSQL 1.57% Batchfile 0.03% TSQL 1.15%

dbschemareader's Introduction

DatabaseSchemaReader

A simple, cross-database facade over .Net 2.0 DbProviderFactories to read database metadata.

Any ADO provider can be read (SqlServer, SqlServer CE 4, MySQL, SQLite, System.Data.OracleClient, ODP, Devart, PostgreSql, DB2...) into a single standard model. For .net Core, we support SqlServer, SqlServer CE 4, SQLite, PostgreSql, MySQL and Oracle.

https://github.com/martinjw/dbschemareader or https://dbschemareader.codeplex.com/

https://dbschemareader.codeplex.com/documentation

Nuget: Install-Package DatabaseSchemaReader Nuget

Appveyor Build Status

  • Database schema read from most ADO providers
  • Simple .net code generation:
    • Generate POCO classes for tables, and NHibernate or EF Code First mapping files
    • Generate simple ADO classes to use stored procedures
  • Simple sql generation:
    • Generate table DDL (and translate to another SQL syntax, eg SqlServer to Oracle or SQLite)
    • Generate CRUD stored procedures (for SqlServer, Oracle, MySQL, DB2)
  • Copy a database schema and data from any provider (SqlServer, Oracle etc) to a new SQLite database (and, with limitations, to SqlServer CE 4)
  • Compare two schemas to generate a migration script
  • Simple cross-database migrations generator

Use

  • Full .net framework (v3.5, v4.0, v4.5-v4.8)
//To use it simply specify the connection string and ADO provider (eg System.Data,SqlClient or System.Data.OracleClient)
const string providername = "System.Data.SqlClient";
const string connectionString = @"Data Source=.\SQLEXPRESS;Integrated Security=true;Initial Catalog=Northwind";

//Create the database reader object.
var dbReader = new DatabaseReader(connectionString, providername);
//For Oracle, you should always specify the Owner (Schema).
//dbReader.Owner = "HR";

//Then load the schema (this will take a little time on moderate to large database structures)
var schema = dbReader.ReadAll();

//There are no datatables, and the structure is identical for all providers.
foreach (var table in schema.Tables)
{
  //do something with your model
}
  • .net (netStandard1.5, netStandard 2.0, net 3.1, net6, net7)
//In .net Core, create the connection with the connection string
using (var connection = new SqlConnection("Data Source=.\SQLEXPRESS;Integrated Security=true;Initial Catalog=Northwind"))
{
    var dbReader = new DatabaseSchemaReader.DatabaseReader(connection);
    //Then load the schema (this will take a little time on moderate to large database structures)
    var schema = dbReader.ReadAll();

    //The structure is identical for all providers (and the full framework).
    foreach (var table in schema.Tables)
    {
      //do something with your model
    }
}

UIs

There are two simple UIs (.net framework 4.8 only for now).

  • DatabaseSchemaViewer. It reads all the schema and displays it in a treeview. It also includes options for
  • code generation, table DDL and stored procedure generation.
  • comparing the schema to another database.
  • CopyToSQLite. It reads all the schema and creates a new SQLite database file with the same tables and data. If Sql Server CE 4.0 is detected, it can do the same for that database. These databases do not have the full range of data types as other databases, so creating tables may fail (e.g. SqlServer CE 4 does not have VARCHAR(MAX)). In addition, copying data may violate foreign key constraints (especially for identity primary keys) and will fail.

Building the Source

  • Use Visual Studio 2022 open DatabaseSchemaReader.sln (includes .net Core)
    • You can also use the command line "build.bat" (msbuild)
    • You cannot use the command line "dotnet build" because Core tooling cannot build v3.5 (see dotnet/msbuild#1333)

dbschemareader's People

Contributors

bernarden avatar danispir avatar dependabot[bot] avatar faddiv avatar jansivans avatar marcosmeli avatar martinjw avatar mysiteptyltd avatar oferns avatar ozzcelikk avatar patrice-dargenton-econocom avatar patricedargenton avatar shiningrise avatar sprotty avatar thomasjoscht avatar tonyliving avatar tostringtheory avatar xwiz avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

dbschemareader's Issues

Connection Name Property

I would like to see the addition of a Connection Name property added to the DatabaseSchema class. When dealing with multiple schemas, it would be nice to be able to apply a name. Also, the addition of a ReadOnly property for the connection name in all of the applicable entities (tables, views, storedprocedures, etc.)

[DatabaseSchema Class]
public string ConnectionName { get; set; }

[Entity Property]
public string ConnectionName => DatabaseSchema.ConnectionName;

DB2 Data Types

Hi All,
Is anyone of you facing issues with datatypes for the columns when we use DB2 Connection string ?

Seems to be fine with tables names and column name, although the data type is not working properlly.

I think that in Db2ISeriesSchemaReader.cs is missing the classes that suppose to return the data types !!

Any idea ??

Thanks in advance

from DatabaseColumn.DataType, can't get unsigned/signed for int to from DatabaseColumn.DataType, can't get unsigned/signed info for int

  1. No signed/unsigned info for DataType
    DbDataType: "mediumint(9) unsigned"

DataType:
{MEDIUMINT = System.Int32}
CreateFormat: "MEDIUMINT"
IsDateTime: false
IsFloat: false
IsInt: true
IsNumeric: true
IsString: false
IsStringClob: false
LiteralPrefix: null
LiteralSuffix: null
NetDataType: "System.Int32"
NetDataTypeCSharpName: "int"
ProviderDbType: 9
TypeName: "MEDIUMINT"

Change database name for connection; Read existing databases

Hi,

I'm struggling a little bit with the database name. I need to set it for a connection at runtime without adjusting connection string (because it is unknown/encrypted), f.e. after creation of a new database.

Currently set the database name for connection is only possible in connection string. In Ado the DbConnection method ChangeDatabase will be very useful. But I see no chance to come to this. The connection is normally created via Factory.CreateConnection and used in a using block, f.e. reading schema. There is no way to change database name in this methods.

My second request is to read all databases existing. I think this is also not possible because it will require specific db provider information.

Have you any suggestions?

Thx Thomas

SQL-Server - index columns order issue

SQL-Server - The retrieved columns's order seems to be incorrect - as a consequence, schema comparison often fail and causes index DROP/Re-CREATE.

May I suggest to perform a change in "DatabaseSchemaReader.ProviderSchemaReaders.Databases.SqlServer.Indexes" constructor:

change "ORDER BY t.name, ind.name, col.name" by "ORDER BY t.name, ind.name, ic.index_column_id"

Thanks,

Xavier

SQLITE on Mono

Hi, I want to use your library, mainly with SQLITE in Mono (OS X) but I don't know what provider to use, can u give me some clue?
I saw in your examples you used a library from devart but it seems is not open source.

Reader v.2.2.0 problem, not read table schema, exception fire

//In .net Core, create the connection with the connection string
using (var connection = new System.Data.SqlClient.SqlConnection("Data Source=.;Integrated Security=true;Initial Catalog=tmy; MultipleActiveResultSets=true;"))
{
var dr = new DatabaseSchemaReader.DatabaseReader(connection);
//Then load the schema (this will take a little time on moderate to large database structures)
dr.Owner = "dbo";
var schema = dr.ReadAll();
}

ReadAll error;

  •   dr.ReadAll()	'dr.ReadAll()' threw an exception of type 'System.IndexOutOfRangeException'	DatabaseSchemaReader.DataSchema.DatabaseSchema {System.IndexOutOfRangeException}
    

StacTRACES :
at System.Data.ProviderBase.BasicFieldNameLookup.GetOrdinal(String fieldName)
at System.Data.SqlClient.SqlDataReader.GetOrdinal(String name)
at System.Data.SqlClient.SqlDataReader.get_Item(String name)
at DatabaseSchemaReader.ProviderSchemaReaders.Converters.RowConverters.ColumnRowConverter.Convert(IDataRecord row)
at DatabaseSchemaReader.ProviderSchemaReaders.Databases.SqlServer.Columns.Mapper(IDataRecord record)
at DatabaseSchemaReader.ProviderSchemaReaders.Databases.SqlExecuter.ExecuteDbReader(DbConnection connection)
at DatabaseSchemaReader.ProviderSchemaReaders.Adapters.SqlServerAdapter.Columns(String tableName)
at DatabaseSchemaReader.ProviderSchemaReaders.Builders.TableBuilder.Execute(CancellationToken ct)
at DatabaseSchemaReader.DatabaseReader.AllTables(CancellationToken ct)
at DatabaseSchemaReader.DatabaseReader.ReadAll(CancellationToken ct)

SqlServer.ConstraintWriter has "primary key magic"

The SqlGen Code in SqlServer.ConstraintWriter.WritePrimaryKey mixes two completely unrelated, orthogonal concepts, namely the UNQIUEIDENTIFIER column type and the concept of CLUSTERED/NONCLUSTERED index. This is unfortunate because it creates wrong SQL when comparing schemas.
I suspect that a native support of CLUSTERED/NONCLUSTERED would be helpful, as not only SqlServer has These concepts (even though they are not as prominent with other DBs).

(As we use my fork of DbSchemaReader, this is not a high-prio item - rather an informal notice that this is a design that will probably break in many use cases).

H.M.

How to add custom Data Annotator

Hi, I'm currently trying to generate poco classes for my NPoco based projects.
Our DB ulgy namer. ie. MT_EQP_INFO for a table name, MT_EQP_ID for column name so far.
Removing 'MT_' part has been achieved with setting my customer INamer class instance to writer settings. Cool.
But unable to find any way to add some NPoco biased annotaion like following

[TableName("MT_EQP_INFO"), ...]
public partial class EqpInfo
{
  [ColumnName("MT_EQP_ID")]
  public EqpId { get; set;}
  // ...
}

How can I generate those attributed properties or classes using your nice tool?

Edited: I found DataAnnotaionWriter but it does not have any interface or injection mechanism. Any idea?

Not working with IBM.Data.DB2.iSeries

I have been trying to use this project with DB2 ISERIES but it is not working.

Running Db2Test i get the next error:

Result Message:
Test method DatabaseSchemaReaderTest.IntegrationTests.Db2.Db2Test threw exception:
System.InvalidOperationException: Not enough parameters specified. The command requires 4 parameter(s), but only 2 parameter(s) exist in the parameter collection.
Result StackTrace:
at IBM.Data.DB2.iSeries.iDB2Command.openCursor()
at IBM.Data.DB2.iSeries.iDB2Command.ExecuteDbDataReader(CommandBehavior behavior)
at System.Data.Common.DbCommand.System.Data.IDbCommand.ExecuteReader(CommandBehavior behavior)
at System.Data.Common.DbDataAdapter.FillInternal(DataSet dataset, DataTable[] datatables, Int32 startRecord, Int32 maxRecords, String srcTable, IDbCommand command, CommandBehavior behavior)
at System.Data.Common.DbDataAdapter.Fill(DataTable[] dataTables, Int32 startRecord, Int32 maxRecords, IDbCommand command, CommandBehavior behavior)
at System.Data.Common.DbDataAdapter.Fill(DataTable dataTable)
at DatabaseSchemaReader.ProviderSchemaReaders.Db2ISeriesSchemaReader.IdentityColumns(String tableName, DbConnection connection) in dbschemareader\DatabaseSchemaReader\ProviderSchemaReaders\DbProviders\Db2ISeriesSchemaReader.cs:line 78

Database first code generation

Hi, I think this project is what I've been looking for a long time now :)

First, sorry if my question is silly, but I would like to know if there is a plan to support database first code generation to generate DbContext code, because that's how we do for now :

  • all with EF Core 2.0
  • make changes to the DB schema an SQL Server (or Oracle)
  • generate code with Devart Entity Developer (which is great ! but expensive), it writes the DbContext and all
  • write Liquibase migrations (a lot by hand, some generated with the diff tool embedded with Liquibase)

I'll probably make an issue about Liquibase script generation, if that's necessary.

Parameter name is wrong

I find an issue in dbschemareader\DatabaseSchemaReader\ProviderSchemaReaders\Databases\SqlServer\Index.cs, In AddParameters method at Line 46, The AddDbParameter(command, "tableName", _tableName); should be AddDbParameter(command, "TableName", _tableName);

MySql Owner

The MySql example requires an owner.

How do you determine the owner? The database I'm using has never required me to have an owner to access it, and I'm not sure if it's the user in the connection string or something else.

PostgreSQL view columns always nullable

When reading a PostgreSQL database schema the returned view columns are marked as nullable always, even if the source table has them set to NOT NULL.

Otherwise this is a fantastic lib!

Add the option to specify Command Timeout

When using the library in a very big DB with thousands of tables and procedures, I'm getting timeout when doing a read all. Please add the ability to specify the Command Timeout option so that we have the ability to tweak as per the size of the DB.

Must declare the scalar variable "@tableName" Exception

The above error is thrown when reading tables, this is because the parameter name seems to be case sensitive.

Within ComputedColumns class you declare @tableName parameter within the Sql, but then add @TableName parameter within the AddParameters method.

Foreign Key pre-loading and performance

The SchemaConstraintLoader.cs constructor pre-loads the foreign key constraints passing a null tablename to the CheckConstraints method. Since a null tableName effectively makes the SQL command table agnostic, the constraints returned will be for all tables in the database. In a typical database that has foreign keys, this load happens once in the constructor call, and is loaded thereafter from the cached constraints (SchemaConstraintLoader.cs Line 69).

However the problem arises when a database has no foreign keys at all. When a database without foreign keys pre-loads the foreign keys, no results are returned. Thus, in the foreach loop spanning all database tables in the DatabaseReader.AllTables() method, loading foreign key constraints tries to load the constraints for every single table, which will always return no results. In the case where foreign key pre-loading returns no foreign key constraints, shouldn't the assumption be that no foreign keys exist?
if (_noFks) return null;

On a SQL Server database with 250 tables (and no foreign keys), this dramatically changed execution time:

Before
image

After
image

Function "return argument" not moved to return type

There is a call to CheckArgumentsForReturnType() missing in ProcedureBuilder.AddArguments(...). This leads to the effect that a freshly read schema still has the function "out parameter" in the arguments list.

The solution is to add the missing call at the end of the for lop over functions.

MigrationGenerator.AddColumn always adds default

The migration generator displays some strange behavior in that it always adds a default '' for string columns. There are many cases where this is not desirable behavior. It seems like the default should be left off completely if the default valueis null and include it if it evaluates to a value or string.empty. The constraints generated by this do not always get dropped correctly when a column is changed as well.

The follow code from the method demonstrates the issue.

      if (string.IsNullOrEmpty(databaseColumn.DefaultValue) && !databaseColumn.Nullable)
      {
        DataType dataType = databaseColumn.DataType;
        if (dataType == null || dataType.IsString)
          str += " DEFAULT ''";
        else if (dataType.IsNumeric)
          str += " DEFAULT 0";
        else if (dataType.IsDateTime)
          str += " DEFAULT CURRENT_TIMESTAMP";
        str = str.Replace(" NOT NULL ", " ") + " NOT NULL";
      }

All databases are readed, not the database specified in the connection string

  • Mysql 5.7 with a user named 'demo', and 'demo' has full rights to several different databases;
  • MySqlConnector
using (var connection = new MySql.Data.MySqlClient.MySqlConnection(
    "Data Source=127.0.1;Port=3306;Database=db1;User ID=demo;Password=demo;Allow User Variables=True;"))
{
    var dsr = new DatabaseReader(connection);
    var schema = dsr.ReadAll();
    // only tables of db1 are expeted to return here, but all tables that 'demo' are privileged are returned here.
}

Comparison Options

Would you guys be interested in a pull request with a bit more in the implementation of some comparison options? Eg, I'd like to be able to filter out database implementation specifics. Comparing a postgres database to a sql server one, I'm getting a lot of comparisons with differeing owners, or pgsql returning a ton of information_schema and pg_catalog entries. I was able to manually filter those differences pretty easily... however another change its suggesting is dropping a table and recreating because public owner vs dbo.

I'd be more than happy to investigate the changes, and if I have luck... I'll submit a pull request for it.

Oracle EF Provider

Hi Martin,

The PocoEntityCodeFirst classes generated from an Oracle database have some issues when we try to use them with the Oracle Entity Framework Provider. For example, a NUMBER(2) column is mapped to the decimal type in .NET, but the Oracle EF Provider fails on this mapping; it seems that it requires an Int16, as specified in this link.

Is it possible to get the generated classes to work with Oracle EF Provider or is it that we have to use the Devart EF Provider for Oracle?

Thanks,

Robert

firebird schema reading fails on version 2.3.0

schema reader runs sql containing
rfr.rdb$generator_name AS SEQUENCE
which is not found from rdb$relation_fields rfr.

sql:

SELECT rel.rdb$owner_name AS OWNER_NAME, rfr.rdb$relation_name AS TABLE_NAME, rfr.rdb$field_name AS COLUMN_NAME, fld.rdb$field_type AS FIELD_TYPE, CASE fld.rdb$field_type WHEN 261 THEN 'BLOB' WHEN 14 THEN 'CHAR' WHEN 40 THEN 'CSTRING' WHEN 11 THEN 'D_FLOAT' WHEN 27 THEN 'DOUBLE' WHEN 10 THEN 'FLOAT' WHEN 16 THEN 'INT64' WHEN 8 THEN 'INTEGER' WHEN 9 THEN 'QUAD' WHEN 7 THEN 'SMALLINT' WHEN 12 THEN 'DATE' WHEN 13 THEN 'TIME' WHEN 35 THEN 'TIMESTAMP' WHEN 37 THEN 'VARCHAR' ELSE '' END AS DATA_TYPE, fld.rdb$field_sub_type AS COLUMN_SUB_TYPE, CAST(fld.rdb$field_length AS integer) AS COLUMN_SIZE, CAST(fld.rdb$field_precision AS integer) AS NUMERIC_PRECISION, CAST(fld.rdb$field_scale AS integer) AS NUMERIC_SCALE, CAST(fld.rdb$character_length AS integer) AS CHARACTER_MAXIMUM_LENGTH, CAST(fld.rdb$field_length AS integer) AS CHARACTER_OCTET_LENGTH, rfr.rdb$field_position AS ORDINAL_POSITION, rfr.rdb$default_source AS COLUMN_DEFAULT, fld.rdb$computed_source AS COMPUTED_SOURCE, coalesce(fld.rdb$null_flag, rfr.rdb$null_flag) AS IS_NULLABLE, rfr.rdb$description AS DESCRIPTION, rfr.rdb$generator_name AS SEQUENCE FROM rdb$relation_fields rfr LEFT JOIN rdb$relations rel ON rfr.rdb$relation_name = rel.rdb$relation_name LEFT JOIN rdb$fields fld ON rfr.rdb$field_source = fld.rdb$field_name WHERE rfr.rdb$system_flag = 0 AND (@Owner IS NULL OR @Owner = rel.rdb$owner_name) AND (@TABLE_NAME IS NULL OR @TABLE_NAME = rfr.rdb$relation_name) ORDER BY rfr.rdb$relation_name, rfr.rdb$field_position

Performance lack retrieving table information of large database

Hey Guys, I have a simple use case loading table information including table names, column names and data types of one database. One of my target databases is a PostgreSql with 900+ tables and 10k+ columns. I've tried:

  • Load information with DatabaseReader.ReadAll() without owner. This will take 3 minutes and more.
  • Load information with DatabaseReader.ReadAll() with owner. This will take up to 1 minute.
  • Load information with DatabaseReader.AllTables(). This will take up to 1 minute
  • Extending lib with custom method: Mainly based on ReaderAdapter.Tables(null), ReaderAdapater.Columns(null) and DatabaseReader.DataTypes(). This will take only few (3 - 5) seconds!

Does a general peformance lack exist in large databases? Have you any other suggestion for retrieving my necessary information? I could create a pull request if no other solution exists.

firebird 2.x problem

FirebirdSql.Data.FirebirdClient.FbException occurred
HResult=0x80004005
Message=Dynamic SQL Error
SQL error code = -206
Column unknown
FLD.RDB$OWNER_NAME
At line 34, column 27

Sql: SELECT
fld.rdb$owner_name AS OWNER,
pp.rdb$procedure_name AS PROCEDURE_NAME,
pp.rdb$parameter_name AS PARAMETER_NAME,
fld.rdb$field_type AS FIELD_TYPE,
CASE fld.rdb$field_type
WHEN 261 THEN 'BLOB'
WHEN 14 THEN 'CHAR'
WHEN 40 THEN 'CSTRING'
WHEN 11 THEN 'D_FLOAT'
WHEN 27 THEN 'DOUBLE'
WHEN 10 THEN 'FLOAT'
WHEN 16 THEN 'INT64'
WHEN 8 THEN 'INTEGER'
WHEN 9 THEN 'QUAD'
WHEN 7 THEN 'SMALLINT'
WHEN 12 THEN 'DATE'
WHEN 13 THEN 'TIME'
WHEN 35 THEN 'TIMESTAMP'
WHEN 37 THEN 'VARCHAR'
ELSE ''
END AS DATA_TYPE,
fld.rdb$field_sub_type AS PARAMETER_SUB_TYPE,
pp.rdb$parameter_number AS ORDINAL_POSITION,
CAST(pp.rdb$parameter_type AS integer) AS PARAMETER_DIRECTION,
CAST(fld.rdb$field_precision AS integer) AS NUMERIC_PRECISION,
CAST(fld.rdb$field_scale AS integer) AS NUMERIC_SCALE,
CAST(fld.rdb$character_length AS integer) AS CHARACTER_MAX_LENGTH,
pp.rdb$description AS DESCRIPTION
FROM rdb$procedure_parameters pp
LEFT JOIN rdb$fields fld ON pp.rdb$field_source = fld.rdb$field_name
WHERE
pp.rdb$system_flag = 0 AND
(@owner IS NULL OR @owner = fld.rdb$owner_name)
ORDER BY pp.rdb$procedure_name, pp.rdb$parameter_type, pp.rdb$parameter_number

More support for DB-specific schema information; especially SQL Server

I have forked your nice project because we might replace a Redgate SchemaCompare with it (for various reasons). However, for this, we need quite a few more attributes for SQL-Server specific Information.

What I do right now in my fork:

  • I have written a (IMHO) simpler comparer, which OTOH can filter out arbitrary objects based on names; see here for the first rough implementation: DatabaseSchemaComparer.txt
  • I have added a class DatabaseStatistics for SQL server's STATISTICS objects;
  • I plan to add a generic feature (probably a Dictionary<string, object>) for "db specific attributes"; the comparer would then get some string API to select which of this attributes should also be compared.

Questions: Are you interested in this development (of course, you can always look in my fork); and do you or anyone else here have ideas and/or already implementations of such additions?

Harald M.

POCO column name error when column is both primary and foreign key

I am using the code generator to create POCO classes for my database schema. I have a Department table with a composite primary key BusinessUnitId + DepartmentId, The BusinessUnitId is also a foreign key to the BusinessUnit table. When the POCO class for Department is created, the Id suffix is stripped from BusinessUnitId (by the Namer.Name() function) and this creates two members with the same name (BusinessUnit) in the Department class. I guess the Id should not be removed when a column is also a primary key. What do you think?

ForeignKeyColumnName as Column property

In the column properties we have ForeignKeyTableName which very useful information, I think it would be even better if we can have ForeignKeyColumnName as well, at the moment I actually run a separate query just to retrieve that information.
Thanks.

CodeGen.CodeWriter.Execute() fails when executed as part of NuGet package

Using the DatabaseSchemaReader NuGet package 2.1.1, I read a schema and tried to use the code writer but got a null reference exception in the ProjectWriter constructor. Instead of using the NuGet package, I downloaded the source code of version 2.1.1 in an attempt to locate the error. However, using the source code, the code writer works perfectly! I think the problem with the NuGet package has something to do with the LoadProjectXml() function of the ProjectWriter class. I suspect that the GetManifestResourceStream() function returns null, and the ProjectWriter constructor does not check whether the return XDocument is null.

I am not sure what the fix would be. It looks like a security issue when trying to retrieve the embedded resource.

Oracle.ManagedDataAccess.Client errors when performing DatabaseReader.ReadAllSchema

It appears that this the library has issues when reading schema using the oracle managed driver.

https://www.nuget.org/packages/Oracle.ManagedDataAccess/

The following code snippet results in an exception.

var connString = @"Data Source=(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.137.10)(PORT=1521)))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=XE)));User Id=MySchemaUser;Password=MyPass;";
var providerName = "Oracle.ManagedDataAccess.Client";

using (var schemaReader = new DatabaseReader(connString, providerName))
{
    schemaReader.ReadAll();
}

Message
ORA-01008: not all variables bound
Stack Trace
at OracleInternal.ServiceObjects.OracleCommandImpl.VerifyExecution(OracleConnectionImpl connectionImpl, Int32& cursorId, Boolean bThrowArrayBindRelatedErrors, OracleException& exceptionForArrayBindDML, Boolean& hasMoreRowsInDB, Boolean bFirstIterationDone)
at OracleInternal.ServiceObjects.OracleCommandImpl.ExecuteReader(String commandText, OracleParameterCollection paramColl, CommandType commandType, OracleConnectionImpl connectionImpl, OracleDataReaderImpl& rdrImpl, Int32 longFetchSize, Int64 clientInitialLOBFS, OracleDependencyImpl orclDependencyImpl, Int64[] scnForExecution, Int64[]& scnFromExecution, OracleParameterCollection& bindByPositionParamColl, Boolean& bBindParamPresent, Int64& internalInitialLOBFS, OracleException& exceptionForArrayBindDML, Boolean isDescribeOnly, Boolean isFromEF)
at Oracle.ManagedDataAccess.Client.OracleCommand.ExecuteReader(Boolean requery, Boolean fillRequest, CommandBehavior behavior)
at Oracle.ManagedDataAccess.Client.OracleCommand.ExecuteDbDataReader(CommandBehavior behavior)
at System.Data.Common.DbCommand.ExecuteReader()
at DatabaseSchemaReader.ProviderSchemaReaders.Databases.SqlExecuter.ExecuteDbReader(DbConnection connection)
at DatabaseSchemaReader.ProviderSchemaReaders.Adapters.OracleAdapter.Tables(String tableName)
at DatabaseSchemaReader.ProviderSchemaReaders.Builders.TableBuilder.Execute(CancellationToken ct)
at DatabaseSchemaReader.DatabaseReader.AllTables(CancellationToken ct)
at DatabaseSchemaReader.DatabaseReader.ReadAll(CancellationToken ct)
at DatabaseSchemaReader.DatabaseReader.ReadAll()
at UserQuery
at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Threading.ThreadHelper.ThreadStart()

oracle How can I set it as int32

hello!
I come from China.
my db is oracle
I want the DataType.NetDataTypeCSharpName to be int32
How do I design field types and lengths?
my email kw88@qqcom
I await your reply.
Thank you very much for your work

OraOleDb.Oracle Not Working

Reader.AllTables() results in an "invalid cast" exception when I create the reader using System.Data.OleDb.OleDbConnection with the OraOleDb.Oracle provider.

Suggestion: add option to reverse Filter exclusion logic to inclusion

I was wondering whether it would be possible to add a bool option to the Filter class to reverse the exclude logic; this would effectively gives us an inclusion filter rather than an exclusion filter. A client would be able to select for each of the 4 Exclusions filters, whether to use exclusion or exclusion. The default would be exclusion. What do you think?

By the way, this is a great package!

Length of PostgreSql ARRAY data types

I'm trying to determine the length of ARRAY data types like "character(30)[]" or "character varying(30)[]" in postgresql. The field character_maximum_length is for this fields NULL. After my investigation this should be not possible with information_schema. The only script I found is following:

SELECT col_attr.attname AS "ColumnName" 
   ,pg_catalog.format_type(col_attr.atttypid, col_attr.atttypmod) AS "DataType"
FROM pg_catalog.pg_attribute col_attr
WHERE col_attr.attnum > 0 AND NOT col_attr.attisdropped AND col_attr.attrelid = (
   SELECT cls.oid
   FROM pg_catalog.pg_class cls
   LEFT JOIN pg_catalog.pg_namespace ns ON ns.oid = cls.relnamespace
   WHERE cls.relname = 'aa');

Could you confirm that this isn't possible only using information_schema? Have you any suggestion to solve this "easily" with an extension of lib?

MigrationGenerator does not output default values for columns.

It appears the migration generator for Postgres Sql does not output any default column information. The sample below demonstrates the issue. It seems like DefaultValue should be supported since it has been available in Postgres Sql since version 9.1.

Code Sample

var gen = new DdlGeneratorFactory(SqlType.PostgreSql).MigrationGenerator();
gen.IncludeSchema = false;

var pkSeqName = "Seq_PK_Generator";
var pkSeq = new DatabaseSequence()
{
    Name = pkSeqName,
    MinimumValue = 1,
    IncrementBy = 1,


};

gen.AddSequence(pkSeq).Replace(";", " CACHE;").Dump();

string.Empty.Dump();

var newTable = new DatabaseTable { Name = "TestTable" };

var idColumn = newTable.AddColumn("Id", DbType.Int64);
idColumn.AddPrimaryKey("PK_TestTable");
idColumn.DefaultValue = $"nextval('{pkSeqName}')";
var summaryColumn = newTable.AddColumn("Summary", DbType.String);
summaryColumn.Length = 100;

gen.AddTable(newTable).Dump();

Output

CREATE SEQUENCE "Seq_PK_Generator" INCREMENT BY 1 MINVALUE 1 CACHE;

CREATE TABLE "TestTable"
(
  "Id" BIGINT NOT NULL, --default value missing
  "Summary" VARCHAR (100)  NOT NULL
);
ALTER TABLE "TestTable" ADD CONSTRAINT "PK_TestTable" PRIMARY KEY ("Id");

Triggers on Views are overlooked

We repaired this in my branch by adding the following code at the end of ViewBuilder.Execute

        var triggers = _readerAdapter.Triggers(null);

        foreach (var view in views) {
            UpdateTriggers(view, triggers);
        }

and a new method at the end

    private void UpdateTriggers(DatabaseView view, IList<DatabaseTrigger> triggers) {
        var viewTriggers = triggers.Where(x => x.SchemaOwner == view.SchemaOwner &&
                                            x.TableName == view.Name);
        view.Triggers.Clear();
        view.Triggers.AddRange(viewTriggers);
    }

}

Moreover, in the SQL for SQL Server in Databases.SqlServer.Triggers, we replaced "sys.tables" with "sys.objects".

Functions and SPs not correctly read on (some?) SQL-Server 2008 instances

This is due to the fact that

ObjectProperty (Object_Id (INFORMATION_SCHEMA.ROUTINE_NAME), 'IsMSShipped')

can return NULL to signal "not MS shipped". In both DatabaseSchemaReader/ProviderSchemaReaders/Databases/SqlServer/Functions.cs and DatabaseSchemaReader/ProviderSchemaReaders/Databases/SqlServer/StoredProcedures.cs, this value is directly compared with = 0, which excludes all functions and procedures from the result.

A solution is to wrap the expression with ISNULL, so that the comparison is

ISNULL(ObjectProperty (Object_Id (INFORMATION_SCHEMA.ROUTINE_NAME), 'IsMSShipped'), 0) = 0

CoreFX implications

CoreFX won't have DataTables, but they are replacing DbConnection.GetSchema which is the core of dbSchemaReader.

CoreFX issue: https://github.com/dotnet/corefx/issues/3423
and https://github.com/dotnet/corefx/issues/5024

The current proposal looks like our data schema model, which is promising. This library could then depend on the CoreFx model and schema reading (a more solid base than the existing GetSchema datatables).

We'll see what the final version looks like before deciding whether we wrap it with our model, or extend their model. Whichever, this library could potentially be a good bridge to support both older full framework and CoreFx.

Not looking forward to the conditional compilation directives though.

MigrationGenerator does not generate correct oracle data type for int64.

The oracle migration generator does not appear to map long integer data types (Int64) correctly. In the example below an Int64 is being mapped to non existent oracle type bigint. It should be NUMBER(19) according to http://docs.oracle.com/cd/B19306_01/gateways.102/b14270/apa.htm

var newTable = new DatabaseTable { Name = "TestTable" };

var idColumn = newTable.AddColumn("Id", DbType.Int64);
idColumn.AddPrimaryKey("PK_TestTable");
//idColumn.DefaultValue = $"next value for {pkSeqName}";

var summaryColumn = newTable.AddColumn("Summary", DbType.String);
summaryColumn.Length = 100;

gen.AddTable(newTable)

--script output
CREATE TABLE "TestTable"
(
"Id" BIGINT NOT NULL,
"Summary" NVARCHAR2 (100) NOT NULL
);
ALTER TABLE "TestTable" ADD CONSTRAINT "PK_TestTable" PRIMARY KEY ("Id");

Included Non-Key Columns in SQL Server Index

Is there any way to see “included” columns in a SQL Server index? For example, to see “nonKeyCol1” and “nonKeyCol2” in the below example. Also, is there any way to see the “with” options?

CREATE NONCLUSTERED INDEX [idx_myIndex] on [myTable]
(
[aKeyCol] ASC
) INCLUDE ([nonKeyCol1], [nonKeyCol2]) WITH (Pad_Index = OFF, SORT_IN_TEMPDB = ON) ON “default”

Thanks!
Yl

Trigger schema not read on SQL server 2008

At least on SQL Server 2008, the SQL for reading triggers always returns NULL both as TRIGGER_SCHEMA and TABLE_SCHEMA. In my branch, I have therefore replaced the SQL with the more straightforward (as it goes to sys.triggers and sys.tables instead of sysobjects)

`SELECT
tr.name AS TRIGGER_NAME,
SCHEMA_NAME(parent.schema_id) AS TRIGGER_SCHEMA,
SCHEMA_NAME(parent.schema_id) AS TABLE_SCHEMA,
parent.name AS TABLE_NAME,
OBJECTPROPERTY(tr.object_id, 'ExecIsUpdateTrigger') AS IS_UPDATE,
OBJECTPROPERTY(tr.object_id, 'ExecIsDeleteTrigger') AS IS_DELETE,
OBJECTPROPERTY(tr.object_id, 'ExecIsInsertTrigger') AS IS_INSERT,
OBJECTPROPERTY(tr.object_id, 'ExecIsAfterTrigger') AS IS_AFTER,
tr.is_instead_of_trigger AS IS_INSTEADOF,
tr.is_disabled AS IS_DISABLED,
OBJECT_DEFINITION(tr.object_id) AS TRIGGER_BODY

FROM sys.triggers AS tr
INNER JOIN sys.tables AS parent
ON tr.parent_id = parent.object_id

WHERE (SCHEMA_NAME(parent.schema_id) = @owner or (@owner is null))
AND (parent.name = @TABLE_NAME or (@TABLE_NAME is null)) `

... which works.

SYSDATETIME() is changed to current_timestamptime()

Hi Martin,

When generating a table sql script (MS SQL), default column value of SYSDATETIME() is changed to current_timestamptime(), which does not execute.

CREATE TABLE [dbo].[tmpTbl]
(
  [Update] DATETIME2(3)  NOT NULL DEFAULT (SYSDATETIME()),
);

so when I try to generate the script for the table above, I get this:

CREATE TABLE [dbo].[tmpTbl]
(
  [Update] DATETIME2(3)  NOT NULL DEFAULT (current_timestamptime()),
);

Thanks

PostgreSql bpchar data type

Hi Martin,
I am using PostgreSql 9.5 and Npsql 3.0.4.0.
When a column with data type "char" , GetSchema("Columns") returns char for data_type. On the other hand a column with data type character is returned as bpchar.

If I add bpchar to PostgreSqlSchemaReader.SchemaDataTypes
list.Add(new DataType("bpchar", typeof(string).FullName));
It seems to work as expected. I'm new to PostgreSql, but character seems like a common data_type, was there a reason this was not part of PostgreSqlSchemaReader.SchemaDataTypes?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.