cmeeren / facil Goto Github PK

Hi,

I have a case where I have two stored procedures that access the same temp table (i.e. the table has the same structure and the same name). What is the best way to configure facil to have only one type generated in F# for the table record?

Here is the issue: I have an F# function that takes an input and it produces a list of temp table records. Facil generates two types for this temp table, one under each stored procedure. I really need one type. I was thinking that I could use a generic type with member constraints, but if the tables are big it is a pain.

Other suggestions?

Thank you!

Hi again,

I have a requirement to process a bunch of sql tables, and I have to do it in a transaction.

Can you please add a static method WithConnection(cn: SqlConnection, tran: SqlTransaction) in the generated code to control the transaction?

Right now, the only way - that I can see - to control the transaction is to use ConfigureCommand or to set the userConfigureCommand property.

You have this code:

              DbGen.Procedures.dbo.ProcInsert
                 .WithConnection(conn)
                 .ConfigureCommand(fun cmd -> cmd.Transaction <- tran)

If you have one statement, that's fine, but to specify this over and over in different parts of the code, it's getting annoying.
The simplest thing to do would be for Facil to provide an override where I can pass the transaction (imo).

What do you think?

I hope I am not missing something, but if you have other suggestions please let me know.

Also, these guys provide such on override: http://fsprojects.github.io/FSharp.Data.SqlClient/transactions.html.

    use tran = conn.BeginTransaction()
    
    //Implicit assumption that
    assert (tran.Connection = conn)
    //Supply connection and transaction 
    use cmd = new InsertCurrencyRate(conn, **tran**)

I have an SP which expects a string as a parameter, but this parameter may be null:

let parameters = {| Foo = 123; OptionalString = Unchecked.defaultof<string> |}
let res = DbGen.Procedures.Schema.MySp.WithConnection(conn).WithParameters(parameters).Execute()

This goes pop:

Procedure or function 'MySp' expects parameter '@optionalString', which was not supplied.

I've tried e.g. null :> string and other things but it just gets ignored. If I supply a string, the SP correctly fires.

Hi,

I am not too sure this doable, but I will put it out there. I have a stored procedure that updates records based on some condition, it puts the modified records in a temp table using an output clause, then it returns the data from the temp table.

The problem is that when the generator runs it gets the columns information from the stored procedure, but it also executes the stored procedure, and it commits the data. Is it possible to place the operation that gets the columns information in a transaction, get the structure of the resultset, then rollback then entire thing? Otherwise it is a bit annoying because I have to reset that data to undo the operation and to go back to the previous state.

Thank you

It seems (just from my experimenting) that you must explicitly ignore views when auto-generating Insert code. I would've thought that this could be automatically done by the tool?

Hello,

Sometimes, we have to deal with existing databases that have been developed by other people/projects. In general, more likely there is inconsistency between naming conventions. Some people use pascal case, other use a c style naming conventions (first_name, last_name). It would be nice if Facil offers the option of specifying a naming conversion scheme when generating the F# records. It would be best if the naming convention can be specified, per connection, per sets of objects selected through the include/exclude options down to the column.

Possible options: AsIs, PascalCase, CStyle, CamelCase ( not sure about this one).
Example:
Database: first_name
AsIs: first_name
PascalCase: FirstName
CStyle: first_name
CamelCase: firstName

I am not sure if we need to go as far as specifying Plural or Singular.

In addition to this, it might be nice to offer the option of overriding the name of a record regardless of the conversion scheme.

What do you think?

Is there an environment variable that we can use to conditionally stop regeneration (opposite of FORCE REGENERATE I suppose)? We're in a position where our build project references the "main" project that contains the Facil generated code etc., and under certain conditions we end up stuck where the build project can't run because the main project needs to build itself first, but that fails because Facil tries to connect to the DB to regenerate itself - but fails because the database doesn't exist, because the build process hasn't yet run...

While trying to run against a project I get the following error:

Facil : error : System.Exception: Error getting table DTOs
 ---> System.Exception: Unknown SQL system type ID: 240
   at Microsoft.FSharp.Core.PrintfModule.PrintFormatToStringThenFail@1433.Invoke(String message) in F:\workspace\_work\1\s\src\fsharp\FSharp.Core\printf.fs:line 1433
   at [email protected](Unit unitVar0) in C:\Dev\Facil\src\Facil.Generator\Db.fs:line 496
   at Facil.Db.getTableDtos(FSharpMap`2 sysTypeIdLookup, SqlConnection conn) in C:\Dev\Facil\src\Facil.Generator\Db.fs:line 490
   --- End of inner exception stack trace ---
   at Facil.Db.getTableDtos(FSharpMap`2 sysTypeIdLookup, SqlConnection conn) in C:\Dev\Facil\src\Facil.Generator\Db.fs:line 527
   at Facil.Db.getEverything(RuleSet cfg, String fullYamlPath, FSharpList`1 scriptsWithoutParamsOrResultSetsOrTempTables, SqlConnection conn) in C:\Dev\Facil\src\Facil.Generator\Db.fs:line 694
   at [email protected](Unit unitVar0) in C:\Dev\Facil\src\Facil.Generator\Program.fs:line 90
   at Facil.Program.main(String[] argv) in C:\Dev\Facil\src\Facil.Generator\Program.fs:line 125

I have a feeling it's the sql spatial type, this project uses the geography::Point type.

Facil currently generates anonymous records for the results from scripts. Consider changing these to "full" records, or at the least aliasing the anonymous record and making the alias public so that it can be utilised elsewhere e.g. in mapping code.

I’m having a problem migration away from the SqlClient type provider to Facil - Our project is so big the switch over to Facil can’t be done in one hit while also keeping up with normal development.

We have mutable databases feeding into mutable projects then mutable apps so changing one database access layer has a knock-on effect.

I was hoping to mix Facil with the SqlClient but as we have to compile our project with FSharp.Compiler.Tools to get around the Type Provider problems it means the anonymous records Facil creates break the build.

error FS0010: Unexpected symbol '|' in expression
error FS0604: Unmatched '{'

I know FSharp.Compiler.Tools is a dead project and everyone should switch away from it but in a production this is easier said than done.

I see we can use Result type to tell Facil not to create the anonymous records but adding all the types for all the queries is a bit too much.

I’m happy to do the work and add an option to the Result type which creates a new create record type but before I do I just wanted to make sure you would be happy with this as I know at this moment in time (the early stages of development) you would rather have complete control of code and any changes especially big ones.

At the moment our migration is dead in the water because we had to branch our whole project which has quickly become behind with ongoing development. This is in no way Facil fault it's a great tool just we can't use it because of our setup. As the type providers are widely used this addition could help others projects move to Facil.

After updating to version 2.3.1, it started raising an error when trying to generate DbGen file, saying:
facil.yaml(0 , 0) : error : The variable $(Db) could not be found in the specified configuration sources

Going back to version 2.3.0 works as expected.

In my facil.yaml file I have this configuration:

rulesets:
  - connectionString: $(Db)

Hello,

I have the following:

    scripts:
      - include: "*.sql"
      - for: TempJobTemplate.sql
        tempTables:
          - definition: CreateTable/TempJobTemplate.sql
      

In the CreateTable/TempJobTemplate.sql script I have the create table statement followed by a create index statement (I did not use a "go" between the statements, I ended the create table statement with semicolon). The create index statement is removed from the script generated code, even though I want it there. Is it normal? What can I do to tell Facil to leave it there?

Thanks

Edited: I added the folder the name to the script name.

Using the standard AdventureWorks sample DB (I think I got it from here), trying to run Facil gives the following error:

   ---> System.Exception: Unsupported SQL system type ID '240' for column 'OrganizationNode' in table 'Employee'
     at [email protected](Unit unitVar0) in C:\projects\facil\src\Facil.Generator\Db.fs:line 736
     at Facil.Db.getTableDtos(RuleSet cfg, FSharpMap`2 sysTypeIdLookup, FSharpMap`2 primaryKeyColumnNamesByTable, SqlConnection conn) in C:\projects\facil\src\Facil.Generator\Db.fs:line 730
     --- End of inner exception stack trace ---
     at Facil.Db.getTableDtos(RuleSet cfg, FSharpMap`2 sysTypeIdLookup, FSharpMap`2 primaryKeyColumnNamesByTable, SqlConnection conn) in C:\projects\facil\src\Facil.Generator\Db.fs:line 799
     at Facil.Db.getEverything(RuleSet cfg, String fullYamlPath, FSharpList`1 scriptsWithoutParamsOrResultSetsOrTempTables, SqlConnection conn) in C:\projects\facil\src\Facil.Generator\Db.fs:line 856
     at Facil.Program.regenerate@108(String yamlFilePath, RuleSet cfg, FSharpList`1 scriptsWithoutParamsOrResultSetsOrTempTables, String hash, String outFile, Unit unitVar0) in C:\projects\facil\src\Facil.Generator\Program.fs:line 117
     at Facil.Program.main(String[] argv) in C:\projects\facil\src\Facil.Generator\Program.fs:line 170

This appears to be a SQL type called HierarchyId.

Perhaps from a practical point of view, it would be best to have Facil simply warn and not generate tables that contains types that it does not support, but continue with the remaining ones? I'm thinking people would still benefit even if e.g. 90% of their DB could be generated and 10% had to be hand-rolled.

Hello!

After adding support for diagrams in my DB, Facil included related types and SPs in the generated file.
At a first glance this is not a problem by itself, but it produced an unexpected behavior, my code started raising a System.AccessViolationException.

The culprit for raising this exception are one or many of these:
type sysdiagrams
type sp_alterdiagram_Executable
type sp_alterdiagram
type sp_creatediagram_Executable
type sp_creatediagram
type sp_dropdiagram_Executable
type sp_dropdiagram
type sp_helpdiagramdefinition_Executable
type sp_helpdiagramdefinition
type sp_helpdiagrams_Executable
type sp_helpdiagrams
type sp_renamediagram_Executable
type sp_renamediagram
type sp_upgraddiagrams

As a workaround, we excluded these DB elements using "except" mechanism (facil.yaml) provided by Facil.

Kind regards,
Mauricio

Hi!

First of all: Sorry, there is no bug or problem here, I just didn't a better way to contact you. So feel free to just delete this entry.

We are working on the business app and had it checked by Isaac (Abraham) and his team. They advised us to check out Facil.
So far we are working with Dapper.. but with F#... well it has it's issues. So here I am.

I played around with Facil for a couple of days and like it lot. It work's like a charm.

But there is one issue that really is an obstacle for us. Of course our database has many id-columns, all of them Guids. To avoid mixing them, Isaac proposed to encapsulate them in single case union types.

type DocumentId = DocumentId of Guid
    with member this.Value = match this with DokumentOcrId id -> id

So in turn our DTOs look like this

type Document= { Id: DocumentId 
                              Date: DateTime
                               Title: string}

Maybe I overlooked something but I didn't find a way to make Facil read data into that type. The problem is that DocumentId isn't a guid - it just encapsulated a guid.

Is there a simple way to do so?

I pondered on this some time. From a C# perspective I thought "Well.. if F# types had constructors, I could define a constructor that takes a guid as first parameter and return a nice Document-type". Unfortunately F# types don't have such constructors.

But maybe it is possible to use the result-option likewise. The idea is to configure Facil in way that is hands each row, that was read from the DB, to a factory-like function that converts it to the desired type.

...
- for: whatever.fs
      result : documentBridgeFunction
...

and in the code (handwritten, not generated)

let documentBridgeFunction id date title =
    { Id = DocumentId id
       Date = date
       Title = title }

Do you think, that's possible? Or is it just a bad idea? Or have I missed a point and there is already a way to achieve what we need?

Thanks in advance and best regards,
Wolfram Bernhardt

At the moment if the case is incorrect within the facil.yaml, generation might not work. I ran into this with the following config:

rulesets:
  - connectionString: Data Source=.;Initial Catalog=Db;Integrated Security=True

    scripts:
      - include: "**/*.sql"
        except: Sql/ReadingsMergeTemp.sql

      - for: Sql/ReadingsMerge.sql
        tempTables: 
          - definition: "SQL/ReadingsMergeTemp.sql"
		  

tempTables/definition Sql is upper case so wasn't found.

Hello @cmeeren

As it happened sometime ago, Microsoft has broken (or at least changed) the reading of environment variables with version 6.0.1 of:
open Microsoft.Extensions.Configuration
open Microsoft.Extensions.Configuration.EnvironmentVariables

Do you mind keeping these references to 6.0.0 until it is fixed?

Kind regards,
Mauricio

Hello,

Is there a way to only load the data in the temp tables without calling anything else?

I want to load a bunch of temp tables using facil, then I want to call a stored procedure that processes the data from the temp tables. How do I do that without making extra calls?

I browsed through the source code and the LoadTempTables*** methods are private.

One option would be to use a query such as select * from #temptable where 1 = 0 but it's redundant.

One other thing I observed, is that that the SqlBulkCopy behaviour is controlled through constructor parameters for which there are no available properties to be set. I would have been more flexible to allow the user to create and pass the SqlBulkCopy object.

Thanks

Hello,

Sometimes there are small tables that I want to load in the memory. Is there a way to generate the type that selects all records from the table, something like - type: select.

Thanks

Today when we use SQL "Date" type, facil generates DateTime type parameter.
With NET6, now we can use DateOnly type.

I didn't find if that type can be overwritten on yaml settings.

Hi again,

Sorry, I wish that I didn't have to create these issues and that everything worked fine.

What I am trying to do? Within a sql transaction:

• Get rows to process <- this has to happen within the transaction
• Process rows and gather data in memory
• Save data in temp tables
• Call stored procedure to merge the data from the temp tables into the real tables

The problem that I encountered is when I save the data to the temp tables:

System.InvalidOperationException: ExecuteNonQuery requires the command to have a transaction when the connection assigned to the command is in a pending local transaction.  The Transaction property of the command has not been initialized.
at Microsoft.Data.SqlClient.SqlCommand.ValidateCommand(Boolean isAsync, String method)
 at Microsoft.Data.SqlClient.SqlCommand.InternalExecuteNonQuery(TaskCompletionSource`1 completion, Boolean sendToPipe, Int32 timeout, Boolean& usedCache, Boolean asyncWrite, Boolean inRetry, String methodName)
 at Microsoft.Data.SqlClient.SqlCommand.ExecuteNonQuery()
 at Facil.Runtime.CSharp.GeneratedCodeUtils.LoadTempTables(SqlConnection conn, IEnumerable`1 tempTableData) in C:\projects\facil\src\Facil.Runtime.CSharp\GeneratedCodeUtils.cs:line 48
 at Facil.Runtime.CSharp.GeneratedCodeUtils.ExecuteQueryEager[T](SqlConnection existingConn, String connStr, Action`1 configureNewConn, Action`1 configureCmd, Action`1 initOrdinals, Func`2 getItem, IEnumerable`1 tempTableData) in C:\projects\fac
il\src\Facil.Runtime.CSharp\GeneratedCodeUtils.cs:line 150

Basically this code ends up being called and it blows up when it tries to execute cmd.ExecuteNonQuery()

        private static void LoadTempTables(SqlConnection conn, IEnumerable<TempTableData> tempTableData)
        {
            foreach (var data in tempTableData)
            {
                using var cmd = conn.CreateCommand();
                // Note: If there is ever a need for letting users configure the command,
                // do not use the configureCmd parameter passed to methods on this class,
                // which also sets any parameters.
                cmd.CommandText = data.Definition;
                cmd.ExecuteNonQuery();

                using var bulkCopy = new SqlBulkCopy(conn) { DestinationTableName = data.DestinationTableName };
                data.ConfigureBulkCopy(bulkCopy);
                var reader = new TempTableLoader(data.NumFields, data.Data);
                bulkCopy.WriteToServer(reader);
            }
        }

Any idea on how to workaround this? I cannot move the processing of the data and the saving of the processed data into the temp tables outside the transaction.

Thanks

Hi,

Because I am stuck to .net 5 for now, I am using Facil 1.4.3. How do I handle multiple databases? According to the docs, I can have multiple yaml files. I created two yaml files facil_db1.yaml, facil_db2.yaml, but the build process complains:

Error No config file found. A minimal config file has been placed in the project directory (D:\TestDbAccess\facil.yaml). Re-build after editing the config. TestDbAccess D:\TestDbAccess\Facil 1

Also, it seems that there is a build hook that calls the facil generator, but I am not sure how I can override that command in case it supports specifying the yaml file name.

Thank you

Update: One option would be to create two library projects, one for each database and then reference these.

Sometimes using sp_describe_first_result_set stored procedure doesn't return accurate results, and/or the structure (or the absence of it) of the resultset returned by a stored procedure cannot be inferred.

It would be nice if the library allows the developer to specify in the configuration yaml file a test sql statement (execute spTest 'validParameterValue') that facil can use as a last resort in the determining the structure of the sql output from a stored procedure or sql script. This avoids hitting branches in the stored procedure execution that return errors, due for instance, to parameter checking, and it allows the developer to override the default parameter values used by facil to discover the structure of the data.

Thanks

No concrete plans exist to support these types at the moment. This is just an issue to track dependency on dotnet/SqlClient#30.

Hi,

I upgraded to Facil 2.2.0 - thank you for making all the changes.
However, now, when I run my console app I get this exception:

Unhandled exception. System.IO.FileNotFoundException: Could not load file or assembly 'Facil.Runtime.CSharp, Version=2.2.0.0, Culture=neutral, PublicKeyToken=null'. The system cannot find the file specified.
File name: 'Facil.Runtime.CSharp, Version=2.2.0.0, Culture=neutral, PublicKeyToken=null'

The version that is in the bin folder is 2.1.2. Any idea on how to fix this? Or is it something that you have to fix?

Thank you

I found this while working with a very, very wide table with lots of nullable columns. The code gen work fine however when it runs you get a stack overflow in the getItem:

Stack overflow.... 1872/2038 -
   at DbGen+Scripts+MenyColumns.getItem(Microsoft.Data.SqlClient.SqlDataReader)
   at [email protected](Microsoft.Data.SqlClient.SqlDataReader)
   at [email protected](Microsoft.Data.SqlClient.SqlDataReader)
   at Facil.Runtime.CSharp.GeneratedCodeUtils+<ExecuteQuerySingleAsync>d__8`1[[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].MoveNext()
   at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[Facil.Runtime.CSharp.GeneratedCodeUtils+<ExecuteQuerySingleAsync>d__8`1[[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]], Facil.Runtime.CSharp, Version=0.2.6.0, Culture=neutral, PublicKeyToken=null]].MoveNext(System.Threading.Thread)
   at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Runtime.CompilerServices.IAsyncStateMachineBox, Boolean)
   at System.Threading.Tasks.Task.RunContinuations(System.Object)
   at System.Threading.Tasks.Task`1[[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].TrySetResult(System.__Canon)
   at System.Threading.Tasks.UnwrapPromise`1[[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].TrySetFromTask(System.Threading.Tasks.Task, Boolean)
   at System.Threading.Tasks.UnwrapPromise`1[[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].ProcessInnerTask(System.Threading.Tasks.Task)
   at System.Threading.Tasks.Task.RunContinuations(System.Object)
   at System.Threading.Tasks.Task.ExecuteWithThreadLocal(System.Threading.Tasks.Task ByRef, System.Threading.Thread)
   at System.Threading.ThreadPoolWorkQueue.Dispatch()

I have create a PR with a test that shows this error: #10

I have seen a similar problem when I ported some code over to dotnet core. I think the F# compiler does some function nesting which ticks over a magic stack allowance and throws the error. In my case it was expect computation expressions test which a restructured into a list.

Hi,
One suggestion, in the yaml you might want to add a comment that the regular expressions used to match sql object names are case sensitive. It kind of took me by surprise, given that the database itself uses case insensitive comparisons, which is the out-of-the-box default for sql server.

You can include the following blurb: The regular expressions used to match sql object names are case sensitive. The regular expression can be prefixed with (?i) to make it case insensitive. Ex: (?i)(dbo\.SomeTableName1|dbo\.SomeTableName2) .
Thanks

@ReedCopsey This is a project for strongly typed access to SQL server. I am considering a logo like the below (not necessarily final colors/proportions). Is that an acceptable use of the F# logo elements? (Related: cmeeren/Felicity#1)

No concrete plans exist to support this type at the moment. This is just an issue to track dependency on dotnet/SqlClient#322.

Hi,

I have another issue - it turns out that .net core doesn't support distributed transactions. See this.

Because the processing that I have to do involves multiple databases, would it be possible to include an option in facil to have it prefix sql object names with the database name? Fortunately, all my databases are on the same server, so if the objects are prefixed with the database name I could in theory use the same connection to do all the processing though in the yaml file the objects are under different connection strings. I tried to prefix a table name with the database name but it doesn't work, facil doesn't find the object, which was kind of what I expected. This is another option, to allow the developer to specify the database name in the yaml file, but I don't know if you want to go there. Well, I don't know what's simpler from an implementation point of view.

If you can't do it, then I will have to write more code manually.

Thanks!

PS. I appreciate greatly the help that provided so far!

Hey. The docs talk about doing everything from within building a .NET project. Is it possible to kick off Facil from the command line / F# script?

Initially from PR #2. I just wanted an issue for this to make it visible that this has been requested and is being worked on.

There's a bug in 2.1.1 where table scripts are missing. I've unlisted that version from NuGet and will publish a fix momentarily.

Hello,

The insertBatch script generated for this table:

CREATE TABLE [dbo].[zzz](
	[PRCo] [tinyint] NOT NULL,
	[Employee] [int] NOT NULL,
	[Document] [varchar](255) NOT NULL,
	....
) ON [PRIMARY]
GO

looks like this:

    let configureCmd sqlParams (cmd: SqlCommand) =
      cmd.CommandText <- """-- zzz
INSERT INTO [dbo].[zzz]
(
  [PRCo],
  [Employee],
  [Document],
....
)
SELECT
  [pRCo],
  [employee],
  [document],
...
FROM #args"""

While the CreateTempTable code is:

    [<EditorBrowsable(EditorBrowsableState.Never)>]
    member this.CreateTempTableData
      (
        ``args``: seq<``zzz``.``args``>
      ) =
      [
        TempTableData
          (
            "#args",
            """
            CREATE TABLE #args (
              [PRCo] TINYINT NOT NULL,
              [Employee] INT NOT NULL,
              [Document] VARCHAR(255) COLLATE Latin1_General_BIN NOT NULL,
              ...
            )
            """,
            (``args`` |> Seq.map (fun x -> x.Fields)),
            12,
            Action<_> this.userConfigureBulkCopy
          )
      ]

When this is executed on a case sensitive sql server, it doesn't work. The case of the column names should be preserved I believe, or am I doing something wrong? The collation of the DB that Facil is generating from is case sensitive, though the SQL Server's collation of that DB's server is not.

Hi,

I am just curios, in the generated code, why do you get the field values by ordinal number instead of using the field names? I assume that it's faster, but how much faster? Did you run any tests?

let mutable ``ordinal_ID`` = 0

let initOrdinals (reader: SqlDataReader) =
        ``ordinal_ID`` <- reader.GetOrdinal "ID"

let getItem (reader: SqlDataReader) =
  let ``ID`` = reader.GetInt32 ``ordinal_ID``

Thanks

See

After adding Facil to a new net5.0 console application on Linux, dotnet build fails, because the Facil.Generator.dll can not be found.

After manual checking I can confirm, that the search path seems correct, but the FacilGenerate.targets file contains an invalid path separator. Changing the file manually allows the build to succeed.

I've not tested on Windows, but I assume, using a slash "/" instead of backslash "\" will work there too. So I suggest replacing the backslash in Facil.Package/FacilGenerate.targets with a slash (like already used in Facil.Generator/FacilGenerate.targets).

Thank you very much.

Hello:

I used to generate a tableScript with type=getAll for a bunch of tables, and I noticed that the same methods are repeated for each table. I attached a screenshot where you can see the code differences for two [tables.](url

)

The ***Execute*** methods have essentially the same bodies though different return values, ultimately depending on the structure of the record.

I think these methods can be moved up in a base class that can be parameterized by the record type.

Here is a possible solution:

  // R is the return type
  [<AbstractClass>]
  type BaseScript<'R> (connStr: string, conn: SqlConnection, sql: string) =
 
    let configureCmd userConfigureCmd (cmd: SqlCommand) =
      cmd.CommandText <- sql
      userConfigureCmd cmd

    abstract member initOrdinals: SqlDataReader -> unit
    abstract member getItem: SqlDataReader -> 'R  


    //abstract member assignOrdinals (reader: SqlDataReader): Unit

    //abstract member readRow (reader: SqlDataReader) : r

    [<EditorBrowsable(EditorBrowsableState.Never)>]
    member val configureConn : SqlConnection -> unit = ignore with get, set

    [<EditorBrowsable(EditorBrowsableState.Never)>]
    member val userConfigureCmd : SqlCommand -> unit = ignore with get, set

    member this.ConfigureCommand(configureCommand: SqlCommand -> unit) =
      this.userConfigureCmd <- configureCommand
      this

    member this.ConfigureConnection(?configureConnection: SqlConnection -> unit) =
      match configureConnection with
      | None -> ()
      | Some config -> this.configureConn <- config
      this

    member this.ExecuteAsync(?cancellationToken) =
      executeQueryEagerAsync connStr conn this.configureConn (configureCmd this.userConfigureCmd) this.initOrdinals this.getItem [] (defaultArg cancellationToken CancellationToken.None)

    member this.AsyncExecute() =
      async {
        let! ct = Async.CancellationToken
        return! this.ExecuteAsync(ct) |> Async.AwaitTask
      }

    member this.ExecuteAsyncWithSyncRead(?cancellationToken) =
      executeQueryEagerAsyncWithSyncRead connStr conn this.configureConn (configureCmd this.userConfigureCmd) this.initOrdinals this.getItem [] (defaultArg cancellationToken CancellationToken.None)

    member this.AsyncExecuteWithSyncRead() =
      async {
        let! ct = Async.CancellationToken
        return! this.ExecuteAsyncWithSyncRead(ct) |> Async.AwaitTask
      }

    member this.Execute() =
      executeQueryEager connStr conn this.configureConn (configureCmd this.userConfigureCmd) this.initOrdinals this.getItem []

    member this.LazyExecuteAsync(?cancellationToken) =
      executeQueryLazyAsync connStr conn this.configureConn (configureCmd this.userConfigureCmd) this.initOrdinals this.getItem [] (defaultArg cancellationToken CancellationToken.None)

    member this.LazyExecuteAsyncWithSyncRead(?cancellationToken) =
      executeQueryLazyAsyncWithSyncRead connStr conn this.configureConn (configureCmd this.userConfigureCmd) this.initOrdinals this.getItem [] (defaultArg cancellationToken CancellationToken.None)

    member this.LazyExecute() =
      executeQueryLazy connStr conn this.configureConn (configureCmd this.userConfigureCmd) this.initOrdinals this.getItem []

    member this.ExecuteSingleAsync(?cancellationToken) =
      executeQuerySingleAsync connStr conn this.configureConn (configureCmd this.userConfigureCmd) this.initOrdinals this.getItem [] (defaultArg cancellationToken CancellationToken.None)

    member this.AsyncExecuteSingle() =
      async {
        let! ct = Async.CancellationToken
        return! this.ExecuteSingleAsync(ct) |> Async.AwaitTask
      }

    member this.ExecuteSingle() =
      executeQuerySingle connStr conn this.configureConn (configureCmd this.userConfigureCmd) this.initOrdinals this.getItem []
   
  type ``barg_assoc_All`` private (connStr: string, conn: SqlConnection) =
    inherit BaseScript<TableDtos.dbo.barg_assoc>(connStr, conn,
        """-- barg_assoc_All
        SELECT
          [barg_assoc_id],
          [barg_assoc_name]
        FROM
          [dbo].[barg_assoc]"""      
    )

    let mutable ``ordinal_barg_assoc_id`` = 0
    let mutable ``ordinal_barg_assoc_name`` = 0

    override this.initOrdinals (reader: SqlDataReader) =
      ``ordinal_barg_assoc_id`` <- reader.GetOrdinal "barg_assoc_id"
      ``ordinal_barg_assoc_name`` <- reader.GetOrdinal "barg_assoc_name"

    override this.getItem  (reader: SqlDataReader)  =
      let ``barg_assoc_id`` = reader.GetInt32 ``ordinal_barg_assoc_id``
      let ``barg_assoc_name`` = reader.GetString ``ordinal_barg_assoc_name``
      {
        ``Barg_assoc_id`` = ``barg_assoc_id``
        ``Barg_assoc_name`` = ``barg_assoc_name``
      }

    [<EditorBrowsable(EditorBrowsableState.Never)>]
    new() =
      failwith "This constructor is for aiding reflection and type constraints only"
      ``barg_assoc_All``(null, null)



    static member WithConnection(connectionString, ?configureConnection: SqlConnection -> unit) =
      ``barg_assoc_All``(connectionString, null).ConfigureConnection(?configureConnection=configureConnection)

    static member WithConnection(connection) = ``barg_assoc_All``(null, connection)




  type ``barg_unit_All`` private (connStr: string, conn: SqlConnection) =

    inherit BaseScript<TableDtos.dbo.barg_unit>(connStr, conn,
           """-- barg_unit_All
           SELECT
             [barg_unit_id],
             [barg_unit_name]
           FROM
             [dbo].[barg_unit]"""
       )



    let mutable ``ordinal_barg_unit_id`` = 0
    let mutable ``ordinal_barg_unit_name`` = 0

    override this.initOrdinals (reader: SqlDataReader) =
      ``ordinal_barg_unit_id`` <- reader.GetOrdinal "barg_unit_id"
      ``ordinal_barg_unit_name`` <- reader.GetOrdinal "barg_unit_name"

    override this.getItem (reader: SqlDataReader) : TableDtos.dbo.barg_unit =
      let ``barg_unit_id`` = reader.GetInt32 ``ordinal_barg_unit_id``
      let ``barg_unit_name`` = reader.GetString ``ordinal_barg_unit_name``
      {
        ``Barg_unit_id`` = ``barg_unit_id``
        ``Barg_unit_name`` = ``barg_unit_name``
      }

    [<EditorBrowsable(EditorBrowsableState.Never)>]
    new() =
      failwith "This constructor is for aiding reflection and type constraints only"
      ``barg_unit_All``(null, null)

    static member WithConnection(connectionString, ?configureConnection: SqlConnection -> unit) =
      ``barg_unit_All``(connectionString, null).ConfigureConnection(?configureConnection=configureConnection)

    static member WithConnection(connection) = ``barg_unit_All``(null, connection)

There can be a lot of methods eliminated from the generated code. Ultimately, thinking about it, what's different about each query that is executed are: the sql query itself, the parameters, and the code that reads the data from the SqlDataReader into the record, and of course the type of the record returned (in case the query returns data). All the other execution methods should be the same, and can be common, and, imo, they should not be repeated for each class that is generated.

The reading of the data could be abstracted through an parameterized interface by a record or records types with two methods, initialize & readRow (I would call it readRow instead of getItem).

If you want to be able to read multiple resultsets (I don't know if the generator currently supports reading multiple resultsets, i.e. scripts or stored procedures can return multiple select statements), it might become a little bit trickier. One would need a list of interfaces, one for each resultset, each interface would have its own record type.

I hope this is useful, and I think the amount of code generated can be reduced.

Thanks

Hey, i am interacting with a SQLServer db that has a ton of tables that have absolutely no primary key.

yeah.. it's exactly as awesome as you might imagine..

anyway, I'm curious what i'm supposed to do to exclude the tables? I can't seem to give it a yaml array to the exclude multiple things, it seems like it wants a regex? i don't really understand how to list individual tables via regex..

also, when i give up on that and ask that it only give the _All output instead of including the byId, i get an exception that sayes error getting outputcolumns for facil-generated table script script ccavgview_All. but that doesn't give me any idea of why or how to fix whatever it's upset about?

what is going on, exactly?

Hello,

What is the recommended way to configure facil when more than one developer works on a project, and developers use individual development databases?

Thanks

In F#, Async<_> computations are lazily evaluated and can normally be run several times if desired, for example for retrying IO operations if they fail due to transient errors.

If all you have is a Task<_>, you can't retry it; you need something like a unit -> Task<_> that produces a fresh task if you want to run it again. This is generally not required for F#'s Async<_>. All the work inside the async computation will be executed anew each time it is run. In actuality however, this of course depends on what the async computation does; if it for example represents a closure over mutable state, it may not be retryable.

Currently the Async<_> computation returned by Facil (from its AsyncExecute methods) is not retryable if that operation has parameters. Running the async computation a second time will result in ArgumentException: SqlParameter is already contained by another SqlParameterCollection.

The reason for this is a combination of the following:

• The SqlParameter objects are instantiated during the call to WithParameters, i.e. outside of the async computation returned by the AsyncExecute methods, and are thus re-used during direct retries of this specific computation.
• The SqlParameter objects are associated with the command they are added to, and this information is not cleared when the command is disposed.

Currently, if wanting to retry a Facil operation with parameters, the entire operation setup (from WithConnection and onwards) must be lazily evaluated (e.g. inside an async computation).

For example, the computation returned by getUser below is currently retryable (and always will be, since the whole chain is executed again each time the computation is run):

let getUser (connStr: string) (UserId userId) : Async<User option> =
  async {
    return!
      GetUserById
        .WithConnection(connStr)
        .WithParameters(userId)
        .AsyncExecuteSingle()
  }

However, the computation returned by the following version of getUser is not retryable (as of 0.2.3):

let getUser (connStr: string) (UserId userId) : Async<User option> =
  GetUserById
    .WithConnection(connStr)
    .WithParameters(userId)
    .AsyncExecuteSingle()

This can be surprising to users, since the signature of getUser is the same.

Facil should make sure that the async computations returned by the AsyncExecute methods are always retryable.

Hi,

I spent at least 2 hours on this one.

What do I want to do? I want to generate the code that calls a stored procedure that processes data that is shoved in a sql server temp table.

Here are the files that can help you reproduce the issue:

TempTable1.sql:

CREATE TABLE #TempTable1
(
    Id int NOT NULL PRIMARY KEY
  , Name varchar(30) NOT NULL
);

This file goes under some directory containing sql (I put it under a folder called CreateTable).

in the yaml I have under procedures:

      - include: (?i)^dbo\.spTestFacil$
        tempTables:
          - definition: CreateTable/TempTable1.sql

Here is the stored procedure code:

-- File Name: dbo.spTestFacil.sql
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO

CREATE OR ALTER PROCEDURE dbo.spTestFacil @psParam nvarchar(50)
as
begin


  DROP TABLE IF EXISTS #TmpJtsSnapshot;

  CREATE TABLE #TmpJtsSnapshot
  (
    Id int NOT NULL PRIMARY KEY
  , Name  varchar(30) NOT NULL

  );


  INSERT INTO #TmpJtsSnapshot ( Id, Name )
  values (1, 'Item 1')

  MERGE INTO #TmpJtsSnapshot AS Target
  USING
  (
    SELECT Id
         , Name
      FROM #TempTable1
  ) AS Source
     ON ( Target.Id = Source.Id )
   WHEN MATCHED THEN
    UPDATE SET Target.Name = Source.Name

   WHEN NOT MATCHED BY TARGET THEN
    INSERT ( Id, Name)
    VALUES
      ( Source.Id, Source.Name );

end
go

When I build the project I get these errors:

   1>Facil : error : System.Exception: Error getting output columns for stored procedure dbo.spTestFacil
          ---> Microsoft.Data.SqlClient.SqlException (0x80131904): Invalid object name '#TempTable1'.
            at Microsoft.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
            at Microsoft.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
            at Microsoft.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
            at Microsoft.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)
            at Microsoft.Data.SqlClient.SqlDataReader.TryConsumeMetaData()
            at Microsoft.Data.SqlClient.SqlDataReader.get_MetaData()
            at Microsoft.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString, Boolean isInternal, Boolean forDescribeParameterEncryption, Boolean shouldCacheForAlwaysEncrypted)
            at Microsoft.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean isAsync, Int32 timeout, Task& task, Boolean asyncWrite, Boolean inRetry, SqlDataReader ds, Boolean describeParameterEncryptionRequest)
            at Microsoft.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, TaskCompletionSource`1 completion, Int32 timeout, Task& task, Boolean& usedCache, Boolean asyncWrite, Boolean inRetry, String method)
            at Microsoft.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method)
            at Microsoft.Data.SqlClient.SqlCommand.ExecuteReader(CommandBehavior behavior)
            at Facil.Db.getColumnsFromQuery(RuleSet cfg, FSharpChoice`3 executable, SqlConnection conn) in C:\projects\facil\src\Facil.Generator\Db.fs:line 336
            at Facil.Db.getColumns(SqlConnection conn, RuleSet cfg, FSharpMap`2 sysTypeIdLookup, FSharpChoice`3 executable) in C:\projects\facil\src\Facil.Generator\Db.fs:line 403

I was curious and I used the sql server profiler to check what facil executes and it looks like facil executes exec dbo.spTestFacil @psParam=N'1' twice for whatever reason, but before it runs the second time it creates a new connection and it does not create the temp tables.

It's very odd. Any idea?

For now, to move on, I am going to comment out the bodies of the stored procedures, run the generator, and them add them back. It is a pain but it works.

Thanks

Update: I forgot to include the usual info: .net 6.0, latest version of facil as of today (2.5.4), windows 2016 server. I use rider.net 2022.1.2, sql server 2017.