Issues with Microsoft.SqlServer.SqlManagementObjects about sqltoolsservice

[shueybubbles]

Do you have the appropriate VC++ runtime installed? Also, what architecture is your build set to use? If you are using AnyCPU, you will need to define your own target to copy the appropriate binary depending on the bitness of your process. The .targets file included with the NuGet package depends on the platform being x64 or x86.

[lumogox]

Hi!
With version 140.17235.0, I'm still getting this issue, Microsoft.SqlServer.BatchParser.dll is not been copied.

System.IO.FileLoadException: Could not load file or assembly 'Microsoft.SqlServer.BatchParser, Version=14.100.0.0, Culture=neutral, PublicKeyToken=89845dcd8080cc91' or one of its dependencies. Attempt to load an unverifiable executable with fixups (IAT with more than 2 sections or a TLS section.) (Exception from HRESULT: 0x80131019) File name: 'Microsoft.SqlServer.BatchParser, Version=14.100.0.0, Culture=neutral, PublicKeyToken=89845dcd8080cc91' ---> System.IO.FileLoadException: Attempt to load an unverifiable executable with fixups (IAT with more than 2 sections or a TLS section.) (Exception from HRESULT: 0x80131019) at System.Reflection.RuntimeAssembly.GetType(RuntimeAssembly assembly, String name, Boolean throwOnError, Boolean ignoreCase, ObjectHandleOnStack type) at System.Reflection.RuntimeAssembly.GetType(String name, Boolean throwOnError, Boolean ignoreCase) at Microsoft.SqlServer.Management.Common.ServerConnection.GetStatements(String query, ExecutionTypes executionType, Int32& statementsToReverse) at Microsoft.SqlServer.Management.Common.ServerConnection.ExecuteNonQuery(String sqlCommand, ExecutionTypes executionType, Boolean retry) at Microsoft.SqlServer.Management.Common.ServerConnection.RollBackTransaction()

[kevcunnane]

@Matteo-T could you help answer this?

[kevcunnane]

@Matteo-T and @shueybubbles could you help on this?

[Matteo-T]

Yeah, we can take look. @Tornhoof: yes, please if you have a repro and some details on what does not seem right, please share that with us. Apologies for not getting back to you on the NuGet site (I wonder if I just forgot to follow up or I never got the notification: I'll check on that as well.

[Tornhoof]

The netstandard2.0 thing is probably easy to resolve as the api surface of netcoreapp2.0 is close to netstandard2.0, so I imagine you just need to change the of the relevant projects to it and recompile.

A more detailed description for the second point:
The obvious goal is to use the nuget package to include the dlls without the need to install the SMO installers on the target systems.
Repro: See attached SMOTest.zip
If you build the repro, the net47 output directory contains x86 native libraries and not x64 native libraries, this is because in your targets file, you copy the x86 native libraries for the case x86 and 'Any CPU', this is obviously wrong, but the test program still runs as those native libraries are apparently not used in this case.
In the netcoreapp2.0 output directory no SMO dlls exist and the test app crashes. In the nuget file the netcoreapp2.0 directory contains only a handfull dlls anyway, you need to find out if thats correct at all.

SmoTest.zip

Additionally, according to the nuget help, if you want a readme, call it Readme.txt (https://docs.microsoft.com/de-de/nuget/create-packages/creating-a-package#adding-a-readme-and-other-files), Readme.md is not shown during package installation

As how to fix the nuspec, I tried a few things, but honestly I didn't get it working properly, the following nuspec should copy the native dlls to the proper output directories, but it does not work on VS 2017 15.5. (It only works if the runtime identifier is set to exactly the same name as in the project and does not honor the runtime identifier dependency path).

<?xml version="1.0"?>
<package>
	<metadata minClientVersion="2.12">
		<id>Microsoft.SqlServer.SqlManagementObjects</id>
		<version>140.17218.5</version>
		<authors>Microsoft</authors>
		<owners>sqltools,Microsoft</owners>
		<requireLicenseAcceptance>true</requireLicenseAcceptance>
		<licenseUrl>https://docs.microsoft.com/sql/relational-databases/server-management-objects-smo/smo-license-terms</licenseUrl>
		<projectUrl>https://docs.microsoft.com/sql/relational-databases/server-management-objects-smo/overview-smo</projectUrl>
		<iconUrl>http://s.gravatar.com/avatar/6f39d8562df0a3509a8240fb281bc5bd?s=80</iconUrl>
		<description>The SQL Server Management Objects (SMO) Framework is a set of objects designed for programmatic management of Microsoft SQL Server and Microsoft Azure SQL Database.</description>
		<summary>All libraries included as part of the SQL Server Management Objects (SMO) Framework</summary>
		<releaseNotes>Initial Release</releaseNotes>
		<copyright>&#xA9; Microsoft Corporation. All rights reserved.</copyright>
		<tags>SQL SMO</tags>
		<dependencies>
			<group targetFramework=".NETCoreApp2.0">
				<dependency id="System.Data.SqlClient" version="4.4.0" exclude="Compile"/>
			</group>
		</dependencies>
	</metadata>
	<files>
		<file src="README.txt" target="" />
		<file src="License_SMO.rtf" target="" />
		<file src="lib\net40\**" target="lib/net40" />
		<file src="lib\netcoreapp2.0\**" target="lib/netcoreapp2.0" />
		<file src="native\win-x86\**" target="runtimes/win-x86/native" />
		<file src="native\win-x64\**" target="runtimes/win-x64/native" />
	</files>
</package>

I think you need someone from the nuget team (e.g. @emgarten) or dotnet team (e.g. @terrajobst) to help you there, because either the documentation in https://docs.microsoft.com/en-us/nuget/create-packages/supporting-multiple-target-frameworks is wrong or the there are actually bugs somewhere. I don't know any nuget packages with native runtime libraries, so I might just be wrong altogether.

[shueybubbles]

Should the Target use $(PlatformTarget) instead of $(Platform) to decide which native DLL to use?
The netcore folder has fewer binaries today because those were the minimum set needed to support SQL Operations Studio. There's still a fair amount of work to do to make the other binaries compile for NetStandard or NetCore. If there are particular pieces of functionality missing from the netcore version of SMO that we should prioritize porting over, please let us know. thanx for your feedback

[Tornhoof]

Thank you for your answer.
If you mean $(PlatformTarget) like this:

<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">

    <ItemGroup Condition=" '$(PlatformTarget)' == 'x64' ">
        <Content Include="$(MSBuildThisFileDirectory)..\..\runtimes\win-x64\native\Microsoft.SqlServer.BatchParser.dll">
            <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
            <Link>Microsoft.SqlServer.BatchParser.dll</Link>
        </Content>
        <Content Include="$(MSBuildThisFileDirectory)..\..\runtimes\win-x64\native\Microsoft.SqlServer.XE.Core.dll">
            <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
            <Link>Microsoft.SqlServer.XE.Core.dll</Link>
        </Content>
        <Content Include="$(MSBuildThisFileDirectory)..\..\runtimes\win-x64\native\Microsoft.SqlServer.XEvent.Linq.dll">
            <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
            <Link>Microsoft.SqlServer.XEvent.Linq.dll</Link>
        </Content>
    </ItemGroup>

    <ItemGroup Condition=" '$(PlatformTarget)' == 'x86'">
        <Content Include="$(MSBuildThisFileDirectory)..\..\runtimes\win-x86\native\Microsoft.SqlServer.BatchParser.dll">
            <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
            <Link>Microsoft.SqlServer.BatchParser.dll</Link>
        </Content>
        <Content Include="$(MSBuildThisFileDirectory)..\..\runtimes\win-x86\native\Microsoft.SqlServer.XE.Core.dll">
            <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
            <Link>Microsoft.SqlServer.XE.Core.dll</Link>
        </Content>
        <Content Include="$(MSBuildThisFileDirectory)..\..\runtimes\win-x86\native\Microsoft.SqlServer.XEvent.Linq.dll">
            <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
            <Link>Microsoft.SqlServer.XEvent.Linq.dll</Link>
        </Content>
        
    </ItemGroup>

</Project>

then this appears to be working, it copies the correct x64 files into the bin dir.

As for using SMO functionality, I can't even get the few lines of code from my example project (attached in the last post) to work on .net core, even with copying all netcoreapp files + native libs into the release dir for netcoreapp:

Server server = new Server("localhost");
foreach (Database serverDatabase in server.Databases)
{
    Console.WriteLine(serverDatabase.Name);
}

this already throws:

Unhandled Exception: Microsoft.SqlServer.Management.Sdk.Sfc.EnumeratorException: Failed to retrieve data for this request. ---> Microsoft.SqlServer.Management.Sdk.Sfc.InvalidConfigurationFileEnumeratorException: File Server.xml was not found.
The file might be missing in sources or misspelled in config.xml, or case sensitivity may not be preserved.
   at Microsoft.SqlServer.Management.Smo.XmlReadDoc.LoadFile(Assembly a, String strFile)
   at Microsoft.SqlServer.Management.Smo.SqlObject.LoadInitDataFromAssemblyInternal(Assembly assemblyObject, String file, ServerVersion ver, String alias, StringCollection requestedFields, Boolean store, StringCollection roAfterCreation, DatabaseEngineType databaseEngineType, DatabaseEngineEdition databaseEngineEdition, Stream configXml)
   at Microsoft.SqlServer.Management.Smo.SqlObject.LoadInitData(String file, ServerVersion ver, DatabaseEngineType databaseEngineType, DatabaseEngineEdition databaseEngineEdition)
   at Microsoft.SqlServer.Management.Sdk.Sfc.ObjectCache.LoadElement(ObjectLoadInfo oli, ServerVersion ver, DatabaseEngineType databaseEngineType, DatabaseEngineEdition databaseEngineEdition)
   at Microsoft.SqlServer.Management.Sdk.Sfc.ObjectCache.GetElement(ObjectLoadInfo oli, ServerVersion ver, DatabaseEngineType databaseEngineType, DatabaseEngineEdition databaseEngineEdition)
   at Microsoft.SqlServer.Management.Sdk.Sfc.ObjectCache.GetAllElements(Urn urn, ServerVersion ver, DatabaseEngineType databaseEngineType, DatabaseEngineEdition databaseEngineEdition, Object ci)
   at Microsoft.SqlServer.Management.Sdk.Sfc.Environment.GetObjectsFromCache(Urn urn, Object ci)
   at Microsoft.SqlServer.Management.Sdk.Sfc.Environment.GetData(Request req, Object ci)
   at Microsoft.SqlServer.Management.Sdk.Sfc.Enumerator.GetData(Object connectionInfo, Request request)
   at Microsoft.SqlServer.Management.Sdk.Sfc.Enumerator.Process(Object connectionInfo, Request request)
   --- End of inner exception stack trace ---
   at Microsoft.SqlServer.Management.Sdk.Sfc.Enumerator.Process(Object connectionInfo, Request request)
   at Microsoft.SqlServer.Management.Smo.ExecutionManager.GetEnumeratorData(Request req)
   at Microsoft.SqlServer.Management.Smo.SqlSmoObject.GetDbCollation(String dbname)
   at Microsoft.SqlServer.Management.Smo.SqlSmoObject.GetDbComparer(Boolean inServer)
   at Microsoft.SqlServer.Management.Smo.SqlSmoObject.InitializeStringComparer()
   at Microsoft.SqlServer.Management.Smo.SqlSmoObject.get_StringComparer()
   at Microsoft.SqlServer.Management.Smo.SimpleObjectCollectionBase.InitInnerCollection()
   at Microsoft.SqlServer.Management.Smo.SmoCollectionBase.InitializeChildCollection(Boolean refresh, ScriptingPreferences sp, String filterQuery, IEnumerable`1 extraFields)
   at Microsoft.SqlServer.Management.Smo.SmoCollectionBase.GetEnumerator(ScriptingPreferences sp)
   at SmoTest.Program.Main(String[] args)

As of what I use of SMO, I use it for creating integration test databases for my projects, with the following steps, this method is quite a bit faster as my db is seeded initially and I save the time to create the db for each test collection

1. Create db
2. detach db
3. copy detached db to test specific files
4. reattach the renamed files

I looked through my code, apparently the only things missing (red in my code) are Backup/Restore:

public string Backup(string dbName)
{
    const string backupFileFormat = "{0:dd-MMM-yyyy-HH-mm-ss}";
    var server = CreateSqlServer();
    if (server.Databases.Contains(dbName))
    {
        var fileName = Path.ChangeExtension(string.Format(dbName + "_" + backupFileFormat, DateTime.UtcNow), ".bak");
        var backupFileName = Path.Combine(DatabasePath, "Backup", fileName);

        var bdi = new BackupDeviceItem(backupFileName, DeviceType.File);
        var bk = new Backup
        {
            Action = BackupActionType.Database,
            BackupSetDescription = $"Full backup of {dbName}",
            BackupSetName = $"{dbName} Backup",
            Database = dbName,
            Incremental = false,
            LogTruncation = BackupTruncateLogType.Truncate
        };
        bk.Devices.Add(bdi);
        bk.SqlBackup(server);
        bk.Devices.Remove(bdi);
        return fileName;
    }

    return null;
}

public void Restore(string dbName, string fileName)
{
    var server = CreateSqlServer();
    if (TryKillAndDetach(server, dbName))
    {
        var rs = new Restore { NoRecovery = true, Database = dbName };
        var localFileName = Path.Combine(DatabasePath, "Backup", fileName);
        var bdi = new BackupDeviceItem(localFileName, DeviceType.File);
        rs.Devices.Add(bdi);
        rs.NoRecovery = false;
        rs.SqlRestore(server);
        rs.Devices.Remove(bdi);
    }
}

But as I can't even get my simple test code to work, I can't say if the steps from above are working on .net core

[shueybubbles]

thanx for the info, it's possible something went haywire in the most recent update. I'll take a look.

[shueybubbles]

it looks like the resources were linked in but with the wrong names.
i think this should be just Server.xml, which it is in our NetFx version. Something is different in the resource linking.

.mresource public Microsoft.SqlServer.Management.Smo.SqlEnum.Server.xml
{
// Offset: 0x000AFF08 Length: 0x00000649
}

[Tornhoof]

It looks like version 140.17224.0 works again, including Backup/Restore. Thanks.

[benrr101]

Since this issue looks resolved, I'll close it.

[TrendyTim]

Ive been getting the
System.IO.FileLoadException: Could not load file or assembly 'Microsoft.SqlServer.BatchParser, Version=14.100.0.0, Culture=neutral, PublicKeyToken=89845dcd8080cc91' or one of its dependencies. Attempt to load an unverifiable executable with fixups

As well.

My structure is
Worker Project references the smo nuget package
WinClient project references the worker project but not the nuget package

It copies most of the needed SqlServer files and probably a few i don't need (no clue what Dmf.Common is or why i need SQLClrProvider, SQLEnum, XE.Core, XEventLinq possibly an extra 2.5meg (excluding xml files i probably wouldn't want), i just want to run scipts with GO in it and the other SMO nuget packages need a hack to work in .net 4 but not batchParser.

If i copied the missing file manually it works fine, not sure why it wont copy the batchparser when it'll do the rest of them.

I was going to say id need to contrive a hack to make it work, but then i thought, i did remove some references because i didn't want them (like Azure a few others, thinking it would add an extra 15 meg to my app), reinstalling the package and not removing the references doesn't seem to have added extra files to the deploy EXCEPT for BatchParser.

So lesson learnt, (in my case) just leave the references alone and put up with the apparent bloat.

Edit: i was confused with BatchParser and BatchParserClient, BatchParser was copied but BatchParserClient wasnt, and still isnt,

Edit2: So i did a hack and referenced BatchParserClient in the main project and that worked, server complained about BatchParser so i had to find out what VC++ Runtime i didnt have (2013) because there doesnt seem to be any mention of the requirements.

It would be nice if you could

1. throw an exception saying "VC++ 2013 x86/x64 needs to be installed" instead failing like it does).
2. in the NuGet package and the site, list VC++ 2013 runtime as a requirement

[gerry123]

+1 for
-throw an exception saying "VC++ 2013 x86/x64 needs to be installed" instead failing like it does).
-in the NuGet package and the site, list VC++ 2013 runtime as a requirement

[shueybubbles]

Thank you for the feedback. The README.md file in the package root lists the VC Runtime requirement and includes a link to the redist. I've submitted a PR to the docs for "Install SMO" to also include this link and requirement.
Note that this package also includes a fully managed SMO stack which can be used by any project built using NetStandard 2.0, avoiding any dependency on these native binaries.
Unsigned/non strong-named binaries with a more granular package dependency chain are available in a new set of preview packages, such as https://www.nuget.org/packages/Microsoft.SQLServer.SMO/150.18040.0-xplat.
If you can use NetStandard binaries, and don't need 100% of the old SMO functionality, I recommend making the switch. There's a UserVoice item asking us to port more SMO functionality to NetStandard at https://feedback.azure.com/forums/908035-sql-server/suggestions/35649403-porting-smo-library-to-net-core. I ask anyone looking for more SMO functionality in NetStandard to add input there.

[shueybubbles]

oh this reminds me I need to add readme.txt, not just readme.md

[gerry123]

I tried using the net core packages, but I miss Microsoft.SqlServer.Management.Trace. I've added this to https://feedback.azure.com/forums/908035-sql-server/suggestions/35649403-porting-smo-library-to-net-core