Can't run Mongo2Go on VSTS / Azure Devops about mongo2go

[galczo5]

All of our tests have the same structure:

[Fact]
public async Task adds_modified_HistoryEntry()
{
    using (var runner = MongoDbRunner.Start())
    {
        var client = new MongoClient(runner.ConnectionString);
        var database = client.GetDatabase(Guid.NewGuid().ToString());
        // test code here
    }
}

[nmurray3]

For anyone who has found this issue recently, I can confirm that at least version 2.2.16 works, but some later versions (eg 3.1.3) do not.

[galczo5]

Ok, but I have that error on my local machine when I'm running tests with dotnet test command.
So it cannot be error for only VSTS hosted agents.

If it helps I have 100+ tests that use Mongo2Go package.

[JohannesHoppe]

No, Mac (#43) and Linux (#47) works now super fine. I tested both successfully.

I have absolutely no clue what happens on VSTS and really hope some user figures out what's wrong. Unfortunately VSTS is pretty 💩, because you can't even login to the agent... Also, time is rare here, too. 😞

[galczo5]

Sorry for the misinformation. We have the same problem on VSTS agents hosted on Windows and Ubuntu, and also on local machines.
The same exception in all cases.

[JohannesHoppe]

@ppumkin It's time to clone the repo and debug this! 😉

[chrisseroka]

I think the problem may be related with .net core. Try to run them against net461.
See: #48

[jbogard]

Should that example have a static field? It seems odd to dispose of a static field in a subclass's instance method.

[JohannesHoppe]

I'm going to close this issue and declare that Mongo2Go is NOT compatible with VSTS / Azure DevOps.
see https://stackoverflow.com/a/44662712

If you're using the hosted agent, you can't open up ports or change anything about the machine's configuration. You'll need to set up your own agent for builds

Mongo2Go must open ports. If this is not possible, then there is absolutely no chance that we will get this up and running.

[lukpep]

nothing special I guess:

netcoreapp2.2

deps:

Base class for every integration test:

public class MongoIntegrationTest<T> : IDisposable
{
    private readonly string _databaseName = "IntegrationTest";
    internal IMongoCollection<T> Collection;
    protected IMongoDatabase Database;
    internal MongoDbRunner Runner;

    internal void CreateConnection(string collectionName)
    {
        CreateConnection();
        Collection = Database.GetCollection<T>(collectionName);
    }

    internal void CreateConnection()
    {
        MongoConfigurator.Initialize();
        Runner = MongoDbRunner.Start();

        var client = new MongoClient(Runner.ConnectionString);
        Database = client.GetDatabase(_databaseName);
    }

    internal void SeedCollection(string collection, string fileName)
    {
        var path = AppContext.BaseDirectory + "../../../../DatabaseSeeds/" + fileName;
        Runner.Import(_databaseName, collection, path, true);
    }

    internal void SetMongoConnection(AutoMock mock)
    {
        mock.Provide(Database);    
    }

    public void Dispose()
    {
        Runner?.Dispose();
    }
}

and example test class:

public class CommentRepositoryTests : MongoIntegrationTest<Comment>
{
    
    [Fact]
    public async Task it_should_be_possible_to_add_and_retrieve_comment()
    {
        CreateConnection(TestConstants.CommentsCollectionName);
        var comment = TestComments.One;
        
        using (var mock = AutoMock.GetLoose())
        {
            SetMongoConnection(mock);
            var repo = mock.Create<CommentRepository>();
            
            await repo.AddAsync(comment);
            var foundComment = await repo.GetOrFailAsync(comment.Id);
            
            foundComment.Should().BeEquivalentTo(comment);
        }
        Runner.Dispose();
    }  
}

task configuration on azure devops:

task: DotNetCoreCLI@2
    displayName: Test
    inputs:
      command: test
      projects: '**/*Tests/*.csproj'
      arguments: '--configuration $(BuildConfiguration) /p:CollectCoverage=true /p:CoverletOutputFormat=cobertura /p:Exclude="[xunit.*]*"'

[Vasistan]

Th 2.2.16 really helped for me as well

[JohannesHoppe]

Client can't connect to the server. That can have various reasons. There is no error message indicating that the server couldn't start. So it should run but the connection is not working. Usually it could be a personal firewall or something. If possible, try to change the build agent to Linux and see if it works there.

Since I don't use Visual Studio Team Services I can't help that much here. Let's see and wait if other people also use VSTS.

[bmarinov]

@JohannesHoppe I have the same problem as him. I tried running the xUnit tests on a linux build agent environment (Ubuntu 16) and on a Windows agent. Same exception. Do you have any ideas how we may troubleshoot the issue? Is there any way we can help troubleshooting this problem? Might it be related to #43 or #47, itseems to be of similar origin?

I would gladly do more testing and experimenting on VSTS. If you want to take a look yourself, you can sign up for a free account with (iirc) 120 build minutes. Let me know if there is anything I can do.

Here the stack trace and some extra output from the failed test run:

2018-09-11T15:17:10.6729867Z 2018-09-11T15:17:10.666+0000 I CONTROL  [initandlisten] MongoDB starting : pid=11535 port=27018 dbpath=/tmp/nwqlejou.pgs22c9dde786aa4c3d893d_27018 64-bit host=factoryvm-az514
2018-09-11T15:17:10.6768142Z 2018-09-11T15:17:10.666+0000 I CONTROL  [initandlisten] db version v3.6.1
2018-09-11T15:17:10.6788394Z 2018-09-11T15:17:10.666+0000 I CONTROL  [initandlisten] git version: 025d4f4fe61efd1fb6f0005be20cb45a004093d1
2018-09-11T15:17:10.6808680Z 2018-09-11T15:17:10.666+0000 I CONTROL  [initandlisten] allocator: tcmalloc
2018-09-11T15:17:10.6828463Z 2018-09-11T15:17:10.666+0000 I CONTROL  [initandlisten] modules: none
2018-09-11T15:17:10.6848468Z 2018-09-11T15:17:10.666+0000 I CONTROL  [initandlisten] build environment:
2018-09-11T15:17:10.6868826Z 2018-09-11T15:17:10.666+0000 I CONTROL  [initandlisten]     distarch: x86_64
2018-09-11T15:17:10.6889777Z 2018-09-11T15:17:10.666+0000 I CONTROL  [initandlisten]     target_arch: x86_64
2018-09-11T15:17:10.6912933Z 2018-09-11T15:17:10.666+0000 I CONTROL  [initandlisten] options: { net: { bindIp: "127.0.0.1", port: 27018 }, storage: { dbPath: "/tmp/nwqlejou.pgs22c9dde786aa4c3d893d_27018", journal: { enabled: false } } }
2018-09-11T15:17:10.6938970Z 2018-09-11T15:17:10.666+0000 I STORAGE  [initandlisten] 
2018-09-11T15:17:10.6959956Z 2018-09-11T15:17:10.666+0000 I STORAGE  [initandlisten] ** WARNING: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine
2018-09-11T15:17:10.6979051Z 2018-09-11T15:17:10.666+0000 I STORAGE  [initandlisten] **          See http://dochub.mongodb.org/core/prodnotes-filesystem
2018-09-11T15:17:10.7002782Z 2018-09-11T15:17:10.666+0000 I STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=2972M,session_max=20000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),statistics_log=(wait=0),verbose=(recovery_progress),,log=(enabled=false),
2018-09-11T15:17:11.1653659Z 2018-09-11T15:17:11.164+0000 I CONTROL  [initandlisten] 
2018-09-11T15:17:11.1673639Z 2018-09-11T15:17:11.164+0000 I CONTROL  [initandlisten] ** WARNING: Access control is not enabled for the database.
2018-09-11T15:17:11.1694609Z 2018-09-11T15:17:11.164+0000 I CONTROL  [initandlisten] **          Read and write access to data and configuration is unrestricted.
2018-09-11T15:17:11.1711068Z 2018-09-11T15:17:11.164+0000 I CONTROL  [initandlisten] 
2018-09-11T15:17:11.1729449Z 2018-09-11T15:17:11.164+0000 I CONTROL  [initandlisten] 
2018-09-11T15:17:11.1746778Z 2018-09-11T15:17:11.164+0000 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled is 'always'.
2018-09-11T15:17:11.1763974Z 2018-09-11T15:17:11.164+0000 I CONTROL  [initandlisten] **        We suggest setting it to 'never'
2018-09-11T15:17:11.1779101Z 2018-09-11T15:17:11.164+0000 I CONTROL  [initandlisten] 
2018-09-11T15:17:11.1799019Z 2018-09-11T15:17:11.164+0000 I STORAGE  [initandlisten] createCollection: admin.system.version with provided UUID: f59d5a2e-4e35-42c4-94cf-5f41b0f3e137
2018-09-11T15:17:11.3317724Z 2018-09-11T15:17:11.330+0000 I COMMAND  [initandlisten] setting featureCompatibilityVersion to 3.6
2018-09-11T15:17:11.3352674Z 2018-09-11T15:17:11.334+0000 I STORAGE  [initandlisten] createCollection: local.startup_log with generated UUID: 35e28118-7e88-4d7e-bb51-c7fd64388199
2018-09-11T15:17:11.4878653Z 2018-09-11T15:17:11.486+0000 I FTDC     [initandlisten] Initializing full-time diagnostic data capture with directory '/tmp/nwqlejou.pgs22c9dde786aa4c3d893d_27018/diagnostic.data'
2018-09-11T15:17:11.4898397Z 2018-09-11T15:17:11.487+0000 I NETWORK  [initandlisten] waiting for connections on port 27018
2018-09-11T15:17:12.5196980Z 2018-09-11T15:17:10.670+0000 I CONTROL  [initandlisten] MongoDB starting : pid=11536 port=27019 dbpath=/tmp/ie2w0l01.tzp38e0b028bb1e4cb7b513_27019 64-bit host=factoryvm-az514
2018-09-11T15:17:12.5215694Z 2018-09-11T15:17:10.670+0000 I CONTROL  [initandlisten] db version v3.6.1
2018-09-11T15:17:12.5230877Z 2018-09-11T15:17:10.670+0000 I CONTROL  [initandlisten] git version: 025d4f4fe61efd1fb6f0005be20cb45a004093d1
2018-09-11T15:17:12.5248302Z 2018-09-11T15:17:10.670+0000 I CONTROL  [initandlisten] allocator: tcmalloc
2018-09-11T15:17:12.5265867Z 2018-09-11T15:17:10.670+0000 I CONTROL  [initandlisten] modules: none
2018-09-11T15:17:12.5284044Z 2018-09-11T15:17:10.670+0000 I CONTROL  [initandlisten] build environment:
2018-09-11T15:17:12.5304992Z 2018-09-11T15:17:10.670+0000 I CONTROL  [initandlisten]     distarch: x86_64
2018-09-11T15:17:12.5325128Z 2018-09-11T15:17:10.670+0000 I CONTROL  [initandlisten]     target_arch: x86_64
2018-09-11T15:17:12.5344048Z 2018-09-11T15:17:10.670+0000 I CONTROL  [initandlisten] options: { net: { bindIp: "127.0.0.1", port: 27019 }, storage: { dbPath: "/tmp/ie2w0l01.tzp38e0b028bb1e4cb7b513_27019", journal: { enabled: false } } }
2018-09-11T15:17:12.5360707Z 2018-09-11T15:17:10.670+0000 I STORAGE  [initandlisten] 
2018-09-11T15:17:12.5378445Z 2018-09-11T15:17:10.670+0000 I STORAGE  [initandlisten] ** WARNING: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine
2018-09-11T15:17:12.5397697Z 2018-09-11T15:17:10.670+0000 I STORAGE  [initandlisten] **          See http://dochub.mongodb.org/core/prodnotes-filesystem
2018-09-11T15:17:12.5416038Z 2018-09-11T15:17:10.670+0000 I STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=2972M,session_max=20000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),statistics_log=(wait=0),verbose=(recovery_progress),,log=(enabled=false),
2018-09-11T15:17:12.5435913Z 2018-09-11T15:17:11.192+0000 I CONTROL  [initandlisten] 
2018-09-11T15:17:12.5452103Z 2018-09-11T15:17:11.192+0000 I CONTROL  [initandlisten] ** WARNING: Access control is not enabled for the database.
2018-09-11T15:17:12.5468416Z 2018-09-11T15:17:11.192+0000 I CONTROL  [initandlisten] **          Read and write access to data and configuration is unrestricted.
2018-09-11T15:17:12.5488124Z 2018-09-11T15:17:11.192+0000 I CONTROL  [initandlisten] 
2018-09-11T15:17:12.5508654Z 2018-09-11T15:17:11.192+0000 I CONTROL  [initandlisten] 
2018-09-11T15:17:12.5532908Z 2018-09-11T15:17:11.192+0000 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled is 'always'.
2018-09-11T15:17:12.5556149Z 2018-09-11T15:17:11.192+0000 I CONTROL  [initandlisten] **        We suggest setting it to 'never'
2018-09-11T15:17:12.5577518Z 2018-09-11T15:17:11.192+0000 I CONTROL  [initandlisten] 
2018-09-11T15:17:12.5595616Z 2018-09-11T15:17:11.192+0000 I STORAGE  [initandlisten] createCollection: admin.system.version with provided UUID: be817d55-7d1d-4659-90eb-603923fa2216
2018-09-11T15:17:12.5611898Z 2018-09-11T15:17:11.348+0000 I COMMAND  [initandlisten] setting featureCompatibilityVersion to 3.6
2018-09-11T15:17:12.5632740Z 2018-09-11T15:17:11.350+0000 I STORAGE  [initandlisten] createCollection: local.startup_log with generated UUID: a98dce42-a03a-4c3b-9f7c-48960dcb70f3
2018-09-11T15:17:12.5651173Z 2018-09-11T15:17:11.500+0000 I FTDC     [initandlisten] Initializing full-time diagnostic data capture with directory '/tmp/ie2w0l01.tzp38e0b028bb1e4cb7b513_27019/diagnostic.data'
2018-09-11T15:17:12.5671878Z 2018-09-11T15:17:11.500+0000 I NETWORK  [initandlisten] waiting for connections on port 27019
2018-09-11T15:17:13.5208889Z 2018-09-11T15:17:13.519+0000 I NETWORK  [listener] connection accepted from 127.0.0.1:56162 #1 (1 connection now open)
2018-09-11T15:17:13.5228096Z 2018-09-11T15:17:13.521+0000 I NETWORK  [listener] connection accepted from 127.0.0.1:52418 #1 (1 connection now open)
2018-09-11T15:17:41.7830998Z 2018-09-11T15:17:41.782+0000 I NETWORK  [conn1] end connection 127.0.0.1:56162 (0 connections now open)
2018-09-11T15:17:41.8770269Z 
2018-09-11T15:17:41.8779274Z 
2018-09-11T15:17:41.9092079Z [xUnit.net 00:00:34.5952973]     ProjectName.IntegrationTests.MongoDatabaseTests+MongoDbCollectionShould.ReturnEmptyCollection [FAIL]
2018-09-11T15:17:41.9496655Z 2018-09-11T15:17:41.948+0000 I CONTROL  [initandlisten] MongoDB starting : pid=11645 port=27020 dbpath=/tmp/atgprmn3.lst22c64ab708ed48b8b778_27020 64-bit host=factoryvm-az514
2018-09-11T15:17:41.9512860Z 2018-09-11T15:17:41.948+0000 I CONTROL  [initandlisten] db version v3.6.1
2018-09-11T15:17:41.9529825Z 2018-09-11T15:17:41.948+0000 I CONTROL  [initandlisten] git version: 025d4f4fe61efd1fb6f0005be20cb45a004093d1
2018-09-11T15:17:41.9546000Z 2018-09-11T15:17:41.948+0000 I CONTROL  [initandlisten] allocator: tcmalloc
2018-09-11T15:17:41.9566289Z 2018-09-11T15:17:41.948+0000 I CONTROL  [initandlisten] modules: none
2018-09-11T15:17:41.9588364Z 2018-09-11T15:17:41.948+0000 I CONTROL  [initandlisten] build environment:
2018-09-11T15:17:41.9608794Z 2018-09-11T15:17:41.948+0000 I CONTROL  [initandlisten]     distarch: x86_64
2018-09-11T15:17:41.9626745Z 2018-09-11T15:17:41.948+0000 I CONTROL  [initandlisten]     target_arch: x86_64
2018-09-11T15:17:41.9647499Z 2018-09-11T15:17:41.948+0000 I CONTROL  [initandlisten] options: { net: { bindIp: "127.0.0.1", port: 27020 }, storage: { dbPath: "/tmp/atgprmn3.lst22c64ab708ed48b8b778_27020", journal: { enabled: false } } }
2018-09-11T15:17:41.9664716Z 2018-09-11T15:17:41.948+0000 I STORAGE  [initandlisten] 
2018-09-11T15:17:41.9685476Z 2018-09-11T15:17:41.948+0000 I STORAGE  [initandlisten] ** WARNING: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine
2018-09-11T15:17:41.9705179Z 2018-09-11T15:17:41.948+0000 I STORAGE  [initandlisten] **          See http://dochub.mongodb.org/core/prodnotes-filesystem
2018-09-11T15:17:41.9728563Z 2018-09-11T15:17:41.948+0000 I STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=2972M,session_max=20000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),statistics_log=(wait=0),verbose=(recovery_progress),,log=(enabled=false),
2018-09-11T15:17:41.9886827Z Failed   ProjectName.IntegrationTests.MongoDatabaseTests+MongoDbCollectionShould.ReturnEmptyCollection
2018-09-11T15:17:41.9905597Z Error Message:
2018-09-11T15:17:41.9925985Z  System.TimeoutException : A timeout occured after 30000ms selecting a server using CompositeServerSelector{ Selectors = MongoDB.Driver.MongoClient+AreSessionsSupportedServerSelector, LatencyLimitingServerSelector{ AllowedLatencyRange = 00:00:00.0150000 } }. Client view of cluster state is { ClusterId : "1", ConnectionMode : "Automatic", Type : "Unknown", State : "Disconnected", Servers : [{ ServerId: "{ ClusterId : 1, EndPoint : "127.0.0.1:27018" }", EndPoint: "127.0.0.1:27018", State: "Disconnected", Type: "Unknown" }] }.
2018-09-11T15:17:41.9943572Z Stack Trace:
2018-09-11T15:17:41.9962203Z    at MongoDB.Driver.Core.Clusters.Cluster.ThrowTimeoutException(IServerSelector selector, ClusterDescription description)
2018-09-11T15:17:41.9984893Z    at MongoDB.Driver.Core.Clusters.Cluster.WaitForDescriptionChangedHelper.HandleCompletedTask(Task completedTask)
2018-09-11T15:17:42.0002398Z    at MongoDB.Driver.Core.Clusters.Cluster.WaitForDescriptionChanged(IServerSelector selector, ClusterDescription description, Task descriptionChangedTask, TimeSpan timeout, CancellationToken cancellationToken)
2018-09-11T15:17:42.0021421Z    at MongoDB.Driver.Core.Clusters.Cluster.SelectServer(IServerSelector selector, CancellationToken cancellationToken)
2018-09-11T15:17:42.0040725Z    at MongoDB.Driver.MongoClient.AreSessionsSupportedAfterServerSelection(CancellationToken cancellationToken)
2018-09-11T15:17:42.0058227Z    at MongoDB.Driver.MongoClient.AreSessionsSupported(CancellationToken cancellationToken)
2018-09-11T15:17:42.0075415Z    at MongoDB.Driver.OperationExecutor.StartImplicitSession(CancellationToken cancellationToken)
2018-09-11T15:17:42.0091983Z    at MongoDB.Driver.MongoCollectionImpl`1.UsingImplicitSession[TResult](Func`2 func, CancellationToken cancellationToken)
2018-09-11T15:17:42.0109828Z    at MongoDB.Driver.MongoCollectionImpl`1.FindSync[TProjection](FilterDefinition`1 filter, FindOptions`2 options, CancellationToken cancellationToken)
2018-09-11T15:17:42.0126983Z    at MongoDB.Driver.FindFluent`2.ToCursor(CancellationToken cancellationToken)
2018-09-11T15:17:42.0143496Z    at MongoDB.Driver.IAsyncCursorSourceExtensions.ToList[TDocument](IAsyncCursorSource`1 source, CancellationToken cancellationToken)
2018-09-11T15:17:42.0161742Z    at ProjectName.IntegrationTests.MongoDatabaseTests.MongoDbCollectionShould.ReturnEmptyCollection() in /home/vsts/work/1/s/WorkDir/Project/ProjectName.IntegrationTests/MongoDatabaseTests.cs:line 49```

[xShivan]

I'm experiencing a very similar problem on my private VSTS build agents (both Linux and Windows Server).

[galczo5]

Same here.
Here is output:

 System.TimeoutException : A timeout occured after 30000ms selecting a server using CompositeServerSelector{ Selectors = MongoDB.Driver.MongoClient+AreSessionsSupportedServerSelector, LatencyLimitingServerSelector{ AllowedLatencyRange = 00:00:00.0150000 } }. Client view of
cluster state is { ClusterId : "3", ConnectionMode : "Automatic", Type : "Unknown", State : "Disconnected", Servers : [{ ServerId: "{ ClusterId : 3, EndPoint : "127.0.0.1:27018" }", EndPoint: "127.0.0.1:27018", State: "Disconnected", Type: "Unknown" }] }.
Ślad stosu:
   at MongoDB.Driver.Core.Clusters.Cluster.ThrowTimeoutException(IServerSelector selector, ClusterDescription description)
   at MongoDB.Driver.Core.Clusters.Cluster.WaitForDescriptionChangedHelper.HandleCompletedTask(Task completedTask)
   at MongoDB.Driver.Core.Clusters.Cluster.WaitForDescriptionChangedAsync(IServerSelector selector, ClusterDescription description, Task descriptionChangedTask, TimeSpan timeout, CancellationToken cancellationToken)
   at MongoDB.Driver.Core.Clusters.Cluster.SelectServerAsync(IServerSelector selector, CancellationToken cancellationToken)
   at MongoDB.Driver.MongoClient.AreSessionsSupportedAfterSeverSelctionAsync(CancellationToken cancellationToken)
   at MongoDB.Driver.MongoClient.AreSessionsSupportedAsync(CancellationToken cancellationToken)
   at MongoDB.Driver.MongoClient.StartImplicitSessionAsync(CancellationToken cancellationToken)
   at MongoDB.Driver.MongoCollectionImpl`1.UsingImplicitSessionAsync[TResult](Func`2 funcAsync, CancellationToken cancellationToken)
   at MongoDB.Driver.MongoCollectionBase`1.InsertOneAsync(TDocument document, InsertOneOptions options, Func`3 bulkWriteAsync)
   at Flexi.Project.BusinessLogic.Tests.ProjectService.UpdateCommand.Updates_Project() in C:\dev\FlexiProject\Flexi.Project.BusinessLogic.Tests\ProjectService\UpdateCommand.cs:line 46
--- End of stack trace from previous location where exception was thrown ---

[JohannesHoppe]

Looks like this is not supported with hosted agents on VSTS:

https://stackoverflow.com/a/44662712

If you're using the hosted agent, you can't open up ports or change anything about the machine's configuration. You'll need to set up your own agent for builds

[JohannesHoppe]

@galczo5 If it fails on you local machine, then it's a different issue.

[JohannesHoppe]

Have you tried to use the "Hosted Windows Container". That guy might have the rights to open ports?

[DavidLievrouw]

I had a similar issue. Fixed it by installing Microsoft Visual C++ 2015 Redistributable Update 3 RC on my private agent.

On a hosted agent (VS2017), it worked just fine.

Here is a link to a sample solution, including a yaml file for the VSTS pipeline: MongoDbBackedIntegrationTests. Hope it helps.

[xShivan]

@DavidLievrouw, thanks for the suggestion. Unfortunately, it hasn't solved the problem on my private Windows build agent, also I'm experiencing this issue on my MacOS dev machines (both 10.13 High Sierra and 10.14 Mojave).

[JohannesHoppe]

On a hosted agent (VS2017), it worked just fine.

So it just worked for you? I'm confused, what is the trick? 😕

[DavidLievrouw]

I'm confused, what is the trick?

Don't know. I just created a minimal setup here. The yaml file with the VSTS build pipeline is also there.
Maybe you can try to start from that project, and when that works, add your tests there?
jm2c

[p10tyr]

Well my xUnit seems to run in "Azure Devops" 🤕 but after the test completes it does not go to the next step. Running it on a paid for dedicated Azure Agent

It just waits around.. and never completes..

So I implemented IDisposable to force disposed the _runner. It fixed it and the build carries on after dotnet test now

 public class MongoIntegrationTest : IDisposable
    {
        internal static MongoDbRunner _runner;
        internal static IMongoCollection<Domain.Read.DTO.Policyholder> _collection;
        internal static string _databaseName = "IntegrationTest";
        internal static string _collectionName = "TestCollection";

        internal static void CreateConnection()
        {
            _runner = MongoDbRunner.Start();

            MongoClient client = new MongoClient(_runner.ConnectionString);
            IMongoDatabase database = client.GetDatabase(_databaseName);
            _collection = database.GetCollection<Domain.Read.DTO.Policyholder>(_collectionName);


        }


        //This must be here for Azure to close the connection otherwise it hangs around forever after running the tests
        public void Dispose()
        {
            _runner.Dispose();
        }
    }

This is the log.. when not forcing dispose. All looks fine

[command]C:\agent\_work\_tool\dncs\2.1.402\x64\dotnet.exe test C:\agent\_work\31\s\.\private.private.Subscriptions.Tests.Integration\private.private.Subscriptions.Tests.Integration.csproj --logger trx --results-directory C:\agent\_work\_temp
Build started, please wait...
Build completed.

Test run for C:\agent\_work\31\s\private.private.Subscriptions.Tests.Integration\bin\Debug\netcoreapp2.1\private.private.Subscriptions.Tests.Integration.dll(.NETCoreApp,Version=v2.1)
Microsoft (R) Test Execution Command Line Tool Version 15.8.0
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
2018-10-10T15:20:39.097+0100 I CONTROL  [initandlisten] MongoDB starting : pid=316 port=27018 dbpath=C:\Windows\ServiceProfiles\NetworkService\AppData\Local\Temp\wuecdb10.qlu3ef1b3da0f2144b5b343_27018 64-bit host=private-build-01

2018-10-10T15:20:39.097+0100 I CONTROL  [initandlisten] targetMinOS: Windows 7/Windows Server 2008 R2

2018-10-10T15:20:39.097+0100 I CONTROL  [initandlisten] db version v3.6.1

2018-10-10T15:20:39.097+0100 I CONTROL  [initandlisten] git version: 025d4f4fe61efd1fb6f0005be20cb45a004093d1

2018-10-10T15:20:39.097+0100 I CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.0.1u-fips  22 Sep 2016

2018-10-10T15:20:39.097+0100 I CONTROL  [initandlisten] allocator: tcmalloc

2018-10-10T15:20:39.097+0100 I CONTROL  [initandlisten] modules: none

2018-10-10T15:20:39.097+0100 I CONTROL  [initandlisten] build environment:

2018-10-10T15:20:39.097+0100 I CONTROL  [initandlisten]     distmod: 2008plus-ssl

2018-10-10T15:20:39.097+0100 I CONTROL  [initandlisten]     distarch: x86_64

2018-10-10T15:20:39.097+0100 I CONTROL  [initandlisten]     target_arch: x86_64

2018-10-10T15:20:39.097+0100 I CONTROL  [initandlisten] options: { net: { bindIp: "127.0.0.1", port: 27018, ssl: { mode: "disabled" } }, storage: { dbPath: "C:\Windows\ServiceProfiles\NetworkService\AppData\Local\Temp\wuecdb10.qlu3ef1b3da0f2144b5b343_27018", journal: { enabled: false } } }

2018-10-10T15:20:39.098+0100 I STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=1535M,session_max=20000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),statistics_log=(wait=0),verbose=(recovery_progress),,log=(enabled=false),

2018-10-10T15:20:39.307+0100 I CONTROL  [initandlisten] 

2018-10-10T15:20:39.307+0100 I CONTROL  [initandlisten] ** WARNING: Access control is not enabled for the database.

2018-10-10T15:20:39.307+0100 I CONTROL  [initandlisten] **          Read and write access to data and configuration is unrestricted.

2018-10-10T15:20:39.307+0100 I CONTROL  [initandlisten] 

2018-10-10T15:20:39.307+0100 I CONTROL  [initandlisten] Hotfix KB2731284 or later update is not installed, will zero-out data files.

2018-10-10T15:20:39.307+0100 I CONTROL  [initandlisten] 

2018-10-10T15:20:39.307+0100 I CONTROL  [initandlisten] ** WARNING: The file system cache of this machine is configured to be greater than 40% of the total memory. This can lead to increased memory pressure and poor performance.

2018-10-10T15:20:39.307+0100 I CONTROL  [initandlisten] See http://dochub.mongodb.org/core/wt-windows-system-file-cache

2018-10-10T15:20:39.307+0100 I CONTROL  [initandlisten] 

2018-10-10T15:20:39.308+0100 I STORAGE  [initandlisten] createCollection: admin.system.version with provided UUID: 2eacfe0b-55d9-4ae6-b015-54884a5597cd

2018-10-10T15:20:39.385+0100 I COMMAND  [initandlisten] setting featureCompatibilityVersion to 3.6

2018-10-10T15:20:39.388+0100 I STORAGE  [initandlisten] createCollection: local.startup_log with generated UUID: 0e6f8380-77be-4a37-bdaa-fd25bc4c6966

2018-10-10T15:20:40.552+0100 I FTDC     [initandlisten] Initializing full-time diagnostic data capture with directory 'C:/Windows/ServiceProfiles/NetworkService/AppData/Local/Temp/wuecdb10.qlu3ef1b3da0f2144b5b343_27018/diagnostic.data'

2018-10-10T15:20:40.554+0100 I NETWORK  [initandlisten] waiting for connections on port 27018

2018-10-10T15:20:41.376+0100 I NETWORK  [listener] connection accepted from 127.0.0.1:18352 #1 (1 connection now open)

2018-10-10T15:20:41.420+0100 I NETWORK  [conn1] received client metadata from 127.0.0.1:18352 conn: { driver: { name: "mongo-csharp-driver", version: "2.7.0.0" }, os: { type: "Windows", name: "Microsoft Windows 6.1.7601 S", architecture: "x86_64", version: "6.1.7601" }, platform: ".NET Core 4.6.26814.03" }

2018-10-10T15:20:41.545+0100 I NETWORK  [listener] connection accepted from 127.0.0.1:18353 #2 (2 connections now open)

2018-10-10T15:20:41.548+0100 I NETWORK  [conn2] received client metadata from 127.0.0.1:18353 conn: { driver: { name: "mongo-csharp-driver", version: "2.7.0.0" }, os: { type: "Windows", name: "Microsoft Windows 6.1.7601 S", architecture: "x86_64", version: "6.1.7601" }, platform: ".NET Core 4.6.26814.03" }

2018-10-10T15:20:41.599+0100 I STORAGE  [conn2] createCollection: IntegrationTest.TestCollection with generated UUID: 7e68191c-e19a-46f0-a1f5-46cabe3c5d16

Results File: C:\agent\_work\_temp\private-BUILD-01$_private-BUILD-01_2018-10-10_15_20_42.trx

Total tests: 1. Passed: 1. Failed: 0. Skipped: 0.
Test Run Successful.
Test execution time: 5.1692 Seconds

[JohannesHoppe]

They just renamed "VSTS" to "Azure Devops". I haven't spotted any relevant difference except the rebranding.

[p10tyr]

@JohannesHoppe Azure DevOps is all the same it is just Azure and VisualStudio.com banged into a new DevOps dashboard. It is pretty cool but in case of functionality still depends what frameworks and OS you are using under the hood.

[JohannesHoppe]

I see.

regarding the MongoDbRunner:

That might be the reason for everything. The runner should always be disposed. See the original example in the README:

    Cleanup stuff = () => _runner.Dispose();

https://github.com/Mongo2Go/Mongo2Go#examples

[p10tyr]

I completely missed that!

Why does it not hang around when I run it on my local machine from the command line? Does that mean I still have several instances of MongoDB running in the background somewhere?

There clearly must be a problem on Azure DevOps not calling the finalizer properly??

[JohannesHoppe]

That might be possible. But chances are high that the garbage collector calls the finalizer wich calls Dispose(false) for you (see here). But it's a bad practice to rely on this.

@ppumkin Do you use the big Visual Studio (not VSCode) while developing and dotnet test on VSTS? If yes, what happens if you forget to dispose locally while running dotnet test?

The overall experience highly depends on the concrete runner. E.g. we also had the opposite problem where the runner suddenly killed process so that nothing was cleaned up. (see #17)

[p10tyr]

When I have no forced Dispose with IDisposable

• OK- I use Visual Studio Pro - integrated Test Runner
• OK - I run from command line outside of any IDE dotnet test Subscriptions.Tests.Integration.csproj --logger trx --results-directory C:\agent\_work which is the same command that is run on Azure - You can check the log above.
• NOK - The same command in Azure will keep the process hanging after Tests completed

When I force Dispose with IDisposable

• OK- I use Visual Studio Pro - integrated Test Runner
• OK - I run from command line outside of any IDE dotnet test Subscriptions.Tests.Integration.csproj --logger trx --results-directory C:\agent\_work which is the same command that is run on Azure - You can check the log above.
• OK - The same command in Azure now works and build continues to next step

SDK installed locally and on Azure build agent = 2.1.402

[p10tyr]

I have just checked my processes.. 😆 I have many mogod.exe instances running after each dotnet test I did.

This looks like some kind of Mongo2Go problem not disposing the connection once the Parent thread goes away (or it is a feature 👍 I think you guys need to decide and document it a bit better on what happens)

[JohannesHoppe]

@ppumkin If the runner kills Mongo2Go then Mondo2Go can't kill the mongod. See here:

Mongo2Go/src/Mongo2Go/Helper/MongoDbProcess.IDisposable.cs

Line 45 in 14dd755

_process.Kill();

@donald2379 Can you investigate on this?
Does the solution (implementing IDisposable correctly) also fixes your problem?

[JohannesHoppe]

Same for @bmarinov, @xShivan and @galczo5. Please report if you have implemented IDisposable correctly!

[xShivan]

Working on the same codebase as @galczo5 here.

[p10tyr]

So by the end of the working day I have more than 20 processes running.

I use xUnit TestCollections in which I start the Mongo2Go session so its available to all subsequent tests in that test collection (class)

After all tests are finished it calls the Dispose of the ICollectionFixture (it actually forces you to implement IDisposable) so I am not sure why there are still processes hanging around?

[p10tyr]

@JohannesHoppe hint.. hint, nudge.. nudge 💻 👨‍💻
I'll give it a try...

[p10tyr]

Maybe its the way tests are run in parallel? The only reason a Singleton would not be a Singleton is if it sand boxed in completely different threads? Correct? I suppose that is what is happening when I keep running tests normally, i just see allot of instances hanging around afterwards.

[lukpep]

hmmm I using mongo2go with hundreds of integration tests on azure DevOps - not a single problem.
Can I help You guys with this somehow?

azure hosted (free zone) agents

[JohannesHoppe]

Could you post your exact configuration here?

[CarmineDeloitte]

Hello guys, is this issue resolved? Because with 3.1.3 we still have the same problem

[mohammad-eydimorad]

The 2.2.16 version worked for me as well. I'm using Gitlab pipelines.