Failing and ignored tests about msbuild

[am11]

My results are varying. Only one assertion fails: ToolLocationHelper_Tests.cs#L804.

However, there are 227 skipped tests, which is due to the [Ignore] attribute, which explicitly disables a test case:
https://msdn.microsoft.com/en-us/library/microsoft.visualstudio.testtools.unittesting.ignoreattribute.aspx.

Ex: ProjectRootElementCache_Tests.cs#L108:

[Ignore] // "This test seems to be flaky depending on when garbage collection happened"

[barbosatek]

Seems some of the tests are failing because of threading issues when running all the tests.

After running only the 4 failing tests, only one fails:

Which actually fails because it's expecting my enviroment to be Windows 10 rather than Wondows 8.1

Test Name:  GetPathToDotNetFrameworkSdk
Test FullName:  Microsoft.Build.UnitTests.ToolLocationHelper_Tests.GetPathToDotNetFrameworkSdk
Test Source:    c:\Users\v-hobarb\Documents\GitHub\msbuild\src\Utilities\UnitTests\ToolLocationHelper_Tests.cs : line 750
Test Outcome:   Failed
Test Duration:  0:00:00.0298146

Result Message: Assert.AreEqual failed. Expected:<C:\Program Files (x86)\Microsoft SDKs\Windows\v8.1A\>. Actual:<C:\Program Files (x86)\Microsoft SDKs\Windows\v10.0A\>.
Result StackTrace:  at Microsoft.Build.UnitTests.ToolLocationHelper_Tests.GetPathToDotNetFrameworkSdk() in c:\Users\v-hobarb\Documents\GitHub\msbuild\src\Utilities\UnitTests\ToolLocationHelper_Tests.cs:line 804

Same results when running them with ReSharper

[AndyGerlicher]

All the tests that are not marked with [Ignore] should be working. There are quite a few I had to disable migrating it out of our internal environment though. Most of those are tagged with a comment like // Ignore: Test requires installed toolset.. Ideally all the tests should be enabled (or deleted if they don't apply anymore), so we are definitely open to PR for fixes there :).

The biggest difficulty was that the tests often targeted the version of MSBuild that is installed on your machine in the GAC. We had a cmd file to update the machine so that you would be (hopefully) testing what you just built and not the retail version, but this is definitely not ideal. This has been around and not changed for ages, so we couldn't get to all of them. And without doing some sort of signing that method doesn't work anyway.

If you do have specific issues I'd like to see more information and if it repros running the test by itself or not. The machine I used for the majority of the testing was Windows 10 Server (9926) and Visual Studio 2015 (CTP6).