InsertionsClient

The InsertionsClient, .NET Core command line executable, is a tool that is used to insert a drop into Visual Studio by updating default.config, .packageconfig and .props files in Visual Studio repo.

The tool works locally, meaning that the Visual Studio repo should already be checked-out and available on the file system. Similarly, the changes made by the tool should be manually committed and pushed back to the repo.

Table of Contents

How it works
Input
Logging
Usage
Output
Codebase

How it works

What InsertionsClient does can be summarized in the following steps:

Iterates through the assets in the manifest file and matches them with the NuGet packages listed in default.config and .packageconfig files.
Version numbers of the matching NuGet packages in config files are updated with the version numbers in the manifest.json file.
If an access token is specified with -a: switch, binaries for the updated packages are downloaded. Contents of the packages are used to update the values of properties defined under PackagePreprocessorDefinitions tag in .props files.
Modified .props files are saved.
Updated default.config and .packageconfig files are saved.

Input

This section explains the options you can specify when invoking the tool. Although there are many, only two of them are required.

Switch	Description	Is Optional	Example
-d:	Path to the default.config file.	❌	`-d:c:\default.config`
-m:	Path to the manifest.json file or to the containing directory.	❌	`-m:c:\files\manifest.json` or `-m:c:\files`
-i:	Path to ignored packages file.	✔️	`-i:c:\files\ignore.txt`
-idut	Indicates that the packages relevant to the .NET Dev UX team are ignored. If both set, -i: option overrides this switch.	✔️	`-idut`
-wl:	Path to the allowlist file. If this is specified, only the packages listed in this file will be updated. Each line in the file represents a regex pattern that will potentially match one or more package IDs. This switch will be ignored if the given file is empty.	✔️	`-wl:c:\files\allowlist.txt`
-p:	Path to the directory to search for .props files. If left unspecified, all the .props files under src\SetupPackages in the local VS repo will be searched.	✔️	`-p:C:\VS\src\SetupPackages`
-a:	Personal access token with "read" access to the packages in the VS feed. If not specified, props files will not be updated.	✔️	`-a:vv8ofhtojf7xuhehrFxq9k5zvvxstrqg2dzsedhlu757`
-w:	Maximum allowed duration in seconds for completing the insertions, excluding downloads.	✔️	`-w:60`
-ds:	Maximum allowed duration in seconds that can be spent downloading nuget packages	✔️	`-ds:240`
-c:	Maximum level of concurrency.	✔️	`-c:10`
-bf:	Filter string to specify which builds from the manifest will be inserted.	✔️	`-bf:repo=.*core-setup`

More on -m:

It is also possible to input multiple manifest files. To do this, use semicolon (;) to separate the paths as such:

-m:c:\files\manifest.json;\relative\folder\\;c:\another\fullpath\manifest.json

If the same package is updated from multiple manifest files, which manifest file will be used to set the final value is nondeterministic.

More on -bf:

A simple build filter can be written as follows:

-bf:repo=.*core-setup

Right side of the equals sign is a regular expression that should fully match the value of the build property that is given on the left side. Thus, this filter only inserts builds where the repo property ends with the word core-setup.

The previous example contained only one rule: repo=.*core-setup. If you want to specify multiple rules, you can separate them with a comma. Such as:

-bf:repo=.*core-setup,channelId=1299

A set of rules separated by commas is called a "ruleset". For a build to pass the filter and get inserted, it should comply with all the rules within any ruleset.

A more complicated example of this could be:

-bf:repo=.*core-setup,channelId=1299;repo=.*,channelId=972

Which means that a build can be inserted if:

a. The repo name ends with "core-setup" and the channels list contains a channel with the id "1299"

b. Or, the repo name can be anything, but one of the channel ids should be "972"

As you can see, multiple rulesets can be specified using semicolons. A build only needs to comply with one of the rulesets to be inserted.

For each rule, the word left of the = sign represents a build property. Build property can have the following values: repo, commit, branch,buildNumber. There are also two special properties named channel and channelId. If channel is used, then the regular expression on the right side of the equals sign should match with the name of any of the channels of the build. Similarly, channelId is used to select builds where any of the channel ids of a build should match the given regular expression.

Warnings

No spaces are allowed in default.config or manifest.json file paths
No spaces are allowed in props file search directory
The default duration and concurrency values should suffice
When using build filters with -bf switch, special characters in the regular expression should be properly escaped.

Logging

The tool creates a log, detailing every step taken.
The logs are placed in the Logs folder under the current directory.
Full path to the generated log file is display at the end of the program.

Log Format

Each log line details:

The time stamp when the message was logged
The id of the thread where the message was logged
The message string

Sample Log Lines

12-3-2020 11:59:16.114133|thread:1|CMD line param. Specified default.config: C:\Users\bozturk\source\repos\VS\.corext\Configs\default.config
12-3-2020 11:59:16.122993|thread:1|CMD line param. Specified manifest.json: C:\Users\bozturk\source\repos\InsertionsClient\tests\InsertionsClientTest\Assets\manifest.json
12-3-2020 11:59:16.349088|thread:1|De-serialized 19 builds from manifest.json.
12-3-2020 11:59:16.380310|thread:1|Loading default.config content from C:\Users\bozturk\source\repos\VS\.corext\Configs\default.config.
12-3-2020 11:59:16.412066|thread:1|Loaded default.config content.
12-3-2020 11:59:16.419213|thread:1|Loading content of .packageconfig at C:\Users\bozturk\source\repos\VS\.corext\Configs\Microsoft.Developer.IdentityService\IdentityService.packageconfig.
12-3-2020 11:59:16.421827|thread:1|Loaded .packageconfig content.

Usage

Launch a command line session.
Navigate to the location of InsertionsClient.exe
Alternatively, add the directory containing the InsertionsClient.exe to the PATH environment variable.
Launch InsertionsClient.exe with proper arguments.

Examples

The examples below rely on the following conditions:

InsertionsClient.exe is located under the directory "\tools"
default.config is located under the directory "\repos\Assets"
manifest.json is located under the directory "\repos\Assets"

Opting to Specify File with NuGet Packages to Ignore

If you have an ignored.txt file located under "\repos\Assets", you can run the following command to do an insertion while preserving the version numbers of packages specified in the ignored.txt:

$ \tools\InsertionsClient.exe -d:\repos\Assets\default.config -m:\repos\Assets\manifest.json -i:\repos\Assets\ignored.txt

Restricting the Affected Packages With an Allowlist

Contents of a simple allowlist.txt can be as follows: ^VS\.Redist\.Common\.NetCore\.SharedFramework\.(x86|x64)\.[0-9]+\.[0-9]+$

If you have an allowlist.txt file located under "\repos\Assets", the following command can be used:

$ \tools\InsertionsClient.exe -d:\repos\Assets\default.config -m:\repos\Assets\manifest.json -wl:\repos\Assets\allowlist.txt

Opting to Ignore .NET Dev UX NuGet Packages

$ \tools\InsertionsClient.exe -d:\repos\Assets\default.config -m:\repos\Assets\manifest.json -idut

Opting Not to Ignore any NuGet Packages

$ \tools\InsertionsClient.exe -d:\repos\Assets\default.config -m:\repos\Assets\manifest.json

Specifying an Access Token to Update .props Files

$ \tools\InsertionsClient.exe -d:\repos\Assets\default.config -m:\repos\Assets\manifest.json -a:vv8ofhtojf7xuhehroaq9k5zvvxstrqg2dzsedhlu757

Specifying a .props File Search Directory

$ \tools\InsertionsClient.exe -d:\repos\Assets\default.config -m:\repos\Assets\manifest.json -a:vv8ofhtojf7xuhehrFxq9 -p:C:\VS\src\SetupPackages\DotNetCoreSDK

Output

InsertionsClient.exe outputs the results of the running operations both to a persistent log file and to the console output.

Every successfully updated package version in a default.config or _.packageconfig file has a corresponding message such as:

12-3-2020 11:59:16.694366|thread:6|Package VS.Redist.Common.WindowsDesktop.SharedFramework.x64.3.1 was updated to version 3.1.2-servicing.20067.4

Completion statistics summarize the time spent on each asset in the manifest file.

Statistics:

ExactMatch - Update for case of matching manifest.json assets with multiple verions; where version of the exact matching NuGet asset was selected
No. Items: 31
Average: 0.0041-ms
Minimum: 0.0005-ms
Maximum: 0.0269-ms

NoMatch - No matching manifest.json assets for a given default.config NuGet
No. Items: 528
Average: 0.0042-ms
Minimum: 0.0003-ms
Maximum: 0.0010-ms

Completion Summary summarizes the total duration of the operation and the updated package versions as follows:

Duration: 480.00-ms.
Successful updates: 27.
Updated default.config NuGet package versions...
        VS.Redist.Common.AspNetCore.SharedFramework.x64.3.1, version: 3.1.2-servicing.20068.1
        VS.Redist.Common.AspNetCore.SharedFramework.x86.3.1, version: 3.1.2-servicing.20068.1
        ...

Codebase

Information about the codebase can be found in Codebase page.

isabella232 / insertions-client Goto Github PK