Installing AWSPowerShell module on hosted agent takes 8 minutes about aws-toolkit-azure-devops

[stevejroberts]

In the traces I've been watching I've never seen a progress bar get output but I'll take a look to see if the suggested setting speeds things up a bit more.

Microsoft came back this morning with a suggestion to use Install-Module with the -AllowClobber switch but on testing I found it was still taking on average 2 minutes for Install-Module to complete, still slower than save+import so I plan on keeping the latter for now.

[r2690698]

any update on this, its still taking 5+ mins to install

[stevejroberts]

Yeah, I'd noticed this too and have not yet been able to find any reason for it since an install-module call in my shells takes nowhere near that long. Right now I recommend installing the module to the build agent outside of Team Services (if you can); the task will detect it's installed and just bypass the installation code.

[edit] Just noticed that you mentioned the hosted agent, so pre-installing won't help. Have you tried doing the install in a separate step in your build, ahead of the use of the AWSPowerShell task? Curious what timings you get then.

[monnecc]

Thanks for your reaction, Steve. Is it possible to install this to a hosted build server somehow of define this as a prerequisite? And will you be actively working to resolve this or should I avoid using the AWS Powershell task?

[stevejroberts]

We'll certainly take another look at it, especially now that PowerShell Core has been made generally available and we're getting requests to do similar operations with the AWSPowerShell.NetCore module.

I just ran two builds to see if there was any difference in having 'our' task do the module installation vs adding a specific PowerShell task to handle it at the start of the build but didn't see any improvement time-wise. In the additional install-specific task, I see a bunch of output from the command and then a very long pause right after the message regarding the Authenticode signature. This is of course buried inside Install-Module and PowerShellGet so no immediate idea what could be causing the long pause.

The command I ran in the new task is identical to that in our task:

Install-Module AWSPowerShell -Scope CurrentUser -Force -Verbose

The output:

---

Starting: PowerShell Script

---

==============================================================================
Task : PowerShell
Description : Run a PowerShell script
Version : 1.2.3
Author : Microsoft Corporation
Help : More Information

. 'd:\a_temp\d3031ffa-fb12-4959-91e6-a66c645d7071.ps1'
VERBOSE: Acquiring providers for assembly: C:\Program
Files\WindowsPowerShell\Modules\PackageManagement\1.1.7.0\fullclr\Microsoft.PackageManagement.MsuProvider.dll
VERBOSE: Acquiring providers for assembly: C:\Program
Files\WindowsPowerShell\Modules\PackageManagement\1.1.7.0\fullclr\Microsoft.PackageManagement.CoreProviders.dll
VERBOSE: Acquiring providers for assembly: C:\Program
Files\WindowsPowerShell\Modules\PackageManagement\1.1.7.0\fullclr\Microsoft.PackageManagement.ArchiverProviders.dll
VERBOSE: Acquiring providers for assembly: C:\Program
Files\WindowsPowerShell\Modules\PackageManagement\1.1.7.0\fullclr\Microsoft.PackageManagement.MetaProvider.PowerShell.d
ll
VERBOSE: Acquiring providers for assembly: C:\Program
Files\WindowsPowerShell\Modules\PackageManagement\1.1.7.0\fullclr\Microsoft.PackageManagement.NuGetProvider.dll
VERBOSE: Acquiring providers for assembly: C:\Program
Files\WindowsPowerShell\Modules\PackageManagement\1.1.7.0\fullclr\Microsoft.PackageManagement.MsiProvider.dll
VERBOSE: Using the provider 'PowerShellGet' for searching packages.
VERBOSE: The -Repository parameter was not specified. PowerShellGet will use all of the registered repositories.
VERBOSE: Getting the provider object for the PackageManagement Provider 'NuGet'.
VERBOSE: The specified Location is 'https://www.powershellgallery.com/api/v2/' and PackageManagementProvider is
'NuGet'.
VERBOSE: Searching repository 'https://www.powershellgallery.com/api/v2/FindPackagesById()?id='AWSPowerShell'' for ''.
VERBOSE: Total package yield:'1' for the specified package 'AWSPowerShell'.
VERBOSE: Performing the operation "Install-Module" on target "Version '3.3.219.0' of module 'AWSPowerShell'".
VERBOSE: The installation scope is specified to be 'CurrentUser'.
VERBOSE: The specified module will be installed in 'C:\Users\VssAdministrator\Documents\WindowsPowerShell\Modules'.
VERBOSE: The specified Location is 'NuGet' and PackageManagementProvider is 'NuGet'.
VERBOSE: Downloading module 'AWSPowerShell' with version '3.3.219.0' from the repository
'https://www.powershellgallery.com/api/v2/'.
VERBOSE: Searching repository 'https://www.powershellgallery.com/api/v2/FindPackagesById()?id='AWSPowerShell'' for ''.
VERBOSE: InstallPackage' - name='AWSPowerShell', version='3.3.219.0',destination='d:\a_temp\784269265'
VERBOSE: DownloadPackage' - name='AWSPowerShell',
version='3.3.219.0',destination='d:\a_temp\784269265\AWSPowerShell\AWSPowerShell.nupkg',
uri='https://www.powershellgallery.com/api/v2/package/AWSPowerShell/3.3.219'
VERBOSE: Downloading 'https://www.powershellgallery.com/api/v2/package/AWSPowerShell/3.3.219'.
VERBOSE: Completed downloading 'https://www.powershellgallery.com/api/v2/package/AWSPowerShell/3.3.219'.
VERBOSE: Completed downloading 'AWSPowerShell'.
VERBOSE: Hash for package 'AWSPowerShell' does not match hash provided from the server.
VERBOSE: InstallPackageLocal' - name='AWSPowerShell', version='3.3.219.0',destination='d:\a_temp\784269265'
VERBOSE: Catalog file 'AWSPowerShell.cat' is not found in the contents of the module 'AWSPowerShell' being installed.
VERBOSE: Valid authenticode signature found in the file 'AWSPowerShell.psd1' for the module 'AWSPowerShell'.

>>>Long pause (several minutes) here

VERBOSE: Module 'AWSPowerShell' was installed successfully to path
'C:\Users\VssAdministrator\Documents\WindowsPowerShell\Modules\AWSPowerShell\3.3.219.0'.

---

Finishing: PowerShell Script

---

[monnecc]

Yes, I tried this as well. Same result.

[stevejroberts]

Unfortunately we've not had resources to look at this further.

I installed the module from the gallery on an EC2 instance this morning (as part of some unrelated work) and noticed that although not 5mins in length, I too saw a long pause after the Install-Module cmdlet reported that the module was Authenticode signed and before the completion message (in verbose mode) the same as you. I've seen variable pauses on other machines, outside of VSTS development work, too.

I think we'll need to open an issue on the PowerShell UserVoice forums to see if Microsoft can clarify what's going on in this area.

[stevejroberts]

At a PowerShell user group meeting last night one of the attendees mentioned perhaps using http download of the module instead of the gallery cmdlets when we were discussing this problem so I'm going to take a look at that once I get back to Seattle at the end of June (currently in Europe).

[karl-barbour]

@steveataws any joy?

[stevejroberts]

Kind of.

I'm looking at using save-module instead of install-module, and updating the task to do an explicit import from the path that save-module takes (which would be in temporary storage for the task). This seems faster on my workstation and doesn't have the long output-less lag that we have with install-module. I have not tested it on a VSTS account yet though.

[stevejroberts]

Small update. Yesterday I tested an approach that used Save-Module to a temp location on the build host, then an import of the module from the download location. Trace data shows a time-to-save of around 50secs. This is in comparison with times I'm seeing when we use Install-Module of 240secs+.

We're prepping a new updated release of the tools for later this month with some other changes and enhancements, we'll be shipping the change to use Save-Module in it too.

[karl-barbour]

@steveataws thanks for the update.

I know from experience that a progress bar can significantly slow down an Invoke-WebRequest in PowerShell so we tend to disable that. I wonder if this is also causing issues with the module cmdlets.

Take a look at this, for example: https://stackoverflow.com/questions/28682642/powershell-why-is-using-invoke-webrequest-much-slower-than-a-browser-download

Could you maybe compare timings with $ProgressPreference = 'SilentlyContinue' to see if it helps further?

[stevejroberts]

Resolving, change to use Save-Module instead of Install-Module was included in v1.1.0 released Aug 2nd. Not sure that we can squeeze any more time out of the PowerShellGet cmdlets but if 'install' time still seems overly long feel free to reopen.