poshcode / configuration

A module to help other modules have settings

PoshCode Package Manager Module (BETA)

PoshCode's purpose is to make it easier to distribute PowerShell modules and scripts over the internet or within local and corporate networks.

With this new project we are focusing on making it easy to distribute modules without explaining module installation to your users.

Additionally, we're supporting the automatic installation of dependencies, so that you can distribute modules which have a dependency on other modules without having to worry about how your users will find and install them.

The module has two main commands for users, and a third for module developers:

1. Find-Module searches configured module registries and repositories
2. Install-Module installs modules from local packages (.nupkg or .zip files), or from URLs or UNC paths
3. Compress-Module creates a redistributable module package from a module on your system

To install the PoshCode module

If your "WebClient" service is running (this is Window's built-in WebDAV client), you can install it straight from our server with a single command in any version of PowerShell:

    \\PoshCode.org\DavWWWRoot\Modules\Install.ps1

If you have problems with that (various things can make Windows WebDAV slow, and the service doesn't seem to be installed by default on server OSes), you will need to download and run our Install.ps1 script. Of course, you can still do that from PowerShell:

On PowerShell 3 and up you can do that using Invoke-WebRequest:

    # First download, then run, then delete the installer:
    iwr http://PoshCode.org/i -OutF PC.ps1; .\PC; rm .\PC.ps1

On PowerShell 2 you need to create and use a WebClient:

    (New-Object System.Net.WebClient).DownloadFile("http://poshcode.org/i","$pwd\pc.ps1"); .\PC; rm .\PC.ps1

The rest of the documentation will be in the wiki, broken into several sections:

1. Installing Modules (user's guide)
2. Creating Module Packages
3. Distributing Module Packages (should include the "how to" for enterprise users)
4. Additional Features of PoshCode (everything else)

Note: the additional features of PoshCode include coverages of some of the mini modules I wrote to support the Package Manager functionality, mostly around Serialization and Configuration.

configuration's Issues

Add-MetadataConverter is always called, even with no converters supplied

This code block from #46 doesn't have the expected effect (https://github.com/PoshCode/Configuration/blob/main/Source/Header/param.ps1):

param(
    $Converters = @{},
    $EnterpriseData,
    $UserData,
    $MachineData
)

if ($Converters) {
    Add-MetadataConverter $Converters
}

Even empty hashtables are truthy, so the if branch is always followed.

I'm trying to use these modules without exporting to the global scope, and this call causes the import of the Metadata module in the global scope. Without this call, I can import the modules within my own module and not leak to the global scope.

Workaround is to import with ArgumentList:

Import-Module Configuration -ArgumentList @($null)

Warning on First Use of Export-Configuration

The first time Export-Configuration is used, it generates a warning message:

WARNING: The Enterprise path C:\Users\<username>\AppData\Roaming\powershell cannot be found

...and it cannot be suppressed with any of the usual methods:

$callerWarningPreference = $WarningPreference
$WarningPreference = 'SilentlyContinue'

$new | Export-Configuration -WarningAction Ignore

$WarningPreference = $callerWarningPreference

I'm trying to present a clean experience to users and this error message is a bit cryptic and not at all helpful: Export-Configuration creates the path and writes the file without issue. Surely this message is better suited to the Verbose stream? Searching the source, it appears to be coming from Get-ConfigurationPath and the flags to suppress warning aren't being pushed down into its scope.

I suppose I could call Get-ConfigurationPath myself first--suppressing the warning there--but I think the issue still deserves attention in either the code or the documentation.

PowerShell 4 Support

Hi!

If it matters, it looks like this is currently tied to PowerShell 5 or later.

I get the following error when attempting to load/use the module:

$KeyValue = Get-Metadata $Path -PropertyName $PropertyName -Passthru
 +                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ [<<==>>] Exception: Cannot find an overload for "ParseInput" and the argument count: "4".

Looking at constructors for that method, I only see the three parameter version:

OverloadDefinitions
-------------------
static System.Management.Automation.Language.ScriptBlockAst ParseInput(
    string input,
    [ref]System.Management.Automation.Language.Token[] tokens,
    [ref]System.Management.Automation.Language.ParseError[] errors)

PSVersionTable:

Name                           Value
----                           -----
PSVersion                      4.0
WSManStackVersion              3.0
SerializationVersion           1.1.0.1
CLRVersion                     4.0.30319.34014
BuildVersion                   6.3.9600.17400
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0}
PSRemotingProtocolVersion      2.2

Guessing the doc here just doesn't include the fact (I'm assuming this is true) that this was introduced in PowerShell 5.

For my use case, I can switch over to PowerShell 5, but just a heads up.

Side note: Have you considered splitting this up to include an independent Metadata module? It seems odd that there are no generic Cmdlets for writing/updating PowerShell data files...

Cheers!

Please add an example for export-configuration -DefaultPath <path>

As I read the parameter description in the help for Export-Configuration, the DefaultPath parameter is used to copy a configuration file that's distributed with the module to one of the three scope-based locations. As such, I'm not sure I'd expect a hashtable to be passed as input. But clearly, -InputObject remains a required parameter value.

Having an example of your intended use would be helpful.

If I understood how it was to be used, I'd submit PR to update the help.

Thanks!

Question: Latest published version not on Powershell gallery

I wonder if it's intentional that 1.3.1 is the latest version on Powershell gallery.
Any reason?

Cannot import-configuration when the module folder isn't the module name

Even though you can happily test a module as "c:\my-powershell\example.psm1" with import-module -force c:\my-powershell\example.psm1 using import-configuration will create the error below. You'd have to rename the path "c:\example\example.psm1". I appreciate that's the name scheme you've got to have when putting the module into psmodulePath but for development it's a little restrictive.

Inside my .psm1 with $Script:Config = import-configuration results in the following error

Get-Module : The specified module 'N:\Documents\src\ps-schoolgroups' was not found. Update the Name parameter to
point to a valid path, and then try again.
At \\homeDirectory\jbennett\Documents\WindowsPowerShell\Modules\Configuration\1.3.0\Configuration.psm1:99 char:27
+ ...            if($mi2 = Get-Module $mi.ModuleBase -ListAvailable | Where ...
+                          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ResourceUnavailable: (N:\Documents\src\ps-schoolgroups:String) [Get-Module], FileNotFou
   ndException
    + FullyQualifiedErrorId : Modules_ModuleNotFoundForGetModule,Microsoft.PowerShell.Commands.GetModuleCommand

error from line 99

I don't know if this is behaviour worth improving but I couldn't make a workaround if you might be able to suggest one. I have 72 projects in a folder it's quite nice organizing the powershell ones as prefixed with ps-<modulename>. Of course they don't get installed to psmodulepath with that pattern.

The -Scope parameter for Export-Configuration has a parameterset (and it should not)

We need to be able to specify a value for Export-Configuration -Scope regardless of ParameterSet (just like you can for Get-StoragePath).

Currently, it has it's ParameterSet to the ManualOverride set, so you can only use it when specifying the -Name and -Company by hand (as in the first example below).

I want all three of these to work:

Scenario: Specifying the scope when listing metadata
    Given a module with the name 'MyModule1' and the author 'MyCompany' with code like this
    """
    $Config = Import-Configuration -Company 'MyCompany' -Name MyModule1
    if(!$Config.ApiKey) {
        $Config =@{ ApiKey = "My Default API Key" }
        Export-Configuration $Config  -Company 'MyCompany' -Name MyModule1 -Scope Machine
    }
    """
    When the module is imported
    Then a settings file named Configuration.psd1 should exist in the Machine folder

Scenario: Specifying the scope when piping moduleinfo
    Given a module with the name 'MyModule1'
    When a user writes (something like) this
    """
    Get-Module MyModule1 | Export-Configuration @{ ApiKey = "MyApiKey" } -Scope Enterprise
    """
    Then a settings file named Configuration.psd1 should exist in the Enterprise folder

Scenario: Specifying the scope implicitly
    Given a module with the name 'MyModule1' and the author 'MyCompany' with code like this
    """
    function Set-ApiKey {
        param($ApiKey)
        $Config =@{ ApiKey = $ApiKey }
        Export-Configuration $Config -Scope Machine
    }
    """
   When I invoke Set-ApiKey "OurCorporateApiKey"
   Then a settings file named Configuration.psd1 should exist in the Machine folder

Exported scriptblocks including a single-quoted string fail to reimport

Found when using PowerLine.

Original Issue: Jaykul/PowerLine#42

When I run this interactively it works, however when I export it and reopen PowerShell and reimport PowerLine it throws at line 6.

Looking at it more closely it looks like it's some kind of bug in the Configuration module maybe? Or maybe just in the export command. Let me know if you'd rather track this on Configuration.

https://gist.github.com/Chirishman/ad7ab9492f9972d77536e89fb1f02053

Error:

Import-Metadata : Exception calling "Create" with "1" argument(s): "At line:1 char:63
+ New-PromptText -Fg White -Bg 20 (((Get-SegmentedPath) -replace
+                                                               ~
You must provide a value expression following the '-replace' operator.
At line:1 char:63
+ New-PromptText -Fg White -Bg 20 (((Get-SegmentedPath) -replace
+                                                               ~
Missing closing ')' in expression."
At C:\Program Files\WindowsPowerShell\Modules\Configuration\1.3.1\Configuration.psm1:417 char:21
+ ...             Import-Metadata $EnterprisePath -ErrorAction Ignore -Orde ...
+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Import-Metadata], MethodInvocationException
    + FullyQualifiedErrorId : ParseException,Import-Metadata

We need to support references in settings

When I'm configuring complicated things, I may want to be able to define variables which I can reuse in other configurations. For instance, in a script which does software installs:

UserName = "Jaykul"
DownloadTo = "C:\Users\{UserName}\Downloads\Installers"
ConfigFolder = "{DownloadTo}\Configuration"

Currently we support tokenizing only for folder names. Maybe we could make it work for other settings by reference, like {Settings.UserName} by requiring them to be prefixed with Settings. and then resolving them after import (like paths in PoshCode) iterating to allow multiple reference levels:

UserName = "Jaykul"
Paths = @{
   DownloadTo = "C:\Users\{Settings.UserName}\Downloads\Installers"
   ConfigFolder = "{Settings.Paths.DownloadTo}\Configuration"
}

Expose ValidVariables as -AllowedVariables

As a pre-cursor or alternative to #3 how about we just get the ability to specify a list of variables that are allowed (in addition to the built-in PSScriptRoot, PSCulture, PSUICulture, True, False, Null)?

Missing signature

When updating I get:

The version '1.5.0' of the module 'Configuration' being installed is not catalog signed. Ensure that      
the version '1.5.0' of the module 'Configuration' has the catalog file 'Configuration.cat' and signed     
with the same publisher 'CN=Joel H. Bennett, O=Joel H. Bennett, L=West Henrietta, S=New York, C=US'       
as the previously-installed module 'Configuration' with version '1.3.1' under the directory               
'%userprofile%\Documents\PowerShell\Modules\Configuration\1.3.1'. If you still want to install or        
update, use -SkipPublisherCheck parameter.                                                                

Trailing empty lines

Update-Metadata calls Set-Content which adds trailing blank lines when writing $ManifestContent, these accumulate when making multiple changes..

The addition of the NoNewLine parameter to Set-Content calls in Update-Metadata is sufficient to fix this issue.

I don't mind making the change and submitting a pull request, but it's such a small change and it may not be a problem for anyone other than me.

Regards,

Chris

We need the "SpecialFolders" feature from PoshCode

I'd like settings (especially settings stored in the "local machine" path) to be stored with tokens instead of folders, so I could set a setting to "{MyDocuments}\WindowsPowerShell" and have it work for other users.

Incompatibility with PS v4

Assumption

• This module should run on PSv4

Description

Set-Content is using a parameter that is not available on PSv4

Details

I found this by using Update-Metadata of BuildHelpers:
Set-Content in PSv4 does not have a -NoNewLine parameter:

Configuration/Source/Metadata.psm1

Line 595 in 244a7cb

Set-Content -Encoding UTF8 -Path $Path -Value $ManifestContent -NoNewline

Export-Configuration Overwriting Existing Nested Hashtable Values for User

Issue description

When using a nested Hashtable with more than 1 value, using Export-Configuration to save the values individually overwrites the previous values for the nested Hashtable.

Sorry if this format is not good for reporting this bug - tried to be as detailed as possible and provide test/example scenario files.

Steps to reproduce the issue

• See the attached Configuration.psd1.txt and Test.ps1.txt files
  • Configuration.psd1.txt
  • Test.ps1.txt
• Place both in the same folder, removing the .txt extensions
• Execute Test.ps1

What's the expected result?

The end of the output to show BOTH values having been overridden in User Configuration file, and Final Config to reflect BOTH overrides:

*** User Configuration File w Foo.Bar AND Foo.Baz overridden

@{
  Foo = @{
    Bar = 42
    Baz = 8675309
  }
}

*** Final Config
{
  "Foo": {
    "Bar": 42,
    "Baz": 8675309
  }
}

What's the actual result?

The end of the output shows only the MOST RECENT value having been overridden in User Configuration file, and Final Config only reflects the MOST RECENT override:

*** User Configuration File w Foo.Bar AND Foo.Baz overridden

@{
  Foo = @{
    Baz = 8675309
  }
}

*** Final Config
{
  "Foo": {
    "Bar": 1,
    "Baz": 8675309
  }
}

Update-Metadata should not keep adding newlines to the psd1

Editing this again: Now that the -NoNewLine switch was removed from the Set-Content call in the Update-MetaData function, there is a new line added to the psd1 each time it is edited. My idea for a fix would be to just add a .Trim() on the value so that it only ends up with a single newline in all cases.

Configuration keys start with an int break manifest import

Following up on SCRT-HQ/PSProfile#28

It looks like PowerShell doesn't like dictionary keys that start with an int and are not wrapped in quotes, resulting in the parser breaking when attempting to import metadata containing keys like this. Sample to replicate:

@{1BadKey = 'Value'}

Digging into this and sending over a PR asap!

Can't use Configuration in a module ScriptsToProcess script

PS C:\git\junkcode\Powershell\Bravia\AlphaBravia> Import-Module .\AlphaBravia.psd1
Test-Path : Cannot bind argument to parameter 'Path' because it is an empty string.
At C:\Users\Adam\Documents\WindowsPowerShell\Modules\Configuration\1.0.2\Configuration.psm1:322 char:22
+         if(Test-Path $DefaultPath -Type Container) {
+                      ~~~~~~~~~~~~
    + CategoryInfo          : InvalidData: (:) [Test-Path], ParameterBindingValidationException
    + FullyQualifiedErrorId : ParameterArgumentValidationErrorEmptyStringNotAllowed,Microsoft.PowerShell.Commands.Test
   PathCommand

Test-Path : Cannot bind argument to parameter 'Path' because it is an empty string.
At C:\Users\Adam\Documents\WindowsPowerShell\Modules\Configuration\1.0.2\Configuration.psm1:327 char:39
+         $Configuration = if(Test-Path $DefaultPath) {
+                                       ~~~~~~~~~~~~
    + CategoryInfo          : InvalidData: (:) [Test-Path], ParameterBindingValidationException
    + FullyQualifiedErrorId : ParameterArgumentValidationErrorEmptyStringNotAllowed,Microsoft.PowerShell.Commands.Test
   PathCommand

Update-Object : Cannot bind argument to parameter 'InputObject' because it is null.
At C:\Users\Adam\Documents\WindowsPowerShell\Modules\Configuration\1.0.2\Configuration.psm1:362 char:26
+         $Configuration | Update-Object $Machine |
+                          ~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidData: (:) [Update-Object], ParameterBindingValidationException
    + FullyQualifiedErrorId : ParameterArgumentValidationErrorNullNotAllowed,Update-Object

The code I'm calling in ScriptsToProcess is:

. ".\AlphaBravia.ps1"

Import-Configuration -Author "Adam Baxter" -Name "AlphaBravia"

I understand this might be backwards - what I'm trying to do is check if the module has already been configured, and prompt for a server name if not.

Update-Metdata should not change file encoding

Current behavior

Metadata is hardcoded to use -Encoding UTF8.
UTF8 is UTF-8 with BOM.

Configuration/Source/Metadata.psm1

Line 595 in 244a7cb

Set-Content -Encoding UTF8 -Path $Path -Value $ManifestContent -NoNewline

Expected behavior

Metadata should not change the file encoding of the manifest.

Eg: I have my manifest files in UTF-8 without BOM

Get-Metadata doesnot return arrays

Version 1.3.1 in the gallery doesnot return properties that are arrays

Are scripts also able to benefit from this configuration module?

Are powershell scripts also able to benefit from this configuration module?

No access to Machine scope

Description

When Import-Configuration tries to load the config from Machine scope, it fails with:

New-Item : Access to the path '/etc/xdg/powershell/AtlassianPS' is denied.
At /Users/travis/.local/share/powershell/Modules/Configuration/1.2.0/Configuration.psm1:209 char:21
+             $null = New-Item $PathRoot -Type Directory -Force
+                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : PermissionDenied: (/etc/xdg/powershell/AtlassianPS:String) [New-Item], UnauthorizedAccessException
+ FullyQualifiedErrorId : CreateDirectoryUnauthorizedAccessError,Microsoft.PowerShell.Commands.NewItemCommand

See logs at: https://travis-ci.org/AtlassianPS/BitbucketPS/jobs/335077778#L307

Details

On Linux and OSX, the fallback path /etc/xdg/powershell needs sudo permissions.
https://github.com/PoshCode/Configuration/blob/master/Source/Configuration.psm1#L209

Allow the PWD to contain a configuration file that can act as another layer.

It would be nice if you clone a project's git repository, and the repository could contain a configuration file for something like a test framework module, that could override specific machine wide settings that may need to be tweaked on a case by case basis.

Doing this at the user level could be difficult because not only would the user need to know which tweaks need to be made to specific projects, something contributors may not know easily, but a settings schema for the module would need to be able to understand which project you are working on and how to jigger the settings for specific modules.

I am willing to help with a PR on this, but I am creating this issue to start the discussion on whether you think this would be a useful capability.

I would like to use this module as the settings manager for a module I am working on, but being able to specify settings overrides that live in the project repository is a high priority objective for me.

Thanks,
Bill

Allow option to merge, or parameters to allow external merging

Hiyo!

In some cases, one might simply want to set, or add a property to an existing configuration. As is, Export-Configuration simply overwrites everything.

I don't see an option to manually merge - if we import-configuration to get current properties, we have no control over what scope, so it seems awkward to assume we can let the winner of import-configuration override the specific scope a user selects. Maybe I'm not thinking it through though.

I see two potential workarounds:

• Allow something like a -Merge switch on the export, which reads the current config, and adds / updates properties as needed
• Allow specifying the scope on Import-Configuration

Both might come in handy depending on your scenario and desire for abstraction.

Cheers!

Cross-platform support?

@Jaykul Any there any plans/desire to add cross-platform support to this module?

Skimming through the source, I don't see anything that would prevent this module from working on Linux/MacOS (other than the use of Windows-specific environment variables for where to store the configuration(s).

1.5.x Breaking Change: Metadata commands are now external

I utilize Configuration in a Build pipeline, and today i started having build failures, with the error "The term
'Add-MetadataConverter' is not recognized as the name of a cmdlet, function, script file, or operable program.".

From looking at the latest commits, it looks like a bunch of commands were removed. Was this intentional? Any reason the major number wasn't updated since it contains breaking changes?

I don't explicitly call add-metadataconverter, but it appears to be called as part of the module loading.

WARNING: TargetObject: Add-MetadataConverter
WARNING: CategoryInfo: ObjectNotFound: (Add-MetadataConverter:String) [], ActionPreferenceStopException
WARNING: FullyQualifiedErrorId: CommandNotFoundException
WARNING: InvocationInfo: System.Management.Automation.InvocationInfo
WARNING: MyCommand:
WARNING: BoundParameters: System.Collections.Generic.Dictionary`2[System.String,System.Object]
WARNING: UnboundArguments:
WARNING: ScriptLineNumber: 11
WARNING: OffsetInLine: 5
WARNING: HistoryId: 1
WARNING: ScriptName: C:\Program Files\WindowsPowerShell\Modules\Configuration\1.5.1\Configuration.psm1
WARNING: Line: Add-MetadataConverter $Converters
WARNING: PositionMessage: At C:\Program Files\WindowsPowerShell\Modules\Configuration\1.5.1\Configuration.psm1:11 char:5

• Add-MetadataConverter $Converters
  

• ~~~~~~~~~~~~~~~~~~~~~
  

WARNING: PSScriptRoot: C:\Program Files\WindowsPowerShell\Modules\Configuration\1.5.1
WARNING: PSCommandPath: C:\Program Files\WindowsPowerShell\Modules\Configuration\1.5.1\Configuration.psm1
WARNING: InvocationName: Add-MetadataConverter
WARNING: PipelineLength: 0
WARNING: PipelinePosition: 0
WARNING: ExpectingInput: False
WARNING: CommandOrigin: Internal
WARNING: DisplayScriptPosition:
WARNING: ScriptStackTrace: at , C:\Program
Files\WindowsPowerShell\Modules\Configuration\1.5.1\Configuration.psm1: line 11

We need to load all relevant files and let them override each other

Feature: Multiple settings files should layer
As a module author, I want to distribute a default config with my module
As a machine administrator, I want to save corporate defaults
As a user I want to save my own preferences
And our custom settings shouldn't be overwritten on upgrade

Scenario: Save shouldn't overwrite default settings
Given MyModule has Config.psd1 default settings file
When I save the configuration
The settings should not overwrite the default file

Scenario: Load should layer settings (and support new settings after upgrades)
Given MyModule has Config.psd1 default settings file
And I set some settings in the LocalMachine 
And I set some settings in the CurrentUser
When I loading the settings
Then CurrentUser settings should overwrite LocalMachine settings
And LocalMachine settings should overwrite default settings
But default settings should still be read, just in case

Scenario: Migrate settings only once
Given MyModule has a new version
And I have some settings from an old version
When I load the settings in the new module
Then the settings from the old version should be copied
And MyModule should be able to migrate them
But they should save only to the new version

Alternate Config Files

I've been thinking that we could easily add a parameter for the file name (defaulted to the current hard-coded Configuration.psd1) which would allow the file name to be specified ...

The idea would NOT be to let you customize the file names -- but to support different "sets" of configurations. That way if a module author wanted to switch between multiple named configurations they could just expose the name to their users ... or users who know you're using Configuration could just:

Get-Module YourModule | Import-Configuration -Set Special

So what should I call the parameter? We can't use Name because that's the parameter I'm using for the Module name.

• Set (has the benefit of starting with a character that's unique among parameters)
• Config
• ConfigSet
• Variant
• ???

Release 1.0.2 not here

I see the release 1.0.2 is not available here as it is in PowershellGallery.com. Would you be able to create a release here in Github?

Dynamic construction of DSC ConfigurationData.psd1 - the right way

Please suggest.
I'm still trying to use unmodified Configuration module, perhaps you know a better solution.