GithubHelp home page GithubHelp logo

powershell / dscresource.tests Goto Github PK

View Code? Open in Web Editor NEW
51.0 27.0 49.0 1.31 MB

Common meta tests for PowerShell DSC resources repositories.

License: MIT License

PowerShell 99.68% JavaScript 0.15% C# 0.17%

dscresource.tests's Introduction

DscResource.Tests

Common meta tests and other shared functions for PowerShell DSC resources repositories.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

Branches

master

Build status codecov

This is the branch containing the latest release - no contributions should be made directly to this branch.

dev

Build status codecov

This is the development branch to which contributions should be proposed by contributors as pull requests. This branch is used by DSC Resource Kit modules for running common tests.

Change Log

A full list of changes in each version can be found in the change log.

Table of Contents

DSC Resource Common Meta Tests

Meta.Tests.ps1

Common Meta Test Opt-In

New tests may run but only produce errors. Once you fix the test, please copy .MetaTestOptIn.json from this repo to the root of your repo. If there is any new problem in the area, this will cause the tests to fail, not just warn.

The following opt-in flags are available:

  • Common Tests - Validate Module Files: run tests to validate module files have correct BOM.
  • Common Tests - Validate Markdown Files: run tests to validate markdown files do not violate markdown rules. Markdown rules can be suppressed in .markdownlint.json file.
  • Common Tests - Validate Example Files: run tests to validate that examples can be compiled without error.
  • Common Tests - Validate Example Files To Be Published: run tests to validate that examples can be published successfully to PowerShell Gallery. See requirements under Publish examples to PowerShell Gallery.
  • Common Tests - Validate Script Files: run tests to validate script files have correct BOM.
  • Common Tests - Required Script Analyzer Rules: fail tests if any required script analyzer rules are violated.
  • Common Tests - Flagged Script Analyzer Rules: fail tests if any flagged script analyzer rules are violated.
  • Common Tests - New Error-Level Script Analyzer Rules: fail tests if any new error-level script analyzer rules are violated.
  • Common Tests - Custom Script Analyzer Rules: fail tests if any custom script analyzer rules are violated.
  • Common Tests - Relative Path Length: fail tests if the length of the relative full path, from the root of the module, exceeds the max hard limit of 129 characters. 129 characters is the current (known) maximum for a relative path to be able to compile a configuration in Azure Automation using a DSC resource module.
  • Common Tests - Validate Markdown Links: fails tests if a link in a markdown file is broken.
  • Common Tests - Spellcheck Markdown Files: fail test if there are any spelling errors in the markdown files. There is the possibility to add or override words in the \.vscode\cSpell.json file.

Common Tests - Validate Markdown Links

The test validates the links in markdown files. Any valid GitHub markdown link will pass the linter.

NOTE! There is currently a bug in the markdown link linter that makes it unable to recognize absolute paths where the absolute link starts in a parent folder. For example, if a markdown file /Examples/README.md, contains an absolute link pointing to /Examples/Resources/SqlAG, that link will fail the test. Changing the link to a relative link from the README.md file's folder, e.g Resources/SqlAG will pass the test. See issue vors/MarkdownLinkCheck#5.

Common Tests - Spellcheck Markdown Files

When opt-in to this test, if there are any spelling errors in markdown files, the tests will fail.

Note: The spell checker is case-insensitive, so the words 'AppVeyor' and 'appveyor' are equal and both are allowed.

If the spell checker (cSpell) does not recognize the word, but the word is correct or a specific phrase is not recognized but should be allowed, then it is possible to add these to a dictionary or tell it to ignore the word or phrases. This is done by adding a \.vscode\cSpell.json in the repository.

The following JSON is the simplest form of the file \.vscode\cSpell.json (see cSpell for more settings).

This settings file will also work together with the Visual Studio Code extension Code Spell Checker. By using this extension the spelling errors can be caught in real-time. When a cSpell.json exists in the .vscode folder, the individual setting in the cSpell.json file will override the corresponding setting in the Visual Studio Code User settings or Workspace settings file. This differs from adding a Code Spell Checker setting to the Visual Studio Code Workspace settings file, as the Workspace settings file will override all the settings in the User settings.

{
    "ignorePaths": [
        ".git/*",
        ".vscode/*"
    ],
    "language": "en",
    "dictionaries": [
        "powershell"
    ],
    "words": [
        "markdownlint",
        "Codecov"
    ],
    "ignoreRegExpList": [
        "AppVeyor",
        "[email protected]",
        "\\.gitattributes"
    ]
}

The key words should have the words that are normally used when writing text.

The key ignoreRegExpList is used to ignore phrases or combinations of words, such as AppVeyor, which will be detected as two different words since it consists of two words starting with upper-case letters. To configure cSpell to ignore the word combination AppVeyor, then we can add a regular expression, in this case AppVeyor. This will cause cSpell to ignore part of the text that matches the regular expression.

Common Tests - Validating Localization

These tests validate the localization folders and files, and also that each localization string key is used and there are no missing or extra localization string keys. These tests will only work if the localization variable is $script:localizedData, and it is a string constant, e.g. $script:localizedData.MyStringKey.

  • Should have an en-US localization folder.
  • The en-US localization folder should have the correct casing.
  • A resource file with the correct name should exist in the localization folder.
  • The resource or module should use all the localization string keys from the localization resource file.
  • The localization resource file should not be missing any localization string key that is used in the resource or module.
  • If there are other localization folders (other than en-US)
    • They should contain a resource file with the correct name.
    • The folders should use the correct casing.
    • All en-US resource file localized string keys must also exist in the resource file.
    • There should be no additional localized string keys in the resource file that does not exist in the en-US resource file.

Markdown Testing

.markdownlint.json gulpfile.js package.json

The DSC Resource Common Meta Tests contains tests for validating that any markdown files in a DSC Resource meet the standard markdown guidelines.

These tests use NPM to download Gulp, which then uses a Gulp file to ensure that the markdown files are correct.

The 'markdown' tests can be excluded when running pester by using:

Invoke-Pester -ExcludeTag @('Markdown')

It is possible to override the default behavior of the markdown validation test. By default the common tests use the settings in the markdownlint settings file .markdownlint.json. If the file '.markdownlint.json' exists in the root path of the module repository, then that file will be used as the settings file. Please note that there are currently only two markdown lint rules allowed to be overridden, and that is lint rule MD013 (line length) and MD024 (Multiple headers with the same content). These are disabled by default, and can be enabled by individual repositories to enforce those linting rules.

Example Testing

The DSC Resource Common Meta Tests contains tests for validating that any included Example files work correctly. These tests are performed by attempting to apply the example DSC Configurations to the machine running the tests. This causes them to behave as extra integration tests.

The 'example' tests can be excluded when running pester by using:

Invoke-Pester -ExcludeTag @('Example')

When a repository is opted-in to example testing, each example file in the 'Examples' folder must have a function named Example which should contain the configuration which will be tested.

An optional configuration data hash table can be added for any specific data that needs to be provided to the example configuration. The configuration data hash table variable name must be $ConfigurationData for the test to pick it up. If no configuration block is provided a default configuration block is used.

PSScriptAnalyzer Rules

The DSC Resource Common Meta Tests also contains tests for PSScriptAnalyzer (PSSA) rules. Along with the built-in PSSA rules, custom rules are tested. Those rules are defined and maintained in this repository in DscResource.AnalyzerRules. These custom rules are built to follow the style guideline, and overriding them should be a temporary measure until the code can follow the style guideline

There will be cases where built-in and/or custom PSSA rules may need to be suppressed in scripts or functions. You can suppress a rule by decorating a script/function or script/function parameter with .NET's SuppressMessageAttribute.

When the Common Tests - PS Script Analyzer on Resource Files test fails on a PSSA rule, Meta.Tests will use Write-Warning to output Rule Name, Script Name, Line Number, and Rule Message. When necessary, the rule name can be used to suppress the rule as needed. For example, the following code would cause the PSAvoidGlobalVars built-in PSSA rule to fail:

function Set-TargetResource
{
    param
    (
        [Parameter(Mandatory = $true)]
        [String]
        $FeatureName
    )

    $windowsOptionalFeature = Dism\Enable-WindowsOptionalFeature -FeatureName $FeatureName -NoRestart $true

    if ($windowsOptionalFeature.RestartNeeded -eq $true)
    {
        Write-Verbose -Message $script:localizedData.RestartNeeded
        $global:DSCMachineStatus = 1
    }
}

In this example, suppression is allowed here because $global:DSCMachineStatus must be set in order to reboot the machine. To suppress the PSAvoidGlobalVars rule for this function, this can be done by using the SuppressMessageAttribute like this:

function Set-TargetResource
{
    [Diagnostics.CodeAnalysis.SuppressMessageAttribute('PSAvoidGlobalVars', '')]
    param
    (
        [Parameter(Mandatory = $true)]
        [String]
        $FeatureName
    )

    $windowsOptionalFeature = Dism\Enable-WindowsOptionalFeature -FeatureName $FeatureName -NoRestart $true

    if ($windowsOptionalFeature.RestartNeeded -eq $true)
    {
        Write-Verbose -Message $script:localizedData.RestartNeeded
        $global:DSCMachineStatus = 1
    }
}

For further details and examples for suppressing PSSA rules, please see the Suppressing Rules documentation.

Skip meta tests (for debug purpose)

For debug purpose it is possible to skip the common tests, the tests in the Meta.Tests.ps1 script file. When debugging a certain unit test or integration test in AppVeyor, it takes quite some time for the common test to run before the actual unit or integration test runs.

To temporarily skip the common tests, the environment variable SkipAllCommonTests can be used.

Add environment variable to the appveyor.yml

Note: This environment variable should not be merged, or a commit pushed into the branches dev or master. This environment variable is purely for debug purposes before sending in a pull request (PR).

Using AppVeyor environment variable.

environment:
  SkipAllCommonTests: True

Or as the first PowerShell line to run when AppVeyor is starting testing.

test_script:
    - ps: Set-Item -Path env:\SkipAllCommonTests -Value $true

MetaFixers Module

MetaFixers.psm1

We are trying to provide automatic fixers where it's appropriate. A fixer corresponds to a particular test.

For example, if Files encoding test from Meta.Tests.ps1 test fails, you should be able to run ConvertTo-UTF8 fixer from MetaFixers.psm1.

TestHelper Module

TestHelper.psm1

The test helper module (TestHelper.psm1) contains the following functions:

  • New-Nuspec: Creates a new nuspec file for NuGet package.
  • Install-ResourceDesigner: Will attempt to download the xDSCResourceDesignerModule using NuGet package and return the module.
  • Initialize-TestEnvironment: Initializes an environment for running unit or integration tests on a DSC resource.
  • Restore-TestEnvironment: Restores the environment after running unit or integration tests on a DSC resource.

Templates for Creating Tests

The Template files that are used for creating Unit and Integration tests for a DSC resource are available in the DSCResource.Template GitHub Repository in the Tests folder

To use these files, see the test guidelines document and the instructions at the top of each template file.

The resource files are:

Example Test Usage

To see examples of the Unit/Integration tests in practice, see the NetworkingDsc MSFT_Firewall resource:

Example Usage of DSCResource.Tests in AppVeyor.yml

To automatically download and install the DscResource.Tests in an AppVeyor.yml file, please see the following sample AppVeyor.yml. https://github.com/PowerShell/DscResource.Template/blob/master/appveyor.yml

AppVeyor Module

AppVeyor.psm1

This module provides functions for building and testing DSC Resources in AppVeyor.

Note: These functions will only work if called within an AppVeyor CI build task.

  • Invoke-AppveyorInstallTask: This task is used to set up the environment in preparation for the test and deploy tasks. It should be called under the install AppVeyor phase (the install: keyword in the appveyor.yml).
  • Invoke-AppveyorTestScriptTask: This task is used to execute the tests. It should be called under test AppVeyor phase (the test_script: keyword in the appveyor.yml).
  • Invoke-AppveyorAfterTestTask: This task is used to perform the following tasks. It should be called either under the test AppVeyor phase (the test_script: keyword in the appveyor.yml), or the after tests AppVeyor phase (the after_test: keyword in the appveyor.yml).
    • Generate, zip and publish the Wiki content to AppVeyor (optional).
    • Set the build number in the DSC Resource Module manifest.
    • Publish the Test Results artefact to AppVeyor.
    • Zip and publish the DSC Resource content to AppVeyor.
  • Invoke-AppVeyorDeployTask: This task is used to perform the following tasks. It should be called under the deploy AppVeyor phase (the deploy_script: keyword in the appveyor.yml).

Using AppVeyor.psm1 with the default shared model

For an example of a AppVeyor.yml file for using the default shared model with a resource module, see the DscResource.Template appveyor.yml.

Using AppVeyor.psm1 with harness model

An example AppVeyor.yml file of using the harness model with a resource module.

version: 3.1.{build}.0
install:
    - git clone https://github.com/PowerShell/DscResource.Tests

    - ps: |
        $moduleName = 'xNetworking'
        $mainModuleFolder = "Modules\$moduleName"
        $harnessModulePath = "Tests\$($moduleName).TestHarness.psm1"
        $harnessFunctionName = "Invoke-$($moduleName)Test"
        Import-Module "$env:APPVEYOR_BUILD_FOLDER\DscResource.Tests\AppVeyor.psm1"
        Invoke-AppveyorInstallTask

build: false

test_script:
    - ps: |
        Invoke-AppveyorTestScriptTask `
            -Type 'Harness' `
            -MainModulePath $mainModuleFolder `
            -HarnessModulePath $harnessModulePath `
            -HarnessFunctionName $harnessFunctionName

deploy_script:
    - ps: |
        Invoke-AppveyorAfterTestTask `
            -Type 'Wiki' `
            -MainModulePath $mainModuleFolder `
            -ResourceModuleName $moduleName

Encrypt Credentials in Integration Tests

Any configuration used for an integration test that uses a configuration that contains credential parameters must be configured to use MOF encryption by providing a certificate file.

The path to the certificate file must be provided in the CertificateFile property in the ConfigurationData.

$ConfigurationData = @{
    AllNodes = @(
        @{
            NodeName        = 'localhost'
            CertificateFile = $env:DscPublicCertificatePath
        }
    )
}

When these tests are run in AppVeyor and the AppVeyor module is being used then the Invoke-AppveyorInstallTask and/or Invoke-AppveyorTestScriptTask will automatically generate an appropriate certificate file and assign the path to the environment variable $env:DscPublicCertificatePath.

To run the same tests outside of AppVeyor, the certificate can be created and the path assigned to the $env:DscPublicCertificatePath variable by running the function New-DscSelfSignedCertificate from the TestHelper module.

$certificate = New-DscSelfSignedCertificate

CodeCoverage reporting with CodeCov.io

This is to enable code coverage reporting through codecov.io which allows you to report on pull request and project code coverage. To use codecov.io, you must have enabled Pester code coverage, which the first two sections cover.

Ensure Code Coverage is enabled

Repository using -Type 'Default' for Invoke-AppveyorTestScriptTask

  1. On the call to Invoke-AppveyorTestScriptTask, make sure you have -CodeCoverage specified. This will enable Pester code coverage.

It is possible to control which relative paths, from the root module folder, are evaluated for code coverage. By specifying one or more relative paths in the parameter -CodeCoveragePath each path is searched for PowerShell modules files (.psm1). For each relative folder it will look in the root of the relative path, and also recursively search the first level subfolders, for PowerShell module files (.psm1). Defaults to the relative paths 'DSCResources', 'DSCClassResources', and 'Modules'.

Repository using -Type 'Harness' for Invoke-AppveyorTestScriptTask

  1. Make sure you are properly generating pester code coverage in the repository harness code.

Enable reporting to CodeCov.io

  1. On the call to Invoke-AppveyorTestScriptTask, specify -CodeCovIo. This will enable reporting to codecov.io

Configure CodeCov.io

  1. Copy .codecov.yml from the root of this repo to the root of your repo.
  2. Adjust the code coverage goals if needed. See the CodeCov.io documentation.

Add the badge to the Readme

Add the following code below the AppVeyor badge in the main repo readme.md, replacing <repoName> with the name of the repository.

[![codecov](https://codecov.io/gh/PowerShell/<repoName>/branch/master/graph/badge.svg)](https://codecov.io/gh/PowerShell/<reproName>/branch/master)
[![codecov](https://codecov.io/gh/PowerShell/<repoName>/branch/dev/graph/badge.svg)](https://codecov.io/gh/PowerShell/<reproName>/branch/dev)

Documentation Helper Module

This module consists of the following three nested modules:

MofHelper

A helper module containing the Get-MofSchemaObject function used to return the contents of the schema.mof files as a PowerShell object to be used in other scripts.

PowerShellHelp

A module containing the function New-DscResourcePowerShellHelp that when run will process all of the MOF based resources in a specified module directory and create PowerShell help files for each resource into the resource's en-US subdirectory. These help files include details on the property types for each resource, as well as a text description and examples where they exist.

A README.md with a text description must exist in the resource's subdirectory for the help file to be generated.

When the DSC resource module is imported, these help files can then be read by passing the name of the resource as a parameter to Get-Help.

WikiPages

A module containing the function New-DscResourceWikiSite that is used by some HQRM DSC Resource modules to produce Wiki Content to be distributed with the DSC Resource module as well as published in the Wiki section of the DSC Resource repo on GitHub.

It is usually called by the Invoke-AppveyorAfterTestTask task in AppVeyor.psm1 when the -type parameter is set to 'Wiki'. For example:

Invoke-AppveyorAfterTestTask `
    -Type 'Wiki' `
    -MainModulePath '.\Modules\SharePointDsc\' `
    -ResourceModuleName 'SharePointDsc'

Run integration tests in order

This is only available for resource modules that are using the shared AppVeyor module model, meaning those resource modules that are calling the helper function Invoke-AppveyorTestScriptTask either without the parameter -Type, or has assigned the value 'Default' to parameter -Type.

Note: Resource modules using the "Harness"-model (e.g SharePointDsc and xStorage) must add this functionality per individual resource module.

To run integration tests in order, the resource module must opt-in by calling helper function Invoke-AppveyorTestScriptTask using the switch parameter -RunTestInOrder.

Also, each integration test file ('*.Integration.Tests.ps1') must be decorated with an attribute Microsoft.DscResourceKit.IntegrationTest containing a named attribute argument 'OrderNumber' and be assigned a numeric value (1, 2, 3,..). The value 0 should not be used since it is reserved for DscResource.Tests, for making sure the common tests are always run first.

Integration tests will be run in ascending order, so integration tests with value 1 will be run before integration tests with value 2. If an integration test does not have a assigned order, it will be run unordered after all ordered tests have been run.

Example showing how the integration test file could look like to make sure an integration test is always run as one of the first integration tests. This should be put a the top of the integration test script file.

[Microsoft.DscResourceKit.IntegrationTest(OrderNumber = 1)]
param()

Run tests in a Docker Windows container

The same parameter RunTestInOrder can also be use to run unit tests or integration tests in a container. This make it possible to run integration tests and unit tests in parallel on the same build worker. The common tests will by default always be run on the AppVeyor build worker.

To run a test in a container, the test must be decorated with the attribute Microsoft.DscResourceKit.IntegrationTest or Microsoft.DscResourceKit.UnitTest.

The Pester output from the container, including errors will be sent to the console in a Pester like format, and they will also be added to the list of tests in AppVeyor portal. There is transcript from the test run that is uploaded as artifact in AppVeyor which can contain more detailed errors why the one test failed.

Note: The transcript catches more output than Pester normally writes to the console since it sees all errors that Pester catches with | Should -Throw.

If the container returns an exit code other than 0, the Docker log for the container is gathered and uploaded as an artifact. This is intended to enable a more detailed error of what happened to be displayed. The Docker log will be searched for any error records. If any are found then an exception will be thrown which will stop the the tests in the build worker.

Named attribute argument

  • ContainerName: The name of the container. If the same container name is used in multiple tests they will be run sequentially in the same container.
  • ContainerImage: The name of the container image to use for the container. This should use the normal Docker format for specifying a Docker image, i.e. 'microsoft/windowsservercore:latest'. If the tag 'latest' is used, then docker pull will always run to make sure the latest revision of the image is in the local image repository. To use the 'latest' local revision, don't suffix the tag 'latest' to the image name. Note: If the same container name is used in multiple test and they have different container images, the first container image that is loaded from at test will be used for all tests.

Example

This example shows how the integration test file would look if the tests should be run in a container and also run before other integration tests. This should be put a the top of the integration test script file.

[Microsoft.DscResourceKit.IntegrationTest(OrderNumber = 1, ContainerName = 'ContainerName', ContainerImage = 'Organization/ImageName:Tag')]
param()

This example shows how the integration test file would look if the tests should be run in a container and not using any specific order. This should be put a the top of the integration test script file.

[Microsoft.DscResourceKit.IntegrationTest(ContainerName = 'ContainerName', ContainerImage = 'Organization/ImageName:Tag')]
param()

This example shows how the unit test file would look if the tests should be run in a container. This should be put a the top of the unit test script file. Note: Unit test does not support ordered testing at this time.

[Microsoft.DscResourceKit.UnitTest(ContainerName = 'ContainerName', ContainerImage = 'Organization/ImageName:Tag')]
param()

Artifacts when running tests in a container

These are the artifacts that differ when running tests using a container.

  • unittest_Transcript.txt - Contains the transcript from the test run that was done in the container.
  • unittest_TestResults.xml - Contains the Pester output in the NUnitXML format from the tests that was tested in the container.
  • unittest_TestResults.json - Contains the serialized object that Pester returned after it finished the test run in the container.
  • unittest_DockerLog.txt - If the container exits with any other exit code than 0 a Docker log is gathered and uploaded as an artifact. This is intended to enable a more detailed view of the error. If the container exits with exit code 0 then the Docker log will not be uploaded.
  • worker_TestsResults - Contains the Pester output in the NUnitXML format from the tests that was tested in the build worker.

Deploy

To run the deploy steps the following must be added to the appveyor.yml. The default is that opt-in is required for all the deploy tasks. See comment-based help for the optional parameters.

Example opt-in to both Example Publishing and Wiki Content Publishing:

deploy_script:
  - ps: |
        Invoke-AppVeyorDeployTask -OptIn PublishExample, PublishWikiContent

Publish examples to PowerShell Gallery

To opt-in to this task, change the appveyor.yml to include the opt-in task PublishExample, e.g. Invoke-AppVeyorDeployTask -OptIn PublishExample.

By opting-in to the PublishExample task, the test framework will publish the examples in the AppVeyor deploy step, but only if it is a 'master' branch build ($env:APPVEYOR_REPO_BRANCH -eq 'master').

Note: It is possible to override the deploy branch in appveyor.yml, e.g. Invoke-AppVeyorDeployTask -Branch @('dev','my-working-branch'). But if building on any other branch than 'master' the task will do a dry run (using -WhatIf).

By adding script metadata to an example (see New-ScriptFileInfo) the resource module automatically opt-in to publish that example (if already opt-in for the deploy tasks in the appveyor.yml).

Requirements/dependencies for publishing to PowerShell Gallery

  • Publish only on 'master' build.
  • Must have opt-in for the example validation common test.
  • Publish only an example that passes Test-ScriptFileInfo.
  • Publish only an example that does not already exist (for example has a newer version).
  • Publish only an example which is located under '\Examples' folder.
  • Publish only an example where file name ends with 'Config'.
  • Publish only an example that where filename and configuration name are the same. Published examples must have the same configuration name as the file name to be able to deploy in Azure Automation.
    • Example files are allowed to begin, be prefixed, with numeric value followed by a dash (e.g. '1-', '2-') to support auto-documentation. The prefix will be removed from the name when publishing, so the filename will appear without the prefix in PowerShell Gallery.
  • Publish only examples that have a unique GUID within the resource module. Note: This is only validated within the resource module, the validation does not validate this against PowerShell Gallery. This is to prevent simple copy/paste mistakes within the same resource module.
  • Publish only an example where the configuration name contains only letters, numbers, and underscores. Where the name starts with a letter, and ends with a letter or a number.

PowerShell Gallery API key

For the Publish-Script to work each repo that opt-in must have the PowerShell Gallery account API key as a secure environment variable in appveyor.yml. For DSC Resource Kit resource modules, this should be the same API key, since it must be encrypted by an account that has permission to the AppVeyor PowerShell organization account.

Note: This key can only be used for resource modules under DSC Resource Kit.

environment:
  gallery_api:
    secure: 9ekJzfsPCDBkyLrfmov83XbbhZ6E2N3z+B/Io8NbDetbHc6hWS19zsDmy7t0Vvxv

Note: There was a problem running Test-ScriptFileInfo on the AppVeyor build worker, because the build worker has the setting core.autocrlf=input which result in the files checked out only have LF as as the end-of-line (eol) character. Test-ScriptFileInfo is unable to parse the files with just LF. To solve this, the best option is to add a .gitattributes file to the root of the repository, with the following content. This will always make git checkout files with CRLF as the end-of-line (eol) characters.

* text eol=crlf

Contributor responsibilities

Contributors that add or change an example to be published must make sure that

  • The example filename is short but descriptive and ends with 'Config'.
    • If the example is for a single resource, then the resource name could be prefixed in the filename (and configuration name) followed by an underscore (e.g. xScript_WatchFileContentConfig.ps1). The thought is to easier find related examples.
  • The Node block is targeting 'localhost' (or equivalent).
  • The filename and configuration name match (see requirements/dependencies above).
  • The example contains script metadata with all required properties present.
  • The example has an unique GUID in the script metadata.
  • The example have comment-based help with at least .DESCRIPTION.
  • The example script metadata version and release notes is updated accordingly.
  • (Optional) The example has a #Requires statement.
Example of script metadata, #Requires statement and comment-based help

Note: The .PRIVATEDATA in the script metadata is optional and it is for a future implementation to be able to run integration test on the examples.

<#PSScriptInfo
.VERSION 1.0.4
.GUID 124cf79c-d637-4e50-8199-5cf4efb3572d
.AUTHOR Microsoft Corporation
.COMPANYNAME Microsoft Corporation
.COPYRIGHT
.TAGS DSCConfiguration
.LICENSEURI https://github.com/PowerShell/xPSDesiredStateConfiguration/blob/master/LICENSE
.PROJECTURI https://github.com/PowerShell/xPSDesiredStateConfiguration
.ICONURI
.EXTERNALMODULEDEPENDENCIES
.REQUIREDSCRIPTS
.EXTERNALSCRIPTDEPENDENCIES
.RELEASENOTES First version.
.PRIVATEDATA 2016-Datacenter,2016-Datacenter-Server-Core
#>

#Requires -module @{ModuleName = 'xPSDesiredStateConfiguration';ModuleVersion = '8.2.0.0'}

<#
    .SYNOPSIS
        Creates a file at the given file path with the specified content through
        the xScript resource.

    .DESCRIPTION
        Creates a file at the given file path with the specified content through
        the xScript resource.

    .PARAMETER FilePath
        The path at which to create the file. Defaults to $env:TEMP.

    .PARAMETER FileContent
        The content to set for the new file.
        Defaults to 'Just some sample text to write to the file'.
#>

Publish Wiki Content

To opt-in to this task, change the appveyor.yml to include the opt-in task PublishWikiContent, e.g. Invoke-AppVeyorDeployTask -OptIn PublishWikiContent.

By opting-in to the PublishWikiContent task, the test framework will publish the contents of a DSC Resource Module Wiki Content artifact to the relevant GitHub Wiki repository, but only if it is a 'master' branch build ($env:APPVEYOR_REPO_BRANCH -eq 'master'). A Wiki Sidebar file will be generated, containing links to all of the markdown files in the Wiki, as well as as a Wiki Footer file. Any files contained within the WikiSource directory of the repository will also be published to the Wiki overriding any auto generated files.

Note: It is possible to override the deploy branch in appveyor.yml, e.g. Invoke-AppVeyorDeployTask -Branch @('dev','my-working-branch').

Requirements/dependencies for publishing Wiki Content

  • Publish only on 'master' build.
  • The Invoke-AppveyorAfterTestTask function must be present in the Appveyor configuration with a Type of 'Wiki' to generate the Wiki artifact.
  • A GitHub Personal Access Token with repo/public_repo permissions for a user that has at least Collaborator access to the relevant DSC Module GitHub repository must be generated and then added as a secure variable called github_access_token to the environment section of the repository's appveyor.yml file.
  • The GitHub Wiki needs to be initialized on a repository before this function is run.

Note: Currently Wiki content files are only added or updated by the function, not deleted. Any deletions must be done manually by cloning the Wiki repository and deleting the required content.

dscresource.tests's People

Contributors

bgelens avatar bhlokana-msft avatar boscoedash avatar brianfarnhill avatar dan1el42 avatar danielboth avatar dcrreynolds avatar dscottraynsford avatar eshaparmar avatar indented-automation avatar johlju avatar karolkaczmarek avatar kusuriya avatar kwirkykat avatar lfshr avatar limiteddenial avatar mbreakey3 avatar mhendric avatar nabrond avatar narrieta avatar plagueho avatar powershellteam avatar pvs043 avatar regedit32 avatar ryanspletzer avatar ssvilen avatar travisez13 avatar vors avatar x-guardian avatar ykuijs avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

dscresource.tests's Issues

test template scripts only run without error in root of module.

When using the template for the integration test. The first thing the script template does is download a DscResource.Test directory which contains TestHelper.psm1 from GitHub. The template script calls TestHelper.psm1 which throws error because it canโ€™t locate the module manifest file which is located in the root folder of the module. TestHelp.psm1 is looking for this file in the Tests\Integration folder. TestHelper.psm1 is using $MyInvocation.MyCommand.Path to get the path to the module manifest. $MyInvocation.MyCommand.Path is going to resolve to whatever folder TestHelper.psm1 is located it. This goes against the testing guidance to create a Tests\Integration folder as the test will only run without error if they ran from the root of the module.

Add Custom PSScriptAnalyzer Test for validating Parameter and Variable naming format

I've recently been doing some work into creating custom PSSA rules. It should be fairly easy to create some rules that validate the Variable naming conventions (e.g. local variables start with lower case, parameter vars start with upper case).

The basis for the custom rule would be code like this:

process
{
    try
    {
        $Fucntions = $ScriptBlockAst.Ast.FindAll( { $args[0] -is [System.Management.Automation.Language.FunctionDefinitionAst] }, $true )
        foreach ($function in $functions)
        {
            $ParamNames = $Function.FindAll( { $args[0] -is [System.Management.Automation.Language.ParamBlockAst] }, $true ).Parameters.Name.VariablePath.UserPath
            $Variables = $Function.FindAll( { $args[0] -is [System.Management.Automation.Language.VariableExpressionAst] }, $true )
        }
    }
    catch
    {
        $PSCmdlet.ThrowTerminatingError( $_ )
    }
}

Needs a bit more work but it should be easy enough. If it is worth implementing, then I'm happy to submit a PR.

Error thrown by Meta.test.ps1 if executed more than once

Caused by line:
https://github.com/PowerShell/DscResource.Tests/blob/master/Meta.Tests.ps1#L64

This occurs because if:

$PSScriptAnalyzerModule | Import-Module -Force

is run more than once, the following errors occur:
2015-12-19_18-30-59

If the PSScriptAnalyzer module is removed from memory then the tests will work again.

Also note the incorrect message in the Version System.Object[] line.

I'll take a look at these issues and raise some PR's for them.

Sorry about not documenting the issue fully, but I'm short on time today but wanted to make sure I didn't forget the problem.

Thanks

Change PSScriptAnalyzer test in Meta.tests.ps1 to eveluate warning level rules

Currently the PSScriptAnalyzer test in Meta.tests.ps1 only evaluates Error level rules. However, there are some PSSA Warning level rules that would also be valuable if they failed on for DSC Resources.

Rules

Some PSSA Warning level rules are should not be applied to DSC resources because they don't make sense for DSC. The following rules I think should not be run:

PSUseShouldProcessForStateChangingFunctions
PSMissingModuleManifestField - this one is triggered because some of the modules contain language data files *.psd1 and PSSA is treating them as manifest files and reporting that version number and other parameters are missing.
PSAvoidUsingInvokeExpression

Coverage

PSSA is currently running against the entire root folder of the resource module (which includes tests and examples folders). From my initial tests a lot of failures are occurring in the tests and examples folders.

Suggestion: Only run the PSSA tests against the DSCResource folder in each module for now and look at extending it. Also, the tests are getting run on the DSCResource.tests folder which seems unnecessary. Some of the larger resource modules also take quite a long time (10-20 minutes) running PSSA so reducing the coverage to only the resource code might be a good idea for now.

Output from running warning level against xNetworking\DSCResource folder:

RuleName                            Severity     FileName   Line  Message
--------                            --------     --------   ----  -------
PSUseShouldProcessForStateChangingF Warning      MSFT_xDnsC 124   Function โ€™Set-TargetResourceโ€™ has verb that could change
unctions                                         onnectionS       system state. Therefore, the function has to support
                                                 uffix.psm1       'ShouldProcess'.
PSPossibleIncorrectComparisonWithNu Warning      MSFT_xFire 658   $null should be on the left side of equality comparisons.
ll                                               wall.psm1
PSPossibleIncorrectComparisonWithNu Warning      MSFT_xFire 540   $null should be on the left side of equality comparisons.
ll                                               wall.psm1
PSPossibleIncorrectComparisonWithNu Warning      MSFT_xFire 286   $null should be on the left side of equality comparisons.
ll                                               wall.psm1
PSAvoidUsingInvokeExpression        Warning      MSFT_xFire 628   Invoke-Expression is used. Please remove Invoke-Expression
                                                 wall.psm1        from script and find other options instead.
PSAvoidUsingInvokeExpression        Warning      MSFT_xFire 627   Invoke-Expression is used. Please remove Invoke-Expression
                                                 wall.psm1        from script and find other options instead.
PSAvoidUsingInvokeExpression        Warning      MSFT_xFire 326   Invoke-Expression is used. Please remove Invoke-Expression
                                                 wall.psm1        from script and find other options instead.
PSAvoidUsingInvokeExpression        Warning      MSFT_xFire 131   Invoke-Expression is used. Please remove Invoke-Expression
                                                 wall.psm1        from script and find other options instead.
PSAvoidUsingInvokeExpression        Warning      MSFT_xFire 120   Invoke-Expression is used. Please remove Invoke-Expression
                                                 wall.psm1        from script and find other options instead.
PSUseSingularNouns                  Warning      MSFT_xFire 578   The cmdlet 'Test-RuleProperties' uses a plural noun. A
                                                 wall.psm1        singular noun should be used instead.
PSUseShouldProcessForStateChangingF Warning      MSFT_xNetB 161   Function โ€™New-TerminatingErrorโ€™ has verb that could change
unctions                                         IOS.psm1         system state. Therefore, the function has to support
                                                                  'ShouldProcess'.
PSUseShouldProcessForStateChangingF Warning      MSFT_xNetB 63    Function โ€™Set-TargetResourceโ€™ has verb that could change
unctions                                         IOS.psm1         system state. Therefore, the function has to support
                                                                  'ShouldProcess'.
PSMissingModuleManifestField        Warning      MSFT_xNetB 1     The member 'ModuleVersion' is not present in the module
                                                 IOS.psd1         manifest. This member must exist and be assigned a version
                                                                  number of the form 'n.n.n.n'. Add the missing member to the
                                                                  file 'C:\Users\Daniel\Source\GitHub\xNetworking\DSCResources
                                                                  \MSFT_xNetBIOS\en-US\MSFT_xNetBIOS.psd1'.
PSUseShouldProcessForStateChangingF Warning      MSFT_xNetw 43    Function โ€™Set-TargetResourceโ€™ has verb that could change
unctions                                         orkTeam.ps       system state. Therefore, the function has to support
                                                 m1               'ShouldProcess'.
PSMissingModuleManifestField        Warning      MSFT_xNetw 1     The member 'ModuleVersion' is not present in the module
                                                 orkTeam.ps       manifest. This member must exist and be assigned a version
                                                 d1               number of the form 'n.n.n.n'. Add the missing member to the
                                                                  file 'C:\Users\Daniel\Source\GitHub\xNetworking\DSCResources
                                                                  \MSFT_xNetworkTeam\en-US\MSFT_xNetworkTeam.psd1'.
PSUseDeclaredVarsMoreThanAssigments Warning      MSFT_xRout 431   The variable 'Subnet' is assigned but never used.
                                                 e.psm1
PSUseShouldProcessForStateChangingF Warning      MSFT_xRout 95    Function โ€™Set-TargetResourceโ€™ has verb that could change
unctions                                         e.psm1           system state. Therefore, the function has to support
                                                                  'ShouldProcess'.

Add functionality to run pester and Automatically generate code coverage visualization

  • Create a common way for which repo can call pester (#90)
  • fix style issues such as putting code blocks starts on new line (where possible). (#116)
  • Add a common way to automatically detect code and non-code files for code coverage (#90)
  • Add a way to override code and non-code coverage file detection filed #114
  • Add a common way to convert Pester test results object into the code coverage visualizer format (I'm planning on using codecov.io, similarly to PowerShell\PowerShell) (#117)
  • Add a common function to upload results to the code coverage visualizer (#117)
  • Add deployment documentation(#117)

System Environment variable PSModulePath shouldn't be updated for unit tests

The current resource test template updates the PSModulePath system variable for LCM to load the resource modules during integration tests. For unit tests, this is not necessary. Doing so and terminating a test without the test environment restore causes ISE to hang (while opening) when the added module path contains a folder that has multiple subfolders. For example, in of my tests, I had the resource modules for unit tests in C:\temp. The test preparation functions added C:\ to the PSModulePath system variable that caused ISE to scan the entire drive for modules.

cioe-utuoaaiysh

For integration tests too, this need not be done. Instead, the modules can be copied over the existing module path and cleaned up later.

Suggest introducing tags

This would allow for easy exclusion of specific test types when the repo is cloned in to other projects

Problem with markdown lint errors MD013 and MD024

In the xSQLServer repo the README.md currently have long lines and using the same header several times (like PSDscResource).

MD013
https://ci.appveyor.com/project/johlju/xsqlserver/build/5.0.342.0#L864
MD024
https://ci.appveyor.com/project/johlju/xsqlserver/build/5.0.342.0#L1139

We have three options as I see it (not taking in account changing the README.md).

  1. Add MD013 and MD024 to the .markdownlint.json file in this module.
  2. Change the code so it uses the .markdownlint.json in the module being tested if it exist. I guess this line need to do change https://github.com/PowerShell/DscResource.Tests/blob/dev/gulpfile.js#L21 with some logic :)
  3. I workaround it by parsing in the additional MD013 and MD024 in the .markdownlint.json after cloning DscResource.Tests in AppVeyor.yml.

I personally like option 2.

Meta.tests.ps1 add incorrect path to PSModulePath

Currently we add

# Modify PSModulePath of the current PowerShell session. 
51 # We want to make sure we always test the development version of the resource 
52 # in the current build directory. 
53 if (($env:PSModulePath.Split(';') | Select-Object -First 1) -ne $pwd) 
54 { 
55     $env:PSModulePath = "$pwd;$env:PSModulePath" 
56 } 

Which in appveyor will add C:\project<repo_name>

Instead, we should be adding:

# Modify PSModulePath of the current PowerShell session.
# We want to make sure we always test the development version of the resource
# in the current build directory.
$script:oldPSModulePath = $env:PSModulePath
$pathToAdd = Split-Path $pwd -Parent
if (($env:PSModulePath.Split(';') | Select-Object -First 1) -ne $pathToAdd)
{
    $env:PSModulePath = "$pathToAdd;$env:PSModulePath"
}

which would add C:\project (containing folder with the DSC resource module).

Another issue is that we should be also reverting value of $env:PSModulePath to the original value after the tests.

Move Import-Module from Initialize-TestEnvironment to *.tests.ps1

See this discussion for context.

The problem with Import-Module being inside the Intialize-TestEnvironment is that the DSC Resouce is loaded into the scope of the TestHelper.psm1 module. This means that tests for *-TargetResource have to be made inside an InModuleScope block.

@iainbrighton has tested the Import-Module cmdlet with -Scope Global set but it results in problems Mocking Get-TargetResource from the Test- and Set-TargetResource.

@narrieta has suggested the solution to this would be to move the Import-Module cmdlet from out of the Initialization and into the Unit Test header for each resource. Import-Module is found here.

This would be a breaking change for any resource unit and integration tests using the current test templates. Fixing these wouldn't be a major issue however but needs to be taken into account.

@KarolKaczmarek, @TravisEz13, @iainbrighton, @tysonjhayes?

Use Pester's TestDrive:/ in tests rather than a custom working directory

I've been looking through the Initialize-TestEnvironment method getting ready to retrofit this in to SharePointDsc and one of the questions I have about it for unit testing is why it sets up a custom path in the temp directory to save files in when Pester does the exact same thing already. In the cases of unit tests in SharePointDsc that we choose to not mock the file functions out, and in our integration tests, all of the path references point off to the TestDrive path. This gives us a fresh TestDrive that gets cleaned up between each Describe/Context block without it needing to be additional logic inside the DscResource.Tests scripts.

I don't see any harm in having it in there, but it does seem a touch redundant given the tool we use for testing already covers this off. Just wanted to throw it out there for discussion to see if there is a use case for this that I'm not seeing or if it's something that could be removed in favour of what Pester gives us natively.

Need to improve how we verify an example MOF should compile

Meta.Tests.ps1, line 439 at r5 (raw file):

                    $params = @{}
                    $command.Parameters.Keys | Where-Object { $_ -like "*Account" -or $_ -eq "Passphrase" } | ForEach-Object -Process {
                        $params.Add($exampleFile, $mockCredential)

I can't see this $mockCredential being assigned anywhere?

See
https://github.com/PowerShell/xSQLServer/blob/dev/Tests/xSQLServerCommon.Tests.ps1#L20


Meta.Tests.ps1, line 441 at r5 (raw file):

                        $params.Add($exampleFile, $mockCredential)
                    }
                    $null = Example @params -OutputPath "TestDrive:\" -ErrorAction Continue -WarningAction SilentlyContinue

Don't we need it to load with a dummy configuration ? Like -ConfigurationData $mockConfigurationData? or is this done in another way?
See https://github.com/PowerShell/xSQLServer/blob/dev/Tests/xSQLServerCommon.Tests.ps1#L51

This can be added later: I also have seen a need for using an example provided ConfigurationData block. It should look for $ConfigurationData variable after dot-sourcing the example. If it exist it should use that to run the example. If not, it should use the default $mockConfigurationData.


Suggest using Save-Module rather than Install-Module

The current script installs the module to the build slave. This can be troublesome because it requires administrative access and could potentially impact the build slave which is an issue if the same environment is re-used in unit testing. Using Save-Module to the current user profile and then deleting the folder on cleanup would be a nice approach.

Running tests with least privlege

Hi,

When running in an environment of least privilege, you cannot run DSC tests making use of TestHelper.psm1.

The helper attempts to make registry changes on line 373. My analysis seems to show that we don't need to make this change at a system level, for all future executions of PowerShell. My recommendation is to remove like 373. If removal cannot occur, then place the correct # requires statement to ensure that it is executed as an Administrator.

The script modifies the execution policy. Once again this creates issues on multiple levels. This logic should be removed, and recommendations made on how the environment should be setup.

There is also calls to install-module that need to be addressed as well.

Initialize-TestEnvironment should change PSModulePath at Process level for Unit Tests

Currently the Initialize-TestEnvironment code changes the PSModuleRoot path to ensure that the correct module is tested. It is changed at the machine level to ensure that the correct path is set during Integration testing. However it should only be changed at the Process level for Unit tests.

These lines:
line https://github.com/PowerShell/DscResource.Tests/blob/master/TestHelper.psm1#L357
https://github.com/PowerShell/DscResource.Tests/blob/master/TestHelper.psm1#L419
should be wrapped in a check so that they only fire when $TestType = 'integration'

As noted by @narrieta in dsccommunity/ActiveDirectoryDsc#96 (comment)

PSScriptAnalyzer errors not shown correctly when more than one occurs in Meta.Tests

If more that one error is detected by PSScriptAnalyzer test in meta.test.ps1 the text that is produced is not correctly formed:
2015-12-19_17-25-27
The problem occurs here:
https://github.com/PowerShell/DscResource.Tests/blob/master/Meta.Tests.ps1#L147

This should be changed to something like this:

Write-Error -Message 'There are PSScriptAnalyzer errors that need to be fixed:'
@($PSScriptAnalyzerErrors).Foreach( { Write-Error -Message $_ } )
Write-Error -Message 'For instructions on how to run PSScriptAnalyzer on your own machine, please go to https://github.com/powershell/psscriptAnalyzer/'

I'll test and submit a PR.

Thanks!

Run Test-xDscResource by default

In the xDSCResourceDesigner module there is a function Test-xDscResource that tests if the resource fails many common tests including some warnings that the Get-TargetResource should return a hashtable and Test-TargetResource should return a boolen.

It would be quite nice if we tested this by default or at least pulled in a number of the functions into these tests to improve the quality of the code we are working on.

DscResourceCommonTests\Meta.Tests.ps1 incorrect module import

Hi, this path seems incorrect now:

# Load the TestHelper module which contains the *-ResourceDesigner functions Import-Module -Name (Join-Path -Path $PSScriptRoot -ChildPath 'TestHelper.psm1') -Force

as this is thrown

Import-Module : The specified module 'C:\projects\xwebadministration\DscResource.Tests\DscResourceCommonTests\TestHelper.psm1' was not loaded because no valid module file was found in any module directory. At C:\projects\xwebadministration\DscResource.Tests\DscResourceCommonTests\Meta.Tests.ps1:22 char:1

from

https://ci.appveyor.com/project/PowerShell/xwebadministration

thanks

Module installation of xDscResourceDesigner failed with exit code at <ScriptBlock>

I often receive the following error when invoking pester testing when the DSCResource.Tests is included in the project:
[-] Error occurred in test script 'C:\projects\xnetworking\DscResource.Tests\Meta.Tests.ps1' 22.92s
Module installation of xDscResourceDesigner failed with exit code
at , C:\projects\xnetworking\DscResource.Tests\Meta.Tests.ps1: line 46

2015-10-13_9-24-57

This error is completely random and only occurs about 50% of the time. If I Invoke the pester tests again it usually works correctly with no error. This wouldn't usually be too much of a problem, but the issue also occurs up in AppVeyor where I might not have permission to re-build a commit - unless I actually commit something - which is more of a problem.

This maybe a connection or download issue but I'm not sure. Any help would be appreciated!

Initialize-TestEnvironment will fail if your box is Domain Joined machine

Description

Initialize-TestEnvironment will failed to change Execution policy if Box is joined to Domain.

Exception

When you run like other DSC Resource doing, you will get Security Error on L383

# Initialize
[String] $moduleRoot = Split-Path -Parent (Split-Path -Parent (Split-Path -Parent $Script:MyInvocation.MyCommand.Path))
if ( (-not (Test-Path -Path (Join-Path -Path $moduleRoot -ChildPath 'DSCResource.Tests'))) -or `
     (-not (Test-Path -Path (Join-Path -Path $moduleRoot -ChildPath 'DSCResource.Tests\TestHelper.psm1'))) )
{
    & git @('clone','https://github.com/PowerShell/DscResource.Tests.git',(Join-Path -Path $moduleRoot -ChildPath '\DSCResource.Tests\'))
}
else
{
    & git @('-C',(Join-Path -Path $moduleRoot -ChildPath '\DSCResource.Tests\'),'pull')
}
Import-Module (Join-Path -Path $moduleRoot -ChildPath 'DSCResource.Tests\TestHelper.psm1') -Force
$TestEnvironment = Initialize-TestEnvironment -DSCModuleName $global:dscModuleName -DSCResourceName $global:dscResourceName -TestType Unit
Initializing Test Environment for Unit testing of Grani_CredentialManager in module GraniResource.
DSC Module Manifest 'GraniResource.psd1' detected in folder 'D:\Github\guitarrapc\GraniResource'.
[-] Error occurred in test script 'D:\Github\guitarrapc\GraniResource\Test\Grani_CredentialManager\Grani_CredentialManager.Tests.ps1' 1.29s
  Security error.
  at Initialize-TestEnvironment, D:\Github\guitarrapc\GraniResource\DSCResource.Tests\TestHelper.psm1: line 383

Reason

Issue happen when Domain Joined machine, or any other condition may apply, had restricted to change ExecutionPolicy. You can reproduce error by following on Domain Joined Box.

Set-ExecutionPolicy -ExecutionPolicy Unrestricted
Execution Policy Change
The execution policy helps protect you from scripts that you do not trust. Changing the execution policy might expose you to the security risks described in the
about_Execution_Policies help topic at http://go.microsoft.com/fwlink/?LinkID=135170. Do you want to change the execution policy?
[Y] Yes  [A] Yes to All  [N] No  [L] No to All  [S] Suspend  [?] Help (default is "N"): y
Set-ExecutionPolicy : Windows PowerShell updated your execution policy successfully, but the setting is overridden by a policy defined at a more specific scope.  Due to the ove
rride, your shell will retain its current effective execution policy of Bypass. Type "Get-ExecutionPolicy -List" to view your execution policy settings. For more information pl
ease see "Get-Help Set-ExecutionPolicy".
At line:1 char:1
+ Set-ExecutionPolicy -ExecutionPolicy Unrestricted
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : PermissionDenied: (:) [Set-ExecutionPolicy], SecurityException
    + FullyQualifiedErrorId : ExecutionPolicyOverride,Microsoft.PowerShell.Commands.SetExecutionPolicyCommand

Proposal fix

You can avoid issue by just change execution policy to only current Process. As this function aim to remove any side-effect with Test, I propose limit to current process is much better.

Set-ExecutionPolicy -ExecutionPolicy Unrestricted -Scope Process -Force

Suppressing the Example Tests results causes "Pending" test results which AppVeyor won't accept

I think we might be running into a problem with the suppressed Example validation tests:
https://ci.appveyor.com/project/PlagueHO/xnetworking/build/3.1.513.0#L2202

They seem to output a test result value of "Pending" which AppVeyor is rejecting.

Looking at the message, Pending is not valid. The valid values are:
None, Running, Passed, Failed, Ignored, Skipped, Inconclusive, NotFound, Cancelled, NotRunnable

So I think we need to implement @TravisEz13 's improved suppression mechanism or use the -ExcludeTag 'Examples' mechanism instead.

ConvertTo-SpaceIndentation Breaks Files in Some Cases

Recently when I submitted a pull request for changes to the xExchange module, the automated tests failed on a couple files, and suggested running Get-TextFilesList | ConvertTo-SpaceIndentation to fix them. I decided to run this against the entire module, instead of just the files it suggested. After doing so and resubmitting the pull request, I got failures in multiple files that weren't otherwise modified other than running these commands against them. The failures are reported here:

https://ci.appveyor.com/project/PowerShell/xexchange/build/1.0.35

One of the files that broke is located here:

https://github.com/PowerShell/xExchange/blob/dev/DSCResources/MSFT_xExchMailboxDatabase/MSFT_xExchMailboxDatabase.psm1

make new common tests warnings until repos have opted-in

There should be a file in each repo that says which common tests the repo has passed. Those are treated as an error. Once someone gets that test passing they add the test to the file so that regressions will cause a failure.

Restore-TestEnvironment remains PSModulePath added by Initialize-TestEnvironment

Description of Issue

Restore-TestEnvironment could not handle Initialize-TestEnvironment added path.

Detail

Initialize-TestEnvironment adding current Test DSC Module's Root Path, which retrieved at here https://github.com/PowerShell/DscResource.Tests/blob/dev/TestHelper.psm1#L347, to the Machine's PSmodulePath at https://github.com/PowerShell/DscResource.Tests/blob/dev/TestHelper.psm1#L376

It's trying to pass $OldModulePath info, https://github.com/PowerShell/DscResource.Tests/blob/dev/TestHelper.psm1#L393 but ModuleRoot is remain afterRestore-TestEnvironment has ran.

Expectation

Should completely remove Test Module Path from PSModulePath.

psd1 files are now UTF16

Hi, PSScriptAnalyzer has just popped this up:

File C:\projects\xwebadministration\DscResource.Tests\DscResourceCommonTests\DscResourceCommonTests.psd1 contains 0x00 bytes. It's probably uses Unicode and need to be converted to UTF-8. Use Fixer 'Get-UnicodeFilesList $pwd | ConvertTo-UTF8'. File C:\projects\xwebadministration\DscResource.Tests\DscResourceTestHelper\DscResourceTestHelper.psd1 contains 0x00 bytes. It's probably uses Unicode and need to be converted to UTF-8. Use Fixer 'Get-UnicodeFilesList $pwd | ConvertTo-UTF8'.

and

File C:\projects\xwebadministration\DscResource.Tests\DscResourceCommonTests\DscResourceCommonTests.psd1 contains 0x00 bytes. It's probably uses Unicode and need to be converted to UTF-8. Use Fixer 'Get-UnicodeFilesList $pwd | ConvertTo-UTF8'. File C:\projects\xwebadministration\DscResource.Tests\DscResourceTestHelper\DscResourceTestHelper.psd1 contains 0x00 bytes. It's probably uses Unicode and need to be converted to UTF-8. Use Fixer 'Get-UnicodeFilesList $pwd | ConvertTo-UTF8'.

from https://ci.appveyor.com/project/PowerShell/xwebadministration/build/1.10.346.0

thanks

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.