GithubHelp home page GithubHelp logo

azure / arm-ttk Goto Github PK

View Code? Open in Web Editor NEW
425.0 26.0 182.0 5.34 MB

Azure Resource Manager Template Toolkit

Home Page: https://aka.ms/arm-ttk

License: MIT License

PowerShell 99.61% Batchfile 0.04% Shell 0.35%

arm-ttk's Introduction

Azure Resource Manager Template Toolkit (arm-ttk)

The code in this repository can be used for analyzing and testing Azure Resource Manager Templates. The tests will check a template or set of templates for coding best practices. There are some checks for simple syntactical errors but the intent is not to re-implement tests or checks that are provided by the platform (e.g. the /validate api).

Note: Starting with the 0.10 release, Bicep now contains all of the deploymentTemplate test cases included in the TTK. We will begin moving the investment in new tests to the Bicep linter. The TTK will remain available to support available JSON and createUiDefinition scenarios.

Using the TTK

For detailed instruction on how to use the arm-ttk, see this readme. More information can be found in the documentation.

For a guided tutorial on the arm-ttk, check out this MS LEARN module.

Philosophy

A little bit about the tests... These are the tests that are used to validate templates for the Azure QuickStart Repo and the Azure Marketplace. The purpose is to ensure a standard or consistent set of coding practices to make it easier to develop expertise using the template language (easy to read, write, debug).

As for the type, number and nature of the tests a test should check for something in the following categories (add more as you think of them :))

  • Validating the author's intent (unused parameters or variables)
  • Security practices for the language (outputting secrets in plain text)
  • Using the appropriate language construct for the task at hand (using environmental functions instead of hard-coding values)

Not everything is appropriate for a universal set of tests and not every test will apply to every scenario, so the framework allows for easy expansion and individual selection of tests.

Running Unit Tests locally before request a PR

Tests can be run directly in PowerShell, or run from the command line using a wrapper script.

You can run all of the unit tests by using .\arm-ttk.tests.ps1.

This will run the full suite of unit tests against the tests json files.

use:

# set your location in the project directory:
Set-Location -Path "$(YourGithubProjectFolder)\arm-ttk\unit-tests"

# import the module from the current branch, use -Force to make sure you have imported any code changes
Import-Module ..\arm-ttk\arm-ttk.psd1 -Force

# These are the same tests that run in the pipeline when doing a commit or a pull request (PR). 
.\arm-ttk.tests.ps1

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repositories using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct.

For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

arm-ttk's People

Contributors

bb-froggy avatar bmoore-msft avatar bobbyangers avatar chgeuer avatar dciborow avatar dvdmtw98 avatar edburns avatar galiacheng avatar github-actions[bot] avatar jakelevinez avatar jf781 avatar konjp avatar microsoftopensource avatar niktripathi avatar psah434 avatar rangers-globecar avatar raosuhas avatar rcousens avatar sam-cogan avatar samrobillard avatar samsmithnz avatar startautomating avatar stephenweatherford avatar thomasyip-msft avatar tiefpunkt avatar v-amolpatil avatar v-rucdu avatar v-rusraut avatar verabe avatar yks0000 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

arm-ttk's Issues

Allow parameter to specify the filename for "the main template"

In the location test for the TTK there is a flag to check if the file is the $IsMainTemplate. This boolean is if the file name is one of 'mainTemplate.json', 'azureDeploy.json', 'prereq.azuredeploy.json'. If so then check the parameter location for the correct definition of resourceGroup.location() being the default.

This prohibits any deployment files from being named differently.

Here is the code for the test:

if ($IsMainTemplate){ 
    if($locationParameter.defaultValue -and "$($locationParameter.defaultvalue)".Trim() -ne '[resourceGroup().location]') {
    Write-Error "The defaultValue of the location parameter in the main template must not be a specific location. The default value must be [resourceGroup().location]. It is 
`"$($locationParameter.defaultValue)`"" -ErrorId Location.Parameter.Hardcoded -TargetObject $parameter
}
# In all other templates:
# if the parameter named "location" exists, it must not have a defaultValue property
# Note that Powershell will count an empty string (which should fail the test) as null if not explictly tested, so we check for it
}else {
    if($locationParameter.defaultValue -ne $null){ 
        Write-Error "The location parameter of nested templates must not have a defaultValue property. It is `"$($locationParameter.defaultValue)`"" -ErrorId Location.Parameter.DefaultValuePresent -TargetObject $parameter
    }   
}

This is problematic as if deploying multiple arm templates it isn't always ideal to be naming the same.

"Virtual Machines Should Not Be Preview" fails for scale set

When a template is creating a scale set, following test is failing :

Virtual Machines Should Not Be Preview (1 ms) Virtual machine resource [variables('vScaleSetName')] has no image to reference

It should not, my guess is in scale set image reference is given through
"virtualMachineProfile": {
"storageProfile": {
"osDisk": {
"caching": "ReadWrite",
"createOption": "FromImage"
},
"imageReference": "[variables('customImage')]"
}

The test is probably looking for storageProfile as it is defined for vm

tags name containing id fails IDs Should Be Derived From ResourceIDs test

Hello,
I have a resource with a tag that is called projectid. This tag has nothing to do with azure IDs of course since it relates to our internal system for tracking projects.
If my understanding is correct I suppose a slight change is needed around here :


To also ignore when path contains "tags".

Thanks

IDs Should Be Derived From ResourceIDs fails in Load Balancer

We have several templates that use a loadbalancer similar to the link in the azure quickstarts below. You can see that we have id parameters that need to be concatenated to reach the properties within the load balancer, e.g. frontend ip config, backend pool, lb probe. From what I can see this is only possible using "concat" or am I missing a feature in ARM templates?

Adding a variable and placing the "concat" in the variable section also triggers a fail.

https://github.com/Azure/azure-quickstart-templates/blob/master/101-internal-loadbalancer-create/azuredeploy.json

"frontendIPConfiguration": {
  "id": "[concat(resourceId('Microsoft.Network/loadBalancers', variables('loadBalancerName')), '/frontendIpConfigurations/loadBalancerFrontEnd')]"
},
"backendAddressPool": {
  "id": "[concat(resourceId('Microsoft.Network/loadBalancers', variables('loadBalancerName')), '/backendAddressPools/loadBalancerBackEnd')]"
},
"probe": {
  "id": "[concat(resourceId('Microsoft.Network/loadBalancers', variables('loadBalancerName')), '/probes/lbprobe')]"
},

Steps to reproduce:

    [-] IDs Should Be Derived From ResourceIDs (35 ms)
        Property: "id" must use one of the following expressions for an resourceId property:                                                                                                                               extensionResourceId,resourceId,subscriptionResourceId,tenantResourceId,if,parameters,reference,variables,subscription,guid                                                                                 
        Property: "id" must use one of the following expressions for an resourceId property:                                                                                                                               extensionResourceId,resourceId,subscriptionResourceId,tenantResourceId,if,parameters,reference,variables,subscription,guid                                                                                 
        Property: "id" must use one of the following expressions for an resourceId property:                                                                                                                               extensionResourceId,resourceId,subscriptionResourceId,tenantResourceId,if,parameters,reference,variables,subscription,guid                                                                                 
        Property: "id" must use one of the following expressions for an resourceId property:                                                                                                                               extensionResourceId,resourceId,subscriptionResourceId,tenantResourceId,if,parameters,reference,variables,subscription,guid

need to be able to run (some) test cases on non-template files

E.g. the test case that checks the templates for hard-coded endpoints in templates, needs to be extended to other files in the sample/project - for example scripts (sh, ps1) need to also be checked for the endpoints...

currently we only check json files with the proper schema

Min And Max Value Are Numbers is not working properly

--------------------MESSAGE FROM ADMIN, DELETE BEFORE SUBMITTING----------------------

Sorry to hear you had a bad experience with one of the templates ๐Ÿ˜Ÿ But, in case you're just asking a question, we're happy to help. You can also check if the question might already have been asked here https://github.com/Azure/azure-quickstart-templates/issues?utf8=%E2%9C%93&q=is%3Aissue

We've created an outline of recommended sections to fill out that will help make this Pull Request awesome!

--------------------MESSAGE FROM ADMIN, DELETE BEFORE SUBMITTING----------------------

[Template Name goes here](Template link goes here)
test/arm-ttk testcases Min And Max Value Are Numbers

Issue Details

I understand this testcase is to ensure using integers for min/max values. Yet, i have a parameter in my template

        "numberOfInstances": {
            "type": "int",
            "defaultValue": 2,
            "minValue": 0,
            "maxValue": 5
        },

then i run the deploymentTemplate test, and it gives me the failures on Min And Max Value Are Numbers saying:

[-] Min And Max Value Are Numbers (1182 ms)
numberOfInstances maxValue is not an [int] (it's a [int])
numberOfInstances minValue is not an [int] (it's a [int])

I couldn't find any suspicious points in the source code as well, please have a look, thanks!

Repro steps (if necessary, delete otherwise)

  1. having a parameter field like the one above in mainTemplate.json
  2. run Test-AzTemplate -TemplatePath $youpath -Test "Min And Max Value Are Numbers"

cannot debug individual tests that use Find-JsonContent or Expand-AzTemplate

tests that use one of our shared functions (e.g. Find-JsonContent) cannot be run and debugged individually because the function is not found/loaded when running scripts individually...

We'd need to conditionally load the fn before running

# First, find all objects with an ID property in the MainTemplate.
$ids = $TemplateObject  | Find-JsonContent -Key *id <#-Value *#> -Like

VMSizes-Must-Match-Template.test.ps1 invalid assumption

Description

Consider https://github.com/edburns/arm-ttk/blob/master/arm-ttk/testcases/CreateUIDefinition/VMSizes-Must-Match-Template.test.ps1 .

As far as I can tell, the intent of line 22 is to define a boolean
variable depending on whether the vmSizeSelect control is located in
"steps" or in "basics". I observe the logic currently used to make this
determination is faulty.

I submit less faulty logic is the following:

$isInSteps = $selector.JSONPath.Contains('steps[')
$lookingFor= if ($isInSteps) { "*steps(*$stepName*).$controlName*"} else {"*basics(*$($controlName)*"} 

I will include two tests inputs below, both with the same
mainTemplate.json but with different createUiDefinition.json files.
Let's label them as vmSizeSelectInSteps and vmSizeSelectInBasics.
In the case of vmSizeSelectInSteps the value of $selector.JSONPath is

JSONPath=[0].parameters[0].steps[0].elements.type

While in the case of vmSizeSelectInBasics the value of
$selector.JSONPath is

JSONPath=[0].parameters[0].basics[2].elements.type

Therefore, using the occurrence of the string steps[ in JSONPath is a
better indicator of the intent than $stepName = $selector.ParentObject[0].name

Test Input

mainTemplate.json

{
    "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "_artifactsLocation": {
            "defaultValue": "[deployment().properties.templateLink.uri]",
            "type": "string",
            "metadata": {
                "description": "The base URI where artifacts required by this template are located. When the template is deployed using the accompanying scripts, a private location in the subscription will be used and this value will be automatically generated."
            }
        },
        "_artifactsLocationSasToken": {
            "defaultValue": "",
            "type": "securestring",
            "metadata": {
                "description": "The sasToken required to access _artifactsLocation.  When the template is deployed using the accompanying scripts, a sasToken will be automatically generated. Use the defaultValue if the staging location is not secured."
            }
        },
        "enableAppGateway": {
            "defaultValue": false,
            "type": "bool",
            "metadata": {
                "description": "If true, deploy an Azure App Gateway in front of the nodes of the cluster"
            }
        },
        "portsToExpose": {
            "defaultValue": "80,443,7001-9000",
            "type": "string",
            "metadata": {
                "description": "Ports and port ranges to expose"
            }
        },
        "wlsDomainName": {
            "defaultValue": "clusterDomain",
            "type": "string",
            "metadata": {
                "description": "Provide Weblogic domain name"
            }
        },
        "managedServerPrefix": {
            "defaultValue": "msp",
            "type": "string",
            "metadata": {
                "description": "Provide managed server prefix names"
            }
        },
        "dnsLabelPrefix": {
            "defaultValue": "wls",
            "type": "string",
            "metadata": {
                "description": "Unique DNS Name for the Public IP used to access the Virtual Machine."
            }
        },
        "dnsNameforApplicationGateway": {
            "defaultValue": "wlsclusterappgw",
            "type": "string",
            "metadata": {
                "description": "DNS for ApplicationGateway"
            }
        },
        "gatewayPublicIPAddressName": {
            "defaultValue": "gatewayPublicIP",
            "type": "string",
            "metadata": {
                "description": "Public IP Name for the Application Gateway"
            }
        },
        "linuxOSVersion": {
            "defaultValue": "7.4",
            "type": "string",
            "allowedValues": [
                "7.4",
                "7.3"
            ],
            "metadata": {
                "description": "The Oracle Linux version for the VM. This will pick a fully patched image of this given Oracle Linux version."
            }
        },
        "numberOfInstances": {
            "defaultValue": 2,
            "type": "int",
            "minValue": 2,
            "maxValue": 5,
            "metadata": {
                "description": "Number of VMs to deploy, limit 5 since this sample is using a single storage account"
            }
        },
        "adminVMName": {
            "defaultValue": "adminVM",
            "type": "string",
            "metadata": {
                "description": "Admin Server hosting VM name."
            }
        },
        "location": {
            "defaultValue": "[resourceGroup().location]",
            "type": "string",
            "metadata": {
                "description": "Location for all resources."
            }
        },
        "vmSizeSelect": {
            "defaultValue": "Standard_A3",
            "type": "string",
            "allowedValues": [
                "Standard_A1",
                "Standard_A2",
                "Standard_A3",
                "Standard_A4"
            ],
            "metadata": {
                "description": "Select appropriate VM Size as per requirement (Standard_A1, Standard_A2, Standard_A3, Standard_A4)"
            }
        },
        "keyVaultName": {
            "defaultValue": "",
            "type": "string",
            "metadata": {
                "description": "KeyVault Name"
            }
        },
        "keyVaultResourceGroup": {
            "defaultValue": "",
            "type": "string",
            "metadata": {
                "description": "Resource group name in current subscription containing the KeyVault"
            }
        },
        "keyVaultSSLCertDataSecretName": {
            "defaultValue": "",
            "type": "string",
            "metadata": {
                "description": "The name of the secret in the specified KeyVault whose value is the SSL Certificate Data"
            }
        },
        "keyVaultSSLCertPasswordSecretName": {
            "defaultValue": "",
            "type": "string",
            "metadata": {
                "description": "The name of the secret in the specified KeyVault whose value is the password for the corresponding SSL Certificate"
            }
        }
    },
    "variables": {
        "keyVaultLinkedTemplateName": "appGatewayNestedTemplate.json",
        "clusterLinkedTemplateName": "clusterTemplate.json",
        "currentSubscription": "[subscription().subscriptionId]"
    },
    "resources": [
        {
            "type": "Microsoft.Resources/deployments",
            "apiVersion": "2018-05-01",
            "name": "clusterLinkedTemplate",
            "properties": {
                "mode": "Incremental",
                "templateLink": {
                    "uri": "[uri(parameters('_artifactsLocation'), concat('nestedtemplates/', variables('clusterLinkedTemplateName')))]",
                    "contentVersion": "1.0.0.0"
                },
                "parameters": {
                    "_artifactsLocation": {
                        "value": "[parameters('_artifactsLocation')]"
                    },
                    "_artifactsLocationSasToken": {
                        "value": "[parameters('_artifactsLocationSasToken')]"
                    },
                    "acceptOTNLicenseAgreement": {
                        "value": "[parameters('acceptOTNLicenseAgreement')]"
                    },
                    "adminPasswordOrKey": {
                        "value": "[parameters('adminPasswordOrKey')]"
                    },
                    "adminUsername": {
                        "value": "[parameters('adminUsername')]"
                    },
                    "adminVMName": {
                        "value": "[parameters('adminVMName')]"
                    },
                    "dnsLabelPrefix": {
                        "value": "[parameters('dnsLabelPrefix')]"
                    },
                    "dnsNameforApplicationGateway": {
                        "value": "[parameters('dnsNameforApplicationGateway')]"
                    },
                    "enableAppGateway": {
                        "value": "[parameters('enableAppGateway')]"
                    },
                    "gatewayPublicIPAddressName": {
                        "value": "[parameters('gatewayPublicIPAddressName')]"
                    },
                    "linuxOSVersion": {
                        "value": "[parameters('linuxOSVersion')]"
                    },
                    "location": {
                        "value": "[parameters('location')]"
                    },
                    "managedServerPrefix": {
                        "value": "[parameters('managedServerPrefix')]"
                    },
                    "numberOfInstances": {
                        "value": "[parameters('numberOfInstances')]"
                    },
                    "otnAccountUsername": {
                        "value": "[parameters('otnAccountUsername')]"
                    },
                    "otnAccountPassword": {
                        "value": "[parameters('otnAccountPassword')]"
                    },
                    "portsToExpose": {
                        "value": "[parameters('portsToExpose')]"
                    },
                    "vmSizeSelect": {
                        "value": "[parameters('vmSizeSelect')]"
                    },
                    "wlsDomainName": {
                        "value": "[parameters('wlsDomainName')]"
                    },
                    "wlsUserName": {
                        "value": "[parameters('wlsUserName')]"
                    },
                    "wlsPassword": {
                        "value": "[parameters('wlsPassword')]"
                    }

                }
            }
        },
        {
            "type": "Microsoft.Resources/deployments",
            "apiVersion": "2018-05-01",
            "name": "keyVaultLinkedAppGatewayTemplate",
            "condition": "[parameters('enableAppGateway')]",
            "properties": {
                "mode": "Incremental",
                "templateLink": {
                    "uri": "[uri(parameters('_artifactsLocation'), concat('nestedtemplates/', variables('keyVaultLinkedTemplateName')))]",
                    "contentVersion": "1.0.0.0"
                },
                "parameters": {
                    "appGatewaySSLCertificateData": {
                        "reference": {
                            "keyVault": {
                                "id": "[resourceId(variables('currentSubscription'),  parameters('keyVaultResourceGroup'), 'Microsoft.KeyVault/vaults', parameters('keyVaultName'))]"
                            },
                            "secretName": "[parameters('keyVaultSSLCertDataSecretName')]"
                        }
                    },
                    "appGatewaySSLCertificatePassword": {
                        "reference": {
                            "keyVault": {
                                "id": "[resourceId(variables('currentSubscription'),  parameters('keyVaultResourceGroup'), 'Microsoft.KeyVault/vaults', parameters('keyVaultName'))]"
                            },
                            "secretName": "[parameters('keyVaultSSLCertPasswordSecretName')]"
                        }
                    },
                    "gatewayPublicIPAddressName": {
                        "value": "[parameters('gatewayPublicIPAddressName')]"
                    },
                    "location": {
                        "value": "[parameters('location')]"
                    },
                    "managedServerPrefix": {
                        "value": "[parameters('managedServerPrefix')]"
                    },
                    "numberOfInstances": {
                        "value": "[parameters('numberOfInstances')]"
                    },
                    "wlsDomainName": {
                        "value": "[parameters('wlsDomainName')]"
                    }
                }
            },
            "dependsOn": [
                "clusterLinkedTemplate"
            ]
        }
    ],
    "outputs": {
        "wlsDomainLocation": {
            "type": "string",
            "value": "[reference('clusterLinkedTemplate').outputs.wlsDomainLocation.value]"
        },
        "adminHostName": {
            "type": "string",
            "value": "[reference('clusterLinkedTemplate').outputs.adminHostName.value]"
        },
        "adminConsole": {
            "type": "string",
            "value": "[reference('clusterLinkedTemplate').outputs.adminConsole.value]"
        },
        "adminSecuredConsole": {
            "type": "string",
            "value": "[reference('clusterLinkedTemplate').outputs.adminSecuredConsole.value]"
        },
        "appGatewayURL": {
            "type": "string",
            "value": "[if(parameters('enableAppGateway'), reference('keyVaultLinkedAppGatewayTemplate').outputs.appGatewayURL.value, '')]"

        }
    }
}

createUiDefinition vmSizeSelectInSteps

{
    "$schema": "https://schema.management.azure.com/schemas/0.1.2-preview/CreateUIDefinition.MultiVm.json#",
    "handler": "Microsoft.Azure.CreateUIDef",
    "version": "0.1.2-preview",
    "parameters": {
        "basics": [
            {
                "name": "dnsLabelPrefix",
                "type": "Microsoft.Common.TextBox",
                "label": "DNS Label Prefix",
                "toolTip": "The string to prepend to the DNS label.",
                "defaultValue": "wls",
                "constraints": {
                    "required": true,
                    "regex": "^[a-z0-9A-Z]{3,10}$",
                    "validationMessage": "The prefix must be between 3 and 10 characters long and contain letters, numbers only."
                }
            },
            {
                "name": "managedServerPrefix",
                "type": "Microsoft.Common.TextBox",
                "label": "Managed Server Prefix",
                "toolTip": "The string to prepend to the name of the managed server.",
                "defaultValue": "msp",
                "constraints": {
                    "required": true,
                    "regex": "^[a-z0-9A-Z]{3,20}$",
                    "validationMessage": "The prefix must be between 3 and 20 characters long and contain letters, numbers only."
                }
            },
            {
                "name": "numberOfInstances",
                "type": "Microsoft.Common.TextBox",
                "label": "Number of VMs",
                "defaultValue": "4",
                "toolTip": "The number of VMs to create, with one WebLogic node per VM",
                "constraints": {
                    "required": true,
                    "regex": "^(2|3|4|5)$",
                    "validationMessage": "Number of VMs to deploy, limit 5 since this offer is using a single storage account."
                },
                "visible": true
            },
            {
                "name": "wlsDomainName",
                "type": "Microsoft.Common.TextBox",
                "label": "WebLogic Domain Name",
                "toolTip": "The name of the WebLogic Domain to create.",
                "defaultValue": "clusterDomain",
                "constraints": {
                    "required": true,
                    "regex": "^[a-z0-9A-Z]{3,20}$",
                    "validationMessage": "The Domain Name must be between 3 and 20 characters long and contain letters, numbers only."
                }
            }
        ],
        "steps": [
            {
                "name": "VirtualMachineConfig",
                "label": "Virtual Machine Settings",
                "subLabel": {
                    "preValidation": "Configure the resources and settings of the virtual machines that comprise the nodes of the WebLogic Domain. ",
                    "postValidation": "Done"
                },
                "bladeTitle": "Node Virtual Machine Settings",
                "elements": [
                    {
                        "name": "vmSizeSelect",
                        "type": "Microsoft.Compute.SizeSelector",
                        "label": "Virtual machine size",
                        "toolTip": "The size of virtual machine to provision.",
                        "recommendedSizes": [
                            "Standard_A1",
                            "Standard_A2",
                            "Standard_A3",
                            "Standard_A4"
                        ],
                        "osPlatform": "Linux",
                        "count": "1"
                    }
                ]
            },
            {
                "name": "Credentials",
                "label": "Credentials for Cluster Creation",
                "subLabel": {
                    "preValidation": "Provide credentials Required for Cluster Creation",
                    "postValidation": "Done"
                },
                "bladeTitle": "Credentials for Cluster Creation",
                "elements": [
                    {
                        "name": "acceptOTNLicenseAgreement",
                        "label": "Accept OTN License Agreement",
                        "type": "Microsoft.Common.TextBox",
                        "toolTip": "A value of N indicates you do not accept the OTN License Agreement.  In that case the deployment will fail.",
                        "defaultValue": "Y",
                        "visible": true,
                        "constraints": {
                            "required": true,
                            "regex": "^[Yy]$",
                            "validationMessage": "The value must be Y/y to proceed with deployment."
                        }
                    },
                    {
                        "name": "adminUsername",
                        "type": "Microsoft.Common.TextBox",
                        "label": "Username for admin account of VMs",
                        "defaultValue": "weblogic",
                        "toolTip": "Use only letters and numbers",
                        "constraints": {
                            "required": true,
                            "regex": "^[a-z0-9A-Z]{1,30}$",
                            "validationMessage": "The value must be 1-30 characters long and must only contain letters and numbers."
                        },
                        "visible": true
                    },
                    {
                        "name": "adminPasswordOrKey",
                        "type": "Microsoft.Common.PasswordBox",
                        "label": {
                            "password": "Password for admin account of VMs",
                            "confirmPassword": "Confirm password"
                        },
                        "toolTip": "Password for admin account of VMs",
                        "constraints": {
                            "required": true,
                            "regex": "^((?=.*[0-9])(?=.*[a-z])(?=.*[A-Z])|(?=.*[0-9])(?=.*[a-z])(?=.*[!@#$%^&*])|(?=.*[0-9])(?=.*[A-Z])(?=.*[!@#$%^&*])|(?=.*[a-z])(?=.*[A-Z])(?=.*[!@#$%^&*])).{12,72}$",
                            "validationMessage": "Password must be at least 12 characters long and have 3 out of the following: one number, one lower case, one upper case, or one special character"
                        },
                        "options": {
                            "hideConfirmation": false
                        },
                        "visible": true
                    },
                    {
                        "name": "otnAccountUsername",
                        "type": "Microsoft.Common.TextBox",
                        "label": "OTN Account Username",
                        "toolTip": "The username of a pre-existing OTN account.",
                        "constraints": {
                            "required": true,
                            "regex": "^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}$",
                            "validationMessage": "The value must be a valid email address"
                        }
                    },
                    {
                        "name": "otnAccountPassword",
                        "type": "Microsoft.Common.PasswordBox",
                        "label": {
                            "password": "Password for OTN Account",
                            "confirmPassword": "Confirm password"
                        },
                        "toolTip": "Password for OTN Account",
                        "constraints": {
                            "required": true,
                            "validationMessage": "Validation constraints for OTN accounts apply here."
                        },
                        "options": {
                            "hideConfirmation": false
                        },
                        "visible": true
                    },
                    {
                        "name": "wlsUserName",
                        "type": "Microsoft.Common.TextBox",
                        "label": "Username for WebLogic Administrator",
                        "defaultValue": "weblogic",
                        "toolTip": "Use only letters and numbers",
                        "constraints": {
                            "required": true,
                            "regex": "^[a-z0-9A-Z]{1,30}$",
                            "validationMessage": "The value must be 1-30 characters long and must only contain letters and numbers."
                        },
                        "visible": true
                    },
                    {
                        "name": "wlsPassword",
                        "type": "Microsoft.Common.PasswordBox",
                        "label": {
                            "password": "Password for WebLogic Administrator",
                            "confirmPassword": "Confirm password"
                        },
                        "toolTip": "Password for WebLogic Administrator",
                        "constraints": {
                            "required": true,
                            "regex": "^(?=.*[A-Z])(?=.*[a-z])(?=.*\\d)[A-Za-z\\d]{12,}$",
                            "validationMessage": "The password must contain at least 12 characters, with at least 1 uppercase letter, 1 lowercase letter and 1 number."
                        },
                        "options": {
                            "hideConfirmation": false
                        },
                        "visible": true
                    }
                ]
            },
            {
                "name": "Network",
                "label": "Network",
                "subLabel": {
                    "preValidation": "Parameters to customzie network settings",
                    "postValidation": "Done"
                },
                "bladeTitle": "Network",
                "elements": [
                    {
                        "name": "portsToExpose",
                        "label": "Ports and port ranges to expose (N or N-N, comma separated)",
                        "type": "Microsoft.Common.TextBox",
                        "toolTip": "Ports and port ranges to expose (N or N-N, comma separated)",
                        "defaultValue": "80,443,7001-9000",
                        "visible": true,
                        "constraints": {
                            "required": true,
                            "regex": "^((([0-9]+-[0-9]+)|([0-9]+))[,]?)+[^,]$",
                            "validationMessage": "Only numbers, hyphen separated ranges of numbers, separated by commas"
                        }
                    }
                ]
            }
        ],
        "outputs": {
            "acceptOTNLicenseAgreement": "[steps('Credentials').acceptOTNLicenseAgreement]",
            "adminPasswordOrKey": "[steps('Credentials').adminPasswordOrKey]",
            "adminUsername": "[steps('Credentials').adminUsername]",
            "dnsLabelPrefix": "[basics('dnsLabelPrefix')]",
            "managedServerPrefix": "[basics('managedServerPrefix')]",
            "numberOfInstances": "[int(basics('numberOfInstances'))]",
            "otnAccountPassword": "[steps('Credentials').otnAccountPassword]",
            "otnAccountUsername": "[steps('Credentials').otnAccountUsername]",
            "portsToExpose": "[steps('Network').portsToExpose]",
            "vmSizeSelect": "[steps('VirtualMachineConfig').vmSizeSelect]",
            "wlsDomainName": "[basics('wlsDomainName')]",
            "wlsPassword": "[steps('Credentials').wlsPassword]",
            "wlsUserName": "[steps('Credentials').wlsUserName]",
            "Location": "[location()]"
        }
    }
}

createUiDefinition vmSizeSelectInBasics

{
    "$schema": "https://schema.management.azure.com/schemas/0.1.2-preview/CreateUIDefinition.MultiVm.json#",
    "handler": "Microsoft.Azure.CreateUIDef",
    "version": "0.1.2-preview",
    "parameters": {
        "basics": [
            {
                "name": "acceptOTNLicenseAgreement",
                "label": "Accept OTN License Agreement",
                "type": "Microsoft.Common.TextBox",
                "toolTip": "A value of N indicates you do not accept the OTN License Agreement.  In that case the deployment will fail.",
                "defaultValue": "Y",
                "visible": true,
                "constraints": {
                    "required": true,
                    "regex": "^[Yy]$",
                    "validationMessage": "The value must be Y/y to proceed with deployment."
                }
            },
            {
                "name": "basicsRequired",
                "type": "Microsoft.Common.Section",
                "label": "Credentails for Virtual Machines and WebLogic",
                "elements": [
                    {
                        "name": "adminUsername",
                        "type": "Microsoft.Common.TextBox",
                        "label": "Username for admin account of VMs",
                        "defaultValue": "weblogic",
                        "toolTip": "Use only letters and numbers",
                        "constraints": {
                            "required": true,
                            "regex": "^[a-z0-9A-Z]{1,30}$",
                            "validationMessage": "The value must be 1-30 characters long and must only contain letters and numbers."
                        },
                        "visible": true
                    },
                    {
                        "name": "adminPasswordOrKey",
                        "type": "Microsoft.Common.PasswordBox",
                        "label": {
                            "password": "Password for admin account of VMs",
                            "confirmPassword": "Confirm password"
                        },
                        "toolTip": "Password for admin account of VMs",
                        "constraints": {
                            "required": true,
                            "regex": "^((?=.*[0-9])(?=.*[a-z])(?=.*[A-Z])|(?=.*[0-9])(?=.*[a-z])(?=.*[!@#$%^&*])|(?=.*[0-9])(?=.*[A-Z])(?=.*[!@#$%^&*])|(?=.*[a-z])(?=.*[A-Z])(?=.*[!@#$%^&*])).{12,72}$",
                            "validationMessage": "Password must be at least 12 characters long and have 3 out of the following: one number, one lower case, one upper case, or one special character"
                        },
                        "options": {
                            "hideConfirmation": false
                        },
                        "visible": true
                    },
                    {
                        "name": "otnAccountUsername",
                        "type": "Microsoft.Common.TextBox",
                        "label": "OTN Account Username",
                        "toolTip": "The username of a pre-existing OTN account.",
                        "constraints": {
                            "required": true,
                            "regex": "^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}$",
                            "validationMessage": "The value must be a valid email address"
                        }
                    },
                    {
                        "name": "otnAccountPassword",
                        "type": "Microsoft.Common.PasswordBox",
                        "label": {
                            "password": "Password for OTN Account",
                            "confirmPassword": "Confirm password"
                        },
                        "toolTip": "Password for OTN Account",
                        "constraints": {
                            "required": true,
                            "validationMessage": "Validation constraints for OTN accounts apply here."
                        },
                        "options": {
                            "hideConfirmation": false
                        },
                        "visible": true
                    },
                    {
                        "name": "wlsUserName",
                        "type": "Microsoft.Common.TextBox",
                        "label": "Username for WebLogic Administrator",
                        "defaultValue": "weblogic",
                        "toolTip": "Use only letters and numbers",
                        "constraints": {
                            "required": true,
                            "regex": "^[a-z0-9A-Z]{1,30}$",
                            "validationMessage": "The value must be 1-30 characters long and must only contain letters and numbers."
                        },
                        "visible": true
                    },
                    {
                        "name": "wlsPassword",
                        "type": "Microsoft.Common.PasswordBox",
                        "label": {
                            "password": "Password for WebLogic Administrator",
                            "confirmPassword": "Confirm password"
                        },
                        "toolTip": "Password for WebLogic Administrator",
                        "constraints": {
                            "required": true,
                            "regex": "^(?=.*[A-Z])(?=.*[a-z])(?=.*\\d)[A-Za-z\\d]{12,}$",
                            "validationMessage": "The password must contain at least 12 characters, with at least 1 uppercase letter, 1 lowercase letter and 1 number."
                        },
                        "options": {
                            "hideConfirmation": false
                        },
                        "visible": true
                    },
                    {
                        "name": "numberOfInstances",
                        "type": "Microsoft.Common.TextBox",
                        "label": "Number of VMs",
                        "defaultValue": "4",
                        "toolTip": "The number of VMs to create, with one WebLogic node per VM",
                        "constraints": {
                            "required": true,
                            "regex": "^(2|3|4|5)$",
                            "validationMessage": "Number of VMs to deploy, limit 5 since this offer is using a single storage account."
                        },
                        "visible": true
                    }
                ],
                "visible": true
            },
            {
                "name": "basicsOptional",
                "type": "Microsoft.Common.Section",
                "label": "Optional Basic Configuration",
                "elements": [
                    {
                        "name": "basicsOptionalAcceptDefaults",
                        "type": "Microsoft.Common.OptionsGroup",
                        "label": "Accept defaults for optional configuration?",
                        "defaultValue": "Yes",
                        "toolTip": "Select 'No' to edit optional basic configuration.",
                        "constraints": {
                            "allowedValues": [
                                {
                                    "label": "Yes",
                                    "value": "false"
                                },
                                {
                                    "label": "No",
                                    "value": "true"
                                }
                            ],
                            "required": true
                        }
                    },
                    {
                        "name": "dnsLabelPrefix",
                        "type": "Microsoft.Common.TextBox",
                        "label": "DNS Label Prefix",
                        "toolTip": "The string to prepend to the DNS label.",
                        "defaultValue": "wls",
                        "constraints": {
                            "required": true,
                            "regex": "^[a-z0-9A-Z]{3,10}$",
                            "validationMessage": "The prefix must be between 3 and 10 characters long and contain letters, numbers only."
                        },
                        "visible": "[bool(basics('basicsOptional').basicsOptionalAcceptDefaults)]"
                    },
                    {
                        "name": "managedServerPrefix",
                        "type": "Microsoft.Common.TextBox",
                        "label": "Managed Server Prefix",
                        "toolTip": "The string to prepend to the name of the managed server.",
                        "defaultValue": "msp",
                        "constraints": {
                            "required": true,
                            "regex": "^[a-z0-9A-Z]{3,20}$",
                            "validationMessage": "The prefix must be between 3 and 20 characters long and contain letters, numbers only."
                        },
                        "visible": "[bool(basics('basicsOptional').basicsOptionalAcceptDefaults)]"
                    },
                    {
                        "name": "wlsDomainName",
                        "type": "Microsoft.Common.TextBox",
                        "label": "WebLogic Domain Name",
                        "toolTip": "The name of the WebLogic Domain to create.",
                        "defaultValue": "clusterDomain",
                        "constraints": {
                            "required": true,
                            "regex": "^[a-z0-9A-Z]{3,20}$",
                            "validationMessage": "The Domain Name must be between 3 and 20 characters long and contain letters, numbers only."
                        },
                        "visible": "[bool(basics('basicsOptional').basicsOptionalAcceptDefaults)]"
                    },
                    {
                        "name": "vmSizeSelect",
                        "type": "Microsoft.Compute.SizeSelector",
                        "defaultValue": "Standard_A3",
                        "label": "Virtual machine size",
                        "toolTip": "The size of virtual machine to provision.",
                        "recommendedSizes": [
                            "Standard_A1",
                            "Standard_A2",
                            "Standard_A3",
                            "Standard_A4"
                        ],
                        "osPlatform": "Linux",
                        "count": "1",
                        "visible": "[bool(basics('basicsOptional').basicsOptionalAcceptDefaults)]"
                    },
                    {
                        "name": "portsToExpose",
                        "label": "Ports and port ranges to expose (N or N-N, comma separated)",
                        "type": "Microsoft.Common.TextBox",
                        "toolTip": "Ports and port ranges to expose (N or N-N, comma separated)",
                        "defaultValue": "80,443,7001-9000",
                        "constraints": {
                            "required": true,
                            "regex": "^((([0-9]+-[0-9]+)|([0-9]+))[,]?)+[^,]$",
                            "validationMessage": "Only numbers, hyphen separated ranges of numbers, separated by commas"
                        },
                        "visible": "[bool(basics('basicsOptional').basicsOptionalAcceptDefaults)]"
                    },
                    {
                        "name": "About",
                        "type": "Microsoft.Common.InfoBox",
                        "visible": true,
                        "options": {
                            "icon": "None",
                            "text": "Template version ${project.version}"
                        },
                        "visible": "[bool(basics('basicsOptional').basicsOptionalAcceptDefaults)]"
                    }
                ],
                "visible": true
            }
        ],
        "steps": [
            {
                "name": "serviceIntegrations",
                "type": "Microsoft.Common.Section",
                "label": "Service Integrations",
                "elements": [
                    {
                        "name": "connectToAGText",
                        "type": "Microsoft.Common.TextBlock",
                        "visible": true,
                        "options": {
                            "text": "Selecting 'Yes' here will cause the template to provision an Azure Application Gateway (WAF_v2 or later SKU), a public IP, and a backend pool consisting of the worker nodes in the cluster.  Further configuration may be necessary after deployment.",
                            "link": {
                                "label": "Learn more",
                                "uri": "https://docs.microsoft.com/en-us/azure/application-gateway/overview"
                            }
                        }
                    },
                    {
                        "name": "enableAppGateway",
                        "type": "Microsoft.Common.OptionsGroup",
                        "label": "Connect to Azure Application Gateway?",
                        "defaultValue": "No",
                        "toolTip": "Select 'Yes' to cause an Azure Application Gateway to be created as the load balancer for the cluster.",
                        "constraints": {
                            "allowedValues": [
                                {
                                    "label": "Yes",
                                    "value": true
                                },
                                {
                                    "label": "No",
                                    "value": false
                                }
                            ],
                            "required": false
                        }
                    },
                    {
                        "name": "keyVaultText",
                        "type": "Microsoft.Common.TextBlock",
                        "visible": "[steps('serviceIntegrations').enableAppGateway]",
                        "options": {
                            "text": "App Gateway integration requires an SSL Certificate to enable SSL termination at the gateway.  End-to-end SSL encryption is not supported by the template.  The template will look for the certificate and its password in the Azure KeyVault specified here.",
                            "link": {
                                "label": "Learn more",
                                "uri": "https://github.com/andrewatfornax/tech-articles/blob/6985d626e9815daf63f705dbe9bd570e38cc0297/azure-app-gateway-ssl-arm-template.md"
                            }
                        }
                    },
                    {
                        "name": "keyVaultName",
                        "type": "Microsoft.Common.TextBox",
                        "label": "Name of the Azure KeyVault containing secrets for the Certificate for SSL Termination",
                        "defaultValue": "",
                        "toolTip": "Use only letters and numbers",
                        "constraints": {
                            "required": true,
                            "regex": "^[a-z0-9A-Z]{1,30}$",
                            "validationMessage": "The value must be 1-30 characters long and must only contain letters and numbers."
                        },
                        "visible": "[steps('serviceIntegrations').enableAppGateway]"
                    },
                    {
                        "name": "keyVaultResourceGroup",
                        "type": "Microsoft.Common.TextBox",
                        "label": "Resource group name in current subscription containing the KeyVault",
                        "defaultValue": "",
                        "toolTip": "Use only letters and numbers",
                        "constraints": {
                            "required": true,
                            "regex": "^[a-z0-9A-Z]{1,30}$",
                            "validationMessage": "The value must be 1-30 characters long and must only contain letters and numbers."
                        },
                        "visible": "[steps('serviceIntegrations').enableAppGateway]"
                    },
                    {
                        "name": "keyVaultSSLCertDataSecretName",
                        "type": "Microsoft.Common.TextBox",
                        "label": "The name of the secret in the specified KeyVault whose value is the SSL Certificate Data",
                        "defaultValue": "",
                        "toolTip": "Use only letters and numbers",
                        "constraints": {
                            "required": true,
                            "regex": "^[a-z0-9A-Z]{1,30}$",
                            "validationMessage": "The value must be 1-30 characters long and must only contain letters and numbers."
                        },
                        "visible": "[steps('serviceIntegrations').enableAppGateway]"
                    },
                    {
                        "name": "keyVaultSSLCertPasswordSecretName",
                        "type": "Microsoft.Common.TextBox",
                        "label": "The name of the secret in the specified KeyVault whose value is the password for the corresponding SSL Certificate",
                        "defaultValue": "",
                        "toolTip": "Use only letters and numbers",
                        "constraints": {
                            "required": true,
                            "regex": "^[a-z0-9A-Z]{1,30}$",
                            "validationMessage": "The value must be 1-30 characters long and must only contain letters and numbers."
                        },
                        "visible": "[steps('serviceIntegrations').enableAppGateway]"
                    }
                    
                ],
                "visible": true
            }
        ],
        "outputs": {
            "Location": "[location()]",
            "acceptOTNLicenseAgreement": "[basics('acceptOTNLicenseAgreement')]",
            "adminPasswordOrKey": "[basics('basicsRequired').adminPasswordOrKey]",
            "adminUsername": "[basics('basicsRequired').adminUsername]",
            "dnsLabelPrefix": "[basics('basicsOptional').dnsLabelPrefix]",
            "enableAppGateway": "[steps('serviceIntegrations').enableAppGateway]",
            "keyVaultName": "[steps('serviceIntegrations').keyVaultName]",
            "keyVaultResourceGroup": "[steps('serviceIntegrations').keyVaultResourceGroup]",
            "keyVaultSSLCertDataSecretName": "[steps('serviceIntegrations').keyVaultSSLCertDataSecretName]",
            "keyVaultSSLCertPasswordSecretName": "[steps('serviceIntegrations').keyVaultSSLCertDataSecretName]",                        
            "managedServerPrefix": "[basics('basicsOptional').managedServerPrefix]",
            "numberOfInstances": "[int(basics('basicsRequired').numberOfInstances)]",
            "otnAccountPassword": "[basics('basicsRequired').otnAccountPassword]",
            "otnAccountUsername": "[basics('basicsRequired').otnAccountUsername]",
            "portsToExpose": "[basics('basicsOptional').portsToExpose]",
            "vmSizeSelect": "[basics('basicsOptional').vmSizeSelect]",
            "wlsDomainName": "[basics('basicsOptional').wlsDomainName]",
            "wlsPassword": "[basics('basicsRequired').wlsPassword]",
            "wlsUserName": "[basics('basicsRequired').wlsUserName]"
        }
    }
}

ARM-TTK: Tests are run against all JSON files in a folder when a full file path is provided

Issue Details

When running the ARM TTK and supplying a full path to a json file for the TemplatePath parameter I would expect tests just to run against that specific file. However, if there are other JSON files in that folder, then tests will be run against them anyway.

When you specify a folder rather than a full path, the TTK will only be run against "maintemplate.json" or "azuredeploy.json" but it seems in this scenario where you supply a full path it will run against any JSON file.

I believe this is coming from the "folderfiles" property, but I am not clear why this is used to get all the files.

Repro steps (if necessary, delete otherwise)

  1. Create a folder with multiple ARM templates in
  2. Run Test-AzTemplate specifying the full path to one of these files for the TemplatePath parameter
  3. See tests run against all files

Textbox testing fails with regex containing UI functions

Hello experts,

I create a Textbox and it's value depends on input of another Textbox.
I create dynamic regex to test its value, expression is dynamic generated according the input, as following:

[concat('^(([a-z0-9A-Z]{1,30}[,]){',string(sub(length(steps('DeletingNodesConfiguration').DeletingNodesInfo.managedServerNames),length(replace(steps('DeletingNodesConfiguration').DeletingNodesInfo.managedServerNames,',','')))),'}[a-z0-9A-Z]{1,30})$')]

I tested the Textbox from Create UI Definition Sandbox, the regex expression works. But it failed when testing with command mvn -Ptemplate-validation-tests clean install, error happens in file Textboxes-Are-Well-Formed.test.ps1, details is:

 Textbox managedComputerNames regex is invalid: Exception calling ".ctor" with "1" argument(s): "parsing '[concat('^(([a-z0-9A-Z]{1,30}[,]){',string(sub(length(steps('DeletingNodesConfiguration').DeletingNodesInfo.managedServerNames),length(replace(steps('DeletingNodesConfiguration').DeletingNodesInfo.managedServerNames,',','')))),'}[a-z0-9A-Z]{1,30})$')]' - Too many )'s."

Root cause is that it fail to parse the expression of UI functions.
Can I pass a value for testing in this cases?

Attaching the UI definition.

{
    "$schema": "https://schema.management.azure.com/schemas/0.1.2-preview/CreateUIDefinition.MultiVm.json#",
    "handler": "Microsoft.Azure.CreateUIDef",
    "version": "0.1.2-preview",
    "parameters": {
        "basics": [ 
			{
                        "name": "About",
                        "type": "Microsoft.Common.InfoBox",
                        "options": {
                            "icon": "None",
                            "text": "Template version 1.0.0"
                        }
                    }
		],
        "steps": [
            {
                "name": "DeletingNodesConfiguration",
                "label": "Details of the deleting nodes",
                "subLabel": {
                    "preValidation": "Configure details of existing WebLogic Cluster and information of Managed Servers to be deleted. ",
                    "postValidation": "Done"
                },
                "bladeTitle": "WeLogic Cluster Settings",
                "elements": [
					{
						"name": "ExistingWLSClusterInfo",
						"type": "Microsoft.Common.Section",
						"label": "Details of the existing WebLogic Cluster",
						"elements": [
							{
								"name": "wlsAdminVMName",
								"type": "Microsoft.Common.TextBox",
								"label": "Computer Name of admin",
                			 	"toolTip": "Computer name of admin server, the value is used to specify the vitural machine of admin server to run script to delete nodes from WebLogic Cluster.",
								"defaultValue": "adminVM",
								"constraints": {
									"required": true,
									"regex": "^[a-z0-9A-Z]{3,20}$",
									"validationMessage": "The prefix must be between 3 and 20 characters long and contain letters, numbers only."
								}
							},
							{
								"name": "wlsAdminPort",
								"type": "Microsoft.Common.TextBox",
								"label": "Port number on the admin host for the admin server",
								"defaultValue": "7001",
								"toolTip": "The port number the admin server of existing cluster.  This value will be used in t3:// urls.",
								"constraints": {
									"required": true,
									"regex": "^([0-9]+)$",
									"validationMessage": "The value must be a valid port number."
								}
							},
							{
								"name": "wlsUserName",
								"type": "Microsoft.Common.TextBox",
								"label": "Username for WebLogic Administrator",
								"defaultValue": "weblogic",
								"toolTip": "Use only letters and numbers",
								"constraints": {
									"required": true,
									"regex": "^[a-z0-9A-Z]{1,30}$",
									"validationMessage": "The value must be 1-30 characters long and must only contain letters and numbers."
								}
							},
							{
								"name": "wlsPassword",
								"type": "Microsoft.Common.PasswordBox",
								"label": {
									"password": "Password for WebLogic Administrator",
									"confirmPassword": "Confirm password"
								},
								"toolTip": "Password for WebLogic Administrator",
								"constraints": {
									"required": true,
									"regex": "^(?=.*[A-Z])(?=.*[a-z])(?=.*\\d)[A-Za-z\\d]{12,}$",
									"validationMessage": "The password must contain at least 12 characters, with at least 1 uppercase letter, 1 lowercase letter and 1 number."
								},
								"options": {
									"hideConfirmation": false
								}
							}
						],
						"visible": true
					},
					{
						"name": "DeletingNodesInfo",
						"type": "Microsoft.Common.Section",
						"label": "Details of the deleting nodes",
						"elements": [
							{
								"name": "managedServerNames",
								"type": "Microsoft.Common.TextBox",
								"label": "Managed Server names to be deleted",
								"toolTip": "Server names of managed servers to be deleted, comma delimited.",
								"constraints": {
									"required": true,
									"regex": "^([a-z0-9A-Z]{1,30}[,]){0,29}[a-z0-9A-Z]{1,30}$",
									"validationMessage": "Each server name must be 1-30 characters long and must only contain letters and numbers, comma delimited. You can delete up to 30 nodes at a time"
								}
							},	
							{
								"name": "managedComputerNames",
								"type": "Microsoft.Common.TextBox",
								"label": "Managed Server computer names to be deleted",
								"toolTip": "Computer names of Vitual Machine that hosts managed server to be deleted, comma delimited.",
								"constraints": {
									"required": true,
									"regex": "[concat('^(([a-z0-9A-Z]{1,30}[,]){',string(sub(length(steps('DeletingNodesConfiguration').DeletingNodesInfo.managedServerNames),length(replace(steps('DeletingNodesConfiguration').DeletingNodesInfo.managedServerNames,',','')))),'}[a-z0-9A-Z]{1,30})$')]",
									"validationMessage": "Each name must be 1-30 characters long and must only contain letters and numbers, comma delimited, number of computer names should be the same with the number of server names."
								}
							},
							{
								"name": "wlsForceShutDown",
								"type": "Microsoft.Common.OptionsGroup",
								"label": "Force to shut down Managed Servers?",
								"defaultValue": "false",
								"toolTip": "Will force to shut down managed server if the tag set to true.",
								"constraints": {
									"allowedValues": [
										{
											"label": "ture",
											"value": "true"
										},
										{
											"label": "false",
											"value": "false"
										}
									],
									"required": true
							}
							}

						],
						"visible": true
					}
				]
            }
        ],
        "outputs": {
            "adminVMName":"[steps('DeletingNodesConfiguration').ExistingWLSClusterInfo.wlsAdminVMName]",
            "wlsUserName":"[steps('DeletingNodesConfiguration').ExistingWLSClusterInfo.wlsUserName]",
            "wlsPassword":"[steps('DeletingNodesConfiguration').ExistingWLSClusterInfo.wlsPassword]",
            "wlsAdminPort":"[steps('DeletingNodesConfiguration').ExistingWLSClusterInfo.wlsAdminPort]",
            "deletingManagedServerNames":"[steps('DeletingNodesConfiguration').DeletingNodesInfo.managedServerNames]",
            "deletingManagedServerMachineNames":"[steps('DeletingNodesConfiguration').DeletingNodesInfo.managedComputerNames]",
			"wlsForceShutDown":"[steps('DeletingNodesConfiguration').DeletingNodesInfo.wlsForceShutDown]",
			"Location": "[location()]"
        }
    }
}

`

Error on large template files

Hi,

I have found an issue with Import-Json that errors out in case of large template files (tested with >300KB). (In case you are wondering, i am trying to run the tests on exported templates, not writing large templates :) )

Splitting up

[IO.File]::ReadAllText("$resolvedPath") | 
            ConvertFrom-Json -ErrorAction SilentlyContinue -ErrorVariable ConvertProblems

Into

$content=[IO.File]::ReadAllText("$resolvedPath");
        ConvertFrom-Json -ErrorAction SilentlyContinue -ErrorVariable ConvertProblems -InputObject $content

seems to fix it, although I am not sure what the limit of this new approach is. Probably some powershell limitations..

adminUsername-Should-Not-Be-A-Literal does not catch hardcoded variables

Taking a note while reviewing a PR... if the adminUsername property uses a variable and the variable is not an expression, this test won't find it... it looks like we are excluding top-level properties (technically second level properties) so we aren't finding vars with Find-JSONContent.

Cannot bind argument to parameter 'Name' because it is an empty string.

Since the changes on 27 May I have been getting:
[-] Variables Must Be Referenced (67 ms) Cannot bind argument to parameter 'Name' because it is an empty string. Unreferenced variable: Cannot bind argument to parameter 'Name' because it is an empty string. Unreferenced variable: Cannot bind argument to parameter 'Name' because it is an empty string. Unreferenced variable: Cannot bind argument to parameter 'Name' because it is an empty string. Unreferenced variable: Cannot bind argument to parameter 'Name' because it is an empty string. Unreferenced variable: Cannot bind argument to parameter 'Name' because it is an empty string. Unreferenced variable: Cannot bind argument to parameter 'Name' because it is an empty string. Unreferenced variable: Cannot bind argument to parameter 'Name' because it is an empty string. Unreferenced variable:

If I pin to commit 9dec6bc4d16d1357a7dab5e64c51185ba1571e46 the issue goes away.

Not sure what additional info might be of use, let me know if need more details.

Find-JSONContent should return a list of matching properties (not csv)

Take the following:

        {
            "resourceId": "[concat()]",
            "id": "[concat('Microsoft.Storage/storageAccounts', 'name')]", //TODO this throws off the error message since we pull the parent's name (in this case 'variables')
            "keyVaultSecretId": "[concat()]" // this is actually a uri Microsoft.Network/applicationGateways sslCertificates - created with reference() and concat /secrets/secretname
        },
        {
            "someId": "[concat()]"
        },
        {
            "keyVaultSecretId": "[concat()]" // this is actually a uri Microsoft.Network/applicationGateways sslCertificates - created with reference() and concat /secrets/secretname
        }

When searching for -like *id and the properties are siblings we return a comma separated list, e.g.

resourceId,id,keyVaultSecretId

Instead of 3 different properties... The offending code seems to be the "join ,"

https://github.com/Azure/arm-ttk/blob/master/arm-ttk/Find-JsonContent.ps1#L126

is Deployment-Resources-Must-Not-Be-Debug correct?

It seems like the new test Deployment-Resources-Must-Not-Be-Debug is not correct. The fail unit test shows debugSetting like:

{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"resources": [
{
"debugSetting": "requestContent",
"type": "Microsoft.Resources/deployments"
}
]
}

But, debugSetting is in the properties and it has a detailLevel property. I would think the following template should fail but it passes:

{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"storageAccountName": {
"type": "string"
}
},
"resources": [
{
"type": "Microsoft.Resources/deployments",
"apiVersion": "2019-10-01",
"name": "nestedTemplate1",
"properties": {
"mode": "Incremental",
"debugSetting": {
"detailLevel": "requestContent"
},
"template": {
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"resources": [
{
"type": "Microsoft.Storage/storageAccounts",
"apiVersion": "2019-04-01",
"name": "[parameters('storageAccountName')]",
"location": "West US",
"sku": {
"name": "Standard_LRS"
},
"kind": "StorageV2"
}
]
}
}
}
],
"outputs": {
}
}

Empty properties are not allowed, or are they? ยฏ\_(ใƒ„)_/

test/arm-ttk/testcases/

Issue Details

My offer passed all tests, as shown here:

                                                                                                Validating arm\createUiDefinition.json
  CreateUIDefinition
    [+] Allowed Values Should Actually Be Allowed (154 ms)

    [+] CreateUIDefinition Must Have Parameters (9 ms)
    [+] CreateUIDefinition Should Have Schema (26 ms)
    [+] Credential Confirmation Should Not Be Hidden (78 ms)
    [+] Handler Must Be Correct (5 ms)
    [+] Location Should Be In Outputs (7 ms)
    [+] Outputs Must Be Present In Template Parameters (7 ms)
    [+] Parameters Without Default Must Exist In CreateUIDefinition (20 ms)
    [+] Password Textboxes Must Be Used For Password Parameters (39 ms)
    [+] Textboxes Are Well Formed (35 ms)
    [+] Tooltips Should Be Present (35 ms)
    [+] Usernames Should Not Have A Default (34 ms)
    [+] VMSizes Must Match Template (39 ms)
Validating arm\mainTemplate.json
  deploymentTemplate
    [+] DeploymentTemplate Schema Is Correct (6 ms)
    [+] IDs Should Be Derived From ResourceIDs (114 ms)
    [+] Location Should Not Be Hardcoded (43 ms)
    [+] ManagedIdentityExtension must not be used (6 ms)
    [+] Min And Max Value Are Numbers (6 ms)
    [+] Outputs Must Not Contain Secrets (21 ms)
    [+] Parameters Must Be Referenced (15 ms)
    [+] Parameters Property Must Exist (6 ms)
    [+] PublicIP DNS Should Not Contain Concat (6 ms)
    [+] ResourceIds should not contain (26 ms)
    [+] Resources Should Have Location (8 ms)
    [+] Secure String Parameters Cannot Have Default (3 ms)
    [+] Template Should Not Contain Blanks (13 ms)
    [+] VM Images Should Use Latest Version (3 ms)
    [+] VM Size Should Be A Parameter (57 ms)
    [+] Variables Must Be Referenced (12 ms)
    [+] Virtual Machines Should Not Be Preview (7 ms)
    [+] adminUsername Should Not Be A Literal (58 ms)
    [+] apiVersions Should Be Recent (96 ms)
    [?] artifacts parameter (25 ms) 
        ENV:SAMPLE_NAME is empty - using placeholder for manual verification: 100-blank-template

    [+] providers apiVersions Is Not Permitted (5 ms)

However, when it came time to go through the marketplace to live offer, the human reviewer @Olgaian stated:

please remove empty properties

I assert that if a human reviewer is going to flag something, then the tests should flag it also.

Repro steps (if necessary, delete otherwise)

  1. Create mainTemplate.json that has an empty property:
      {
         "type": "Microsoft.Storage/storageAccounts",
         "apiVersion": "2019-06-01",
         "name": "[variables('storageAccountName')]",
         "location": "[parameters('location')]",
         "sku": {
            "name": "[variables('storageAccountType')]"
         },
         "kind": "Storage",
         "properties": {}
      },
  1. run the tests
  2. observe they do not fail.

Note that this test seems to address this issue:

https://github.com/Azure/azure-quickstart-templates/blob/master/test/arm-ttk/testcases/deploymentTemplate/Template-Should-Not-Contain-Blanks.test.ps1

Line 18 - 32 on this test seem to address this.

VM Sizes Must Match template test not working

I'm using the latest commit (9dec6bc4d16d1357a7dab5e64c51185ba1571e46 by the time of writing) and the test VM Sizes Must Match template test is not working.

Steps to reproduce:

  • Clone the repository
  • Go to unit-tests/VMSizes-Must-Match-Template/Pass/Element-Nested-In-Section
  • Execute Test-AzTemplate.sh

Error shown:

    [-] VMSizes Must Match Template (49 ms)                 
                                                                          Exception calling "Create" with "1" argument(s): "At line:1 char:28                                                   
+ $CreateUIDefinitionObject.[0].parameters[0].steps[0]                                                               
         +                            ~                                                                                  
              Missing type name after '['.                                                                                                                                                                                                                                At line:1 char:27                                                                                                        
     + $CreateUIDefinitionObject.[0].parameters[0].steps[0]
+                           ~
Missing property name after reference operator."

For some reason, the JSONPath has this [0] at the beginning, which is causing issues.

Details about my system:

  • Mac OS Catalina
  • PowerShell 7.0.1
  • Installed coreutils and powershell using brew

apiVersion test is confused when preview and non-preview version are the same

Microsoft.DBforMySQL/servers has 2 apiVersions listed in the cache...

2017-12-01-preview
2017-12-01

And when the non-preview version is used we throw an error.

  {
      "apiVersion": "2017-12-01",
      "type": "Microsoft.DBforMySQL/servers",
      "name": "[variables('databaseName')]",
      "location": "[parameters('location')]",
      "properties": {
        "administratorLogin": "[variables('databaseLoginName')]",
        "administratorLoginPassword": "[variables('databaseLoginPassword')]",
        "createMode": "Default",
        "sslEnforcement": "Disabled"
      },
  },

Using the non-preview version here is correct but the ttk throws an error

[Question] Could arm-ttk testsuite use Pester?

ARM-TTK

Issue Details

The ARM-TTK uses a custom testing suite. But PowerShell community standardizes on using Pester as its default testing framework.

Is this a possibility for the ARM-TTK?

IMO this would open up a lot of contributions to the test-suite from the community.

test cases in "test" folder has unexplained dependancies

I am following instructions in this video.

I did a new install of powershell on my macos. This is for the simplest ARM template for a single instance based on "100-marketplace-sample". When I try to run the test case in the "test" folder in powershell I get this error:

Test-AzureRMTemplate -TemplatePath ../../my-offer/azuredeploy.jsonsdf
Import-Json : The term 'Import-Json' is not recognized as the name of a cmdlet, function, script file, or operable program.
Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
At /Users/pratap/cloud-images/azure/dai-azure-managed-app/test/template-tests/Test-AzureRMTemplate.ps1:145 char:76

  • ... script:AlreadyLoadedCache[$cacheFile.Name] = $cacheFile | Import-Json
  •                                                           ~~~~~~~~~~~
    
  • CategoryInfo : ObjectNotFound: (Import-Json:String) [], CommandNotFoundException
  • FullyQualifiedErrorId : CommandNotFoundException

--------------------MESSAGE FROM ADMIN, DELETE BEFORE SUBMITTING----------------------

Sorry to hear you had a bad experience with one of the templates ๐Ÿ˜Ÿ But, in case you're just asking a question, we're happy to help. You can also check if the question might already have been asked here https://github.com/Azure/azure-quickstart-templates/issues?utf8=%E2%9C%93&q=is%3Aissue

We've created an outline of recommended sections to fill out that will help make this Pull Request awesome!

--------------------MESSAGE FROM ADMIN, DELETE BEFORE SUBMITTING----------------------

[Template Name goes here](Template link goes here)

Issue Details

Repro steps (if necessary, delete otherwise)

AdminUsername should allow for keyvault reference

Basically the title. If I define an adminUsername like so:

"adminUsername": {
                        "reference": {
                            "keyvault": {
                                "id": "[variables('keyvault').resourceId]"
                            },
                            "secretName": "[parameters('integration').adminUsernameSecretName]"
                        }
                    },

it should be valid (at least I think so).

I tried messing with the regex like this: \[[^\]]+\] to this: [\[|\{]+[^\]|\}]+ and that pass for the above example, but I'm not sure if that's the best way to account for a case like this..

Problem in VMSizes-Must-Match-Template test when using Microsoft.Common.Section

Hi.

When the VMSize selector is inside a Microsoft.Common.Section, the script does not correctly retrieve the name of the step.
Instead, when it is retrieving the parent object name it will get the name of the section:

Later this name is used as the step name (even if it is the section name) and therefore the validation fails, even if it is correct:

$lookingFor= if ($isInSteps) { "*steps(*$stepName*).$controlName*"} else {"*basics(*$($controlName)*"}

"Template Should Not Contain Blanks" tests for blanks but some Azure Resources necessitate a blank

I'm not sure if this is somehow actually just my fault, but when using a userAssignedIdentity, you have to declare it in the ARM template like so:

 "identity": {
            "type": "UserAssigned",
            "userAssignedIdentities": {
                "[resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', 
variables('integration').userAssignedIdentity.name)]": {}
            }
        },

That little {} at the end there is causing the "Template Should Not Contain Blanks" to fail. This seems odd.

Perhaps the test should check strictly for newline characters?

Textboxes-Are-Well-Formed.test.ps1: fails, but no helpful output

On the attached input, I observe that Textboxes-Are-Well-Formed.test.ps1 fails, and produces this output

[-] Textboxes Are Well Formed (107 ms)

Looking at the test, I see a bunch of Write-Error outputs but I sure
don't see them above. I instrumented the test like so:

param(
# The contents of CreateUIDefintion, converted from JSON.
[Parameter(Mandatory=$true,Position=0)]
[PSObject]
$CreateUIDefinitionObject
)

# First, find all textboxes within CreateUIDefinition.

$allTextBoxes = $CreateUiDefinitionObject | Find-JsonContent -Key type -value Microsoft.Common.TextBox

Write-Host "debug: edburns: allTextBoxes $($allTextBoxes)"

foreach ($textbox in $allTextBoxes) { # Then we walk over each textbox.
    Write-Host "debug: edburns: $($textbox.Name)"
    if (-not $textbox.constraints) { # If constraints was missing or blank,
        Write-Host "debug: edburns: missing constraints"
        Write-Error "Textbox $($textbox.Name) is missing constraints" -TargetObject $textbox # error
        continue # and continue (since additional failures would be noise).
    }    
    if (-not $textbox.constraints.regex) { # If the constraint didn't have a regex,
        Write-Host "debug: edburns: missing regex"
        Write-Error "Textbox $($textbox.Name) is missing constraints.regex" -TargetObject $textbox #error.
    } else {
        Write-Host "debug: edburns: has regex"
        try { # If it did,
            [Regex]::new($textbox.constraints.regex) # try to cast to a regex
        } catch {
            Write-Host "debug: edburns: exception caught"
            $err = $_ # if that fails, 
            Write-Error "Textbox $($textbox.Name) regex is invalid: $($err)" -TargetObject $textbox #error.
        }
        Write-Host "debug: edburns: no exception"
    }
    if (-not $textbox.constraints.validationMessage) { # If there's not a validation message
        Write-Host "debug: edburns: missing validationMessage. Textbox $($textbox.Name) is missing constraints.validationMessage"
        Write-Error "Textbox $($textbox.Name) is missing constraints.validationMessage" -TargetObject $textbox #error.
    } else {
        Write-Host "debug: edburns: has validationMessage"
    }
}

And when I re-run it, I see this output.

debug: edburns: missing validationMessage. Textbox dsConnectionURL is missing constraints.validationMessage

If I had seen this output at the outset, I would have saved a bunch of
time. I'm not sure why I don't see the output, but I don't think using
Write-Host is the correct choice. Can you please make it so the error
explanations are written out?

Thanks,

Ed

Add exception for policy definitions in "Template Should Not Contain Blanks"

In Azure policy definitions, you sometime have to check against empty strings that will fail this test automatically even if it's wanted. For now, I skip this test but I was wondering if there was a better way to handle this use case or add an exception in the test itself.

Example from a policy definition:

{
"value": "[[resourceGroup().tags[parameters('tagName')]]",
"notEquals": ""
}

Skip parameter ignored

Repro:

Test-AzTemplate -TemplatePath $TemplateFileOrFolder -Skip "IDs-Should-Be-Derived-From-ResourceIDs"

Result: test still performed (there are no any references to $skip in the code)

Reasons we need to use "empty" or null values.

On occasion we need to use empty values, as per the following example:

"empty": [
        ],
"subnets": "[union(variables('defaultSubnets'),if(parameters('deployVnetGateway'), variables('gatewaySubnet'),variables('empty')),if(parameters('deployBastion'), variables('bastionSubnet'),variables('empty')))]",

This failed arm-ttk validation. I have managed to workaround as follows, but the logic is a lot harder to follow!

"subnetLogicGateway": "[if(parameters('deployVnetGateway'),union(variables('defaultSubnets'),variables('gatewaySubnet')),variables('defaultSubnets'))]",
        "subnetLogicBastionGateway": "[if(parameters('deployBastion'),union(variables('subnetLogicGateway'),variables('bastionSubnet')),variables('subnetLogicGateway'))]",
        "subnets": "[variables('subnetLogicBastionGateway')]",

Leaf tests support parameters, but the top level mechanism doesn't allow values to be passed thru

Consider artifacts-parameter.test.ps1. This test has parameters:

[string]$SampleName = "$ENV:SAMPLE_NAME",
[string]$RawRepoPath = "https://raw.githubusercontent.com/Azure/azure-quickstart-templates/master/",

However, there is no way to pass values for these parameters through to the test. For example:

./Test-AzTemplate.sh -RawRepoPath https://raw.githubusercontent.com/edburns -SampleName /arm-oraclelinux-wls/master/arm-oraclelinux-wls/src/main/scripts -TemplatePath /mnt/c/Users/edburns/Documents/azure/workareas/20190801-jacob-thomas/arm-oraclelinux-wls/target/arm -Test all

gives

Test-AzTemplate : A parameter cannot be found that matches parameter name 'RawRepoPath'.
At line:1 char:156
+ ... ates/test/arm-ttk/arm-ttk.psd1'; Test-AzTemplate -RawRepoPath https:/ ...
+                                                      ~~~~~~~~~~~~
+ CategoryInfo          : InvalidArgument: (:) [Test-AzTemplate], ParameterBindingException
+ FullyQualifiedErrorId : NamedParameterNotFound,Test-AzTemplate

This is because the list of parameters in Test-AzTemplate.ps1 appears to be hard coded.

I tried adding the parameters I'm interested in:

    # When asserting the correct _artifactsLocation, this value is part of the definition of correctness
    [string]
    $RawRepoPath = "https://raw.githubusercontent.com/Azure/azure-quickstart-templates/master/",

    # When asserting the correct _artifactsLocation, this value is part of the definition of correctness
    [string]
    $SampleName = "$ENV:SAMPLE_NAME",

But I quickly realized it is very complex to hack these all the way through to the leaf.

do we need an -isQuickStart switch (staging and other practices)

I'm using ARM-TTK to validate templates outside of the quickstart repository. Firstly please confirm that this is a valid use case?

Linked templates are staged prior to deployment to a storage account.

This means the default values specified here:

are not appropriate.

The staging location needs to be specified at deploy time.

Of the two, a GitHub URL is not appropriate, and if I use [deployment().properties.templateLink.uri] the templates are not deployable for obvious reasons.

Is this a case of needing an isQuickstart switch or similar?

Thanks.

Don't allow root directories as a template path

@(Get-ChildItem -Path $templateFolder.FullName -Recurse |

As this uses the -Recurse parameter, storing the .json file in the root of your system will cause unexpected behavior. Windows will just look and eventually go banana's on memory. Linux will return something similar to (for the files it can't deal with):
MethodInvocationException: /arm-ttk/arm-ttk/Expand-AzTemplate.ps1:192
Line |
192 | $fileObject = [Ordered]@{
| ~~~~~~~~~~~~~~~~~~~~~~~~~
| Exception calling "ReadAllText" with "1" argument(s): "Could not find file '/usr/share/man/man1/sh.1.gz'."

Why this matters? If you're running this in a container it is very likely you're doing something similar to "COPY azuredeploy.json /". I'm not saying this is the best idea but for containers with a short lifespan this happens, specially in dev/test scenarios. Works fine when using a directory such as Test-AzTemplate -TemplatePath /test/

Test 'Textboxes Are Well Formed' produces weird output

One of the tests seems to mess up the test output.

I don't know what Options, MatchTimeout and RightToLeft and values under them are, but they should probably not be in the default output when the test passes.

Validating package-empty\createUiDefinition.json
  CreateUIDefinition
    [+] Allowed Values Should Actually Be Allowed (444 ms)

    [+] CreateUIDefinition Must Have Parameters (2 ms)
    [+] CreateUIDefinition Should Have Schema (17 ms)
    [+] Credential Confirmation Should Not Be Hidden (224 ms)
    [+] Handler Must Be Correct (3 ms)
    [+] Location Should Be In Outputs (8 ms)
    [+] Outputs Must Be Present In Template Parameters (39 ms)
    [+] Parameters Without Default Must Exist In CreateUIDefinition (13 ms)
    [+] Password Textboxes Must Be Used For Password Parameters (166 ms)
    [+] Textboxes Are Well Formed (113 ms)

Options RightToLeft MatchTimeout
------- ----------- ------------
   None       False -00:00:00.0010000



Options RightToLeft MatchTimeout
------- ----------- ------------
   None       False -00:00:00.0010000



Options RightToLeft MatchTimeout
------- ----------- ------------
   None       False -00:00:00.0010000



Options RightToLeft MatchTimeout
------- ----------- ------------
   None       False -00:00:00.0010000



    [+] Tooltips Should Be Present (134 ms)

It's reproducible by executing ./arm-ttk/Test-AzTemplate.sh ./unit-tests/Common/Pass/100-marketplace-sample. It seems to get worse based on the number of TextBoxes present in CUID.

Add check for correct use of list*() functions

Can we add a check for use of a list*() function - for example match the resourceId type and resource with /list*/actions available for the RP

We can cache the actions (as we do for apiVersions) since they don't change often.

Matching the different formats of the resourceId may be tricky - since a var or param may be used instead of the resourceId function. Solvable though.

allow old apiVersion if every newer version is preview

Hi,

I have a case where I use Application Insights (Microsoft.Insights/components), and the possible versions are:

  1. 2020-02-02-preview
  2. 2018-05-01-preview
  3. 2015-05-01
    See: https://docs.microsoft.com/en-us/azure/templates/microsoft.insights/allversions

However I am not interested in using preview versions, I prefere the non-preview.
When I run the test of my arm template then it tells me that I am using too old version of this resource:

[-] apiVersions Should Be Recent (47 ms)
        Api versions must be the latest or under 2 years old (730 days) - API version 2015-05-01 of Microsoft.Insights/components is 1823 days old
        Valid Api Versions:
        2018-05-01-preview

Example:

{
      "name": "[variables('applicationInsightsName')]",
      "type": "Microsoft.Insights/components",
      "apiVersion": "2015-05-01",
      "kind": "web",
      "location": "[parameters('location')]",
      "properties": {
        "Application_Type": "[variables('applicationInsightsApplicationType')]",
        "Flow_Type": "[variables('applicationInsightsFlowType')]",
        "Request_Source": "[variables('applicationInsightsRequestSource')]"
      }
    }

Is it possible to configure the tool to ignore preview when there is a non-preview version?

Is it possible to disallow preview versions completly?

Thanks
Mikkel

Arrays with only one element is converted into a string

When passing an TemplateObject to the Format-AzTemplate the following happens

Arrays with only one element is converted into a string when running Format-AzTemplate. This will create dependsOn with a string value, and not an Array with one string element.

I suspect that this is because the Property is being piped (and therefore converted to string by poweshell) to the sortProperties scriptBlock.

in.$propName | & $MyInvocation.MyCommand.ScriptBlock @recurseSplat

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.