Unexpected parameter "build_type" when running deploy-model-training-pipeline in CV pattern about mlops-v2

[patcharees]

Hi. I am having this problem as well. Any workaround?

[setuc]

The template needs a fix

You will notice that the build_type was not setup or passed as a parameter to the register environment file that can be found here.
https://github.com/Azure/mlops-templates/blob/34dc9a07612cf920db5e1377a90f672b43ab8afd/.github/workflows/register-environment.yml

You can remove the build type or use a similar approach from classical template.

https://github.com/Azure/mlops-project-template/blob/acd9c8afbcd477cf85da819ace5e3ccd1dadd105/classical/aml-cli-v2/mlops/devops-pipelines/deploy-model-training-pipeline.yml#L51

I will need to retest the pipelines to fix the error. Give me some time to test it and address this issue.

[changwei0314]

Hi @Dan-RT, If you don't necessarily need to use Bicep and Azure DevOps, I would suggest referring to the guidance provided by this repository.

I have successfully built an MLOps engine using this guidance. However, please note that this project utilizes Terraform and GitHub Actions.

[patcharees]

Hi @setuc,
Thanks for info. I found another issue. It is missing /templates/aml-cli-v2/register-dataset.yml in mlops-templates repo. Any workaround?

[setuc]

@patcharees let me review the CV templates to see what is causing these issues?

There is register dataset but CV doesn’t use that as we download the data on the fly. Do you have the error message that you get

[patcharees]

Below is my template mlops-project-template\cv\aml-cli-v2\mlops\devops-pipelines\deploy-model-training-pipeline.yml. You can see it requires register-dataset.yml. When I run this pipeline to deploy model training, I got this error

/mlops/devops-pipelines/deploy-model-training-pipeline.yml: File /templates/aml-cli-v2/register-dataset.yml not found in repository https://dev.azure.com/xxx/xxx/_git/mlops-templates branch refs/heads/main version fe222c36eaa8c0949d99e97e6ba5f17361f1f108.

as register-dataset.yml does not really exist in aml-cli-v2 folder

---

name: deploy-model-training-pipeline

variables:

• ${{ if eq(variables['Build.SourceBranchName'], 'main') }}:
  • template: ../../config-infra-prod.yml
• ${{ if ne(variables['Build.SourceBranchName'], 'main') }}:
  • template: ../../config-infra-dev.yml
• name: version
  value: aml-cli-v2

trigger: none

pool:
vmImage: ubuntu-20.04

resources:
repositories:
- repository: mlops-templates # Template Repo
name: mlops-templates
type: git
ref: main

stages:

• stage: DeployTrainingPipeline
  displayName: Deploy Training Pipeline
  jobs:
  • job: DeployTrainingPipeline
    steps:
    • checkout: self
      path: s/
    • task: Bash@3
      displayName: "Create checkout repository folder(s)"
      inputs:
      targetType: "inline"
      script: |
      set -e
      mkdir "$(Build.Repository.Name)"
      mkdir "mlops-templates"
    • checkout: mlops-templates
      path: s/templates/
    • template: templates/${{ variables.version }}/install-az-cli.yml@mlops-templates
    • template: templates/${{ variables.version }}/install-aml-cli.yml@mlops-templates
    • template: templates/${{ variables.version }}/connect-to-workspace.yml@mlops-templates
    • template: templates/${{ variables.version }}/create-compute.yml@mlops-templates
      parameters:
      cluster_name: gpu-cluster
      size: Standard_NC6
      min_instances: 0
      max_instances: 1
      cluster_tier: dedicated
    • template: templates/${{ variables.version }}/register-environment.yml@mlops-templates
      parameters:
      build_type: docker
      environment_name: nvidia_pytorch # Not used for docker builds
      environment_file: mlops/azureml/train/train-env.yaml
    • template: templates/${{ variables.version }}/register-dataset.yml@mlops-templates
      parameters:
      data_type: training
      environment_file: mlops/azureml/train/create_stanford_dogs_dataset.yaml
    • template: templates/${{ variables.version }}/run-pipeline.yml@mlops-templates
      parameters:
      pipeline_file: mlops/azureml/train/pipeline.yaml
      experiment_name: $(environment)cv_train$(Build.SourceBranchName)
      display_name: $(environment)cv_run$(Build.BuildID)

[Dan-RT]

Hi guys, I am facing the same bug with the same settings, @setuc did you get the chance to look at this issue?
@changwei0314 @patcharees did one of you figure out a workaround? @changwei0314 I saw you have a similar project but using terraform and github

[setuc]

@Dan-RT Try removing build-type from the line 58 here

https://github.com/Azure/mlops-project-template/blob/acd9c8afbcd477cf85da819ace5e3ccd1dadd105/cv/aml-cli-v2/mlops/devops-pipelines/deploy-model-training-pipeline.yml#L58

I will issue a patch for it.

[setuc]

@changwei0314 @Dan-RT Is Bicep really a priority for you?