Job naming issues with AML CLI training pipeline about mlops-v2

[cynthiajiangatl]

@lokijota , thank you for the advice. I deleted the pipeline and created a new one, it did take the updated code and ran the pipeline successfully. Could someone from Engineering take a look at the Github code checkout issue? Thanks.

[cynthiajiangatl]

I replaced all the hyphens with underscores, it didn't fix the error. Please let me know how to get the pipeline to run using AML CLI2

[lostmygithubaccount]

@cynthiajiangatl what is the error you are seeing? If you replaced the hyphens with underscores, you shouldn't be seeing the same error

[cynthiajiangatl]

I still see the same error, not sure why it was not using the updated code. When I tried to use a particular commit to run the devops pipeline, it ran into another issue, just not checking out the latest code for some reason.

[lokijota]

Hi @cynthiajiangatl . For some reason ADO didn't pick up the changes with me either, I had to recreate the pipeline in ADO, but then it worked. Just make sure you replace all the references in the yml file, in the job stage names and references.

[buswrecker]

I have the same error also. Are '-' dashes allowed? the MLOps-v2 uses dashes. https://github.com/Azure/mlops-v2/blob/main/data-science-regression/pipeline.yml. The official AML Examples - https://github.com/Azure/azureml-examples/blob/main/cli/jobs/pipelines/nyc-taxi/pipeline.yml YML uses underscores "_". What's the best practice?

[msdpalam]

I had the same issue; I renamed the job names under data-science-regression/pipeline.yml, and then made sure the other step jobs reflect the changes. This resolved my issue. Thought I would share:
jobs:
prep_job:
type: command
component: file:./components/prep/prep.yml
inputs:
raw_data: ${{parent.inputs.pipeline_job_input}}
outputs:
transformed_data: ${{parent.outputs.pipeline_job_transformed_data}}

train_job:
type: command
component: file:./components/train/train.yml
inputs:
training_data: ${{parent.jobs.prep_job.outputs.transformed_data}}
outputs:
model_output: ${{parent.outputs.pipeline_job_trained_model}}
....
....

[andreipurdila]

Regarding the ADO pipeline not picking up the latest changes, this is due to the presence of the Cache@2 task in the install-az-cli.yml template. Remove that step (or properly configure it) and the ADO pipeline will pick up the latest changes.

[cindyweng]

removed Cache@2 task in the cli-v2 template as Andrei specified, code changes elsewhere now will be reflected when you rerun the same pipeline