microsoftlearning / dp100

Labs for Course DP-100: Designing and Implementing Data Science Solutions on Microsoft Azure

DP-100: Designing and Implementing a Data Science Solution on Azure

Important Notice!

This repo was replaced by a new repo on December 12th 2020. The corresponding courseware was also updated at this time. We're making this change to:

• Consolidate the labs used in the DP-100 instructor-led course and the self-paced exercises in the equivalent online modules on Microsoft Learn.
• Update labs to reflect recent changes in the Azure Machine Learning service and SDK.
• Add new labs on topics related to responsible machine learning.

After a short interval, this repo will be archived.

This repo contains the lab files for Microsoft course DP-100T01-A: Designing and Implementing a Data Science Solution on Azure.

The lab instructions are here.

Contributions

At this time, we are not accepting external contributions to this repo. If you have suggestions or spot any errors, please report them as issues.

dp100's Issues

Lab 07B - UserError: User program failed with NameError: name 'ResourceNotFoundError' is not defined

I get an error saying "UserError: User program failed with NameError: name 'ResourceNotFoundError' is not defined" when trying to run the Batch Inferencing experiment. SDK version 1.9.0. I tried to re-create the cluster but it still failed.

Cannot find 'Create inference pipeline drop-down list' in Designer UI

In the Create inference pipeline drop-down list, click Real-time inference pipeline.

I see instructions like this (e.g. in Lab 2B) but I'm unable to find a 'Create inference pipeline drop-down list' in the current Designer UI - has the UX changed recently? I'm pretty new to it so seeing it for pretty much the first time...

Inference Cluster CPU minimum now 12

Lab 2B > Task 1 > Step 3

Instructions say 2 nodes, following error indicates that "The number of nodes multiplied by the virtual machine’s number of cores (vCPUs) must be greater than or equal to 12"

NOTE: Azure Pass Subscriptions limit Total CPU count to 10

Invalid Template Deployment for Standard_DS2_v2 in Lab 02B

Provisioning error: InvalidTemplateDeployment: Details: RequestId: c3cff3fd-133c-404d-9b98-8cac9ca81eb4 Error code: 'InvalidTemplateDeployment'. Target: ''. Message: 'The template deployment 'd81164b2-df14-4f24-970e-f2d17180eaa7' is not valid according to the validation procedure. The tracking id is 'c3cff3fd-133c-404d-9b98-8cac9ca81eb4'. See inner errors for details.' Error code: 'BadRequest'. Target: ''. Message: 'Provisioning of resource(s) for container service aks-clustereb14719489 in resource group Parichart-resource-group failed. Message: { "code": "BadRequest", "message": "The VM size of AgentPoolProfile:agentpool is not allowed in your subscription in location 'canadaeast'. The available VM sizes are Standard_M128,Standard_M128-32ms,Standard_M128-64ms,Standard_M128m,Standard_M128ms,Standard_M128s,Standard_M16-4ms,Standard_M16-8ms,Standard_M16ms,Standard_M32-16ms,Standard_M32-8ms,Standard_M32ls,Standard_M32ms,Standard_M32ts,Standard_M64,Standard_M64-16ms,Standard_M64-32ms,Standard_M64ls,Standard_M64m,Standard_M64ms,Standard_M64s,Standard_M8-2ms,Standard_M8-4ms,Standard_M8ms For more details, please visit https://aka.ms/cpu-quota" }. Details: '

Client Request ID : 30454edc-7aa6-4611-a722-016152db689a Service Request ID : |44ee677735154ead828df1dc698689d2.155bdabf05274172.4131b239_

I've got the above error.
What do I need to do to deploy aks-cluster using VM size: Standard_DS2_v2, please?

Lab 2B : Issue while deploying the model

Hi,

I am facing issues in deploying the model using designer (as per instructions from lab 2b) . I have done this lab several times, i have not faced any issue until now. I have also checked that inference cluster is created successfully. While deploying the model i get this error :

Deploy: Failed on step WaitServiceCreating. Details: AzureML service API error.
Your container application crashed. This may be caused by errors in your scoring file's init() function.
Please check the logs for your container instance: pipeline-created-on-07-28-2020-r.
From the AML SDK, you can run print(service.get_logs()) if you have service object to fetch the logs.
You can also try to run image viennaglobal.azurecr.io/azureml/azureml_6ae744633f749472feb283065055dc2c:latest locally.
Please refer to http://aka.ms/debugimage#service-launch-fails for more information.

Lab 5A Error: Building docker image failed with exit code 247

I've created all the necessary prerequisites for lab 5A but everytime I try to run the experiment in Lab 5A it fails with the following log:

Streaming log file azureml-logs/60_control_log.txt
Starting the daemon thread to refresh tokens in background for process with pid = 6489
Running: ['/bin/bash', '/tmp/azureml_runs/diabetes-training_1586236752_b6da4d68/azureml-environment-setup/docker_env_checker.sh']

Materialized image not found on target: azureml/azureml_df12a7544160ecd42f585376cb683f4c

Logging experiment preparation status in history service.
Running: ['/bin/bash', '/tmp/azureml_runs/diabetes-training_1586236752_b6da4d68/azureml-environment-setup/docker_env_builder.sh']
Running: ['docker', 'build', '-f', 'azureml-environment-setup/Dockerfile', '-t', 'azureml/azureml_df12a7544160ecd42f585376cb683f4c', '.']
Sending build context to Docker daemon 433.7kB
Step 1/15 : FROM mcr.microsoft.com/azureml/base:intelmpi2018.3-ubuntu16.04@sha256:a1b514f3ba884b9a7695cbba5638933ddaf222e8ce3e8c81e8cdf861679abb05
---> 93a72e6bd1ce
Step 2/15 : USER root
---> Using cache
---> 0d71445890ae
Step 3/15 : RUN mkdir -p $HOME/.cache
---> Using cache
---> fccad9720028
Step 4/15 : WORKDIR /
---> Using cache
---> 9abe918a1283
Step 5/15 : COPY azureml-environment-setup/99brokenproxy /etc/apt/apt.conf.d/
---> Using cache
---> 221d1e7cadfd
Step 6/15 : RUN if dpkg --compare-versions conda --version | grep -oE '[^ ]+$' lt 4.4.11; then conda install conda==4.4.11; fi
---> Using cache
---> 44a9199c4d25
Step 7/15 : COPY azureml-environment-setup/mutated_conda_dependencies.yml azureml-environment-setup/mutated_conda_dependencies.yml
---> Using cache
---> 2378786fc0ae
Step 8/15 : RUN ldconfig /usr/local/cuda/lib64/stubs && conda env create -p /azureml-envs/azureml_5d419b151e9c1ce888d07b7a6d7737fd -f azureml-environment-setup/mutated_conda_dependencies.yml && rm -rf "$HOME/.cache/pip" && conda clean -aqy && CONDA_ROOT_DIR=$(conda info --root) && rm -rf "$CONDA_ROOT_DIR/pkgs" && find "$CONDA_ROOT_DIR" -type d -name pycache -exec rm -rf {} + && ldconfig
---> Running in c770aabb1907
The command '/bin/sh -c ldconfig /usr/local/cuda/lib64/stubs && conda env create -p /azureml-envs/azureml_5d419b151e9c1ce888d07b7a6d7737fd -f azureml-environment-setup/mutated_conda_dependencies.yml && rm -rf "$HOME/.cache/pip" && conda clean -aqy && CONDA_ROOT_DIR=$(conda info --root) && rm -rf "$CONDA_ROOT_DIR/pkgs" && find "$CONDA_ROOT_DIR" -type d -name pycache -exec rm -rf {} + && ldconfig' returned a non-zero code: 247

Solving environment: ...working...

CalledProcessError(247, ['docker', 'build', '-f', 'azureml-environment-setup/Dockerfile', '-t', 'azureml/azureml_df12a7544160ecd42f585376cb683f4c', '.'])

Building docker image failed with exit code: 247

Logging error in history service: Failed to run ['/bin/bash', '/tmp/azureml_runs/diabetes-training_1586236752_b6da4d68/azureml-environment-setup/docker_env_builder.sh']
Exit code 1
Details can be found in azureml-logs/60_control_log.txt log file.

Uploading control log...

Lab 8A: BanditPolicy is not used

Unused policy BanditPolicy. Is imported in the 6th cell, but in HyperDriveConfig the parameter policy is set to None.

Error in lab 8B

When running the automl step will it fail with the following:

�[91mWARNING: pip is being invoked by an old script wrapper. This will fail in a future version of pip.
Please see https://github.com/pypa/pip/issues/5599 for advice on fixing the underlying issue.
To avoid this problem you can invoke Python with '-m pip' instead of running pip directly.
ERROR: Could not find a version that satisfies the requirement dotnetcore2>=2.1.9 (from azureml-dataprep[fuse,pandas]<1.2.0a,>=1.1.35a->azureml-train-automl==1.0.83.*->-r /azureml-environment-setup/condaenv.2ab_3h75.requirements.txt (line 1)) (from versions: none)
�[0m�[91mERROR: No matching distribution found for dotnetcore2>=2.1.9 (from azureml-dataprep[fuse,pandas]<1.2.0a,>=1.1.35a->azureml-train-automl==1.0.83.*->-r /azureml-environment-setup/condaenv.2ab_3h75.requirements.txt (line 1))
�[0m�[91m

CondaValueError: pip returned an error

�[0mThe command '/bin/sh -c ldconfig /usr/local/cuda/lib64/stubs && conda env create -p /azureml-envs/azureml_d5db24e95bda7d8ce518fbfff093cbd3 -f azureml-environment-setup/mutated_conda_dependencies.yml && rm -rf "$HOME/.cache/pip" && conda clean -aqy && CONDA_ROOT_DIR=$(conda info --root) && rm -rf "$CONDA_ROOT_DIR/pkgs" && find "$CONDA_ROOT_DIR" -type d -name __pycache__ -exec rm -rf {} + && ldconfig' returned a non-zero code: 1
2020/01/23 12:52:11 Container failed during run: acb_step_0. No retries remaining.
failed to run step ID: acb_step_0: exit status 1

Run ID: cgc failed after 5m34s. Error: failed during run, err: exit status 1

Issue In Lab 9A & 9B about AutoML

the error is cannot import name 'AutoMLConfig'
I tried to reinstall automl but it doesn't work
I used !pip install azureml-train-automl
If anyone has an answer....

Lab 05A : Running Estimator Causing File Save Internal Server Error

Upon executing the code for assigning the experiment to an Estimator, I am getting File Save Error as in the screenshot below

.

Estimators are deprecated

On Dec 7th 2020, Azure ML SDK 1.19.0 was released. In this release, estimators are deprecated; so any labs that use estimators will fail. We'll be releasing a refreshed version of the course next week, which will address this (and other) issues. In the meantime, you can apply either of the following workarounds:

• Pin the SDK version to 1.18.0 (so in steps to install the SDK, use pip install azureml-sdk==1.18.0. However, you might experience some compatibility issues in later labs)
• Use the updated labs (which will be the "official" labs for the refreshed course from next week onwards) at https://aka.ms/mslearn-dp100 - they more or less follow the same flow as the existing labs, but do not use estimators.

Issues with workspace authentication

Often (quite always) I got authentication errors during the connection to the ML workspace, using the config.json:

from azureml.core import Workspace
ws = Workspace.from_config()
print(ws.name, "loaded")

The only way I have found to resolve it is using the interactive authentication option:

from azureml.core import Workspace
from azureml.core.authentication import InteractiveLoginAuthentication

interactive_auth = InteractiveLoginAuthentication(tenant_id="my_tenant-id")

ws = Workspace(subscription_id="my_subscrition_id",
resource_group="my_ResGroup",
workspace_name="my_workspace_name",
auth=interactive_auth)

print(ws.name, "loaded")

Is there another way?

Error in lab 7B

EDIT: SOLVED IN LATEST HEAD

when running cell 12 does the following errors occur and therefore is the step not able to complete

ERROR: Could not find a version that satisfies the requirement dotnetcore2>=2.1.9 (from azureml-dataprep[fuse,pandas]~=1.1->-r /azureml-environment-setup/condaenv.iamp9drf.requirements.txt (line 7)) (from versions: none)
ERROR: No matching distribution found for dotnetcore2>=2.1.9 (from azureml-dataprep[fuse,pandas]~=1.1->-r /azureml-environment-setup/condaenv.iamp9drf.requirements.txt (line 7))

Error in Lab 8B

Getting the error at 6th block of the code of the Lab 8B in DP100 labs

"ModuleNotFoundError: No module named 'sklearn.impute._base'; 'sklearn.impute' is not a package"

Lab 08A - Tuning Hyperparameters

While Determine the Best Performing Run getting following error:

4 best_run = run.get_best_run_by_primary_metric()
----> 5 best_run_metrics = best_run.get_metrics()
6 parameter_values = best_run.get_details() ['runDefinition']['arguments']
7

AttributeError: 'NoneType' object has no attribute 'get_metrics'

Because of which I couldn't proceed further.

Lab 1B: Error when opening Visual Studio Online

Visual Studio Online is currently displaying the following error when opening an environment.

Failed to connect to the remote extension host server (Error: Connection error: Version mismatch, client refused.)

We're investigating. In the meantime, the Visual Studio Online exercise in Lab 1B may not work.

Error in Lab 3B

Hi

When running the Lab 3B (and also 3A) will I get the following error:

---------------------------------------------------------------------------
TrainingException                         Traceback (most recent call last)
<ipython-input-4-1819b83d8616> in <module>
     14 
     15 # Run the experiment based on the estimator
---> 16 run = experiment.submit(config=estimator)
     17 run.wait_for_completion(show_output=True)

/anaconda/envs/azureml_py36/lib/python3.6/site-packages/azureml/_jupyter_common/__init__.py in submit(self, config, tags, **kwargs)
     84 def _experiment_submit_notebook_decorator(original_submit):
     85     def submit(self, config, tags=None, **kwargs):
---> 86         run = original_submit(self, config, tags, **kwargs)
     87         _update_run_created_from(run)
     88         return run

/anaconda/envs/azureml_py36/lib/python3.6/site-packages/azureml/core/experiment.py in submit(self, config, tags, **kwargs)
    200         submit_func = get_experiment_submit(config)
    201         with self._log_context("submit config {}".format(config.__class__.__name__)):
--> 202             run = submit_func(config, self.workspace, self.name, **kwargs)
    203         if tags is not None:
    204             run.set_tags(tags)

/anaconda/envs/azureml_py36/lib/python3.6/site-packages/azureml/train/_estimator_helper.py in _estimator_submit_method(estimator, workspace, experiment_name, **kwargs)
     67             source_directory_data_store=source_directory_data_store)
     68 
---> 69     experiment_run = estimator._fit(workspace, experiment_name)
     70 
     71     if override_params:

/anaconda/envs/azureml_py36/lib/python3.6/site-packages/azureml/train/estimator/_mml_base_estimator.py in _fit(self, workspace, experiment_name)
    152         self._last_submitted_runconfig = self.run_config
    153 
--> 154         return self._submit(workspace, experiment_name, telemetry_values)
    155 
    156     def _override_params(self, script_params=None, inputs=None, source_directory_data_store=None):

/anaconda/envs/azureml_py36/lib/python3.6/site-packages/azureml/train/estimator/_mml_base_estimator.py in _submit(self, workspace, experiment_name, telemetry_values)
    146                 return experiment_run
    147             except AzureMLException as e:
--> 148                 raise TrainingException(e.message, inner_exception=e) from None
    149 
    150     def _fit(self, workspace, experiment_name):

TrainingException: TrainingException:
	Message: {
    "error_details": {
        "correlation": {
            "operation": "efd224e10e18cf42b5a0c11b81054b1a",
            "request": "648ce197a1ddb5d5"
        },
        "environment": "northeurope",
        "error": {
            "code": "UserError",
            "innerError": {
                "code": "ForbiddenError"
            },
            "message": "Exception of type 'Microsoft.MachineLearning.Common.WebApi.Exceptions.ForbiddenException' was thrown."
        },
        "location": "northeurope",
        "time": "2020-01-22T09:19:18.0091392+00:00"
    },
    "status_code": 403,
    "url": "https://northeurope.experiments.azureml.net/execution/v1.0/subscriptions/05a11c00-3baa-427a-9d5f-442b1c24f711/resourceGroups/dp100new/providers/Microsoft.MachineLearningServices/workspaces/dp100ws/experiments/diabetes-training/localrun?runId=diabetes-training_1579684757_15f503fc"
}
	InnerException ExperimentExecutionException:
	Message: {
    "error_details": {
        "correlation": {
            "operation": "efd224e10e18cf42b5a0c11b81054b1a",
            "request": "648ce197a1ddb5d5"
        },
        "environment": "northeurope",
        "error": {
            "code": "UserError",
            "innerError": {
                "code": "ForbiddenError"
            },
            "message": "Exception of type 'Microsoft.MachineLearning.Common.WebApi.Exceptions.ForbiddenException' was thrown."
        },
        "location": "northeurope",
        "time": "2020-01-22T09:19:18.0091392+00:00"
    },
    "status_code": 403,
    "url": "https://northeurope.experiments.azureml.net/execution/v1.0/subscriptions/05a11c00-3baa-427a-9d5f-442b1c24f711/resourceGroups/dp100new/providers/Microsoft.MachineLearningServices/workspaces/dp100ws/experiments/diabetes-training/localrun?runId=diabetes-training_1579684757_15f503fc"
}
	InnerException None
	ErrorResponse 
{
    "error": {
        "message": "{\n    \"error_details\": {\n        \"correlation\": {\n            \"operation\": \"efd224e10e18cf42b5a0c11b81054b1a\",\n            \"request\": \"648ce197a1ddb5d5\"\n        },\n        \"environment\": \"northeurope\",\n        \"error\": {\n            \"code\": \"UserError\",\n            \"innerError\": {\n                \"code\": \"ForbiddenError\"\n            },\n            \"message\": \"Exception of type 'Microsoft.MachineLearning.Common.WebApi.Exceptions.ForbiddenException' was thrown.\"\n        },\n        \"location\": \"northeurope\",\n        \"time\": \"2020-01-22T09:19:18.0091392+00:00\"\n    },\n    \"status_code\": 403,\n    \"url\": \"https://northeurope.experiments.azureml.net/execution/v1.0/subscriptions/05a11c00-3baa-427a-9d5f-442b1c24f711/resourceGroups/dp100new/providers/Microsoft.MachineLearningServices/workspaces/dp100ws/experiments/diabetes-training/localrun?runId=diabetes-training_1579684757_15f503fc\"\n}"
    }
}
	ErrorResponse 
{
    "error": {
        "message": "{\n    \"error_details\": {\n        \"correlation\": {\n            \"operation\": \"efd224e10e18cf42b5a0c11b81054b1a\",\n            \"request\": \"648ce197a1ddb5d5\"\n        },\n        \"environment\": \"northeurope\",\n        \"error\": {\n            \"code\": \"UserError\",\n            \"innerError\": {\n                \"code\": \"ForbiddenError\"\n            },\n            \"message\": \"Exception of type 'Microsoft.MachineLearning.Common.WebApi.Exceptions.ForbiddenException' was thrown.\"\n        },\n        \"location\": \"northeurope\",\n        \"time\": \"2020-01-22T09:19:18.0091392+00:00\"\n    },\n    \"status_code\": 403,\n    \"url\": \"https://northeurope.experiments.azureml.net/execution/v1.0/subscriptions/05a11c00-3baa-427a-9d5f-442b1c24f711/resourceGroups/dp100new/providers/Microsoft.MachineLearningServices/workspaces/dp100ws/experiments/diabetes-training/localrun?runId=diabetes-training_1579684757_15f503fc\"\n}"
    }
}

Python Module Failing

Thank you Graeme Malcolm for your invaluable contribution. Much appreciated. David

I am sorry to add to your workload but it seems that the new python module is failing. I had previously used the SQL Transformation module and that worked in the pipeline ok but did not allow deployment as a result of the pandas update. I would like to come here with a solution to the problem rather than a question but unfortunately my knowledge of Python is minimal or non-existent.

If you can help us (the general community) I / we would be most grateful.

CODE BENEATH
import pandas as pd
def azureml_main(dataframe1 = None, dataframe2 = None):
scored_results = dataframe1[['PatientID', 'Scored labels', 'Scored Probabilities']]
scored_results.rename(columns={'Scored Labels': 'DiabetesPrediction', 'Scored Probabilities':'Probability'},
inplace=True)
return scored_results

Unable to use Azure Pass

I received mail for assessment at 6th Oct. I checked it at afternoon. But, now I'm trying to redeem my Azure Pass. And it's showing me that's it's already used by someone even I never shared it with anyone. So please guide me.

Lab 10A AppInsights doesn't report logged events

ACI integration with AppInsights is currently not working.
The Azure ML Engineering team have confirmed the issue and are working on a fix.

Juypter Notebooks Written Scripts Missing Import OS

Is missing the import os from the second code cell.

Lab 7B no longer works

This Lab a couple od weeks ago worked perfectly.

Now, in the block of code:
from azureml.pipeline.core import Pipeline

pipeline = Pipeline(workspace=ws, steps=[parallelrun_step])
pipeline_run = Experiment(ws, 'batch_prediction_pipeline').submit(pipeline)
pipeline_run.wait_for_completion(show_output=True)

I left it running for almost 4 hours and does not finish.

The log does not advance

Change Lab 02B Task 2 (Code Spaces) to Optional

Can we please change the Visual Studio Code Spaces tasks in Lab 01B to optional?

The lab causes a number of headaches for my students with no real benefit. By making it optional, if someone is interested in seeing VSCode in the browser they can.

To be honest, I would rather use VSCode installed on my desktop than in a browser. Most students would feel the same way.

As it is, for the last two classes I have made it an optional part of the course.

A better lab would be deploying one of the Data Science Virtual Machines.

Error in Lab2B

Getting 403 Forbidden error. Unable to deploy in lab 2B. Getting rbac authentication issue in Lab2B.md Task 4.

Lab 09A - text and code disagree

The text in 09A says that it will use local compute to save time, but the code uses the cluster

Run an Automated Machine Learning Experiment

To reduce time in this lab, you'll run an automated machine learning experiment on local compute with only three iterations.

automl_config = AutoMLConfig(name='Automated ML Experiment',
                             task='classification',
                             compute_target=training_cluster,

Only the creator can access a compute instance Error

Hi,

I am currently using Azure pass subscription and currently working on DP-100 labs. I got the "Only the creator can access a compute instance" error when i clicked the Jupyter link from the compute instance i created.

1. I am using, my personal Outlook Account
2. I have assigned the Role Owner as mentioned in the lab instruction
3. I deleted the history and everything, login again but same issue persists
4. I tried and restarting the compute, created a new compute instance but same issue persisted.

Please help!!

Lab 04A - Data upload from local computer fails!

I'm using local access to azure workspace using lab files that I cloned from this github repo to my local drive.

I tried to run the following part:

ws.get_default_datastore().upload_files(files=['./data/diabetes.csv', './data/diabetes2.csv'], # Upload the diabetes csv files in /data
target_path='diabetes-data/', # Put it in a folder path in the datastore
overwrite=True, # Replace existing files of the same name
show_progress=True)

I received the following error:
UserErrorException: UserErrorException:
Message: relative_root: '.\data' is not part of the file_path: './data/diabetes.csv'.
InnerException None
ErrorResponse
{
"error": {
"code": "UserError",
"message": "relative_root: '.\data' is not part of the file_path: './data/diabetes.csv'."
}
}

Then I specified file paths fully:

default_ds.upload_files(files=['C:/Users/Emre/Documents/Python Scripts/AzureMLRepository/DP100/DP100/data/diabetes.csv',
'C:/Users/Emre/Documents/Python Scripts/AzureMLRepository/DP100/DP100/data/diabetes2.csv'], # Upload the diabetes csv files in /data
relative_root='C:/Users/Emre/Documents/Python Scripts/AzureMLRepository/DP100/DP100/', #
target_path='/diabetes-data/', # Put it in a folder path in the datastore
overwrite=True, # Replace existing files of the same name
show_progress=True)

Now I receive another long error which contains:

logger.error("Upload failed, please make sure target_path does not start with invalid characters.", e)
Message: 'Upload failed, please make sure target_path does not start with invalid characters.'
Arguments: (AzureHttpError('Forbidden\n\r\n<TITLE>Forbidden</TITLE>\r\n\r\n

Forbidden URL

\r\n

---

HTTP Error 403. The request URL is forbidden.

\r\n\r\n'),)

Please help me complete this lab by successfully uploading data files from local drive to azure blob storage.

Emre

Lab 2B error when calling deployed endpoint

An update to a dependency (pandas) is causing pipelines that contain some modules (including SQL Transformation) to fail.
The Azure ML Engineering team have confirmed this issue and are working on a fix. In the meantime, omit the SQL Transformation from the inference pipeline (so the service returns all fields).

Add a license

Environment and CondaDependencies need to be updated

It appears that the notebooks here (at least 07A) include the previous method to set up an environment for deployment:

from azureml.core.conda_dependencies import CondaDependencies 

# Add the dependencies for our model (AzureML defaults is already included)
myenv = CondaDependencies()
myenv.add_conda_package('scikit-learn')

# Save the environment config as a .yml file
env_file = folder_name + "/diabetes_env.yml"
with open(env_file,"w") as f:
    f.write(myenv.serialize_to_string())
print("Saved dependency info in", env_file)

# Print the .yml file
with open(env_file,"r") as f:
    print(f.read())

...but this method now throws an error on the current AMLS SDK version.

Perhaps this should be updated to:

from azureml.core.environment import Environment
from azureml.core.conda_dependencies import CondaDependencies

myenv = Environment(name="myenv")
conda_dep = CondaDependencies()

conda_dep.add_conda_package("scikit-learn")

# Adds dependencies to PythonSection of myenv
myenv.python.conda_dependencies=conda_dep

Lab 2B: Test tab and sample code not present in published endpoint

Previously, a published endpoint from a designer real-time pipeline included a Test interface and some sample code to call the service. This is no longer appearing in the UI.
This is a known issue that the Azure ML engineering team are currently troubleshooting.
As a workaround, the lab has been updated to provide the code, so only the endpoint and key from the Consume tab need to be copied.

Task 4, Step 3 Lab 2A:

When trying to view visualization of normalized data set, we get the following error "Unable to profile this dataset. This might be because your data is stored behind a virtual network or your data does not support profile."

Regards
SS

Lab 6A Cell 9 fails

The code in cell 9, after the markup "OK, you're ready build the pipeline from the steps you've defined and run it as an experiment." errors

"!pip install azureml-datadrift" required for lab10b

see title

The 08B notebook (AutoML) fails when azureml-sdk == 1.5.0

At least for me the execution of the '08B - Using Automated Machine Learning' notebook fails at execution of the Auto ML experiment. Exception is:
ActivityFailedException: ActivityFailedException: Message: Activity Failed: { "error": { "code": "UserError", "message": "User program failed with AttributeError: 'str' object has no attribute 'name'"

However, when I downgraded the azureml-sdk packages back to 1.4.0, everything executed smoothly.

Query - Relevancy of the course material

Taking the exam soon. Would like to check the relevancy of the materials: is it up-to-date?

Thanks.

Lab 2B: Deploying a Service with the Azure ML Designer -issue

Hi,
In Lab 2B: Deploying a Service with the Azure ML Designer

Task 2: Create an Inference Pipeline
The inference pipeline is failing and the issue persists due to the python script execution.

Could you please check into this?

Thanks, And Regards
Bony

Lab 10B - Datadrift install/import deprecation issue

Lab 10B - when using the pip install --upgrade to install the data drift package, it throws a deprecation error (something about an Artist class in matplotlib) when trying to import the package. After the --upgrade was removed, the lab worked fine.

Losing access to COMPUTE instance

Greetings

After using the Azure Machine Learning Studio for the last 2 days and today when i started up the compute instance, i am getting the following:

User live.com#[email protected] does not have access to compute instance mlcomputeinstancedp100.

Only the creator can access a compute instance.

Click here to sign out and sign in again with a different account.

What could be the issue?

SS

Unable to pull to MicrosoftLearning/DP100 into codespaces

Hi @GraemeMalcolm

I understand that my issue might not be related to github itself but something to do with pull requests done when creating new codespaces. I seem to be getting a weird error

Failed to connect to the remote extension host server (Error: Time limit reached)

I'm currently on a Free subscription and created this account like approximately 1.5 weeks ago.

Steps:

• Created new plan using my "Free" subscription

• created new code space by entering git repo https://github.com/MicrosoftLearning/DP100

Any help would be highly appreciated more especially that I'm practicing implementing Data Sciences solutions implementation on Azure. Please :-(

azureml.widgets giving an error in JupyterLab

The widgets work fine in Jupyter but give module not found error in JupyterLab. ( You can reproduce this error by running cell 5 in notebook 03A - Running Experiments)

insert import os into the 3B lab

Presently the code reads:

%%writefile $training_folder/diabetes_training.py

Import libraries

from azureml.core import Run
import pandas as pd
import numpy as np
import joblib
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import roc_auc_score
from sklearn.metrics import roc_curve

If the cells are run separately you get an error from line 47 saying that the os is not recognised.
os.makedirs('outputs', exist_ok=True)

To avoid the error make sure you include the line import os as follows:

import joblib
import os
from sklearn.model_selection import train_test_split

More to Tips.md

Maybe add that a reboot of the compute target might also solve some issues.

I experienced a lot of errors until I rebooted the compute target.

Or restarting the kernel between each lab is also a good idea. Just to make sure that all variables is completely wiped.