azure / aml-real-time-ai

Easily deploy models to FPGAs for ultra-low latency with Azure Machine Learning powered by Project Brainwave

Microsoft Azure Machine Learning Hardware Accelerated Models Powered by Project Brainwave

IMPORTANT!

This service is now generally available, and this repo will be shut down. Please use the updated notebooks and read the updated documentation.

Easily create and train a model using various deep neural networks (DNNs) as a featurizer for deployment on Azure for ultra-low latency inferencing. These models are currently available:

• ResNet 50
• ResNet 152
• DenseNet-121
• VGG-16

How to get access

Azure ML Hardware Accelerated Models is currently in preview.

Step 1: Create an Azure ML workspace

Follow these instructions to install the Azure ML SDK on your local machine, create an Azure ML workspace, and set up your notebook environment, which is required for the next step.

Note: Only workspaces in the East US 2 region are currently supported.

Once you have set up your environment, install the contrib extras:

pip install --upgrade azureml-sdk[contrib]

Currently only tensorflow version<=1.10 is supported, so install it at the end:

pip install "tensorflow==1.10"

Go to the documentation page for any questions.

Step 2: Deploy your service

Check out the sample notebooks here.

Note: You can deploy one FPGA service. If you want to deploy more than one service, you must request quota by submitting the form. You will need information from your workspace created in Step 1 (learn how to get workspace information). You will receive an email if your quota request has been successful.

Support

Read the docs or visit the forum.

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.microsoft.com.

When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

Build Status

System	Unit tests
Ubuntu 16.04

aml-real-time-ai's Issues

This repo is missing important files

There are important files that Microsoft projects should all have that are not present in this repository. A pull request has been opened to add the missing file(s). When the pr is merged this issue will be closed automatically.

Microsoft teams can learn more about this effort and share feedback within the open source guidance available internally.

Merge this pull request

Image Request Isn't reusable

Describe the bug
The image request object can only have MakePredictRequest() called once.

To Reproduce
Construct an ImageRequest with a stream - each call of MakePredictRequest() will try to read the complete stream,

Expected behavior
ImageRequest will read the stream on construction and each call to MakePredictRequest will return the same result.

Notebook fails on model register phase

When running step 0 of 00_QuickStart notebook ran into this error:

Creating storage account fpga7ae4fcb810a4c6b6cfdb

---

TypeError Traceback (most recent call last)
in ()
----> 1 model_id = deployment_client.register_model(model_name, service_def_path)

e:\work\aml-real-time-ai\pythonlib\amlrealtimeai\deployment_client.py in register_model(self, model_name, service_def)
45
46 def register_model(self, model_name, service_def):
---> 47 storage_account_key = self.__create_storage_account_and_get_key()
48 cloud_storage_account = CloudStorageAccount(self.__storage_account_name, storage_account_key)
49

e:\work\aml-real-time-ai\pythonlib\amlrealtimeai\deployment_client.py in __create_storage_account_and_get_key(self)
277 self.__storage_account_name,
278 StorageAccountCreateParameters(
--> 279 sku=Sku(SkuName.standard_ragrs),
280 kind=Kind.storage,
281 location=self.__location

TypeError: init() takes 1 positional argument but 2 were given

Quickstart Notebook fails on service registration

I am trying to follow the quickstart notebook (https://github.com/Azure/aml-real-time-ai/blob/master/notebooks/project-brainwave-quickstart.ipynb)

I have my azure ML workspace and I registered it locally. The workspace can be loaded.
The problem that I have is after registering the models(which is sucessfull, I can see the in my Azure workspace) the service cannot be published due the to "Response Code: 403"

The exception I get:
WebserviceException: Received bad response from Model Management Service:
Response Code: 403
Headers: {'Date': 'Wed, 29 May 2019 11:13:37 GMT', 'Content-Type': 'application/json', 'Transfer-Encoding': 'chunked', 'Connection': 'keep-alive', 'Vary': 'Accept-Encoding', 'Request-Context': 'appId=cid-v1:2d2e8e63-272e-4b3c-8598-4ee570a0e70d', 'api-supported-versions': '1.0, 2018-03-01-preview, 2018-11-19', 'x-ms-client-request-id': 'e2eb89ddcd4a423b81eba57d2cf9ee90', 'x-ms-client-session-id': '', 'Strict-Transport-Security': 'max-age=15724800; includeSubDomains; preload', 'Content-Encoding': 'gzip'}
Content: b'{"code":"Forbidden","statusCode":403,"message":"Forbidden","details":[{"code":"Forbidden","message":"AI Fabric returned Bad Request. Reason: "}]}'

For more information on this error, contact Microsoft Support or get help on the Azure ML forum (https://aka.ms/aml-forum) with the request ID .

The trace:

Creating image
Image creation operation finished for image imagenet-infer:10, operation "Succeeded"
Creating service

WebserviceException Traceback (most recent call last)
in
6 try:
----> 7 service = Webservice(ws, service_name)
8 except WebserviceException:

~/Programs/Anaconda/envs/tf-gpu/lib/python3.6/site-packages/azureml/core/webservice/webservice.py in new(cls, workspace, name)
82 raise WebserviceException('WebserviceNotFound: Webservice with name {} not found in provided '
---> 83 'workspace'.format(name))
84 else:

WebserviceException: WebserviceNotFound: Webservice with name imagenet-infer not found in provided workspace

During handling of the above exception, another exception occurred:

HTTPError Traceback (most recent call last)
~/Programs/Anaconda/envs/tf-gpu/lib/python3.6/site-packages/azureml/core/webservice/webservice.py in _deploy_webservice(workspace, name, webservice_payload, webservice_class)
372 resp = requests.post(mms_endpoint, params=params, headers=headers, json=webservice_payload)
--> 373 resp.raise_for_status()
374 except requests.exceptions.HTTPError:

~/Programs/Anaconda/envs/tf-gpu/lib/python3.6/site-packages/requests/models.py in raise_for_status(self)
939 if http_error_msg:
--> 940 raise HTTPError(http_error_msg, response=self)
941

HTTPError: 403 Client Error: Forbidden for url: https://eastus2.modelmanagement.azureml.net/api/subscriptions/*****my subscription id*******/resourceGroups/demo-rg/providers/Microsoft.MachineLearningServices/workspaces/demo/services?api-version=2018-11-19

During handling of the above exception, another exception occurred:

WebserviceException Traceback (most recent call last)
~/Programs/Anaconda/envs/tf-gpu/lib/python3.6/site-packages/azureml/contrib/brainwave/brainwave_webservice.py in _deploy(workspace, name, image, deployment_config)
118 try:
--> 119 return Webservice._deploy_webservice(workspace, name, create_payload, BrainwaveWebservice)
120 except WebserviceException as e:

~/Programs/Anaconda/envs/tf-gpu/lib/python3.6/site-packages/azureml/core/webservice/webservice.py in _deploy_webservice(workspace, name, webservice_payload, webservice_class)
378 'Content: {}'.format(resp.status_code, resp.headers, resp.content),
--> 379 resp.status_code)
380 if resp.status_code != 202:

WebserviceException: Received bad response from Model Management Service:
Response Code: 403
Headers: {'Date': 'Wed, 29 May 2019 11:13:37 GMT', 'Content-Type': 'application/json', 'Transfer-Encoding': 'chunked', 'Connection': 'keep-alive', 'Vary': 'Accept-Encoding', 'Request-Context': 'appId=cid-v1:2d2e8e63-272e-4b3c-8598-4ee570a0e70d', 'api-supported-versions': '1.0, 2018-03-01-preview, 2018-11-19', 'x-ms-client-request-id': 'e2eb89ddcd4a423b81eba57d2cf9ee90', 'x-ms-client-session-id': '', 'Strict-Transport-Security': 'max-age=15724800; includeSubDomains; preload', 'Content-Encoding': 'gzip'}
Content: b'{"code":"Forbidden","statusCode":403,"message":"Forbidden","details":[{"code":"Forbidden","message":"AI Fabric returned Bad Request. Reason: "}]}'

During handling of the above exception, another exception occurred:

WebserviceException Traceback (most recent call last)
in
9 image_config = BrainwaveImage.image_configuration()
10 deployment_config = BrainwaveWebservice.deploy_configuration()
---> 11 service = Webservice.deploy_from_model(ws, service_name, [registered_model], image_config, deployment_config)
12 service.wait_for_deployment(True)

~/Programs/Anaconda/envs/tf-gpu/lib/python3.6/site-packages/azureml/core/webservice/webservice.py in deploy_from_model(workspace, name, models, image_config, deployment_config, deployment_target)
247 image.image_build_log_uri,
248 image.generated_dockerfile_uri))
--> 249 return Webservice.deploy_from_image(workspace, name, image, deployment_config, deployment_target)
250
251 @staticmethod

~/Programs/Anaconda/envs/tf-gpu/lib/python3.6/site-packages/azureml/core/webservice/webservice.py in deploy_from_image(workspace, name, image, deployment_config, deployment_target)
279 if child._webservice_type == ACI_WEBSERVICE_TYPE:
280 return child._deploy(workspace, name, image, deployment_config)
--> 281 return deployment_config._webservice_type._deploy(workspace, name, image, deployment_config)
282
283 else:

~/Programs/Anaconda/envs/tf-gpu/lib/python3.6/site-packages/azureml/contrib/brainwave/brainwave_webservice.py in _deploy(workspace, name, image, deployment_config)
131 .format(error_message, request_id[0])
132 if error_message:
--> 133 raise WebserviceException(e.message + "\n\n" + error_message)
134 raise
135

OS: Ubuntu 18.04.2 LTS
Language: Python 3.7
Name: tensorflow
Version: 1.10.1
Name: azureml-sdk
Version: 1.0.39