serverless-mxnet-server's Introduction

Serverless-MXNet-Server

This is a work in progress. Check out the outstanding problems section below.

AWS has developed MXNet, a deep learning framework. Recently, they released a model server for MXNet, which makes it easy to set up an endpoint for serving your models.

This is a project to deploy the MXNet server in a Serverless application. Autoscaling, low-management model serving FTW ⚡️.

Usage

Install the Serverless Framework:
```
$ npm install -g serverless
```

Install this repository as a template:

$ sls install --url https://github.com/alexdebrie/serverless-mxnet-server && cd serverless-mxnet-server
$ npm install

Deploy!
```
$ sls deploy
```

Call your endpoint!

# Set your BASE_URL with the endpoint given with sls deploy
$ export BASE_URL="https://58kx6wsxej.execute-api.us-east-1.amazonaws.com/dev"
$ curl -X POST ${BASE_URL}/squeezenet/predict -F "[email protected]"

Outstanding problems

I'm having trouble getting the image data through API Gateway in the format expected by the MXNet server.

To get it working locally:

Create a virtual environment and install all required packages:

$ virtualenv venv --python=python3 && source venv/bin/activate
$ pip install -r requirements.txt

Start your app locally:
```
$ sls wsgi serve
```

Make a request against it:

curl -X POST http://127.0.0.1:5000/squeezenet/predict -F "[email protected]"
{
  "prediction": [
    [
      {
        "class": "n02124075 Egyptian cat",
        "probability": 0.9408263564109802
      },
      {
        "class": "n02127052 lynx, catamount",
        "probability": 0.05596580728888512
      },
      {
        "class": "n02123045 tabby, tabby cat",
        "probability": 0.0025502473581582308
      },
      {
        "class": "n02123159 tiger cat",
        "probability": 0.00034319929545745254
      },
      {
        "class": "n02123394 Persian cat",
        "probability": 0.000268968433374539
      }
    ]
  ]
}

Recommend Projects