typesafehub / fdp-modelserver

An umbrella project for multiple implementations of model serving

Model serving

This is an umbrella project for all things model serving that is comprised of multiple projects

-akkaserver - implementation of model scoring and statistics serving using Akka streams and Akka HTTP

-flinkserver - implementation of model scoring and queryable state using Flink. Both key-based and partition-based approach are implemented here

-kafkaclient - generic client used for testing of all implementations (except serving samples) Reads data files, split them into records, converts to protobuf implementations and publishes them to Kafka

-kafkaconfiguration - simple module containing class with Kafka definitions - server location, topics, etc. used by all applications

-kafkastreamserver - implementation of model scoring and queryable state using Kafka streams Also includes implementation of custom Kafka streams store.

-model - implementation of support classes representing model and model factories used by all applications. Because Kafka streams is Java and the rest of implementations are Scala, there are two versions of these classes - Java and Scala

-serving samples - This module contains simple implementations of model scoring using PMML and tensorflow model definitions. It is not using any streaming frameworks - just straight Scala code

-protobufs - a module containing protobufs that are used for all streaming frameworks. This protobufs describe model and data definition in the stream. Because Kafka streams is Java and the rest of implementations are Scala, both Java and Scala implementations of protobufs are generated

-sparkML - examples of using SparkML for machine learning and exporting results to PMML using JPMML evaluator for Spark - https://github.com/jpmml/jpmml-evaluator-spark

-sparkserver - implementation of model scoring using Spark

-utils - a module containing some utility code. Most importantly it contains embedded Kafka implementation which can be used for testing in the absence of kafka server. In order to use it, just add these lines to your code:

// Create embedded Kafka and topics
EmbeddedSingleNodeKafkaCluster.start()                      // Create and start the cluster 
EmbeddedSingleNodeKafkaCluster.createTopic(DATA_TOPIC)      // Add topic
EmbeddedSingleNodeKafkaCluster.createTopic(MODELS_TOPIC)    // Add topic

If you are using both server and client add kafka embedded only to server and start it before the client In addition to embedded kafka this module there are some utility classes used by all applications. Because Kafka streams is Java and the rest of implementations are Scala, there are two versions of these classes - Java and Scala

-data - a directory of data files used as sources for all applications

Not included in this project are:

-Beam implementation - Beam Flink runner is still on Scala 2.10 so it is in its own separate project - https://github.com/typesafehub/fdp-beam-modelServer

-Python/Tensorflow/Keras - is it is a Python so it is in its own separate project - https://github.com/typesafehub/fdp-tensorflow-python-examples

fdp-modelserver's Issues

Libraries to compile and run ModelServerWithStore.java

What are all the libraries required to compile and build ModelServerWithStore.java?

javac on this class gave 37 errors:

...................................
...................................
symbol: class KafkaStreams
location: class ModelServerWithStore
src/main/java/com/lightbend/modelserver/withstore/ModelServerWithStore.java:78: error: cannot find symbol
final QueriesRestService restService = new QueriesRestService(streams);
^
symbol: class QueriesRestService
location: class ModelServerWithStore
src/main/java/com/lightbend/modelserver/withstore/ModelServerWithStore.java:78: error: cannot find symbol
final QueriesRestService restService = new QueriesRestService(streams);
^
symbol: class QueriesRestService
location: class ModelServerWithStore
37 errors

Compilation error

servingsamples/src/main/scala/com/lightbend/tensorflow/WineModelServingBundle.scala:9: error: object framework is not a member of package org.tensorflow
import org.tensorflow.framework.{MetaGraphDef, SignatureDef, TensorInfo, TensorShapeProto}
^
servingsamples/src/main/scala/com/lightbend/tensorflow/WineModelServingBundle.scala:49: error: not found: value MetaGraphDef
val metaGraphDef = MetaGraphDef.parseFrom(bundle.metaGraphDef())
^
servingsamples/src/main/scala/com/lightbend/tensorflow/WineModelServingBundle.scala:97: error: not found: type SignatureDef
def parseSignature(signatureMap : Map[String, SignatureDef]) : Seq[Signature] = {
^
servingsamples/src/main/scala/com/lightbend/tensorflow/WineModelServingBundle.scala:71: error: not found: type TensorInfo
def convertParameters(tensorInfo: Map[String,TensorInfo]) : Seq[Parameter] = {