MNIST example - Exception in TF background thread about tensorflowonspark

[leewyang]

Are you able to run standard Spark jobs on this cluster with those same settings? The Driver requested a total number of 0 executors error seems to be occurring at the Spark level and not the TFoS level.

Please take a look at the Spark executor logs for any exceptions, since you're only showing the driver logs, which aren't showing the root cause of the exception.

Finally, you are using a very old version of tensorflow (1.9.0) and you should be using at least tensorflowonspark==1.4.4 (the last version of TFoS intended for TF 1.x branch). That said, If you are starting from scratch, I'd recommend using TF2.x along with the latest tensorflowonspark.

[Ipsedo]

The problem was a conflict between .local and conda environment folders when distribute the python environment.

I switch to the 1.4.4 TFoS version working with TF 1.13.1 and all works fine.

Thanks for the quick response, I close the issue.