No Operation named [init_all_tables] in the Graph when using GPT2Transformer about spark-nlp

[mdrobena]

Hi @DevinTDHa,

Thanks for your reply. I just tried to run the first task (GPT2 Pipeline) as shown here.

I had updated both, pypi spark-nlp and the maven library. This is my current environment:

1. Azure Databricks cluster with 14.0 ML runtime
2. Spark 3.5.0
3. Spark Maven library com.johnsnowlabs.nlp:spark-nlp_2.12:5.2.3 following Install Spark NLP on Databricks
4. pypi spark-nlp==5.2.3

However, I still get the same error:
org.apache.spark.SparkException: Job aborted due to stage failure: Task 2 in stage 23.0 failed 4 times, most recent failure: Lost task 2.3 in stage 23.0 (TID 173) (10.139.64.12 executor 1): java.lang.IllegalArgumentException: No Operation named [init_all_tables] in the Graph

I attach my notebook GPT2Transformer_ OpenAI Text-To-Text Transformer.zip to reproduce the error.

[DevinTDHa]

Hi @mdrobena,

Thanks for the thorough description! I was able to reproduce it with your instructions. Important is also that it needs to be run on a muli-node setup. I am looking into this problem.

[mdrobena]

Hi,

I have the exact same issue.

My environment:

Databricks runtime version 14.0 ML (includes Apache Spark 3.5.0, Scala 2.12)
Spark NLP: 5.2.2

[DevinTDHa]

Hi @SidWeng @mdrobena,

I tried to reproduce it, but there isn't an issue with my side. Can you perhaps detail your steps to recreate this issue? Additionally, can you perhaps try just to update Spark NLP to the latest version and see if there is any difference?

I tried these following combinations of settings:

1. Run the notebook on colab with your specified version, applying the config
2. Run the notebook locally with specified versions, applying the config
3. Run the notebook on databricks with the same runtime and specified versions, following Install Spark NLP on Databricks

But in all of them, I get the results without any problems. Thanks for reporting!