Zero-Shot NER gives wrong entities with labels about spark-nlp

[RakshaRaoKaraya]

Can someone please help me with this issue?

[RakshaRaoKaraya]

@maziyarpanahi, @josejuanmartinez : I have tried the example in https://sparknlp.org/2023/02/08/zero_shot_ner_roberta_en.html . It still gives me the same issue. It picks the next entity and gives it the label. The labels are right, but the entities are wrong.

[maziyarpanahi]

@RakshaRaoKaraya

There seems to be an issue with NerConverter and ZeroShotNER. As you can see, the ZeroShot detected the entities and their labels correctly: https://colab.research.google.com/drive/1xX_9Rrnvxb7O-C9ICvTs9rSmA-rLSB_E?usp=sharing

result.select("zero_shot_ner.result").show(1, False)
+-------------------------------------------------------------------------------------+
|result                                                                               |
+-------------------------------------------------------------------------------------+
|[O, O, O, O, O, O, O, O, O, O, O, O, O, B-NAME, O, O, O, O, O, O, O, O, O, O, B-CITY]|
+-------------------------------------------------------------------------------------+

result.select("zero_shot_ner").show(1, False)
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|zero_shot_ner                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|[{named_entity, 0, 5, O, {sentence -> 0, word -> Hellen}, []}, {named_entity, 7, 11, O, {sentence -> 0, word -> works}, []}, {named_entity, 13, 14, O, {sentence -> 0, word -> in}, []}, {named_entity, 16, 21, O, {sentence -> 0, word -> London}, []}, {named_entity, 22, 22, O, {sentence -> 0, word -> ,}, []}, {named_entity, 24, 28, O, {sentence -> 0, word -> Paris}, []}, {named_entity, 30, 32, O, {sentence -> 0, word -> and}, []}, {named_entity, 34, 39, O, {sentence -> 0, word -> Berlin}, []}, {named_entity, 40, 40, O, {sentence -> 0, word -> .}, []}, {named_entity, 42, 43, O, {sentence -> 0, word -> My}, []}, {named_entity, 45, 48, O, {sentence -> 0, word -> name}, []}, {named_entity, 50, 51, O, {sentence -> 0, word -> is}, []}, {named_entity, 53, 57, O, {sentence -> 0, word -> Clara}, []}, {named_entity, 58, 58, B-NAME, {sentence -> 0, word -> ,, confidence -> 0.94376206, question -> What is my name?}, []}, {named_entity, 60, 60, O, {sentence -> 0, word -> I}, []}, {named_entity, 62, 65, O, {sentence -> 0, word -> live}, []}, {named_entity, 67, 68, O, {sentence -> 0, word -> in}, []}, {named_entity, 70, 72, O, {sentence -> 0, word -> New}, []}, {named_entity, 74, 77, O, {sentence -> 0, word -> York}, []}, {named_entity, 79, 81, O, {sentence -> 0, word -> and}, []}, {named_entity, 83, 88, O, {sentence -> 0, word -> Hellen}, []}, {named_entity, 90, 94, O, {sentence -> 0, word -> lives}, []}, {named_entity, 96, 97, O, {sentence -> 0, word -> in}, []}, {named_entity, 99, 103, O, {sentence -> 0, word -> Paris}, []}, {named_entity, 104, 104, B-CITY, {sentence -> 0, word -> ., confidence -> 0.3440236, question -> Which is the city?}, []}]|
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
only showing top 1 row

But it failed to put them together via NerConverter. Seems to be a bug, we'll take a look and fix it for the next release.

[RakshaRaoKaraya]

@maziyarpanahi : Thank you very much for your reply. Would you be able to give me a tentative date when this could be resolved? Or is there any other NER models that I could use meanwhile to detected entities based on Question Answer Model?

[maziyarpanahi]

You are very welcome. We have many NER models both based on our own architecture or all-in-one from Hugging Face. (trained by us)

They are not however zero-shot, meaning the labels are pre-defined. You can have a look for the meantime while we fix this small part:

• This is for NER based on BERT: https://sparknlp.org/models?task=Named+Entity+Recognition&annotator=BertForTokenClassification
• I personally suggest this one from OntoNotes 5.0 and DeBerta that has the highest accuracy and 18 entities: https://sparknlp.org/2022/05/06/deberta_v3_base_token_classifier_ontonotes_en_3_0.html

[RakshaRaoKaraya]

@maziyarpanahi : Thank you for the suggestions. I was hoping to get a question answer type of NER. Even though I am capturing pre-defined entities, I need to capture answers to specific questions in the text. I think Zero-shot would help better. I didn't come across any other model that does that.

[RakshaRaoKaraya]

Hello @maziyarpanahi : Could you please tell me when can I expect a resolution of this bug?

[maziyarpanahi]

Hello @maziyarpanahi : Could you please tell me when can I expect a resolution of this bug?

Hi, we are still investigating this issue. I'll update here once we have a fix for it

[RakshaRaoKaraya]

@maziyarpanahi : Thank you so much for your response. I will wait for the resolution then.

[RakshaRaoKaraya]

Hello @maziyarpanahi : Thank you for looking into this issue. Is the model good to use now in python environment? Do I need to change my Spark NLP version for this to work?

[maziyarpanahi]

Hello @maziyarpanahi : Thank you for looking into this issue. Is the model good to use now in python environment? Do I need to change my Spark NLP version for this to work?

You are welcome. You must have Spark NLP 5.3.0 and everything would work without any change.

[RakshaRaoKaraya]

Works perfectly well now. Thank you once again 👍