Not able to run basic inference using canary-1b model about nemo

[LL-AI-dev]

@P15V make sure that there are no line breaks between the curly brackets. I believe this will solve your error

nemo manifests despite being saved as .json are a bit fiddly at times and cannot be directly loaded in via the json package. each line within the manifest should be a complete json string.

I think the error is occurring because it is trying to read the 2nd line as a json string, but because you have line breaks within the curly brackets the 2nd line is not a valid json string on its own.

[titu1994]

This is the right answer, we read a file as a jsonl, though we (wrongly) call it a .json file. We'll attempt to make the parsing logic a bit more robust in the future, and log an appropriate warning instead of crashing. Fyi @stevehuang52

[krishnacpuvvada]

And directly fixing the errors with the JSON file directly, leads to this error:
"-packages/nemo/collections/common/data/lhotse/nemo_adapters.py", line 84, in iter
text=data[self.text_field],
KeyError: 'answer'

to fix this, please modify the input lines file to add 'answer' field.

{
  "audio_filepath": "PathRemovedDueToPersonalName",  
  "duration": 30.0,  
  "taskname": "asr",  
  "source_lang": "en", 
  "target_lang": "en", 
  "pnc": 'yes', 
  "answer": 'na',
}

also, we recently updated .transcribe signature, so if you are using main branch

transcript = canary_model.transcribe(paths2audio_files="/home/pjstimac/NvidiaCanaryTest/transcribe_manifest.json", batch_size=16)

should be updated to
transcript = canary_model.transcribe(audio="/home/pjstimac/NvidiaCanaryTest/transcribe_manifest.json", batch_size=16)

[titu1994]

The notebook above is not the way to do inference for Canary, it's for beam search with CTC models, and is a deprecated notebook in general.

[titu1994]

Glad it worked. We'll iron out these issues in the pre-release. It shouldn't be so difficult to do inference

[P15V]

Hello all,

Thanks for your time & replies; I can't express how much I genuinely appreciate it!! :)

So after I made this post, I went home and tried all night on my personal time; still no luck unfortunately.

I found that "answer: 'na' " key via the Hugging face documentation and included that.

@krishnacpuvvada, @LL-AI-dev Running that format JSON, with the updated transcript variable, still prints out the json error:
"json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 2 column 1 (char 2)"

I go off the error, and correct that with the JSON, and still get this error (and ran into this last night as well)
" assert isinstance(cut, MonoCut), "Expected MonoCut."
AssertionError: Expected MonoCut."

Tried both a JSON & JSONL file, same result :
" assert isinstance(cut, MonoCut), "Expected MonoCut."
AssertionError: Expected MonoCut."

My updated code with that updated variable:
"" # Load Canary model
from nemo.collections.asr.models import EncDecMultiTaskModel
canary_model = EncDecMultiTaskModel.from_pretrained('nvidia/canary-1b')

transcript = canary_model.transcribe(audio="/home/NameRemoved/NvidiaCanaryTest/transcribed_manifest.jsonl", batch_size=16) ""

Last night, I thought, let's try the tutorial Google Colab Notebooks right from the Nvidia website for any NeMo model...
Not even that could run all the way through right on google colab.

"https://colab.research.google.com/github/NVIDIA/NeMo/blob/stable/tutorials/asr/Offline_ASR.ipynb"

It was error out at this variable "paths2audio_files=files"

Thanks for everyone's time; much appreciated!!!! :D

[P15V]

@titu1994 Well that would explain it! After much trial and error, I finally got it running in a notebook & python shell. with 3 lines of code, skipping the JSON/JSONL manifest entirely:
""
import nemo.collections.asr as nemo_asr
nemoasr_model = nemo_asr.models.EncDecCTCModelBPE.from_pretrained("nvidia/canary-1b")
nemoasr_model.transcribe(['AudioClipDirectly.wav'])
""

Once that was working, I just wrote a Python loop to loop through an audio directory and output the transcription results to a JSON for my viewing/model comparison setup app.

Thanks for all the attempts at help though, @titu1994 & @krishnacpuvvada. I genuinely appreciate it!! Wish the documentation was better so I would not have had to bother you guys, oh well.

be well!!! :D

[P15V]

@titu1994 That would be so great to see!! I've been playing around with Whisper for the past few months, and this was unexpectedly annoying to get going in comparison.
the code is simple enough looking at it; the documentation though, is a different story. But it's going now on my end, yay!! :) Thanks again for the input and help @krishnacpuvvada & @titu1994 , much appreciated!!