Missing json config files when converting model from the Hub about transformers.js

[xenova]

Ah yes, you are completely right. I had this problem before. Initially, I just added a check for which models have processors:

transformers.js/scripts/convert.py

Lines 193 to 195 in 6078e1c

	if model.config.model_type in MODELS_WITH_PROCESSORS:
	processor = AutoProcessor.from_pretrained(model_path)
	processor.save_pretrained(output_model_folder)

but I removed it since it was extremely hacky, and would be difficult to maintain.

The other option would be to just copy all .json files:

transformers.js/scripts/convert.py

Line 203 in 492c117

# TODO copy all .json files

Unless there is a better way? I assume this is a problem with optimum and the transformers' save_pretrained function, which seems to ignore the tokenizer.json

[xenova]

I did some digging, and looks like transformers has a handy function for this! I've updated the conversion script to handle this. Let me know if you know of other files which aren't copied by save_pretrained

[xenova]

Closing as this was fixed in 1f14ca0