Possibility to finetune language model only (unsupervised) about finetune

[madisonmay]

Yup! This is absolutely possible -- just a matter of calling model.fit(text) without providing labels. See issue #56.

I would expect this unsupervised finetuning process to be useful but haven't had the opportunity to run substantive experiments on this front, so definitely let us know how it goes!

[thinline72]

Oh, great, thanks! So I can use any of the classes (Classifier, Regressor, etc.) to just initialize/load model and then finetune without providing labels, right?

And if I want to firstly finetune LM and then finetune on concrete task, I can do the following:

model = Classifier()
model.fit(unlabelled_text)
model.fit(labelled_text, labels)

Does it make sense?

[madisonmay]

Yup, your understanding is correct!

[thinline72]

Great, thank you a lot for your help!