Metrics for sequence tagging tasks dependant on the max_seq_length parameter about xtreme

[stefan-it]

Hi @codedecde ,

I hope this is not the case, because RemBERT has this max length of 256.

But as far as I can tell from the pre-processing code, longer sequences are split into multiple sentences:

xtreme/utils_preprocess.py

Lines 67 to 74 in da430dd

	if (subword_len_counter + current_subwords_len) > max_seq_len:
	fout.write(f"\n{token}\t{label}\n")
	fidx.write(f"\n{idx}\n")
	subword_len_counter = current_subwords_len
	else:
	fout.write(f"{token}\t{label}\n")
	fidx.write(f"{idx}\n")
	subword_len_counter += current_subwords_len

(see extra new line before token) 🤔