error about docextractor

[marouamehri]

Hi @monniert

Thank you very much for sharing your work !

I obtain the following error when tester.py was run.

File "/content/drive/MyDrive/docExtractor/src/utils/metrics.py", line 56, in _fast_hist
minlength=self.n_classes ** 2).reshape(self.n_classes, self.n_classes)
ValueError: cannot reshape array of size 3236 into shape (4,4)

What can I do?

Thank you in advance

[monniert]

Hi @nada0698 @marouamehri

The script src/tester.py is made to evaluate quantitatively the segmentation results on given labels and it requires appropriate ground truth masks. The traceback suggests that there are many labels in your ground truth masks, depending on what you would like to evaluate, you should only have 1 or 2 labels (illustration and text) in your ground truth masks.

If you only want to get some qualitative segmentation masks inference, you can take a look at the notebook at demo/demo.ipynb

[nada0698]

@monniert thank you very much for your answer
I obtain the same error when tester.py was run.
i need the result of the test

!python /content/drive/MyDrive/docExtractor/src/trainer.py -wt --tag test1 --config syndoc.yml

Traceback (most recent call last):
File "/content/drive/MyDrive/docExtractor/src/trainer.py", line 336, in
tester.run()
File "/content/drive/MyDrive/docExtractor/src/tester.py", line 60, in run
self.single_run(image, label)
File "/usr/local/lib/python3.7/dist-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
return func(*args, **kwargs)
File "/content/drive/MyDrive/docExtractor/src/tester.py", line 81, in single_run
self.metrics.update(gt, pred)
File "/content/drive/MyDrive/docExtractor/src/utils/metrics.py", line 52, in update
self.confusion_matrix += self._fast_hist(lt_flat, lp_flat)
File "/content/drive/MyDrive/docExtractor/src/utils/metrics.py", line 56, in _fast_hist
minlength=self.n_classes ** 2).reshape(self.n_classes, self.n_classes)
ValueError: cannot reshape array of size 1284 into shape (4,4)

What can I do?

[marouamehri]

Hi @monniert

I want to generate a dataset having only text (4) and illustration (1) labels when I run syndoc_generator.py -n 10 --dataset_name syndoc -m

Should I only edit restricted_labels: [1, 4] in docExtractor/configs/syndoc.yml ?

Thanks for taking time to answer us.

[monniert]

Hi

• @nada0698: similarly, the traceback suggests there are too many labels in you ground truth, you should check there is the same number of labels in your prediction / GT pair examples
• @marouamehri: so you mean without the text border labels? using -m generates documents with illustration, text and text_border labels, I have just pushed the feature you need, you should use python src/syndoc_generator.py -n 10 --dataset_name toto -m -ntb to generate documents without borders. Otherwise, you can also generate as you did but not use border labels during training by specifying the appropriate restricted_labels argument

[monniert]

I assume the issue is fixed so I close it for now, please reopen if needed