erykdarnowski / ts-test-extractor Goto Github PK

Description

The currently implemented quick & dirty fix, doesn't fully resolve the issue and in fact introduces a new one (when a word starts with a diacritic, it gets combined with the word previous to it).

The proposed solution is a switch from PyPDF2 to using fitz from PyMuPDF.
Another thing is the need for encoding="utf-8" when writing the out.txt file on Windows.

Description

Add the encoding parameter when writing the JSON file, like was done before:
...('ts.json', 'w') as... -> ...('ts.json', 'w', encoding='utf-8')...

Description

Add an error message to the part of the script that extracts pure text from PDF files, when there are none.

Description

Actually, don't really like the all caps.

TODO

• Info about input files + included example + that they should be numbered
• Run instructions
• That the regex patterns and formatting fixes are really specific so this repo won't be really usefull cause it was made with a really specific usecase in mind (it will most likely be only usefull as inspiration).

TODO

*Make sure each expression gets 171 matches

• Add clean up
  • Titles: /(\s\n){0,2}(^Test\sz\s.*?)(\s\n){2,}/gms (replace)
• Add extraction
  • Tasks: /(^[0-9]{1,}\.\n?\s?)(?![0-9])(.*?)(?=\n?\s\n[A-Z]\.)/gms (G2)
  • Answers (A, B, C): /(^[A-Z]\..*?)(?=\s\n?Odp\.)/gms (G1)
  • Correct answers: /(?<=^Odp\.)(.*?)([A-Z])(?=\.?)/gm (G2)
  • Answer expls.: /(^Odp\.(\:|\s)\s?[A-Z])(\.?\s+\n?)(.*?)(?=((^[0-9]{1,}\.\n?\s?)(?![0-9]))|(\s\n){3})/gms (G4)

Resources

• https://regex101.com/
• https://docs.python.org/3/library/re.html

Description

Basically compare the output JSON file of the script with the PDFs to make sure that the tasks, answers, correct answers and so on are correctly grouped / joined / aligned. In addition could also look for formatting stuff to fix that was missed previously.

Description

Add a PDF file with some sample syntax resembling the tests, to show how the script works.

TODO

• Add venv instructions to README.md
• Implement (for all files at once)
• Fix issue with Polish diacritic characters

Resources

• https://www.geeksforgeeks.org/extract-text-from-pdf-file-using-python/
• https://stackoverflow.com/questions/70848337/how-to-read-multiple-pdf-from-a-folder-one-by-one

TODO

• Add check for misaligned values (based on lengths)
• Add formatting fixes
  • Answers (A, B, C): /\s(?=[A-Z]\.)/gm (replace)
• Implement writing to JSON file

Resources

• https://regex101.com/
• https://note.nkmk.me/en/python-split-rsplit-splitlines-re/#split-a-string-by-regex-resplit

Description

Fix this : -> :.

Description

For the last time...
Just change Instructions to Usage and remove info about performing setup.

Description

After messing around with the PDF files, I've noticed that when they get found by glob they don't have to be in order.