matze-dd / tex2txt Goto Github PK
View Code? Open in Web Editor NEWA LaTeX filter with Python
License: GNU General Public License v3.0
A LaTeX filter with Python
License: GNU General Public License v3.0
The replacement collection in variable parms.display_math works
quite well, if German is the main language. Requirements for replacements are summarized in the script in function set_language_de().
Till now, we could not yet select replacements that work equally well
with the English version of LanguageTool. For example, sensitivity is not good with the collection provided in function set_language_en() in these cases:
Currently, the central substitution function mysub() operates on a text string and an array of line numbers.
Instead, one could use an array of character offsets: for each character in the current text string, the corresponding array entry would indicate the position of that character in the input file.
Instead of storing line numbers with option --nums, one would write a file with character offset numbers.
With an adapted output filter for the proofreader, one then could also display corrected column numbers.
Bugs:
Inside of
leads to a warning. It is unclear, whether a macro is expanded and which text results from it.
For the same reason, in displayed equations, inclusion of & and \\ in {} braces will lead to warnings.
A problem similar to the one in script function split_sec() occurs in general. Assume that \xyz is a macro without arguments that yields an empty string and is expanded after \textcolor. Then
\textcolor{red}{This\xyz} is a problem.
will create the script output
Thisis a problem.
Looking for a preferably general solution.
Things like
\begin{align}
\input{equ.tex} % contains only equation body
\end{align}
are not treated correctly.
The Python script shell.py sometimes marks too much.
An extreme example is (\texorpdfstring skips its second argument)
\texorpdfstring{Thisx}{is a problem} is OK.
Here, only 'Thisx' should be highlighted.
Instead, 'Thisx}{is a problem}' is marked.
The checks from issue #5 still can fail if something like that is used in a heavily nested manner:
\begin{\xxx{itemize}}
We do not plan to catch such a case.
The delimiter & in displayed equations will always create white space in the text output.
With tex2txt.py --lang en ...
, the LaTeX input
We know that
\[
f(n) = 0 \text{ for all } n.
\]
this completes the proof.
currently results in this plain text version:
We know that
U for all V.
this completes the proof.
It seems that the dot at 'V.' is not recognised as sentence splitter by LanguageTool (LT), since it might be the acronym of a first name. Consequently, LT will not complain about the lower-case 'this' starting a new sentence.
According to some experiments, the following settings for maths replacements are more appropriate in English texts.
parms.inline_math = ('B-B-B', 'C-C-C', 'D-D-D', 'E-E-E', 'F-F-F', 'G-G-G')
parms.display_math = ('U-U-U', 'V-V-V', 'W-W-W', 'X-X-X', 'Y-Y-Y', 'Z-Z-Z')
Now, LT's sensitivity seems to be almost as good as for German texts with the current replacement collections ('D1D', 'I1I', ...). Word repetitions due to missing interpunction in equations and missing white space in connection with \text{...} parts are detected as before.
Still, there is at least one difference to the German version. In the following snippet, the missing dot is not detected in the English variant. LT does not complain about the capital 'This'.
We know that
\[
f(n) = 0 \text{ for all } n
\]
This completes the proof.
But LT also won't generate a message for
This Is a pity.
Pairing of [] brackets is checked without recognition of enclosed
{} braces.
Since the function modifies members of the global objects 'defs' and 'parms', as well as function variables like 'text_add_frame', it may only be active once at each point in time.
When I use the python script of tex2txt in combination with shell.py and use LanguageTool in version 5.1 some random paragraphs are ignored and they do not appear in the html output anymore.
I use the following command:
python3 shell.py --html --language de-DE chapter-4.tex > report-4test.html
I have not found any pattern for this so I can't provide an example for this issue. With LanguageTool 4.7 it does work, so I don't know what the problem is. I would like to use 5.1 because there are some updates and more grammar rules included.
Thanks and regards
Matthias
The Python script is only structured by comments.
It wildly mixes definitions of variables and functions with statements that actually perform text replacement operations.
This should be improved, for instance by usage of classes.
The handling has to be restructured.
Assume declaration
Macro(name='swap', args='AA', repl=r'\2\1')
Then the input
\swap
{A}
{B}
correctly leads to output 'BA', but the corresponding line number from option --nums is '3+' instead of '2+'.
This only happens on swapping of arguments, and it apparently has been introduced with the simplification of mysub() in release 1.5.6.
There is no line-number tracking inside of parsed displayed equations.
Instead, all corresponding lines in the output text carry the line number of the opening \begin{...}.
Thus, error location may be cumbersome for large equation environments.
A similar problem as in issue #2 appears with mulit-line arguments of macros, for instance in:
\textcolor{red}{
XX
YY
ZZ
}
All of the second argument is related to the line numer of the opening \textcolor.
Via the file from option --repl, one can insert character sequences
internally used for coding of verbatim text.
--> reject inclusion of raw % signs (escaped version \% is resolved
afterwards)
Since \verb has to be handled at the very beginning, a macro declaration
like Simple('textbackslash', r'\\verb?\\?')
does not work.
Similarly, something as
\comment{\begin{verbatim}}
...
\comment{\end{verbatim}}
will not work.
In the section on implementation issues of README.md, checks for nesting depths of braces and environments have been mentioned. They do fail, if braces or environment frames are inserted via expansion
of a declared macro.
One could reject recursive declaration of macros, but this would reduce
flexibility.
Short-hand notation without {} braces is practical for macros just taking the next character as argument.
Although the implementation of text-mode accents supports both this and the style with braces, declaration of such macros is not possible via Simple() or Macro().
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.