Oddity in "title"; false positives for German about vmcp-upconversion

[LucasHorseshoeBend]

The "title" issue and the false positive for German are probably distinct issues.
The title that displays in
http://vmcp.conaltuohy.com/xtf/view?docId=tei/Mueller%20letters/1850-9/1858/58-10-07a-final.xml
is disconcerting, as it is counterintuitive to have to click upon [...] to open the file
There are two others with the same feature in 1858 files:
http://vmcp.conaltuohy.com/xtf/view?docId=tei/Mueller%20letters/1850-9/1858/58-11-00-final.xml
and
http://vmcp.conaltuohy.com/xtf/view?docId=tei/Mueller%20letters/1850-9/1858/58-12-00a-final.xml

but I can't pick up a common feature of those files in Word.
There are eight 1858 files coded German that do have German text that behave as expected.
The three identified above are the only files in 1858 where this issue arises.

I did spot checks:
1840s folders, no such cases when German selected, nor when English selected

1868 folder, one case of the problem when faceted as English, and is in English,
http://vmcp.conaltuohy.com/xtf/view?docId=tei/Mueller%20letters/1860-9/1868/68-12-06-draft.xml
BUT the same file appears with same "title" when I select the in German facet!!

So there is something odd about the recognition of languages also.

[LucasHorseshoeBend]

The resaving did not eliminate the "title" as [...]
the four files all report as being both English and German
I do not understand this behaviour.

[Conal-Tuohy]

Thanks for your detective work, here @LucasHorseshoeBend ; based on your analysis I have got to the bottom of the missing title and the language mis-classification, and they are indeed related, as you surmised.

The immediate cause of the problem in 58-10-07a-final is that the letter contains an empty trailing paragraph which is styled with the style t-letter.

The chain of events leading from that empty paragraph to the text being classified as German and also having a missing title is this:

1. The paragraphs whose style names begin with t- are assumed to be translations (into English), and the remainder of the letter is, by elimination, assumed to be in German. So that single blank t-letter para is what causes all the text of the letter to be tagged as German.
2. Because XTF irritatingly requires every TEI document to have a title, which these letters don't generally have, my pipeline has a step which adds a heading which it derives from the opening lines of the letter (an "incipit") suffixed with an ellipsis. In order to give the documents English language titles, the incipit includes only text which has been tagged as English, but in this case the only English language paragraph is that empty t-letter paragraph, so the incipit also ends up empty.

I think that I can fix the whole thing by a small tweak to the translation-recognition step so it ignores paragraphs which are styled as a translation if they are empty.

[LucasHorseshoeBend]

Thanks Conal A tweak if easy to do would be a welcome safety net. Now that I know what to look for I will be able to correct the issues when found. Interestingly the empty trailing t-letter paragraph in 58-10-07a did not show up in the Word style mapping tool I use to pick up departures from the agreed styles. I have put a copy of 58-10-07a into the quarantine folder, and named it 58-10-07a-Title Prob for you to use as a test file. I have looked to see whether I can pick up issues where the language designation is falsely "German" but without the "title" problem. I have not yet tried to exhaust the set, but I have found one quickly, a copy of which I have also put into quarantine with the same empty t-letter paragraph fault left intact: 85-09-21e-False German, but I have corrected it in the main set. A second case was more interesting, where an empty t-letter paragraph had been used to insert spaces below headings, rather than using the standard extra space style we created for such usage. It is detected in the facets correctly as German, but has a proper title. I have now replaced those empty t-letter paragraphs by paragraphs coded as extra space. A copy of that file before correction is also in the quarantine folder, as 88-02-07 incorrect t-letter usage Because I would like to have the files as clean as possible, I have removed the offending empty paragraphs in my example cases: 58-10-07a-final 58-11-00-final 58-12-00a-final 68-12-06-draft 85-09-21e 88-02-07 Best wishes Arthur

…

On 23 Feb 2021, at 05:41, Conal Tuohy ***@***.***> wrote: Thanks for your detective work, here @LucasHorseshoeBend <https://github.com/LucasHorseshoeBend> ; based on your analysis I have got to the bottom of the missing title and the language mis-classification, and they are indeed related, as you surmised. The immediate cause of the problem in 58-10-07a-final is that the letter contains an empty trailing paragraph which is styled with the style t-letter. The chain of events leading from that empty paragraph to the text being classified as German and also having a missing title is this: The paragraphs whose style names begin with t- are assumed to be translations (into English), and the remainder of the letter is, by elimination, assumed to be in German. So that single blank t-letter para is what causes all the text of the letter to be tagged as German. Because XTF irritatingly requires every TEI document to have a title, which these letters don't generally have, my pipeline has a step which adds a heading which it derives from the opening lines of the letter (an "incipit") suffixed with an ellipsis. In order to give the documents English language titles, the incipit includes only text which has been tagged as English, but in this case the only English language paragraph is that empty t-letter paragraph, so the incipit also ends up empty. I think that I can fix the whole thing by a small tweak to the translation-recognition step so it ignores paragraphs which are styled as a translation if they are empty. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#40 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AF3IGTWRW5RG65HQXH25YFLTAM52PANCNFSM4DIUNJCQ>.

[Conal-Tuohy]

Cheers @LucasHorseshoeBend; since it was a trivial tweak I just went ahead and did it.

[LucasHorseshoeBend]

Thanks Best wishes Arthur

…

On 24 Feb 2021, at 06:03, Conal Tuohy ***@***.***> wrote: Closed #40 <#40>. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#40 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AF3IGTSWIEED4LXAIL3ZWI3TASJDVANCNFSM4DIUNJCQ>.