Comments (9)
Ok, there is a wealth of problems here.
Most of them are NOT an ucto problem but FoLiA issues.
folialint more or less issues the same messages. But more clear, not as a stack trace:
metPart.xml isn't a problem too, but:
> folialint zonderPart.mini.xml
zonderPart.mini.xml failed:
inconsistent text: node item(TEI.1.text.1.body.1.div1.1.p.3.list.1.item.1) has a mismatch for the text in set:default
the element text ='La reformation de l'instruction du Conseil, qui peut cependant agir selon qu'il est instruict.'
the deeper text ='1.'
foliavalidator in fact states the same, stripping the slack:
line 79: [InconsistentText] Text for ListItem, ID TEI.1.text.1.body.1.div1.1.p.3.list.1.item.1, class default, is inconsistent:
expected (after normalization): '1.',
got (after normalization): 'La reformation de l'instruction du Conseil, qui peut cependant agir selon qu'il est instruict.'
This is correct imho: the text of the structure element 'item' should be the composition of the texts from it's structure element children.
- for zonderPart the items text is the french text, and embedded is the label with text "1."
- for metPart the items text is empty at first, but it can be composed of the children 'label and 'part'
leading to composed text of: 1. La reformation de l'instruction du Conseil, qui peut cependant agir selon qu'il est instruict.'
This might come as a surprise :)
It would be insightful tot move the node in zonderPart at it's canonical position:
<item xml:id="TEI.1.text.1.body.1.div1.1.p.3.list.1.item.1">
<t class="default">La reformation de l'instruction du Conseil, qui peut cependant agir selon qu'il est instruict.</t>
<label>
<t class="default">1.</t>
<lang class="nld"/>
</label>
....
So this all needs some thought about wat you want to express....
Then there is the ucto problem..
ucto tries to tokenize the text of the 'label', (1.) which works and creates a paragraph with one sentence.
But a paragraph, NOR a sentence may be attached to a label.
This might be an oversight in the properties of FoLiA.
But is also raises an interesting idea:
Would it be a nice extension to be able to specify in ucto to skip certain FoLiA nodes ?
from ucto.
Wait.. this is not at all that complex. FoLiA has a generic "n" attribute which is used for labels in ordering. This is especially useful for list items. Example from the FoLiA documentation:
<head><t>My grocery list</t></head>
<list xml:id="example.list.1">
<item xml:id="example.list.1.item.1" n="A"><t>Apples</t></item>
<item xml:id="example.list.1.item.2" n="B"><t>Pears</t></item>
</list>
There is no <label>
element in FoLiA. The labels covered by n
are not considered text content subject to linguistic annotation, but more stylistic, which is I think precisely what we want.
from ucto.
Correction: we do have a label element it seems!.... It's not in the documentation though, so that's bad. I'll see if I can figure out what happened.
from ucto.
Thanks! I can probably use the n=... option to encode the labels in my current Nederlab batch.
I was looking for a translation for the corresponding TEI tag, and took label from the folia schema as a possible translation of the TEI tag for list labels. In TEI, label is an element and may contain some markup (http://www.tei-c.org/release/doc/tei-p5-doc/en/html/ref-label.html), so translation by n=... will not always work.
from ucto.
Well, the Label tag is there, but seems only halfway implemented.
I think that allowing <p> and <s> nodes in a <label> would be enough to do the trick.
But still remember: it's text is part of the document text then!
from ucto.
To clarify a bit: @JessedeDoes using the n attribute can be handy. And although it's name might suggest otherwise, it may contain any kind of text (within the XML limits for attributes).
On the downside: the content of the n attribute is opaque to the libraries. It is not taken into account by ucto, text retrieving functions etc.
from ucto.
@proycon : so what is the status of label now?
do we want to allow <p> en <s> nodes (or any abstract structure) in them?
if yes ==> a new FoLiA issue would be handy
in no ==> close the issue
from ucto.
Issue still open indeed, haven't had time to document this properly yet (bit of a lower priority). s
might make some far-fetched sense for a label, but p
probably not.
from ucto.
Ah wait, this is indeed a FoLiA issue rather than an ucto one, new issue makes sense. Closing this one.
from ucto.
Related Issues (20)
- passthru mode should not be combined with other language options
- ucto creates invalid folia HOT 2
- Update debian package for v0.21
- Byte-order mark followed by space or tab results in Folia error HOT 7
- is this correct handling of FoLiA paragraphs with embedded Part nodes? HOT 4
- -T full option produces invalid FoLiA HOT 1
- Tokenization of t-style element that has font_typeface Feature HOT 19
- Validation of ucto output fails due to space character in FoLiA output from Piereling HOT 7
- ucto sometimes misses out on the <t> for <p> HOT 3
- IDs in UCTO in concert with tei2folia HOT 3
- Language detection default for 'unknown' language HOT 9
- Ucto with 'detectlanguages' : failure HOT 3
- remove some deprecated options HOT 6
- Ucto aborts on FoLiA creation
- Question: Concatenating word parts at soft hyphens HOT 77
- Develop a tokenizer for Premodern Slavic
- Implement (soft)hyphen handling in Ucto analogues to foliautils
- Ucto fails on some UTF-8 characters in tei2folia generated FoLiA HOT 12
- add a batch option HOT 6
- Setting -m in container does not supress punctuation-based sentence splitting HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ucto.