Comments (6)
Excellent point. Yeah, I was wondering about that too...
Here's some documentation on the JSON-LD API. The gist seems to be an API that "restructuring data according to the defined transformations" using methods compact
, expand
, andflatten
. The end result is still JSON-LD, but reformatted structurally. extruct's JsonLdExtractor
is not implementing that API.
I found an example of expand
(example 2 going to example 3). That API looks like it'd be really useful to have to transform JSON-LD into a more standardized format before processing, but that's a separate issue. expand
would make the output closer (the same, even?) to the output from RDFaExtractor
.
I guess that note is there because after processing JSON-LD into other JSON-LD, the end result may or may not be in an HTML context. I don't think it applies to the output of extruct.
from extruct.
I’m not sure what the expected behavior is.
The standard that you link says:
- That only
<script>
,</script>
,<!--
and-->
need escaping. - That the escaping will remain after processing.
To me it sounds like extruct is currently acting according to the standard.
I am unsure, though. Also, even in that case, I wonder if such decoding should be an optional feature at least.
from extruct.
The "Note" seems to imply encoding the "usual suspects" of HTML entities:
The reasoning behind it is specifically to avoid having <script>
and/or <!--
in the middle of the script
block without escaping. However, it seems they decided to cover all the characters commonly escaped when within an HTML attribute.
The standard is about rendering the JSON+LD into an HTML document, but extruct is for extracting the data back into python data structures so it should be doing the opposite. Right now extruct is not unescaping back into regular text.
Here's an example:
from extruct.jsonld import JsonLdExtractor
from extruct.xmldom import XmlDomHTMLParser
from lxml.html import fromstring
domparser = XmlDomHTMLParser(encoding='UTF8')
content = """
<script type="application/ld+json">
{
"@context": "http://schema.org/",
"@type": "Thing",
"name": "<script> <!-- &"'"
}
</script>
"""
tree = fromstring(content, parser=domparser)
jsonld = JsonLdExtractor().extract_items(tree)
print(jsonld[0]['name']) # expected output *should* be `<script> <!-- &"'`
from extruct.
I wonder what it means for us that it says there “the content will remain escaped after processing through the JSON-LD API”.
from extruct.
Is this the same issue?
jslde = JsonLdExtractor()
schema = jslde.extract(article.html)
Expecting ',' delimiter: line 7 column 54 (char 281)
from extruct.
@gk2go - The issue is about unescaping escaped HTML entities. The issue you're having is about malformed JSON because the person didn't quote the "
as either \"
or "
.
"headline": "Argentina designates Hezbollah "terrorist" organisation",
Please use https://search.google.com/structured-data/testing-tool/ to test structured data first before suspecting a bug with extruct.
from extruct.
Related Issues (20)
- Example from the README does not work any more
- Extruct not matching up with Schema.org structured data testing tool (Incorrect image Urls) HOT 3
- Some websites put meta tags outside the head. HOT 2
- Very slow extraction for specific string HOT 5
- LD+JSON outside HTML element HOT 1
- error extracting json-ld for validated json
- [suggestion] adding type hints? HOT 7
- Should not Depends on python3 (<< 3.7) HOT 6
- lxml.etree.ParserError: Document is empty HOT 5
- " in application/ld+json gives exception
- Consider switching from lxml's clean_html for enhanced security (and possibly performance) HOT 7
- Selectolax benchmarks
- Unable to get meta tag value from inside body
- SyntaxWarning invalid escape sequence '\s'
- Package breaking due to change in lxml HOT 2
- ImportError: cannot import name '_ElementStringResult' from 'lxml.etree' HOT 1
- chore: Remove Python 2 specific code
- feat: Add dependabot for github actions HOT 3
- cannot import name '_ElementStringResult' from 'lxml.etree HOT 2
- Latest release on PyPi (0.16.0) breaks with lxml>5.1.0: import extruct throws ImportError: cannot import name '_ElementStringResult' HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from extruct.