Comments (6)
Original comment by [email protected]
on 2 Jul 2012 at 9:48
- Changed state: Accepted
- Added labels: ****
- Removed labels: ****
from go-html-transform.
I'm not sure the correct behavior here is to ignore the end tag. My reading of
the html5 spec is that this is invalid html5. However I can see how using this
library for scraping pages could be useful.
Perhaps I should use a flag to govern whether this should be strict html5
parsing or best-effort forgiving parsing. I'm still thinking about it.
Original comment by JeremyMZHS
on 18 Jul 2012 at 8:20
- Added labels: ****
- Removed labels: ****
from go-html-transform.
Yes, it's invalid HTML. However, the HTML5 spec defines in painful detail what
is supposed to happen in that situation. See
http://www.whatwg.org/specs/web-apps/current-work/multipage/tree-construction.ht
ml#adoptionAgency
The HTML5 spec standardizes the handling of bad HTML, so that user agents all
do it the same way. It is a huge pain to implement properly, but parsers that
don't follow the spec should not be called HTML5 parsers.
Original comment by [email protected]
on 7 Jan 2013 at 7:24
- Added labels: ****
- Removed labels: ****
from go-html-transform.
As a test, I wrote a simple Go program to parse web pages and report errors.
Here are a few major sites. The quality of HTML out there is amazingly low, as
anyone involved in web crawling knows. An HTML5 parser has to handle all the
error cases, or it's just a toy. Some examples:
C:\projects\go>go run src\examples\htmlreaderdemo\htmlreaderdemo.go
HTML reader demo.
Enter URL: http://code.google.com
Error parsing 'http://code.google.com': Parse error: Strange characters in end
tag: [:] switching to BogusCommentState
(Note: This fails because "code.google.com" has XML namespace tags in an HTML
document. The document DOCTYPE probably should be XHTML, but it isn't.
<g:plusone> is not valid HTML. There's also trouble with the character set
header disagreeing with the actual document content. Fail.)
Enter URL: http://www.go.com
Error parsing 'http://www.go.com': Parse error: NotSameTag: End Tag does not
match Start Tag start:[td] end:[div]
(Note: Disney's home page. W3C validator finds 79 errors.)
Enter URL: http://www.gm.com
Error parsing 'http://www.gm.com': Parse error: NotSameTag: End Tag does not
match Start Tag start:[head] end:[div]
Enter URL: http://www.ford.com
Error parsing 'http://www.ford.com': Parse error: NotSameTag: End Tag does not
match Start Tag start:[head] end:[body]
Enter URL: http://www.intel.com
OK.
Enter URL: http://www.iana.org
OK.
Enter URL: http://www.nyt.com
OK.
Original comment by [email protected]
on 7 Jan 2013 at 8:14
- Added labels: ****
- Removed labels: ****
from go-html-transform.
I'll take a stab at these later this week.
Original comment by JeremyMZHS
on 16 Jan 2013 at 9:43
- Added labels: ****
- Removed labels: ****
from go-html-transform.
I finally got around to porting go-html-transform over the the exp/html parser
in go tip. h5 was always a stop-gap for me while they got that parser
shipshape. This bug should now no longer apply.
Original comment by [email protected]
on 28 Jan 2013 at 5:10
- Changed state: Fixed
- Added labels: ****
- Removed labels: ****
from go-html-transform.
Related Issues (19)
- [deleted issue]
- Faster selector matching using indexes HOT 2
- exp/html moved to net subrepo HOT 2
- undefined: h5.New HOT 6
- Remove dependency for syscall package, so that GAE apps can import this lib. HOT 3
- List is reassigned instead of appended in Chain.Find (should probably be a set though) HOT 1
- Full CSS parser.
- Missing return types HOT 3
- Go sub repository import pathes are changed HOT 2
- Use new import paths for go sub directories HOT 2
- go1 tag missing import fixes HOT 1
- transform.Replace panics on root nodes HOT 1
- Memory leak HOT 3
- transform.Replace fails when selected node's parent is non-nil HOT 5
- Unclosed tags are not supported HOT 3
- Err on tag namespace HOT 2
- Descendent selectors not working as expected HOT 2
- De-referenced nil from parse, inside bogusCommentHandler HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from go-html-transform.