Parse error in CSS 2.1 test for escaped url() functional notation about css-validator

[ylafon]

invalid escapes and non [a-zA-Z] escapes are not processed so that the regular tokeniser can do its job. Current state is at https://github.com/w3c/css-validator/blob/functionescape/org/w3c/css/util/UnescapeFilterReader.java and seems to work well (will do more tests before merging).

[ylafon]

So doing it properly is pushing JavaCC to its limit, which is the size of generated code. Will have to do a workaround...

[dd8]

This might be useful: We use a C++ parser generator for our CSS stuff, but use the https://www.w3.org/TR/css-syntax-3/#consume-url-token algorithm to parse url() directly from the input stream when we hit a 'url(' token. We let let the parser continue from the current position in the input stream from the end of the url token.

There might be a way of doing something similar with the JavaCC input stream

[ylafon]

I was thinking of an FilterReader in charge of doing all the unicode unescaping for [a-zA-Z] as it is used in most constructions (url(), var() and other functions, @​rules etc...)

[dd8]

That could work.

One potential issue is dealing with invalid escapes - for example if you hit an invalid escape in the middle of url() you need to return the baduri token in https://www.w3.org/TR/2011/REC-CSS2-20110607/grammar.html#scanner or run the 'consume remnants of bad url' algorithm in https://www.w3.org/TR/css-syntax-3/#consume-url-token

[ylafon]

Fixed by #349