werkzeug.Href behaves strange for names with colons about werkzeug

[ThomasWaldmann]

A patch for moin by a moin user is attached to that page:

http://moinmo.in/MoinMoinBugs/PageEditorAndPageNamesWithColons

[ThomasWaldmann]

After looking at werkzeug.urls, urllib, urlparse, I think the issue is the strange safe='/:' default of werkzeug.url_quote.

That default is different btw. from werkzeug._quote and also different from urllib.quote, which both have safe='/' default.

When reading Href docstring, it tells it is only dealing with path stuff (so no absolute urls with scheme: part are expected in the *path). But, if you have "foo:bar" there, it does not urlquote the colon, because colon is in the default safe list .

Thus, urllib.urljoin (which is MADE to deal with absolute URLs as 2nd argument) will get it wrong and think foo: is some scheme.

I suggest to change the werkzeug.url_quote() to use safe='/' default.

That will bring it in sync again with stdlib urllib.quote() again which also has safe='/' as default.

This will fix such issues as colons will become %3a then, so urljoin / urlparse don't think there is a scheme. There can't be a scheme, we are dealing with pathes, see Href docs.

[ThomasWaldmann]

From urlparse.urljoin(base, url) docs:

If url is an absolute URL (that is, starting with // or scheme://), the url‘s host name and/or scheme will be present in the result.

If you do not want that behavior, preprocess the url with urlsplit() and urlunsplit(), removing possible scheme and netloc parts.

[ThomasWaldmann]

diff -r c12c28282eab MoinMoin/support/werkzeug/utils.py
--- a/MoinMoin/support/werkzeug/utils.py        Sun Apr 17 19:47:04 2011 +0200
+++ b/MoinMoin/support/werkzeug/utils.py        Sun Apr 17 23:10:36 2011 +0200
@@ -643,6 +643,11 @@
         if path:
             if not rv.endswith('/'):
                 rv += '/'
+            _scheme, _netloc, _path, _query, _fragment = urlparse.urlsplit(path)
+            if _scheme or _netloc:
+                # we only wanted path, ... but got something that would be
+                # misinterpreted as having a scheme and/or a netloc. Fix this:
+                path = './' + path
             rv = urlparse.urljoin(rv, path)
         if query:
             rv += '?' + url_encode(query, self.charset, sort=self.sort,

[mitsuhiko]

Can you provide a patch with testcase for pulling?

[ThomasWaldmann]

I don't use git, sorry, thus no git pull from me.

Do you want a patch?

[mitsuhiko]

Mainly a testcase :)

[ThomasWaldmann]

    We only accept path components as arguments, it is not expected that
    you give a full url with scheme and netloc (or something that looks
    similar to one) when calling a Href instance:
    >>> href = Href('/base')
    >>> href('foo:bar')  # 'foo:' is NOT a scheme here!
    '/base/foo:bar'
    >>> href('//bar')  # '//bar' is NOT a netloc here!
    '/base/bar'
    >>> href('foo://bar')  # 'foo://bar' is NOT scheme and netloc here!
    '/base/foo://bar'

While the first one is as wanted and expected, the second and third test case show weird stuff / problems:

For '//bar', it loses a slash, while for 'foo://bar' it does not.

So I think we have found a design problem here:

If *path would only be expected to get single path COMPONENTS, we could expect that these components usually do not contain slashes. If they in fact do contain slashes, we maybe could escape them (because that would mean the component has a slash in its name, not that the component is meant to be two path components in a single string).

Needs more thinking...