Simple example fails with Python 2 about wsgi-intercept

[cdent]

Thanks for noticing this. I'm surprised it has not come up before. I believe your fix is correct. A few lines above your patch there's a section of code where the value in the dict is coerced into a native str (thus unicode in py3 but not in py2), but that's supposed to be true for the key as well. I'll be pushing up a patch, using your code and test as the basis, in a few minutes. If you're able to confirm that is working as expected, we can get that merged and a new release made.

[nboullis]

Hi Chris,
It does work with my trivial patch and with my “real” one.
To be honest, I used your code where the value is coerced into a native string to design my patch.
However, I’m not that confident it is correct for Python 2.
As I understand it, the make_environ function receives the request, as bytes, in the inp parameter. Here, the headers are supposed to be ISO-8859-1-encoded.
The line

v = str(v.decode('ISO-8859-1'))

first decodes the value to Unicode, and the re-encodes it to the system’s default encoding.
But it seems to me that this may be wrong when ISO-8859-1 is not the system’s default encoding.
Shouldn’t you rather do something like

if six.PY3:
    v = v.decode('ISO-8859-1')

to only decode the value for Python 3 (like you’re doing, a few lines above, for method_line)?
As for the HTTP_* headers, I guess you should use something like

if six.PY3:
    h = h.decode('ISO-8859-1')
environ['HTTP_' + h] = v

Cheers,
Nicolas

[cdent]

However, I’m not that confident it is correct for Python 2.

PEP 3333 is very confusing on this front, and that confusion compounds in make_environ as it is trying to behave like a server. And what happens in the actual headers versus the environ gets very messy.

The reason to always decode from ISO-8859-1 is because it allows us to produce an error (decode's default error mode is strict) if any of the headers produced by the requesting user-agent are not valid ISO-8859-1. This allows a test (or anything else) using wsgi-intercept to error out early, indicating that the test (or any of the libs used to create the test) itself is not valid.

It's debatable if this should be the job of the fake server in wsgi-intercept, but it is the pattern that's already been set with the value handling, so may as well also be true for the key handling.

The guarded method line is a special case that comes about as a result of it potentially coming in differently depending on the caller. I can't remember the details but cf94bc1 was the original change. In any case the method line is handled differently throughout the stack from the headers.

Yes, totally bewildering, and as a result the tendency in wsgi-intercept is to tweak to be maximally capable and minimally invasive, adjusting as we learn new things or encounter new bugs. In this particular instance you found a way in which things are broken, and the potential fix doesn't appear to add more breakage, so until it does expose more breakage it is probably okay to go with it.

There's more stuff related to this (but more in regard to responses than requests) in 34eb639 and #43

[nboullis]

Thanks for your answer.
I get your point about the need to check that the headers are valid ISO-8859-1 strings, and about you will to keep your changes minimally invasive.
Cheers,
Nicolas

[cdent]

fix has been released as 1.10.0