utf8 in parse_qs, application/x-www-form-urlencoded about mochiweb

[etrepum]

mochiweb does not use "unicode strings" anywhere, even though it is using lists of integers they are treated as lists of bytes (more like iolist() than string()). At the time mochiweb was written, Erlang binaries were a lot slower, and they were not supported in HTTP header decoding.

You probably shouldn't fork the code and it can't be fixed without drastically changing the APIs, which isn't something that makes sense at this point. If you want a cleaner all-binary API you'll want to switch web servers. A "fixed" mochiweb (regardless of whether unicode strings or binaries are used) wouldn't be compatible with webmachine either.

[defunerik]

I found a google group posting from a couple of years back where it was explained.

I realize that there is a history there, and, now, that this behavior is intentional.

However, there is also confusion. Take webmachine, for instance. Webmachine utilizes parse_qs directly to instantiate the url query field in the request object it creates. It then specifies in several places that the keys and values of the proplist that results from parse_qs are "strings". Well, if you decide not to use your own knowledge of what a Erlang string is, since that would be too sensible or you have hit a bug, you might instead look up the webmachine definition (nonstandard by their own admission):

"a list() with all elements in the ASCII range".

Really? Is that 7 bit standard ASCII or US-ASCII? Or 8 bit "extended" ASCII?

Of course, nobody is telling me that I need to treat these as strings, so I can use than as lists of bytes.

[etrepum]

At the time mochiweb was written, Erlang didn't even support Unicode except for a few functions hidden away in xmerl. Strings were either latin1 or bytes of some other kind, and since these are largely compatible no real distinction was made. The documentation you've found that says ASCII is simply wrong.