spinoco / protocol Goto Github PK

In my application, I get requests from mobile browsers that specify referrer urls with some http scheme, for example

Referer: android-app://com.google.android.gm

This will not parse and all users using this mobile browser can't use the application. What do you think about adding a „generic scheme” or some other fallback variant? I may be mistaken, but I think the spec is open in that regard (allowing any alphanumeric value, regarding this spec).

this could be done with compiletime evidence
https://github.com/Spinoco/protocol/blob/master/http/src/main/scala/spinoco/protocol/http/header/DefaultHeader.scala

e.g. shapeless.Typeable

Observation

Using 0.1.7 to make a http cliient request to the Slack API server I see:

java.lang.Throwable: Failed to decode http response :Headers/Expires: Invalid format :  Mon, 26 Jul 1997 05:00:00 GMT : For input string: "mon, 26 jul 1997 05:00:00 gmt"
	at spinoco.fs2.http.HttpResponse$$anonfun$fromStream$2$$anonfun$apply$10.apply(HttpRequestOrResponse.scala:310)
	at spinoco.fs2.http.HttpResponse$$anonfun$fromStream$2$$anonfun$apply$10.apply(HttpRequestOrResponse.scala:308)
	at fs2.Stream$$anonfun$flatMap$1.apply(Stream.scala:151)
  ...etc...

Investigation

Mon, 26 Jul 1997 05:00:00 GMT is not a valid date in the sense that 26 Jul 1997 was a Saturday.
However, the HTTP Spec allows for this:

HTTP/1.1 clients and caches MUST treat other invalid date formats, especially including the value "0", as in the past (i.e., "already expired").

Expected Behaviour

Hmm.... not an exception. I'm not making use of the expires header, so don't have an opinion on what the type of the value should be.

Hi,
from some Android device running Chrome 73, the following Accept header is sent

Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3

The parser fails on the last v= part, saying expecting a q= instead:

Decoding of the request header failed: Headers/Accept/6: expected constant BitVector(16 bits, 0x713d) but got BitVector(16 bits, 0x763d)

I don't know what v= means. The user agent string is Mozilla/5.0 (Linux; Android 7.0; Lenovo TB-7304F) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.90 Safari/537.36 and I'm using protocol-http version 0.3.12.

Hi,
I wrote a little http server using fs2-http and then a client to consume this service with fs2-http, too. But the client failed to parse the response generated from my fs2-http endpoint. The problem was, that I was too lazy to think about the reason line in the http status line and put an empty string in it:

HttpResponse(
      HttpResponseHeader(
        status = status,
        reason = "",
        headers = Nil),
      Stream.empty
    )

Most other http parsers I used with this endpoint (like browsers and Elm http) parsed the response without problems, but the HttpResponseHeaderCodec expects a reason after the status code. Once I used reason = status.label in the above snippet, my problems were solved. But it took a while to figure that out :-)

So my question is, whether we can improve the api in that it either is not possible to specify an empty reason or it I would also be fine with throwing an exception. Or make the reason line optional in the response parser? (I'm not sure about the exact spec here)

Uri has no facilities to render to string. That's unfortunate both for debugging and serialization

scodec is now published for 2.13. so this project could move as well.
I could offer a help with it, if you are interested

scala> val x = "http://x.com/123?a=1&b=2;c=3"
x: String = http://x.com/123?a=1&b=2;c=3

scala> Uri.parse(x)
res0: scodec.Attempt[spinoco.protocol.http.Uri] = Failure(1: Cannot decode query parameter into key-value pair: List(b, 2;c, 3))

The above is compliant according to

• https://tools.ietf.org/html/rfc7230#page-82
• https://tools.ietf.org/html/rfc3986
• https://tools.ietf.org/html/rfc6570

and I've definitely seen URLs like this a lot over my career

?a=1&b=foo;bar;baz;gaz=daz&c=lolz

(not saying I endorse using them... but we have to work with what we're given...)

In both cases the correct parsing of the values is 2;c=3 and foo;bar;baz;gaz=daz respectively, with userland expected to deal with further parsing of the semicolons. I've also seen URL encoded & used as the separator at this layer.

Would it be hard to support this? Where should we start looking?

It seems the inverse works, i.e. constructing a Uri with these expected values and then .stringifying them.

Hello,

I'm coming from fs2-http (thanks btw for both!) and some of my requests didn't made it to my server. I found two things. For example a trailing slash is required:

scala> spinoco.protocol.http.Uri.parse("http://bla.com")
res5: scodec.Attempt[spinoco.protocol.http.Uri] = Failure(Expected value with optional parameter, got none)

scala> spinoco.protocol.http.Uri.parse("http://bla.com/")
res6: scodec.Attempt[spinoco.protocol.http.Uri] = Successful(Uri(http,HostPort(bla.com,None),Path(true,false,WrappedArray()),Query(List())))

Or, query parameters must have a value:

scala> spinoco.protocol.http.Uri.parse("http://bla.com/?a=")
res7: scodec.Attempt[spinoco.protocol.http.Uri] = Failure(0: Expected tuple, got List(Right(a)))

scala> spinoco.protocol.http.Uri.parse("http://bla.com/?a=1")
res8: scodec.Attempt[spinoco.protocol.http.Uri] = Successful(Uri(http,HostPort(bla.com,None),Path(true,false,WrappedArray()),Query(List((a,1)))))

Does this make sense, or am I missing something here?

As discussed on Gitter, cookie values can contain equals signs. This leads to a runtime exception. For example, using the echo service in the fs2-http readme:

$ http :9090/echo 'Cookie:name=value=1'
HTTP/1.1 400 Bad Request

...results in:

[ERROR] java.lang.Throwable: Decoding of the request header failed: 
    Headers/Cookie: Expected tuple, got List(Right(name), Right(value), Right(1))
[ERROR] 	at spinoco.fs2.http.HttpRequest$.$anonfun$fromStream$2(HttpRequestOrResponse.scala:218)
[ERROR] 	at fs2.Stream.$anonfun$flatMap$1(Stream.scala:148)
...

A "real world" cookie that has this structure is Google Analytics __utmz.