crowdhailer / ace

HTTP web server and client, supports http1 and http2

Ace

HTTP web server and client, supports http1 and http2

• Install from Hex
• Documentation available on hexdoc
• Discuss on slack

See Raxx.Kit for a project generator that helps you set up a web project based on Raxx/Ace.

Get started

Hello, World!

defmodule MyApp do
  use Ace.HTTP.Service, port: 8080, cleartext: true
  use Raxx.SimpleServer

  @impl Raxx.SimpleServer
  def handle_request(%{method: :GET, path: []}, %{greeting: greeting}) do
    response(:ok)
    |> set_header("content-type", "text/plain")
    |> set_body("#{greeting}, World!")
  end
end

The arguments given to use Ace.HTTP.Service are default values when starting the service.

Start the service

config = %{greeting: "Hello"}

MyApp.start_link(config, port: 1234)

Here the default port value has been overridden at startup

Raxx

Ace implements the Raxx HTTP interface. This allows applications to be built with any components from the Raxx ecosystem.

Raxx has tooling for streaming, server-push, routing, api documentation and more. See documentation for details.

The correct version of raxx is included with ace, raxx does not need to be added as a dependency.

TLS/SSL

If a service is started without the cleartext it will start using TLS. This requires a certificate and key.

config = %{greeting: "Hello"}
options = [port: 8443, certfile: "path/to/certificate", keyfile: "path/to/key"]

MyApp.start_link(application, options)

TLS is required to serve content via HTTP/2.

Supervising services

The normal way to run services is as part of a projects supervision tree. When starting a new project use the --sup flag.

mix new my_app --sup

Add the services to be supervised in the application file lib/my_app/application.ex.

defmodule MyApp.Application do
  @moduledoc false

  use Application

  def start(_type, _args) do
    import Supervisor.Spec, warn: false

    children = [
      {MyApp, [%{greeting: "Hello"}]}
    ]

    opts = [strategy: :one_for_one, name: MyApp.Supervisor]
    Supervisor.start_link(children, opts)
  end
end

Start project using iex -S mix and visit http://localhost:8080.

Testing

mix test --include ci:true

Will run h2spec as one of the tests.

If the h2spec is not specified in the environment variable $H2SPEC_PATH or on the $PATH, it will be downloaded into test/support/h2spec/ and executed.

Alternative HTTP servers

Other servers you can use for Elixir/erlang applications are Cowboy and Elli.

Of the three Cowboy is the most widely used. Both Elli and Cowboy are written in erlang, Ace is written in Elixir.

Elixir HTTP Benchmark - Ace vs Cowboy

ace's Issues

Exposing connection information to the application.

Certain applications require knowledge of the connection and not just the HTTP message that was sent. For example inspecting a client ssl certificate or blacklisting particular ip addresses.

Adding this to the request is not desirable because:

1 it conflates the transport information with the HTTP protocol
2 The request struct sent by the client should be identical to the request struct handled by the server. Assuming both client and server use Raxx.
3 It is also possible to have HTTP without a connection. Mailing Requests and Responses would work if anyone wanted to do that.

Option 1

Ace could have an entry in the process dictionary of the worker. The ace module could the expose several reader functions that work only in the context of a worker.

defmodule MyApp do
  def handle_request(_request, _state) do
    peer_ip = Ace.peer_ip()

    response(:ok)
  end
end

Somewhat ugly to use the process dictionary, however as all exposed functions are for derived properties the cannot be used to cause side effects.

Option 2

handle_request/2 (and all other callbacks) can have a handle_request/3 partner that takes request, channel, state with the default behaviour to fallback to the arity 2 implementation.

The downside of this approach is raxx is understanding both HTTP and some concept of the transport layer. e.g. would the channel object be an Ace.Connection or a Raxx.Connection.

Option 3

A server specific callback for connecting. e.g.

defmodule MyApp do
  use Ace.HTTP.Service, [port: 8080]

  @impl Ace.HTTP.Server
  def connect(channel, config) do # 1.
    Logger.metadata(client_ip: channel.ip)
    {:ok, config}
  end

  @impl Raxx.Server
  def handle_request(_request, _state) do
    response(:ok)
  end
end

Advantages this gives people an answer to what does init look like, it's replaced with connect.

There is a choice at 1. do we pass a connection or channel object. if channel then no error case should stop the complete connection, but the callback is executed in the same process as the handle_*. If connection then its only executed once for potentially many requests. but it would be possible to use it to stop the connection. i.e. blacklisted ips

handle restarting a listen socket

To put endpoints under a supervisor within the Ace application

hot code upload example

echo server,

switch state from {:message, message} -> %{message: message}

Support graceful close

Useful for blue green deploys, using docker/whatever

• https://github.com/fishcakez/acceptor_pool/tree/master/examples/telnet_echo
• phoenixframework/phoenix#1925

Bubble errors from startup

Suggestion from fishcakez

If there are two many errors when starting servers they should bubble. However once running then failure does not count towards error intensity

Send a 100 Continuation response

Update `Ace.HTTP` to Raxx 0.12.0

This version of Raxx replaces handle_request/2 with the combination of handle_headers/2, handle_fragment/2 and handle_trailers/2

Most changes will need to be made within Ace.HTTP.Handler,

• Handler.process_buffer should be split in to two functions. one to process the request head and one for the content.
• Handler.handle_packet needs to call mod.handle_headers with a request with body set to true or false before reading the request body
• Use content-length to decide if request has body, content-length of 0 is still no body.
• host header is set as authority key in Raxx.Request

NOTE: I think we should also give up on using Raxx.Verify. it was a nice idea to have a common set of tests that can be used across adapters, but I think at this point it is an extra level of indirection and complexity at the moment

@samphilipd If you have the time to pick this up great.

Follow up: can be subsequent PR's

• Support correct behaviour for keep-alive for both HTTP/1.0 and HTTP/1.1
• Start worker process for each request if supporting pipelining
• Remove headers that are for connection only from request struct passed to application]
• Remove reference to connection and transfer-encoding from raxx
• Add support for transfer-encoding in place of content-length, return error if both present.

Create a server behaviour

server behaviour callbacks and a __using__ macro with defaults.

property test

https://www.youtube.com/watch?v=ZW2Pl0bIN5U

https://vimeo.com/104498647

Implement push promise

• check stream state before sending,
• ensure create stream_id is for local peer
• start worker for stream

Ace.HTTP explodes on absolute URLs

See stacktrace:

Elixir.CaseClauseError

no case clause matching: {:ok, {:http_request, :GET, {:absoluteURI, :http, "localhost", :undefined, "/status"}, {1, 1}}, "User-Agent: check_http/v2.1.2 (monitoring-plugins 2.1.2)\r\nConnection: close\r\nHost: localhost:3006\r\nUser-Agent: Nested-Internal-HealthChecker\r\n\r\n"}

Note:

code is here

Ace/extensions/http(s)/lib/ace/http/handler.ex

Line 70 in 338775d

case :erlang.decode_packet(:http_bin, buffer, []) do

would need to handle the remaining cases here http://erlang.org/doc/man/erlang.html#decode_packet-3 in the case clause.

Return error for invalid headers

we want to return an error not raise one when headers are invalid.

Should be included when moving away from hpack as is

docker run --net="host" summerwind/h2spec http2/4.3 --port 8443 -t -k

Clarify `state` in `Ace.HTTP.Server`

Ace/lib/ace/http/server.ex

Lines 51 to 68 in e39bfbf

	def handle_call({:accept, {:tcp, listen_socket}}, from, state) do
	case :gen_tcp.accept(listen_socket) do
	{:ok, socket} ->
	:ok = :inet.setopts(socket, active: :once)
	state = %{state | socket: socket}
	{:ok, worker} = Supervisor.start_child(state.worker_supervisor, [:the_channel])
	monitor = Process.monitor(worker)
	state = %Ace.HTTP1.Endpoint{
	status: {:request, :response},
	socket: {:tcp, socket},
	# Worker and channel could live on same key, there is no channel without a worker
	channel: {:http1, self(), 1},
	worker: worker
	}
	GenServer.reply(from, {:ok, self()})
	:gen_server.enter_loop(Ace.HTTP1.Endpoint, [], {"", state})

Is the state from Line:51 the same state that's being overwritten on line on Line:59?

If yes, then line Line:55 is unnecessary ... same appears again further down.
This would also make :socket unnecessary in defstruct [:worker_supervisor, :settings, :socket]

add behaviour to example in README

ssl support

To match exactly the callbacks, but options to be passed might be different.

Ace.HTTP should return 400 if request is missing or has invalid/multiple Host header

See: https://tools.ietf.org/html/rfc7230#section-5.4

Unify HTTP/1.1 and HTTP/2 interface

Steps needed to unify interface from Ace (for HTTP/2) and Ace.HTTP (for HTTP/1.1)

• Use governor pattern with HTTP/2 connections
• Have single type of worker, ace_http can fetch from dependency
• Introduce Ace.Service that will serve both HTTP/1.1 and HTTP/2
• Use Ace.TCP (or not) same for both http1 and http2
• Run h2spec against service that upgraded
• Update Features list on the README
• Use Raxx 1.0
• Move helpers used only in endpoints to private funcs

Naming
Service -> collection of servers
Endpoint -> Server or Client process managing a single socket to a single connection to a single peer
Worker -> (OR Channel) ephemeral that exists to manage one HTTP exchange. Note they only exist because HTTP defines a stateless protocol. Other application protocols if they included stateful location information could direct messages from endpoint directly to target
Channel -> part of a connection, either stream in HTTP/2 or item in HTTP/1.1 pipeline, connects the two agents(OR actors) involved in the exchange.
Socket -> tcp or ssl socket

Helpful stuff

License, Travis, badges etc

Motivation post

Call for Assistance

why Ace
why Raxx

Plans

• Raxx on cowboy (new stream handlers)
• Ace design is to push purity so can be checked by session types
• Integration with APNS GRPC graphql

Roadmap

Hey @CrowdHailer ,

This is an awesome idea.
I have always wanted to learn how HTTP servers works, creating one with a functional language looks a good way to learn it.

A Roadmap with checkboxes maybe a good idea to make the contribution easy!

type specs + Dialyzer for Ace.HTTP

Ace core has typespecs we should extend Ace.HTTP to have the same

• typespecs
• mod docs

timeouts on inactive connections

If the server stops receiving messages it should timeout. Use the same pattern as genserver timeouts

Seems this can be simplified further

Ace/lib/ace/http/worker.ex

Lines 88 to 100 in 58d19db

	defp normalise_reaction(response = %Raxx.Response{}, state) do
	case response.body do
	false ->
	{[response], state}
	true ->
	{[response], state}
	_body ->
	# {[%{response | body: true}, Raxx.data(response.body), Raxx.tail], state}
	{[response], state}
	end
	end

Regardless of the value of response.body, you return {[response], state}

missing service tests

  test "start multiple connections" do
    {:ok, endpoint} = Ace.TCP.start_link({CounterServer, 0}, port: 0)
    {:ok, port} = Ace.TCP.port(endpoint)
    {:ok, client1} = :gen_tcp.connect({127, 0, 0, 1}, port, [{:active, false}, :binary])
    {:ok, client2} = :gen_tcp.connect({127, 0, 0, 1}, port, [{:active, false}, :binary])
    :ok = :gen_tcp.send(client1, "TOTAL\r\n")
    assert {:ok, "0\r\n"} = :gen_tcp.recv(client1, 0)
    :ok = :gen_tcp.send(client2, "TOTAL\r\n")
    assert {:ok, "0\r\n"} = :gen_tcp.recv(client2, 0)
  end
  
  test "will register the new enpoint with the given name" do
    {:ok, endpoint} = Ace.TCP.start_link({EchoServer, []}, port: 0, name: NamedEndpoint)
    assert endpoint == Process.whereis(NamedEndpoint)
  end
  
  test "there are n servers accepting at any given time" do
    {:ok, endpoint} = Ace.TCP.start_link({EchoServer, []}, port: 0, acceptors: 10)
    {_, _, governor_supervisor} = :sys.get_state(endpoint)
    assert %{active: 10} = Supervisor.count_children(governor_supervisor)
  end

drain connections

New governor cannot be interupted to properly drain connections

  test "drain connection pool" do
    {:ok, server_supervisor} = Server.Supervisor.start_link({EchoServer, :explode})
    {:ok, socket} = :gen_tcp.listen(0, @socket_options)
    socket = {:tcp, socket}
    {:ok, port} = Connection.port(socket)

    {:ok, governor_supervisor} = Governor.Supervisor.start_link(server_supervisor, socket, 1)

    # Establish connection
    {:ok, client} = :gen_tcp.connect({127, 0, 0, 1}, port, [{:active, false}, :binary])
    :ok = :gen_tcp.send(client, "blob\n")
    assert {:ok, "ECHO: blob\n"} = :gen_tcp.recv(client, 0)

    # Drain connections
    :ok = Governor.Supervisor.drain(governor_supervisor)
    assert [] = Supervisor.which_children(governor_supervisor)
    assert [_1] = Supervisor.which_children(server_supervisor)

    # New connection not made
    {:ok, client2} = :gen_tcp.connect({127, 0, 0, 1}, port, [{:active, false}, :binary], 100)
    :ok = :gen_tcp.send(client2, "blob\n")
    assert {:error, :timeout} = :gen_tcp.recv(client2, 0, 100)

    # Establish connection still available
    :ok = :gen_tcp.send(client, "blob\n")
    assert {:ok, "ECHO: blob\n"} = :gen_tcp.recv(client, 0)
  end

  def drain(supervisor) do
    Supervisor.which_children(supervisor)
    |> Enum.map(fn({_i, pid, :worker, _}) ->
      ref = Process.monitor(pid)
      true = Process.exit(pid, :shutdown)
      ref
    end)
    |> Enum.map(fn(ref) ->
      receive do
        {:DOWN, ^ref, :process, _pid, _reason} ->
          :ok
      end
    end)
    :ok
  end

Do some benchmarking V cowboy

Would help anyone wanting to run with this as it is not something Ive done before. For the moment here is just a list of resources

https://elixirforum.com/t/the-worth-of-us-creating-our-own-benchmarking-tests/5045/16?u=crowdhailer
https://gatling.io/docs/current/quickstart/
the-benchmarker/web-frameworks#58
https://github.com/tbrand/which_is_the_fastest
https://flood.io/
http://www.ostinelli.net/a-comparison-between-misultin-mochiweb-cowboy-nodejs-and-tornadoweb/
https://github.com/mroth/phoenix-showdown
https://littlelines.com/blog/2014/07/08/elixir-vs-ruby-showdown-phoenix-vs-rails

Startup checks that application has behaviour

Performance measurement

Mention Tobias

test connection information passed into init callback

should be of a form that is recognizable to raxx. i.e. %{peer: {ip, port}}

HTTP2 API

Ace will need an updated API to support HTTP/2.
I intend for this to be a pure functional interface, as is the case with Raxx (And the standard interaction with GenServer.
It must obviously support streaming and the stream must be routable based on headers alone.

I'm proposing a two step solution where:

1. a server handles routing to stream handlers
2. stream handlers provide callbacks for stages of a stream

example: simple request -> response.

Because the request consists of only headers the body flag is false.

# home_page.ex

defmodule HomePage do
  use Ace.HTTP2.Stream
  import Ace.HTTP2.Stream.Response

  def handle_open(request, response, _config) do
    if Request.accept?(request, "text/html") do
      response
      |> set_status(:ok)
      |> put_header("content-type", "text/html")
      |> send_data(render("home_page.html"))
      |> push(:GET, "/main.css", %{"content-type", "text/css"})
      |> finish()
    else
      response
      |> set_status(:not_acceptable)
      |> finish()
    end
  end

end

There too many possibilities for a simple return value to encapsulate all possibilities. Therefore we use the builder pattern.

example: streaming up a file

The request passed to handle_open has body flag set to true

# upload.ex

defmodule FileUpload do
  use Ace.HTTP2.Stream
  import Ace.HTTP2.Stream.Response
  
  def handle_open(request, response, _state) do
    file = File.open("/tmp.png")

    response
    |> Response.update_state(%{file: file})
  end

  def handle_data(data, response, %{file: file}) do
    :ok = File.write(data)
    response
  end

  def handle_end(_trailers, response, %{file: file}) do
    :ok = File.close(file)

    response
    |> set_status(:created)
    |> put_header("location", "/tmp.png")
    |> finish()
  end
end

example: streaming data from the server

# user_updates.ex

defmodule UserUpdates do
  use Ace.HTTP2.Stream
  import Ace.HTTP2.Stream.Response

  def handle_open(request, response, config) do
    EventSource.subscribe(self())

    response
    |> set_status(:ok)
    |> put_header("content-type", "text/event-stream")
  end

  def handle_info({:update, update}, response, _config) do
    data = serialize(update)

    response
    |> send_data(data)
  end

end

normalise_reaction

Ace/lib/ace/http/worker.ex

Lines 88 to 95 in a895cc0

	defp normalise_reaction(response = %Raxx.Response{}, state) do
	{[response], state}
	end
	defp normalise_reaction({parts, new_state}, _old_state) do
	parts = Enum.map(parts, &fix_part/1)
	{parts, new_state}
	end

Since we cannot control what is returned from a user's mod.handle_head, mod.handle_body or mod.handle_tail, we could use this catch all to throw a controlled exception informing that the expected return format is violated:

  defp normalise_reaction(_any, _state) do
    throw "Invalid return format"
  end

Looking at the complaints of crashes, maybe this might help narrow down the source of the crashes

Useful examples

Need some illuminating examples. perhaps

• static assets built at compile time.
• examples on top of raxx

Un-handled cases causes match errors.

hey, firstoff thanks for starting a pure elixir project for http/2 support. there are some great erlang libs but given that I'm new erlang/elixir I find this much more convienient.

I'm using Ace for writing a http/2 server to handle some of the requests that are coming in. During my tests, I found couple of cases which are not handled, at least not that well.
This results some of the ugly messages such as

10:08:29.740 [error] GenServer #PID<0.247.0> terminating
** (MatchError) no match of right hand side value: {:error, :closed}
    (ace) lib/ace/http2/connection.ex:242: Ace.HTTP2.Connection.do_send_frames/2
    (ace) lib/ace/http2/connection.ex:114: Ace.HTTP2.Connection.handle_info/2
    (stdlib) gen_server.erl:616: :gen_server.try_dispatch/4
    (stdlib) gen_server.erl:686: :gen_server.handle_msg/6
    (stdlib) proc_lib.erl:247: :proc_lib.init_p_do_apply/3
Last message: :timeout
State: {:listening, {:sslsocket, nil, {#Port<0.7082>, {:config, {:ssl_options, :tls, [{3, 3}, {3, 2}, {3, 1}], :verify_none, {#Function<8.18432334/3 in :ssl.handle_verify_options/2>, []}, #Function<9.18432334/1 in :ssl.handle_verify_options/2>, false, false, :undefined, 1,"<<REDACTED>>", :undefined, "<<REDACTED>>", :undefined, [], :undefined, "", :undefined, :undefined, :undefined, :undefined, :undefined, [<<192, 44>>, <<192, 48>>, <<192, 36>>, <<192, 40>>, <<192, 46>>, <<192, 50>>, <<192, 38>>, <<192, 42>>, <<0, 159>>, <<0, 163>>, <<0, 107>>, <<0, 106>>, <<0, 157>>, <<0, 61>>, <<192, 43>>, <<192, 47>>, <<192, 35>>, <<192, 39>>, <<192, ...>>, <<...>>, ...], #Function<2.18432334/4 in :ssl.handle_options/3>, true, 268435456, false, true, :infinity, false, :undefined, ["h2", "http/1.1"], :undefined, :undefined, true, :undefined, [], :undefined, false, true, :one_n_minus_one, :undefined, false, ...}, [active: false, mode: :binary, packet: :raw, reuseaddr: true], #PID<0.246.0>, :undefined, [reuseaddr: true, packet_size: 0, packet: 0, header: 0, active: false, mode: :binary], {:gen_tcp, :tcp, :tcp_closed, :tcp_error}, :tls_connection}}}, #PID<0.248.0>, %Ace.HTTP2.Settings{enable_push: false, initial_window_size: 65535, max_concurrent_streams: 10000, max_frame_size: 16384}}
10:08:29.741 [error] GenServer #PID<0.248.0> terminating

my log files are full of such messages which are the results for unhandled cases.

Here are the ones I've identified during timeouts:
defmodule Ace.HTTP2.Connection
def handle_info(:timeout, {:listening, listen_socket, stream_supervisor, local_settings})

Scenario#1

Connection close error because client has terminated connection w/o sending any data even before SSL negotiation.

Reproduce:

I encountered this while testing my server using cURL (poor me didn't know that curl doesn't support http/2). try

$ curl https://localhost:8443/ping 
<<hangs>>
[cntrl+c to terminate] 

the above will raise {:error, :closed} at :ssl.ssl_accept(socket) instead of :ok

Scenario#2

Connection close error when the connection is still open(?) on the client but gets terminated on the server before the timeout. When the client tries to send any frame after the termination you'd see another {:error, :closed}

Reproduce:
restart the server after initial request. Curiously, I've noticed this only in chrome.

Scenario#3

Protocol negotiation failed during SSL negotiation between client and server. For some reasons, a simple health check from AWS LB fails causing {:error, :protocol_not_negotiated}

the likely-hood of the above cases might be less nevertheless, should be handled by the lib. @CrowdHailer your thoughts?

Test protection against slow client attack

Ace Endpoint doesn't handle downed workers in some state correctly

GenServer #PID<0.568.0> terminating
** (FunctionClauseError) no function clause matching in Ace.HTTP1.Endpoint.handle_info/2
    lib/ace/http1/endpoint.ex:55: Ace.HTTP1.Endpoint.handle_info(
      {:DOWN, #Reference<0.2967040181.409993220.21312>, :process, #PID<0.772.0>, :shutdown}, 
      {"", %Ace.HTTP1.Endpoint{channel: {:http1, #PID<0.568.0>, 1}, keep_alive: nil, monitor: nil, socket: {:tcp, #Port<0.18548>}, status: {:complete, :chunked_body}, worker: #PID<0.772.0>}}
     )
    (stdlib) gen_server.erl:616: :gen_server.try_dispatch/4
    (stdlib) gen_server.erl:686: :gen_server.handle_msg/6
    (stdlib) gen_server.erl:636: :gen_server.try_handle_call/4
    (stdlib) gen_server.erl:665: :gen_server.handle_msg/6
    (stdlib) proc_lib.erl:247: :proc_lib.init_p_do_apply/3
Last message: {:DOWN, #Reference<0.2967040181.409993220.21312>, :process, #PID<0.772.0>, :shutdown}

Unified state and writable structure

We can change {:send, data, state} | {:nosend, state} with {state, writable}. If writeable is a binary or io_list it can be written to the socket. When :erlang.iolist_size(writeable) is equal to 0 then it is equivalent to no send.

Therefore both handle_packet/2 and handle_message/2 can both return the same writeable and state structure.

This is the first step towards eliminating all side effects and side causes and using a statemachine and comm library.

In the final case every callback has the following signature

callback(args) :: {{mod, config}, writeable, [other sends]}

Ace.HTTP responses from callbacks should look as close as possible to what the domain expects, response, chunked_response etc.

Ace.HTTP.Response.serialize(response) :: writeable
Ace.HTTP.Response.serialize(chunked_response) :: {:more, writeable}
# just call next
Ace.HTTP.Response.next_state(response) :: maybe(app)
Ace.HTTP.Response.to_outtray(chunked_response) :: {app, [conn:, writable, pid: "m1"] 

HTTP/2 connection docs

Notes from section 3 of rfc7540

Check keep alive status for subsequent requests

Should a response that was chunked be keep alive once finished

HTTP/2 left out

Items not needed for MVP

• push promise: A client may request not to receive push promises, a server can always not have any to push
• priority: priority request are guidlines and can be ignored
• Server settings, always assume the defaults
• flow control https://tools.ietf.org/html/rfc7540#section-5.2.2

handle callbacks can stop a connection

By default tcp connections should be closed by the client.
However we need the ability to terminate in the case of unresponsive clients.

Follow the same patterns as gen server

Unable to remove padding in h2spec test case

A warning is thrown for removing padding however the test does not fail. It is possible the frame is incorrect because it is offer the core of the test. should be investigated.

Run to reproduce

docker run --net="host" summerwind/h2spec http2/6.2 --port 8443 -t -k

Accessing connection PIDs

Is there a way to retrieve the current client connection PID in the handle_connect function so it can be put in the state variable, as I need to be able to be able to make use of it later to be able to send messages from the server to specific clients on demand, not simply reply to clients in reaction to incoming messages.

Apologies if this is not the right place to ask a question.

Plug stream handler

Make endpoint a supervisor

Ace.TCP and Ace.TLS start a genserver that is linked to some supervisors. This works as it has more functionality that to simply start workers, i.e. start the socket. However not being a supervisor means that introspectin in the observer is not done well

(FunctionClauseError) no function clause matching in :lib.is_op/2

We're seeing this occasionally in the logs. Doesn't seem to be correlated with anything:

(FunctionClauseError) no function clause matching in :lib.is_op/2

Exception(most recent call first)

Elixir.FunctionClauseError: no function clause matching in :lib.is_op/2
  File "lib/rested/router.ex", line 19, in Rested.Router.handle_head/2
  File "lib/ace/http/worker.ex", line 24, in Ace.HTTP.Worker.handle_info/2
  File "gen_server.erl", line 616, in :gen_server.try_dispatch/4
  File "gen_server.erl", line 686, in :gen_server.handle_msg/6
  File "proc_lib.erl", line 247, in :proc_lib.init_p_do_apply/3
  Module "Elixir.Ace.HTTP.Worker", in Ace.HTTP.Worker.init/1

More detail:

Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]: ** (FunctionClauseError) no function clause matching in Rested.Router.handle_head/2
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:     (rested) lib/rested/router.ex:19: Rested.Router.handle_head(%Raxx.Request{authority: "localhost:8081", body: false, headers: [{"user-agent", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML,
like Gecko) Chrome/62.0.3202.94 Safari/537.36"}, {"accept", "image/webp,image/apng,image/*,*/*;q=0.8"}, {"referer", "http://localhost:8081/status"}, {"accept-encoding", "gzip, deflate, br"}, {"accept-language", "en-GB,en-US;q=0.9,en;q=0.8"}], method: :GET, mount: [], pa
th: ["favicon.ico"], query: %{}, scheme: :https}, [])
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:     (ace) lib/ace/http/worker.ex:24: Ace.HTTP.Worker.handle_info/2
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:     (stdlib) gen_server.erl:616: :gen_server.try_dispatch/4
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:     (stdlib) gen_server.erl:686: :gen_server.handle_msg/6
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:     (stdlib) proc_lib.erl:247: :proc_lib.init_p_do_apply/3
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]: Last message: {{:http1, #PID<0.4491.0>, 1}, %Raxx.Request{authority: "localhost:8081", body: false, headers: [{"user-agent", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36"}, {"accept", "image/webp,image/apng,image/*,*/*;q=0.8"}, {"referer", "http://localhost:8081/status"}, {"accept-encoding", "gzip, deflate, br"}, {"accept-language", "en-GB,en-US;q=0.9,en;q=0.8"}], method: :GET, mount: [], path: ["favicon.ico"], query: %{}, scheme: :https}}
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]: State: {Rested.Router, [], nil}
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]: =CRASH REPORT==== 4-Dec-2017::11:17:24 ===
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:   crasher:
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:     initial call: Elixir.Ace.HTTP.Worker:init/1
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:     pid: <0.4527.0>
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:     registered_name: []
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:     exception error: no function clause matching
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:                      'Elixir.Rested.Router':handle_head(#{'__struct__' =>
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:                                                               'Elixir.Raxx.Request',
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:                                                           authority =>
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:                                                               <<"localhost:8081">>,
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:                                                           body => false,
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:                                                           headers =>
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:                                                               [{<<"user-agent">>,
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:                                                                 <<"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36">>},
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:                                                                {<<"accept">>,
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:                                                                 <<"image/webp,image/apng,image/*,*/*;q=0.8">>},
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:                                                                {<<"referer">>,
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:                                                                 <<"http://localhost:8081/status">>},
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:                                                                {<<"accept-encoding">>,
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:                                                                 <<"gzip, deflate, br">>},
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:                                                                {<<"accept-language">>,
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:                                                                 <<"en-GB,en-US;q=0.9,en;q=0.8">>}],
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:                                                           method => 'GET',
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:                                                           mount => [],
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:                                                           path =>
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:                                                               [<<"favicon.ico">>],
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:                                                           query => #{},
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:                                                           scheme => https},
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:                                                         []) (lib/rested/router.ex, line 19)
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:       in function  'Elixir.Ace.HTTP.Worker':handle_info/2 (lib/ace/http/worker.ex, line 24)
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:       in call from gen_server:try_dispatch/4 (gen_server.erl, line 616)
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:       in call from gen_server:handle_msg/6 (gen_server.erl, line 686)
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:     ancestors: [<0.2422.0>,'Elixir.Rested.Router.ClearText',
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:                   'Elixir.Rested.Supervisor',<0.2419.0>]
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:     message_queue_len: 0
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:     messages: []
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:     links: [<0.2422.0>]
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:     dictionary: []
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:     trap_exit: false
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:     status: running
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:     heap_size: 610
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:     stack_size: 27
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:     reductions: 293
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:   neighbours:
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]: =SUPERVISOR REPORT==== 4-Dec-2017::11:17:24 ===
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:      Supervisor: {<0.2422.0>,'Elixir.Supervisor.Default'}
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:      Context:    child_terminated
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:      Reason:     {function_clause,
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:                      [{'Elixir.Rested.Router',handle_head,
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:                           [#{'__struct__' => 'Elixir.Raxx.Request',
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:                              authority => <<"localhost:8081">>,body => false,
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:                              headers =>
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:                                  [{<<"user-agent">>,
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:                                    <<"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36">>},
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:                                   {<<"accept">>,
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:                                    <<"image/webp,image/apng,image/*,*/*;q=0.8">>},
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:                                   {<<"referer">>,
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:                                    <<"http://localhost:8081/status">>},
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:                                   {<<"accept-encoding">>,
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:                                    <<"gzip, deflate, br">>},
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:                                   {<<"accept-language">>,
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:                                    <<"en-GB,en-US;q=0.9,en;q=0.8">>}],
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:                              method => 'GET',mount => [],
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:                              path => [<<"favicon.ico">>],
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:                              query => #{},scheme => https},
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:                            []],
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:                           [{file,"lib/rested/router.ex"},{line,19}]},
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:                       {'Elixir.Ace.HTTP.Worker',handle_info,2,
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:                           [{file,"lib/ace/http/worker.ex"},{line,24}]},
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:                       {gen_server,try_dispatch,4,
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:                           [{file,"gen_server.erl"},{line,616}]},
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:                       {gen_server,handle_msg,6,
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:                           [{file,"gen_server.erl"},{line,686}]},
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:                       {proc_lib,init_p_do_apply,3,
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:                           [{file,"proc_lib.erl"},{line,247}]}]}
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:      Offender:   [{pid,<0.4527.0>},
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:                   {id,'Elixir.Ace.HTTP.Worker'},
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:                   {mfargs,{'Elixir.Ace.HTTP.Worker',start_link,undefined}},
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:                   {restart_type,temporary},
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:                   {shutdown,500},
Dec  4 11:17:24 dc1-live-appserver1 eggl[21496]:                   {child_type,worker}]

Enforce styleguide

Thanks to #44 I have discovered that I have style preferences. To make sure style does not become part of conversations we should have a step that automatically applies a style guide. Perhaps exfmt is the answer.

Until then style guidance

• alias not import other modules within this library
• new line between case match and case body
• comments on their own line, above commented code

Handle Client errors.

Errors that prevent Ace from creating a request to pass to the application.

• url too long
• request too large
• request too slow
• request invalid format

Extend Raxx with a handle_error and a set of errors a that adapters should use to pass through.

def handle_error(%Raxx.MalformedRequest{content: malform}) do
  
end

Endpoint configuration

Name process and set number of listeners

implement client

#```elixir

start connection under application supervisor if not provided

{:ok, } = Client.connect("blah:8080", Connection.start_link())

{:ok, tag} = Client.request(headers, :end)
{:ok, headers} = Client.await_headers(client, tag)
{:ok, body} = Client.read_body()

or

Client.stream(body)

Adopt Raxx Interface

Ace will be the first server to use the upgraded Raxx interface that supports streaming and preserves purity.

Request

Changes to the Raxx.Request to be HTTP/2 focused.

• enforce all keys on the request struct and use constructor always when building
• Rename host -> authority. it should include the port if specified.
• Create function Raxx.Request.port/1 that will extract port from authority if present or else use default
• Drop peer as a key it is part of the connection and not the request

Server callbacks

use Raxx.Server

def handle_request(request, state) do
   
end

def handle_fragment(data, state) do
end

def handle_trailers(trailers, state) do
end

def handle_info(message, state) do
end