meh / elixir-socket Goto Github PK

Hi Meh,

I see in the README this library should wrap gen_sctp, but I don't see this in the code. Is this a WIP?

Thanks,
Lee

Hi - would it be possible to post an updated release on hex.pm? No changes - just the latest and greatest in git right now, so it seems like it would be pretty safe. Thanks!

That would be super cool if we can listen with create a Unix Domain Socket ? Because in my case, i make an application that use Elixir Backend and Electron for Frontend, open a port is not very terrible. But for the rest, it's amazing ! 👍

Here is the output of my mix reps.get :

* Getting socket [git: "https://github.com/meh/elixir-socket.git"]
* Compiling socket
Compiled lib/socket/address.ex
Compiled lib/socket/manager.ex
Compiled lib/socket.ex
Compiled lib/socket/host.ex
Compiled lib/socket/tcp.ex
Compiled lib/uri/ws.ex
Compiled lib/socket/ssl.ex
Compiled lib/socket/udp.ex
Compiled lib/uri/wss.ex
== Compilation error on file lib/socket/web.ex ==
could not compile dependency socket, mix compile failed. In case you want to recompile this dependency, please run: mix deps.compile socket
** (CompileError) /Users/glejeune/Dev/whisky/deps/socket/lib/socket/web.ex:94: function connnect/2 undefined
    erl_eval.erl:934: :erl_eval.guard_test/4
    erl_eval.erl:918: :erl_eval.guard0/4
    erl_eval.erl:906: :erl_eval.guard1/4
    erl_eval.erl:691: :erl_eval.eval_fun/6
    src/elixir_dispatch.erl:189: :elixir_dispatch.expand_macro_fun/7
    src/elixir_dispatch.erl:141: :elixir_dispatch.do_expand_import/6
    src/elixir_dispatch.erl:82: :elixir_dispatch.dispatch_import/5
    lists.erl:1329: :lists.mapfoldl/3

I'm on OSX 10.8.4 with Elixir 0.9.3, Erlang R16B (erts-5.10.1). And here is my mix configuration :

{:socket,"0.0.1",[github: "meh/elixir-socket", tag: "v0.0.1"]}

Socket.Web only has recv/1, meaning there's no way to pass useful things like timeout to it

I know it's not out yet, but on the current master I get

** (UndefinedFunctionError) undefined function: Socket.TCP.packet!/2
    (socket) Socket.TCP.packet!(:raw, {Socket.TCP, #Port<0.8029>, nil, {:resource, 353638984, ""}})
    (socket) lib/socket/web.ex:210: Socket.Web.connect!/3
    (socket) lib/socket/web.ex:140: Socket.Web.connect/3

Hey @meh,

you probably have no time anymore to maintain this. I've taken the freedom to fork this and push a new hex package socket2 with recent updates. It's here: https://github.com/dominicletz/elixir-socket2/

Thanks for the package
Cheers!

If I inspect the accepted connection and decide I want to close it, the client connect! blocks. Unless I'm closing it wrong, I would expect that the client would not block after the server closes the client connection. Here's how I'm closing the connection in the server after I've call accept on the server socket.

Socket.Web.close(client, {:normal, "unknown route"}, [{:wait, false},{:reason, :true}])

https://guides.github.com/features/mastering-markdown/

I stumbled upon an error in elixir-dns with version 0.3.6, so I wanted to see what changes have been introduced, but... the github source is 0.3.5.

Hi , sometimes this exception throw one by one on trying to connect to websocket. And I should restart all application to fix it. What happen? Function Socket.Web.connect throws this exception :

%{__exception__: true, __struct__: UndefinedFunctionError, arity: 1, function: :exception, module: Socket.Error, reason: nil}}

Is there a way to use the websocket client if I have to use a proxy to connect to the websocket server? Looks like there is an additional request to the proxy needed before you're allowed to start the websocket connection.

First message sent from client to proxy:

CONNECT echo.websocket.org:80 HTTP/1.1
Host: echo.websocket.org:80
Proxy-Connection: keep-alive
User-Agent: Mozilla/5.0 ...

Response from proxy to client:

HTTP/1.1 200 Connection established

Message sent from client to server over proxy, followed by response.

GET /?encoding=text HTTP/1.1
Host: echo.websocket.org
Connection: Upgrade
Pragma: no-cache
Cache-Control: no-cache
Upgrade: websocket
Origin: http://www.websocket.org
Sec-WebSocket-Version: 13
User-Agent: Mozilla/5.0 ...
Accept-Encoding: gzip, deflate, sdch
Accept-Language: de-DE,de;q=0.8,en-US;q=0.6,en;q=0.4
Cookie: xxxx
Sec-WebSocket-Key: xxxx
Sec-WebSocket-Extensions: permessage-deflate; client_max_window_bits

HTTP/1.1 101 Web Socket Protocol Handshake
Access-Control-Allow-Credentials: true
Access-Control-Allow-Headers: content-type
Access-Control-Allow-Headers: authorization
Access-Control-Allow-Headers: x-websocket-extensions
Access-Control-Allow-Headers: x-websocket-version
Access-Control-Allow-Headers: x-websocket-protocol
Access-Control-Allow-Origin: http://www.websocket.org
Date: Wed, 25 Jan 2017 12:58:37 GMT
Sec-WebSocket-Accept: xxxx
Server: Kaazing Gateway
Upgrade: websocket
Connection: Upgrade

For me it looks like the second part should be the same in an environment without a proxy. So is there a way to inject this first block where the client talks to the proxy?

Thanks for looking into this.

I'm trying to make a simple app based on the web socket server example.

require Socket.Web

defmodule Echo do
  use Application

  def start(_type, _args) do
    server = Socket.Web.listen! 8080
    spawn fn -> accept(server) end
    {:ok, self()}
  end

  defp accept(server) do
    client = server |> Socket.Web.accept!
    spawn fn -> echo client end
  end

  defp echo(client) do
    client |> Socket.Web.accept!
    msg = client |> Socket.Web.recv!
    client |> Socket.Web.send!(msg)
    echo(client)
  end
end

When a client initiates a connection, client |> Socket.Web.accept! throws this exception:

{#{'__exception__' => true,
   '__struct__' => 'Elixir.Socket.Error',
   message => <<"invalid argument">>},
 [{'Elixir.Socket','packet!',2,[{file,"lib/socket.ex"},{line,238}]},
  {'Elixir.Socket.Web','accept!',2,[{file,"lib/socket/web.ex"},{line,451}]},
  {'Elixir.Echo',echo,1,[{file,"lib/echo.ex"},{line,18}]}]}

Hi, I'm new to the world of Elixir, and haven't had much exposure to Erlang before. To give myself a task, I figured I'd write a simple telnet client. For that I need to connect via TCP to a port at an address, so elixir-socket seemed good, as "h :gen_tcp" said no doc due to being an Erlang module.

So in my mix.exs, I have:

defp deps do
[ { :Socket, "~> 0.1", git: "https://github.com/meh/elixir-socket.git" } ]
end

but when I do mix deps.get, I get:

Unchecked dependencies for environment dev:

• Socket (https://github.com/meh/elixir-socket.git)
  the app file at deps/Socket/ebin/Socket.app is invalid
  ** (Mix) Can't continue due to errors on dependencies

How can I solve that?

To try to poke around a bit, I grabbed git master, but upon running "mix test", I got 5 errors. Is this normal? For good measure, I put the output here: https://www.dropbox.com/s/okcd2clvxnhr929/socket_output.txt

Cheers

Nik

Hi,

I need to pass some advanced SSL options (like cert, key, etc) to the websocket module connection functions but it seems to be currently not possible.
So I just patched the Socket.Web module to transmit a new optional "ssl_options" (via the connect! function) down to the Socket.SSL module and it seems to work but it's a bit quick'n'dirty for now and before going further and propose a PR, I would like to know if I'm on a good track or if it's something that would interest somebody else than me?

http://erlang.org/doc/man/ssl.html supports this option but this library does not. https://hexdocs.pm/socket/Socket.SSL.html#module-options

Since this library relies on it, support for this option can be easily added.

Hello @meh ! I am using this library with a WebSocket server and my experience has been very good this far. Thank you!
I have a use case where the WebSocket server is sending multiple messages in response to each message sent to it (but independent of the timing of the incoming messages) and I need to capture them all (preferably in the order they are sent by the server).
The docs helpfully mention the following way to deal with multiple messages from the server.

socket = Socket.Web.connect! "echo.websocket.org"
case socket |> Socket.Web.recv! do
  {:text, data} ->
    # process data
  {:ping, _ } ->
    socket |> Socket.Web.send!({:pong, ""})
end

However, I believe this will work for only the first message it receives.
How can I make this continuously listen for incoming messages, so that I can process them in the order they were sent by the server?

If this is not currently possible (perhaps due to #19 ), could you comment on whether a polling based solution would solve this problem?

Is there a way to have an active websocket client? I can set the underlying socket to be active, but then I believe the messages I get are the raw over the socket messages.

Would it make sense for Socket.Web to implement Socket.Protocol?

Thanks!

The error:

{{case_clause,nil},[{'Elixir.Socket.Web',recv,1,[{file,"lib/socket/web.ex"},{line,665}]}

I've run into this in a few places where doing a recv on a closed socket returns an { :ok, nil } that can't be matched against. What's the thought behind translating {:error, :closed} to {:ok, xx}, rather than passing it up verbatim? Is this something easily patched by passing theerror, closedup, or is the{:ok, nil}` a special flag somewhere that needs an OK even when the socket's closed?

I get the following error on new project created with elixir 0.13.1

[root@ucx50 socket_test]# mix deps.compile

• Compiling socket
  Compiled lib/socket/address.ex
  Compiled lib/socket/helpers.ex
  Compiled lib/socket/datagram.ex
  Compiled lib/socket/host.ex
  == Compilation error on file lib/socket.ex ==
  could not compile dependency socket, mix compile failed. You can recompile this dependency with mix deps.compile socket or update it with mix deps.update socket
  ** (CompileError) deps/socket/lib/socket.ex:52: URI.struct/0 is undefined, cannot expand struct URI
  (elixir) src/elixir_map.erl:55: :elixir_map.translate_struct/4
  (stdlib) lists.erl:1352: :lists.mapfoldl/3
  (elixir) src/elixir_clauses.erl:26: :elixir_clauses.match/3
  (elixir) src/elixir_clauses.erl:35: :elixir_clauses.clause/7
  (elixir) src/elixir_def.erl:182: :elixir_def.translate_clause/7
  (elixir) src/elixir_def.erl:171: :elixir_def.translate_definition/8
  (elixir) src/elixir_def.erl:89: :elixir_def.store_definition/9

This repository is a little hard to find as it does not have any topic tags associated with it. Adding the tags elixir, websockets, websocket-client, etc... may prove useful for others looking for an Elixir Websockets library though GitHub and Google search.

% elixir --version
Erlang/OTP 25 [erts-13.1.5] [source] [64-bit] [smp:12:12] [ds:12:12:10] [async-threads:1] [jit]

Elixir 1.14.3 (compiled with Erlang/OTP 25)

Trying to compile:

warning: :ssl.cipher_suites/0 is undefined or private, use cipher_suites/2,3 instead
  lib/socket/ssl.ex:55: Socket.SSL.ciphers/0

warning: :ssl.ssl_accept/2 is undefined or private, use ssl_handshake/1,2,3 instead
  lib/socket/ssl.ex:272: Socket.SSL.handshake/2

warning: :ssl.ssl_accept/3 is undefined or private, use ssl_handshake/1,2,3 instead
  lib/socket/ssl.ex:252: Socket.SSL.accept/2

When running:

** (UndefinedFunctionError) function :ssl.ssl_accept/2 is undefined or private, use ssl_handshake/1,2,3 instead
    (ssl 10.8.7) :ssl.ssl_accept({:sslsocket, {:gen_tcp, #Port<0.6>, :tls_connection, [option_tracker: #PID<0.354.0>, session_tickets_tracker: :disabled, session_id_tracker: #PID<0.355.0>]}, [#PID<0.358.0>, #PID<0.357.0>]}, :infinity)
    (socket 0.3.13) lib/socket/ssl.ex:239: Socket.SSL.accept/2
    echo-server.exs:19: Echo.loop_acceptor/1

Any non-standard code/reason causes:

(CaseClauseError) no case clause matching: {code}
    (socket) lib/socket/web.ex:59: Socket.Web.recv/2
    (socket) lib/socket/web.ex:739: Socket.Web.recv!/2

Could you implement a fallback that fires :close with :unknown (or :abnormal) atom, and maybe provide the close code?

When calling recv like this:

Socket.Web.recv(connection, [timeout: 100])

and a timeout happens, this exception is raised:

** (CaseClauseError) no case clause matching: {:error, :timeout}
    (socket) lib/socket/web.ex:666: Socket.Web.recv/2

socket version: 0.3.12
elixir version: 1.6.1

No method shown to supply timeouts to at least TCP and SSL connections.

When a license projects a vulgarity and says DWTFYWT, the first thing I do is change the license to read "Do Whatever You Want To." :)

More seriously, some of us working saps can actually get hit with sexual harassment suits over including profanity in assemblies of code..

Please consider the following snippet, which is to poll a WebSocket server for data using a tail recursive loop:

def loop(socket, user_id) do
    case socket |> Socket.Web.recv! do
      {:text, data} ->      
         #received incoming data from server
      {:ping, _ } ->
        socket |> Socket.Web.send!({:pong, ""})
      _ -> 
        IO.puts("catch all")
    end
    loop(socket, user_id)
  end

This continues to print the "catch all" message even when the server is not really sending any messages. Doesn't the call socket |> Socket.Web.recv! block until data is available from the server? I tried to answer this by going through the code, but I couldn't find anything conclusive.

I tried `Socket.TCP.connect("localhost", 80, {:raw, 6, 15, :true}) and received:

** (FunctionClauseError) no function clause matching in Access.get/3

The following arguments were given to Access.get/3:

    # 1
    {:raw, 6, 15, true}

    # 2
    :timeout

    # 3
    nil

Attempted function clauses (showing 5 out of 5):

    def get(%{__struct__: struct} = container, key, default)
    def get(map, key, default) when is_map(map)
    def get(list, key, default) when is_list(list) and is_atom(key)
    def get(list, key, _default) when is_list(list)
    def get(nil, _key, default)

(elixir) lib/access.ex:302: Access.get/3
(socket) lib/socket/tcp.ex:105: Socket.TCP.connect/3

Using elixir 1.3, Used {:socket, github: "meh/elixir-socket"} in mix.exs.

iex(1)> socket = Socket.Web.connect! "echo.websocket.org"
** (MatchError) no match of right hand side value: %{true: [as: :binary]}
    (socket) lib/socket/tcp.ex:264: Socket.TCP.arguments/1
    (socket) lib/socket/tcp.ex:108: Socket.TCP.connect/3
    (socket) lib/socket/tcp.ex:98: Socket.TCP.connect!/2
    (socket) lib/socket/web.ex:230: Socket.Web.connect!/3


iex(1)> socket = Socket.Web.connect! "echo.websocket.org", secure: true
** (MatchError) no match of right hand side value: %{}
    (socket) lib/socket/ssl.ex:343: Socket.SSL.arguments/1
    (socket) lib/socket/ssl.ex:147: Socket.SSL.connect/3
    (socket) lib/socket/ssl.ex:131: Socket.SSL.connect!/2
    (socket) lib/socket/web.ex:230: Socket.Web.connect!/3

I couldn't figure out what's wrong.

It would be useful to be able to send custom opcodes (like custom binary ops). Libraries like https://github.com/uNetworking/uWebSockets/tree/v0.14 are able to expose received opcodes which would be super helpful for different operations.

https://hexdocs.pm/socket/Socket.html#listen/1-example

{ :ok, server } = Socket.listen "tcp://*:1337"
client = server.accept!(packet: :line)
client.send(client.recv)
client.close

Shouldn't those lines use the protocol? E.g. something like Socket.Protocol.Stream.accept!(server, packet: :line)?

Hi there,

I have been trying to open a socket to some secured endpoints using the following command:

Socket.Web.connect! "ws-feed.gdax.com", secure: true

But with all secured URIs I get the following connection error:

** (Socket.Error) TLS Alert: bad record mac

Any clue on what parameter to use to solve this? I've been searching the doc of SSL as well about this, without much success yet.

Thanks!

I'm trying to connect to a socket (ssl)

Socket.Web.connect! "stream-api.betfair.com", secure: true

But I'm facing with this error:

** (MatchError) no match of right hand side value: {:http_error, "{\"op\":\"connection\",\"connectionId\":\"203-270420013200-944388\"}\r\n"}
    (socket 0.3.13) lib/socket/web.ex:251: Socket.Web.connect!/3

But its not an error. The server accepts my connection, but elixir-socket returns an error.

When accepting a TLS session, we can retrieve some information with info but some are missing. In the returned list, there is a sni_hosts but it is always en empty list so I don't know how to get the SNI. Also, there is noway to retrieve the ALPN?

With Elixir 1.12.2 (compiled with Erlang/OTP 24), there are warnings:

warning: ^^^ is deprecated. It is typically used as xor but it has the wrong precedence, use Bitwise.bxor/2 instead
  lib/socket/web.ex:553

warning: ^^^ is deprecated. It is typically used as xor but it has the wrong precedence, use Bitwise.bxor/2 instead
  lib/socket/web.ex:559

warning: ^^^ is deprecated. It is typically used as xor but it has the wrong precedence, use Bitwise.bxor/2 instead
  lib/socket/web.ex:565

warning: ^^^ is deprecated. It is typically used as xor but it has the wrong precedence, use Bitwise.bxor/2 instead
  lib/socket/web.ex:571

I recall OTP 19 supports encrypted UDP via DTLS, any guides on how to start with that?

Hi @meh thanks very much for this superb package!
When compiling using Elixir v.1.6 we see the following warnings:

Compiling 11 files (.ex)
warning: String.to_char_list/1 is deprecated, use String.to_charlist/1
  lib/socket/address.ex:19

warning: the char_list() type is deprecated, use charlist()
  lib/socket/address.ex:12

warning: String.to_char_list/1 is deprecated, use String.to_charlist/1
  lib/socket/host.ex:61

warning: String.to_char_list/1 is deprecated, use String.to_charlist/1
  lib/socket/host.ex:75

warning: String.to_char_list/1 is deprecated, use String.to_charlist/1
  lib/socket/host.ex:89

warning: String.to_char_list/1 is deprecated, use String.to_charlist/1
  lib/socket/host.ex:103

warning: the char_list() type is deprecated, use charlist()
  lib/socket/host.ex:101

warning: String.to_char_list/1 is deprecated, use String.to_charlist/1
  lib/socket/datagram.ex:54

warning: String.to_char_list/1 is deprecated, use String.to_charlist/1
  lib/socket/ssl.ex:141

warning: String.to_char_list/1 is deprecated, use String.to_charlist/1
  lib/socket/tcp.ex:108

Would you accept a Pull Request updating the relevant function invocations to avoid the warnings?
(thanks in advance)

i had to modify the source code in my way to limit big transfers, how could i do it in a right way?
i never receive fragment packets or anything, if i send for example a file of 100MB it is done in one step by Socket.Web.recv() in memory

this is what i did, in file web.ex, line 577:

@SPEC recv(t, boolean, non_neg_integer, Keyword.t) :: { :ok, binary } | { :error, error }
defp recv(%W{socket: socket, version: 13}, mask, length, options) do
length = cond do
length == 127 ->
case socket |> Socket.Stream.recv(8, options) do
{ :ok, << length :: 64 >> } ->
if length > 5_000_000 do
{:error, :transfer_too_large}
else
length
end

The example at https://hexdocs.pm/socket/Socket.html#listen/1 states:
{ :ok, server } = Socket.listen "tcp://*:1337"

My code:

defmodule Rudy.Server do
    def start() do
        { :ok, server } = Socket.listen "tcp://*:1337"
    end
end

Error:

iex(3)> x = Rudy.Server.start()
** (FunctionClauseError) no function clause matching in anonymous fn/1 in Socket.TCP.arguments/1
    (socket) lib/socket/tcp.ex:308: anonymous fn({:address, "0.0.0.0"}) in Socket.TCP.arguments/1
    (elixir) lib/enum.ex:966: Enum.flat_map_list/2
    (elixir) lib/enum.ex:966: Enum.flat_map_list/2
    (elixir) lib/enum.ex:967: Enum.flat_map_list/2
    (socket) lib/socket/tcp.ex:282: Socket.TCP.arguments/1
    (socket) lib/socket/tcp.ex:163: Socket.TCP.listen/2
      (rudy) lib/rudy/server.ex:3: Rudy.Server.start/0

When trying to create a secure websocket connection I get this error:

** (UndefinedFunctionError) undefined function: String.to_char_list!/1
    (elixir) String.to_char_list!("websocketurl.com")
             lib/socket/ssl.ex:134: Socket.SSL.connect/3
             lib/socket/ssl.ex:126: Socket.SSL.connect!/2
             lib/socket/web.ex:225: Socket.Web.connect!/3

looks like you should be using String.to_char_list instead (no !).

There aren't any docs up on hex for this--would you want a PR to add more docs?

Before I tried for a PR I wanted to run this past you. Correct me if I'm wrong, I'm new to elixir

Web.ping passes {:ping, cookie} to Web.send. send has a guard when data?(opcode) which stops it from matching this call.

I also noticed that the next guard is and opcode != :close, which as far as I can tell can't serve a use as :close doesn't pass data?

Thanks

https://github.com/msantos/gen_unix

Websocket I'm trying to connect to is behind a proxy that takes a basic authentication during connection. Is there some way of specifying headers during the initial connect phase? My .connect! call is returning a 401 because I haven't been able to supply credentials yet.

I can't seem to find sub-protocol support based on:
https://tools.ietf.org/html/rfc6455#page-12

Is this supported or not?

Thanks,
mirko

Do I store the socket as part of a GenServer state, or do I start it as a child of a supervisor and let the GenServer reference it by a name and vice versa, socket to GenServer?

Is there a way to listen (like handle_info) for new messages instead of blocking with receive/1?

I'm trying to use the ipv6_v6only option to my socket but I always get a FunctionClauseError. Reading the source code, it seems there is no way to send the full set of :gen_udp options through Socket.UDP.open?

May be #83 shows the same issue.

Hi!I am a fresher on elixir.How should I receive fragmented data through by websocket?
{ :fragmented, :text, data } ->
recv(socket)
{ :fragmented, :continuation, data } ->
recv(socket)
{ :fragmented, :end, data } ->
recv(socket)

I want to get total data in the end.

Lots of unsafe variable messages as well as unknown PosixError exception on elixir 1.3