docker / libchan Goto Github PK

In libchan we need a way to seamlessly carry trace, debugging, cancellation and other data across boundaries - whether that be process, goroutine or RPC.

The code.google.com/p/go.net/context package provides wiring to do this.
References http://blog.golang.org/context

If libchan can be seamlessly used over unix socket, go channels, etc a standardized mechanism for propagating scoped values, cancelation signals, and deadlines, etc across the boundaries would be awesome.

I'm happy to do the work but not sure if this is something people feel a need for at the moment

@shykes @dmcgowan ?

EOM

Following on from #22 and #26: I think we need to define the channel semantics. Saying "the ordering depends on the underlying transport you use" isn't helpful for developers who want to build software that is independent of the underlying transport.

For transports that don't guarantee ordering, perhaps we should enforce ordering at the libchan level? Perhaps we should not use transports that have undefined ordering?

Some other things that would be useful to know when developing on top of libchan, regardless of transport:

• Are messages dropped or repeated?
• How reliably are messages delivered? (i.e. given a 100%, 99.99%, 99.9% etc reliable network, what is the reliability of libchan?)
• Are there buffers that can overflow?

@aphyr @shykes @dmcgowan @aanand – thoughts?

(I know little about protocol design, so all credit to @andrewgodwin for thoughts on this.)

I want to use Sourcegraph code search and code review with libchan. A project maintainer needs to enable it to set up a webhook so the code is up-to-date there.

Could you please enable libchan on @sourcegraph by going to https://sourcegraph.com/github.com/docker/libchan and clicking on Settings? (It should only take 15 seconds.)

Thank you!

Currrently libchan.Pipe does not use Go channels to communicate. Pipe should be updated to use Go channels and add options for buffering the pipe.

sdpy has (sender, receiver), whereas libchan has (receiver, sender). These should match and should probably follow (receiver, sender), similar to the io package (reader, writer).

I'm having an hard time to figure out how to implement the libchan over SPDY in nodejs for jschan mainly because there are at least 3 different sources of information: the code, the PROTOCOL file and #38.

What is not clear is exactly how msgpack and the 'libchan-ref' header works.
On master there is no msgpack, just urlenconding strings. In the PROTOCOL spec is stated that all messages should be in msgpack format, and #38 implements that.

On master each message gets its own spdy stream, plus a spdy stream for response: master uses the 'parent' relationship of SPDY streams to create all those streams (e.g. the response of a message travels over a SPDY stream that has the current one as parent). This is different to what is specified in PROTOCOL.md, as the 'libchan-ref' header should play some role: however nesting is not documented, and there is no way to 'reply' to a message as in PROTOCOL.md.

In #38, which from what I understand implements PROTOCOL.md, channels are unidirectional but are nestable thanks to an extension of msgpack. In practice, this means that the transport leverage the capabilities in msgpack v5 that allows extension points. This means an extendable msgpack library is needed (which we don't have on node, but that's not the point of the discussion).

Where is the project going? What are the plans? What is the spec of libchan protocol? Will it be all-msgpack for messages (but the README and the purpose of the protocol says otherwise)?
As an environment is spawning up around libchan and microservices, I suggest we do drafts like RFC, e.g. we can know library X implements libchan-1, and iterate over that.

Anyway, thanks for all the good work, there is enough stuff here to blow up my head :).

cc @Vertice @peger.

Moving protocol related discussion from #38 to here. This would follow the model Docker is adopting for large design changes, discussion is on the protocol and the PR is for code review and discussion related to whether it implements the proposal as defined.

Please read the PROTOCOL.md before commenting on this issue.

On going discussions

• Nested channels ids are not defined, should stream ids, reference ids, or something else be used?
• msgpack custom formats
• Dealing with net.Conns and file descriptors
• Channel semantics similarity to go channels (do we need to provide back pressure semantics in the protocol)

Background

Currently the libchan protocol does not define the mechanism for connecting both ends of a channel. This leaves it up the implementer to either use the underlying spdy stream id or something custom. Using the spdy stream id should not be used since the identifier is part of a lower and separate layer and depending on the spdy implementation may not be accessible. In order for clients to be compatible the protocol should have channels define an identifier in the same way as byte streams and communicate that identifier through the spdy headers. The channel header can use the same reference id space as byte streams and be distinguished by the existance of a parent identifier (which byte streams do no have since they do not have hiearchy).

Proposal

I am proposing adding a 'libchan-parent-ref' spdy header to be used when creating channels and requiring both channels and byte streams to make use of the existing 'libchan-ref' header. The decision on whether a new stream is a channel will then be made by the existance of 'libchan-parent-ref' rather the absence of 'libchan-ref'. The msgpack object will then communicate the reference identifer rather than anything custom or the spdy stream id.

It seems when using anonymous embedded structs, field values are not transmitted. For example, given the following types:

type B struct {
    I int
}

type A struct {
    B
    J int
}

If A is sent on a channel, values for the anonymously embedded B struct are not sent.

The currently Go msgpack library does not allow for encoding based on interfaces. This limits the ability of the Send operation to dynamically create underlying streams for non-bytestream ReadWriteCloser (any ReadWriteCloser not created through Sender.CreateByteStream). As a workaround, libchan.ByteStreamWrapper was added to allow the encoder to recognize any ReadWriteCloser type and transparently copy to the transport's byte stream. Ideally this extra wrapping should not need to exist and the encoder could take care of it. It is the encoders responsibility as owning the send type reflection process and to avoid having to do extra type reflection before encoding.

Add in interface to the network transport for creating synchronous and buffered channels.

hi

i trying run the test for http2 and they block forever.
look, i do in directory libchan/http2:

 $ go test # Block forever!!!
 $ go test listener_test.go stream.go spdy.go listener.go # Block forever!!
 $ go test stream_test.go stream.go spdy.go listener.go # Pass OK

Add a contributing.md instructing contributors to use Linux style Signed-off-by and http://developercertificate.org/

I'm trying to do the build of https://github.com/bfosberry/banano/blob/master/server/main.go code and I am getting the following errors:

• undefined: spdy.NewTransportListener
• undefined: spdy.NoAuthenticator

The code is:

listener, err := net.Listen("tcp", fmt.Sprintf(":%s", port))
if err != nil {
    log.Fatal(err)
}

tl, err := spdy.NewTransportListener(listener, spdy.NoAuthenticator)
if err != nil {
    log.Fatal(err)
}

This code run with v.0.1.0 of libchain

I haven't found documentation of changes to be made in code when we want to migrate to the current version of libchain.

Over at jschan we are busy discussing allowing peer-to-peer connections via webRTC.
GraftJS/jschan#27

We're currently planning to do this the same way we did the websocket support (#56), with a custom transport, because re-implementing SPDY is just too big a task for us right now.

I'm starting to work on a node.js / javascript implementation using streams as the base abstraction (less elegant than goroutines, but I can see them working).

Other than the websocket support missing, as per #5, are there any specific issues you can see with the browser environment?

Have you considered browserchannel as another option for a fallback?

The default SpdyTransport uses spdy streams for messages as well as byte streams. A Dual transport would allow for using spdy streams for messages, but a different network type for byte streams. This would most likely involve adding a transport type which is created with a network connection for channels over spdy, but contains a listener/dialer for creation of byte streams. This would allow calls to CreateByteStream on the transport's sender to return real network connections. Any new protocol should be clearly defined and referenced by changes to the Protocol document to ensure different implementation of libchan are in agreement of the transport protocol.

The readme says:

network services communicate in the same way that goroutines communicate using channels:

And the Golang spec, regarding asynchronous channels, says:

elements are received in the order they are sent

How does libchan enforce the ordering/delivery properties of golang channels when the underlying transport is disrupted--e.g., when re-establishing a TCP link that dropped, how does the channel state machine negotiate redelivery of incomplete in-flight messages? Do libchan channels yield at-least-once delivery? At-most-once? Exactly once? Are there any ordering invariants, or can messages be arbitrarily reordered? If ordering is preserved, what are the space costs on both sides of the network?

Add support for encoding and decoding Go channels using the channel extended type with the new msgpack library.

The tag line on github for this project states that it is "Like Go channels over the network".

In Go, channels have two purposes: communication and synchronisation. The above phrase creates the impression that libchan addresses both aspects, whereas in reality it handles only the former. The phrase is therefore deeply misleading and the source of much confusion. To make matters worse, the README even explicitly states that libchan provides "Synchronization for concurrent programming", when this evidently isn't the case. I have spoken to a few folks who had looked at libchan and nearly all of them mistakenly though it dealt with synchronisation.

The phrase need changing.

As I wasn't at the DockerCon I haven't heard how libchan is going to be used, but if multiple streams on top of these transports is a goal I suggest adding a datagram-based transport as well.

The reason is because of head-of-line blocking.

Obviously, HTTP/2 or SPDY is already better than HTTP/1.x or something like WebSocket at dealing with multiple streams.

But at the TCP level you can still get head-of-line blocking because if packets are lost they can not be reordered.

It's similar to the TCP tunneling in TCP problem:
http://sites.inka.de/~W1011/devel/tcp-tcp.html

It is also (one of ?) the reasons https://en.wikipedia.org/wiki/QUIC is being developed.

An other example WebRTC which uses DTLS ('SSL'/TLS over UDP) at the lowerlevel and SCTP as the datachannel protocol for the multiplexing.

Obviously I wouldn't suggest implementing all of WebRTC, it's a pretty big specification.

But they are just examples of current protocols that already do this.

Is there any documentation for this? Am I missing something?

hi,

i'm implementing a server with libchan and http2, they work well, but they fail after some minutes with the error fd too many files open,when debug see the descriptor for libchan.Message.Fd create by the http2.StreamReceiver.Receive method used on the server, they grow and grow..
look, simple test,
ulimit -n 80
then running de server the Fd grow:

• adialer-handler2014/07/09 00:01:14 msg.Fd &{%!s(*os.file=&{9 0})}
• adialer-handler2014/07/09 00:01:15 msg.Fd &{%!s(*os.file=&{13 0})}
• adialer-handler2014/07/09 00:01:15 msg.Fd &{%!s(*os.file=&{15 0})}
• adialer-handler2014/07/09 00:01:17 msg.Fd &{%!s(*os.file=&{23 0})}
• adialer-handler2014/07/09 00:01:29 msg.Fd &{%!s(*os.file=&{29 0})}
• ...
• adialer-handler2014/07/09 00:01:34 msg.Fd &{%!s(*os.file=&{72 0})}

and boom!, i'm find a solution and it's closing the descriptor (msg.Fd.Close()) when then server finish the response,

there a better way?

thanks

I need to move moby/swarmkit#49 over and also fix the error with the websocket handler where a server close hangs the client.

I'm in the process of researching and designing a node.js/javascript implementation of libchan (i'm calling it Graft).

I'm planning to use node streams as the base of the abstraction, and i'm aiming for a api similar in usage to gulp.

What I'm wondering is, what you would consider the simplest proof of concept to bootstrap this implementation, so that I can iterate on the various components from there.

So I guess, what does 'hello world' look like for libchan?

Implement the stream provider interface with an SSH connection

Add an example which demonstrates libchan being used over a websocket. Since the spdy protocol is separated from the transport, this should be possible to implement using the existing code.

Relevant to #5

Most of the power of Go channels come from flexibility provided by select clauses and the analogous reflect package API. An efficient select functionality in libchan would bring it further into parity with Go channels and could support some powerful use cases:

1. A select server where different request types are sent over different channels. The server blocks on all channels until one can proceed and it handles the specific request type (See #73 (comment) for a more detail description).
2. Provide synchronization of distributed request control flow (lots of words). Imagine propagating a timeout through a pipeline of processes, using an operation channel and a cancellation channel.

An implementation could be as simple as providing a select call that spawns multiple goroutines for each sender and receiver passed in and dispatches a callback based on it. More complex implementations would add methods to Sender and Receiver that indicate whether the operation can proceed. It may make sense to even bridge it with process-local Go channels. The API could take hints from the reflect package channel API.

I was using v0.1.0 version and my program ran OK.

transport, err := spdy.NewClientTransport(client)

Now, using origin master, I get this error:

undefined: spdy.NewClientTransport

Which method I need to use instead ?

After seen the work done in #61 and #63, I think it's worth linking in here that I created a Graft/jschan pubsub broker: https://github.com/GraftJS/graft-broker. Graft-broker can also be backed by Redis, thanks to https://github.com/mcollina/mqemitter-redis, however you can't send channels through it.

Now that SPDY is being set aside for HTTP/2 in chromium, we should consider what an http/2 implementation looks like.

CBOR should be an option to use for serialization along with msgpack.

Is there a recommended pattern for implementing a server that can handle a multitude of different requests? The question I have is how to represent the request. For example, suppose I have the following requests:

type EchoRequest struct {
  Cmd: string // "echo"
  Text: string
  Ret: libchan.Sender
}
type AddRequest struct {
  Cmd: string // "add"
  A, B: int
  Ret: libchan.Sender
}

What would I Receive into on the server side without going through map[interface{}]interface{} hell? The cleanest I can come up with is to give each request type its own (well-typed) channel and to make the root channel just send a map of request-type=>channel. If these were Go channels I would receive into an interface{}, look up the command, and then coerce the interface{} to EchoRequest or AddRequest. But that doesn't work with libchan as far as I can see. I'm probably missing something obvious...

Libchan can be difficult to debug network interactions through traditional packet sniffing. Because of the lack of tooling which currently exists around debugging spdy, libchan should have a debug mechanism which is spdy stream and channel aware. Debug level logging as well as efficient info level logging will be needed by applications using libchan.

Example usage would be for docker-archive/docker-registry#635

There is 6Mb binary examples/beamsh/beamsh in repository. It is not very cool, and it is became less cooler with every commit.
May be we should remove it through filter-branch or at least just git rm?
ping @shykes

Is the websocket transport missing or am I missing something obvious about how to hook in one of the other websocket packages to just use the conn transport?

The spdy channel.Send method will return a spdy.InvalidDataFrame error if the serialized size of the message is greater than 16MB (the spdy max data frame size). This could be fixed with a combination of automatic data frame chunking (moby/spdystream#31) and using the underlying spdystream object as an io.Reader for msgpack deserialization on the Receive side.