libp2p / go-libp2p-quic-transport Goto Github PK
View Code? Open in Web Editor NEWAn implementation of a libp2p transport using QUIC
License: MIT License
An implementation of a libp2p transport using QUIC
License: MIT License
vyzo@carbon6 go-libp2p-daemon $ go get github.com/libp2p/go-libp2p-quic-transport
go: finding github.com/libp2p/go-libp2p-quic-transport v0.2.9
go: downloading github.com/libp2p/go-libp2p-quic-transport v0.2.9
go: extracting github.com/libp2p/go-libp2p-quic-transport v0.2.9
# github.com/libp2p/go-libp2p-quic-transport
../../../../pkg/mod/github.com/libp2p/[email protected]/stream.go:15:31: s.Stream.CancelRead(0) used as value
../../../../pkg/mod/github.com/libp2p/[email protected]/stream.go:18:29: s.Stream.CancelWrite(0) used as value
../../../../pkg/mod/github.com/libp2p/[email protected]/transport.go:22:62: undefined: quic.VersionMilestone0_10_0
seems we have an erroneous tag. Where did this v0.2.9 come from?
I want to implement quic-go for my project in samsung and I need it to be unreliable which I am not getting. Can you tell me how to make it unreliable stream. contact me at [email protected]
See the second half of #63 (comment).
Running an IPFS node on a virtual server equipped with 2 GB of RAM leads to OOM within 2 - 5 minutes when qlog is running. The reason is that each zstd.Encoder
consumes at least 8 MB of memory for an internal buffer, so just handling a little more than 200 QUIC connections simultaneously would consume all available memory.
I reported this to the zstd implementation in klauspost/compress#316.
I hope that this can be fixed there, otherwise we'd probably have to log the uncompressed qlog to disk first, and do the compression in a post-processing step when the respective connection is closed. This would reduce the number of simultaneously allocated zstd.Encoder
s.
Currently, if a peer uses their peer ID to sign a certificate, QUIC will allow the signed key to act on behalf of the peer. The peer should have to explicitly state that they want to delegate their peer ID.
Motivation: If we don't do this, we won't be able to use peer IDs to sign any certificates except delegation certificates.
Proposal: Put /p2p/QmPeerID
in the certificate's subject field (or some other field if that doesn't make sense). The certificate would only be valid if there exists some certificate higher up the chain signed by that peer ID.
Versions were registered here: https://github.com/quicwg/base-drafts/wiki/QUIC-Versions
panic: qtls.ClientSessionState not compatible with tls.ClientSessionState
goroutine 1 [running]:
github.com/lucas-clemente/quic-go/internal/handshake.init.0()
/home/ubuntu/go/pkg/mod/github.com/lucas-clemente/[email protected]/internal/handshake/unsafe.go:20 +0x113
Travis should be able to do that.
This would have caught #82.
Currently, we're using ConnectionState()
to get the peer certificates. This has the disadvantage that the server doesn't learn about the client rejecting the connection on peer ID mismatch.
By using tls.Config.VerifyPeerCertificate
instead of conn.ConnectionState()
we can make sure the handshake fails before the handshake completes.
Multiaddrs explicitly include the IP version so we need to make sure to listen/dial with udp#
instead of just udp
.
Attempting to use this transport with a libp2p node using ed25519 keys results in the following error:
transport constructor github.com/libp2p/go-libp2p-quic-transport.NewTransport failed: unsupported key type for TLS
Go version: 1.13.1
go-libp2p-quic-transport: v0.1.1
Prerequisite: quic-go/quic-go#1324.
QUIC should try to pick a port we're listening on when dialing instead of opening a new one. We may want to extract some of the logic in https://github.com/libp2p/go-reuseport-transport.
The libp2p stream muxer assumes that when calling Close()
on a Stream
that both read and write direction are closed. In QUIC, Close()
only closes the write direction.
Transports should apply the "address filters", either directly on all incoming packets or, at least, on the initial handshake packets. Applying them on all incoming packets could be a performance hit at the moment as the filter is very unoptimized.
The transport can get a copy of the filter set by accepting a *github.com/libp2p/go-maddr-filter.Filters
in the constructor.
This transport currently ignores the context when dialing.
Compressing qlogs uses huge amounts of memory.
Running v0.10.0 (which uses gzip, not zstd), so this is unrelated to #191.
The problem is that we're running as many compression contexts in parallel as we have active QUIC connections. This number can easily reach hundreds.
We are seeing a slow goroutine build up in our relays, stuck in dialing through quic:
165407 runtime.gopark
runtime.goparkunlock
sync.runtime_notifyListWait
sync.(*Cond).Wait
github.com/lucas-clemente/quic-go.(*outgoingBidiStreamsMap).OpenStreamSync
github.com/lucas-clemente/quic-go.(*streamsMap).OpenStreamSync
github.com/lucas-clemente/quic-go.(*session).OpenStreamSync
github.com/libp2p/go-libp2p-quic-transport.(*conn).OpenStream
github.com/libp2p/go-libp2p-swarm.(*Conn).NewStream
github.com/libp2p/go-libp2p-swarm.(*Swarm).NewStream
github.com/libp2p/go-libp2p/p2p/host/basic.(*BasicHost).newStream
github.com/libp2p/go-libp2p/p2p/host/basic.(*BasicHost).NewStream
github.com/libp2p/go-libp2p-circuit.(*Relay).DialPeer
github.com/libp2p/go-libp2p-circuit.(*Relay).Dial
github.com/libp2p/go-libp2p-circuit.(*RelayTransport).Dial
github.com/libp2p/go-libp2p-swarm.(*Swarm).dialAddr
github.com/libp2p/go-libp2p-swarm.(*dialLimiter).executeDial
The reason for dialing these peers in the first place is yet unknown, but the build up is more alarming and indicative of a bug.
The daemon is running with the latest go-libpp2p-quic-transport
relase, v0.0.3
This is based on the discussion in libp2p/specs#37.
This document describes how two libp2p peers authenticate each other over QUIC. Note that we can use the same handshake not only for QUIC, but for any connection that uses TLS. It works with TLS 1.2 as well as TLS 1.3.
At initialisation, each peer generates a certificate chain consisting of two certificates:
ConnSecurity
interface)Remember that in IPFS, the peer ID is derived from the key pair, and a node's identity can be verified by checking that their peer ID matches their public key.
Now each node has a certificate chain consisting of two certificates: 1. the root certificate, tied to the node's private key, and 2. an ephemeral certificate. We use TLS client authentication, so during the handshake each node will learn about the other's certificate chain.
A client verifies a server by setting the tls.Config.VerifyPeerCertificate
callback, which is executed during the handshake (as soon as the server's cert chain is received). It does so by
The server request TLS client authentication, but it doesn't run set the VerifyPeerCertificate
callback. As soon as the TLS handshake completes, it
When the client sends an invalid cert chain (e.g. a chain that only contains one certificate), the server will not abort the handshake (since it only parses the cert chain after the handshake completes), but close the connection immediately after accepting. For the client this will look like the server first accepted the connection, and then closed it shortly afterwards. If we deem this unacceptable, we'd have to parse the cert chain twice (once using the VerifyPeerCertificate
callback, and once after the handshake completes).
Do we need to version this? For example, if we ever want to change the length of the certificate chain to 3, this handshake would fail. We can't really negotiate a handshake protocol here, since authentication happens so early during connection establishment.
go get now fails with
# github.com/libp2p/go-libp2p-quic-transport ..\..\..\..\..\github.com\libp2p\go-libp2p-quic-transport\listener.go:68:24: cannot use err (type error) as type string in argument to sess.CloseWithError
Should the passed parameter be changed to string?
See #152 (review).
@willscott just wrote a cross-platform package (github.com/libp2p/go-netroute) for getting the route to a target IP. We should replace the current dependency on the netlink package.
This will give proper reuse support on windows.
I have a problem with quic in the windows environment
my code is like this:
sourceMultiAddr, _ := ma.NewMultiaddr(node.cfg.P2P.Listen)
q,err:=quic.NewTransport(node.privateKey)
if err!=nil{
return err
}
host, err := libp2p.New(
node.ctx,
libp2p.ListenAddrs(sourceMultiAddr),
libp2p.Identity(node.privateKey),
libp2p.BandwidthReporter(node.reporter),
libp2p.Ping(false),
libp2p.Transport(q),
)
if err != nil {
return err
}
the error log is like this:
# github.com/vishvananda/netns
D:\goproject\pkg\mod\github.com\vishvananda\[email protected]\netns.go:27:13: undefined: syscall.Stat_t
D:\goproject\pkg\mod\github.com\vishvananda\[email protected]\netns.go:28:12: undefined: syscall.Fstat
D:\goproject\pkg\mod\github.com\vishvananda\[email protected]\netns.go:31:12: undefined: syscall.Fstat
D:\goproject\pkg\mod\github.com\vishvananda\[email protected]\netns.go:39:8: undefined: syscall.Stat_t
D:\goproject\pkg\mod\github.com\vishvananda\[email protected]\netns.go:43:12: undefined: syscall.Fstat
D:\goproject\pkg\mod\github.com\vishvananda\[email protected]\netns.go:52:8: undefined: syscall.Stat_t
D:\goproject\pkg\mod\github.com\vishvananda\[email protected]\netns.go:56:12: undefined: syscall.Fstat
D:\goproject\pkg\mod\github.com\vishvananda\[email protected]\netns.go:70:29: cannot use int(*ns) (type int) as type syscall.Handle in argument to syscall.Close
pls help me,thanks you!
quic-go uses interface assertion to test for functions specific to the net.UDPConn
, e.g. SetReadBuffer()
. Since our net.PacketConn
wraps the UDPConn
in multiple layers, the interface assertion will fail (see https://play.golang.org/p/GHDrrvwaI2c for a minimal example), thus disabling optimization that should be possible.
Note that this might require changes to quic-go.
Currently, this doesn't build against the current quic-go. While we use gx internally, we prefer to keep everything building with go get
. It's not that much of an issue here as QUIC is still experimental but it would be nice to be able to build with go get
.
The current max connection receive window is 4.5MiB. That limits the connection to ~240mbps (30MiB/s) given a 150ms RTT. Ideally, we'd be able to max out a gigabit link. Given that the actual window is adaptive and this is just the max, I'd suggest setting each stream to a ~20MiB by default and the connection to 24MiB. (or something like that).
getRemotePubKey
and keyToCertificate
currently "forget" the exact key type (the one used by libp2p). The exact key type is implied by RSA (because the type is RSA) but this won't work for Ed25519/ECDSA keys (as we support both generic "ECDSA" keys and Ed25519 keys).
This will also end up forgetting the hash function (and any additional metadata) after libp2p/specs#111.
So, we need to make sure to encode enough information to accurately reconstruct the public key. Ideally, we'd be able to send a custom key object datastructure but I'm pretty sure that won't work.
Once libp2p/go-libp2p#999 is merged.
We should actually use vendoring for quic-go. This will massively increase the size of this repo, but make installing this package without gx possible.
Trying to bring up some relays using ed25519 keys, and I run into this:
May 01 18:14:27 ip-172-31-47-2 relay.sh[13132]: 2019/05/01 18:14:27 transport constructor github.com/libp2p/go-libp2p-quic-transport.NewTransport failed: unsupported key type for TLS
Connection
dials to two servers at the same time
D:/a/go-libp2p-quic-transport/go-libp2p-quic-transport/conn_test.go:230
using an ECDSA key: QmUoxHQ1XEoxx7djJHsj4atge6khZzzVUr7wsTnGkXeB2w
using an ECDSA key: QmTtpKFpp8GMMaLCZsAbos8Ebj6xEZZEyexwXkv3ncCvQ2
using an ECDSA key: QmYDEmCANdqfYQb2hyb6QCZtezrPCFt7nhtApZgzdgr9fP
+ Failure [5.073 seconds]
Connection
D:/a/go-libp2p-quic-transport/go-libp2p-quic-transport/conn_test.go:27
dials to two servers at the same time [It]
D:/a/go-libp2p-quic-transport/go-libp2p-quic-transport/conn_test.go:230
Timed out after 5.000s.
Expected
<chan struct {} | len:0, cap:2>: 0x124eb3c0
to receive something.
D:/a/go-libp2p-quic-transport/go-libp2p-quic-transport/conn_test.go:287
Full Stack Trace
github.com/libp2p/go-libp2p-quic-transport.glob..func2.10()
D:/a/go-libp2p-quic-transport/go-libp2p-quic-transport/conn_test.go:287 +0x984
github.com/onsi/ginkgo/internal/leafnodes.(*runner).runSync(0x1260c5c0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
C:/Users/runneradmin/go/pkg/mod/github.com/onsi/[email protected]/internal/leafnodes/runner.go:113 +0x7a
github.com/onsi/ginkgo/internal/leafnodes.(*runner).run(0x1260c5c0, 0x3, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
C:/Users/runneradmin/go/pkg/mod/github.com/onsi/[email protected]/internal/leafnodes/runner.go:64 +0xb3
github.com/onsi/ginkgo/internal/leafnodes.(*ItNode).Run(0x12404b20, 0x12bf420, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
C:/Users/runneradmin/go/pkg/mod/github.com/onsi/[email protected]/internal/leafnodes/it_node.go:26 +0x44
github.com/onsi/ginkgo/internal/spec.(*Spec).runSample(0x12601880, 0x0, 0x12bf420, 0x1254ad00)
C:/Users/runneradmin/go/pkg/mod/github.com/onsi/[email protected]/internal/spec/spec.go:215 +0x509
github.com/onsi/ginkgo/internal/spec.(*Spec).Run(0x12601880, 0x12bf420, 0x1254ad00)
C:/Users/runneradmin/go/pkg/mod/github.com/onsi/[email protected]/internal/spec/spec.go:138 +0xd3
github.com/onsi/ginkgo/internal/specrunner.(*SpecRunner).runSpec(0x1268e000, 0x12601880, 0x10)
C:/Users/runneradmin/go/pkg/mod/github.com/onsi/[email protected]/internal/specrunner/spec_runner.go:200 +0xd1
github.com/onsi/ginkgo/internal/specrunner.(*SpecRunner).runSpecs(0x1268e000, 0x1)
C:/Users/runneradmin/go/pkg/mod/github.com/onsi/[email protected]/internal/specrunner/spec_runner.go:170 +0xf7
github.com/onsi/ginkgo/internal/specrunner.(*SpecRunner).Run(0x1268e000, 0x124253e0)
C:/Users/runneradmin/go/pkg/mod/github.com/onsi/[email protected]/internal/specrunner/spec_runner.go:66 +0xf1
github.com/onsi/ginkgo/internal/suite.(*Suite).Run(0x124b1bc0, 0x123b00b0, 0x124388c0, 0x10d7661, 0x1b, 0x12406280, 0x1, 0x1, 0x12c7ac0, 0x1254ad00, ...)
C:/Users/runneradmin/go/pkg/mod/github.com/onsi/[email protected]/internal/suite/suite.go:79 +0x3e6
github.com/onsi/ginkgo.RunSpecsWithCustomReporters(0x12bfc00, 0x124388c0, 0x10d7661, 0x1b, 0x1244bf84, 0x1, 0x1, 0x9abddc)
C:/Users/runneradmin/go/pkg/mod/github.com/onsi/[email protected]/ginkgo_dsl.go:219 +0x181
github.com/onsi/ginkgo.RunSpecs(0x12bfc00, 0x124388c0, 0x10d7661, 0x1b, 0x1652960)
C:/Users/runneradmin/go/pkg/mod/github.com/onsi/[email protected]/ginkgo_dsl.go:200 +0x128
github.com/libp2p/go-libp2p-quic-transport.TestLibp2pQuicTransport(0x124388c0)
D:/a/go-libp2p-quic-transport/go-libp2p-quic-transport/libp2pquic_suite_test.go:20 +0x7a
testing.tRunner(0x124388c0, 0x121c9d4)
C:/hostedtoolcache/windows/go/1.15.10/x64/src/testing/testing.go:1123 +0xc4
created by testing.(*T).Run
C:/hostedtoolcache/windows/go/1.15.10/x64/src/testing/testing.go:1168 +0x211
We need to cut a quic release and bubble it up to go-libp2p. However, we've made some significant changes (protocol breaking) so I'd rather not just go ahead and do this myself.
Specifically, it doesn't like :
in the RFC3339Nano
format. Looks like we have to roll out our own time stamp format.
Currently, we require length two chains where the bottom key (the one used directly in the handshake) is ephemeral. However, RSA signature verification is a bit expensive, especially when rapidly forming many connections (e.g., in the DHT).
It would be nice if we could instead make this configurable:
Motivation:
A 4 byte connection ID would allow us to be connected to up to 4 billion peers, which is more than enough.
Prerequisite: quic-go/quic-go#1425.
minor nit, but could a license be added at your convenience? Thanks!
Certificates are currently valid for 30 days.
The addr returned by RemoteMultiaddr should be fully resolved, even if the user dials with a /dns address.
We can't dial an IPv4 addr from an IPv6 UDP connection and vice versa.
To fix the deadlock. See: ipfs/kubo#6168.
Ideally, we'd get this into the impending go-ipfs release.
当我想go get 获取时
我遇到
/work/pkg/mod/github.com/libp2p/[email protected]/conn.go:42:9: cannot use &stream literal (type *stream) as type mux.MuxedStream in return argument:
*stream does not implement mux.MuxedStream (missing CloseRead method)
/work/pkg/mod/github.com/libp2p/[email protected]/conn.go:48:9: cannot use &stream literal (type *stream) as type mux.MuxedStream in return argument:
*stream does not implement mux.MuxedStream (missing CloseRead method)
/work/pkg/mod/github.com/libp2p/[email protected]/stream.go:41:5: cannot use &stream literal (type *stream) as type mux.MuxedStream in assignment:
*stream does not implement mux.MuxedStream (missing CloseRead method)
请问有遇到过类似问题的朋友吗
While using this library in IPFS I am getting these current errors
../../libp2p/go-libp2p-quic-transport/stream.go:15:31: s.Stream.CancelRead(0) used as value
../../libp2p/go-libp2p-quic-transport/stream.go:18:29: s.Stream.CancelWrite(0) used as value
../../libp2p/go-libp2p-quic-transport/transport.go:22:62: undefined: quic.VersionMilestone0_10_0
libp2p/admin should probably be admins and libp2p/go-team should probably have write access.
(don't worry, I won't mess with master, I just want to be able to create branches)
Given that QUIC is using the standard golang TLS config type, it would be awesome if we could extract this code out into a new library and then re-use the same handshake for TLS over TCP. Something like:
type Identity tls.Config
func NewIdentity(privKey ic.PrivKey) (*Identity, error) { return generateConfig(privKey) }
// for dialing
func (i *Identity) ConfigForPeer(remote ic.PubKey) (*tls.Config, error) {
// clone the config and set the validator.
}
func KeyFromChain([]*x509...) (ic.PubKey, error) { ... }
(and any other useful helper functions you can think of)
Not sure why it's hanging, but out of the ~100 runs, which each run the test ~25 times, about one fails.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.