GithubHelp home page GithubHelp logo

matthias247 / http2dotnet Goto Github PK

View Code? Open in Web Editor NEW
96.0 12.0 24.0 358 KB

HTTP/2 support for .NET standard

License: MIT License

C# 100.00%
http2 network csharp dotnet http hpack rfc-7541 rfc-7540

http2dotnet's People

Contributors

matthias247 avatar perksey avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

http2dotnet's Issues

Memory leak on byte-range response

Thanks for this nice library, it's very easy to use it!
I'm currently implementing h2 proto handlers in my own server lib and had encountered some memory leaks on sending byte-range response.
IStream is in state Closed - all content bytes are sent with endOfStream=true.
And it's not disposed properly even after Dispose called


image
image

Instructions on how to cleanly handle streams and connections

I am using the http2dotnet library to implement a mitm proxy for h2 traffic. I am using chrome to visit https://www.httpvshttps.com/ to do simple load testing. (I'm using dotnet core 2.1-preview for ALPN support)

So far, I am able to view that page a few times, the first couple times the performance is acceptable (roughly 1/10th the speed of a direct connection), and then the performance plummets to about 1/100th, and then finally it just freezes my console app entirely.

This leads to me to believe that I am not disposing of something properly, but from the examples I cannot determine what I am missing, aside from disposing of the IStream implementations.

My code looks as follows:

private async Task HandleHttp2TrafficAsync(
  Stream clientStream, // proxy client
  Stream serverStream, // remote server
  CancellationToken cancellationToken)
{
  var clientConfiguration = new ConnectionConfigurationBuilder(false)
    .UseHuffmanStrategy(HuffmanStrategy.IfSmaller)
    .Build();

  var serverStreams = serverStream.CreateStreams();

  var clientConnection = new Connection(
    clientConfiguration,
    serverStreams.ReadableStream,
    serverStreams.WriteableStream);

  var serverConfiguration = new ConnectionConfigurationBuilder(true)
    .UseStreamListener(stream =>
    {
      OnStreamAsync(
	    stream, 
		clientConnection.CreateStreamAsync, 
		cancellationToken)
      .ContinueWith(tt => { }, cancellationToken);

      return true;
    })
    .Build();

  var clientStreams = clientStream.CreateStreams();

  var serverConnection = new Connection(
    serverConfiguration,
    clientStreams.ReadableStream,
    clientStreams.WriteableStream);

  await Task.WhenAll(serverConnection.Done, clientConnection.Done);
}

private static async Task OnStreamAsync(
  IStream serverStream, 
  Func<IEnumerable<HeaderField>, bool, Task<IStream>> createClientStreamAsync, 
  CancellationToken cancellationToken)
{
  var buffer = ArrayPool<byte>.Shared.Rent(8 * 1024);

  try
  {
    // start reading the request
    var requestHeaders = await serverStream.ReadHeadersAsync();

    using (var requestBodyStream = new MemoryStream())
    {
      while (true)
      {
        var result = await serverStream.ReadAsync(buffer);

        requestBodyStream.Write(buffer, 0, result.BytesRead);

        if (result.EndOfStream)
        {
          break;
        }
      }
      
      // start sending the request
      using (var clientStream = await createClientStreamAsync(
	    requestHeaders, 
		requestBodyStream.Length == 0))
      {
        if (requestBodyStream.Length > 0)
        {
          await clientStream.WriteAsync(requestBodyStream.ToArray(), true);
        }

        // start reading the response
        var responseHeaders = await clientStream.ReadHeadersAsync();

        using (var responseBodyStream = new MemoryStream())
        {
          while (true)
          {
            var result = await clientStream.ReadAsync(buffer);

            responseBodyStream.Write(buffer, 0, result.BytesRead);

            if (result.EndOfStream)
            {
              break;
            }

            // start sending the response
            await serverStream.WriteHeadersAsync(
			  responseHeaders, 
			  responseBodyStream.Length == 0); // line 351

            if (responseBodyStream.Length > 0)
            {
              await serverStream.WriteAsync(
			    responseBodyStream.ToArray(), 
				true);
            }
          }
        }
      }
    }
  }
  finally
  {
    serverStream.Dispose(); // should I dispose this?
    ArrayPool<byte>.Shared.Return(buffer);
  }
}

And my general run through I receive the following exceptions after setting up a global exception handler

      An unhandled exception occured
System.AggregateException: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (Attempted to write headers after data) ---> System.Exception: Attempted to write headers after data
   at Http2.StreamImpl.<WriteValidatedHeadersAsync>d__31.MoveNext()
   at MyApp.Proxy.Core.Http.HttpSocketProcessor.<OnStreamAsync>d__12.MoveNext() in C:\Projects\myapp-proxy\src\MyApp.Proxy.Core\Http\HttpSocketProcessor.cs:line 351
   --- End of inner exception stack trace ---
---> (Inner Exception #0) System.Exception: Attempted to write headers after data
   at Http2.StreamImpl.<WriteValidatedHeadersAsync>d__31.MoveNext()
   at MyApp.Proxy.Core.Http.HttpSocketProcessor.<OnStreamAsync>d__12.MoveNext() in C:\Projects\myapp-proxy\src\MyApp.Proxy.Core\Http\HttpSocketProcessor.cs:line 351<---
      An unhandled exception occured
System.AggregateException: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (Attempted to read past the end of the stream.) ---> System.IO.EndOfStreamException: Attempted to read past the end of the stream.
   at Http2.ByteStreamExtensions.<ReadAll>d__0.MoveNext()
   at Http2.ClientPreface.<ReadAsync>d__5.MoveNext()
   --- End of inner exception stack trace ---
---> (Inner Exception #0) System.IO.EndOfStreamException: Attempted to read past the end of the stream.
   at Http2.ByteStreamExtensions.<ReadAll>d__0.MoveNext()
   at Http2.ClientPreface.<ReadAsync>d__5.MoveNext()<---
      An unhandled exception occured
System.AggregateException: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (One or more errors occurred. (Unable to transfer data on the transport connection: The I/O operation has been aborted because of either a thread exit or an application request.)) ---> System.AggregateException: One or more errors occurred. (Unable to transfer data on the transport connection: The I/O operation has been aborted because of either a thread exit or an application request.) ---> System.IO.IOException: Unable to transfer data on the transport connection: The I/O operation has been aborted because of either a thread exit or an application request. ---> System.Net.Sockets.SocketException: The I/O operation has been aborted because of either a thread exit or an application request
   --- End of inner exception stack trace ---
   at System.Net.Security.SslStreamInternal.<<FillBufferAsync>g__InternalFillBufferAsync|36_0>d`1.MoveNext()
   at System.Net.Security.SslStreamInternal.<ReadAsyncInternal>d__32`1.MoveNext()
   --- End of inner exception stack trace ---
   at Http2.IoStreamExtensions.IoStreamWrapper.<>c.<ReadAsync>b__2_0(Task`1 tt)
   at System.Threading.Tasks.ContinuationResultTaskFromResultTask`2.InnerInvoke()
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot)
--- End of stack trace from previous location where exception was thrown ---
   at Http2.ByteStreamExtensions.<ReadAll>d__0.MoveNext()
   at Http2.ClientPreface.<ReadAsync>d__5.MoveNext()
   --- End of inner exception stack trace ---
---> (Inner Exception #0) System.AggregateException: One or more errors occurred. (Unable to transfer data on the transport connection: The I/O operation has been aborted because of either a thread exit or an application request.) ---> System.IO.IOException: Unable to transfer data on the transport connection: The I/O operation has been aborted because of either a thread exit or an application request. ---> System.Net.Sockets.SocketException: The I/O operation has been aborted because of either a thread exit or an application request
   --- End of inner exception stack trace ---
   at System.Net.Security.SslStreamInternal.<<FillBufferAsync>g__InternalFillBufferAsync|36_0>d`1.MoveNext()
   at System.Net.Security.SslStreamInternal.<ReadAsyncInternal>d__32`1.MoveNext()
   --- End of inner exception stack trace ---
   at Http2.IoStreamExtensions.IoStreamWrapper.<>c.<ReadAsync>b__2_0(Task`1 tt)
   at System.Threading.Tasks.ContinuationResultTaskFromResultTask`2.InnerInvoke()
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot)
--- End of stack trace from previous location where exception was thrown ---
   at Http2.ByteStreamExtensions.<ReadAll>d__0.MoveNext()
   at Http2.ClientPreface.<ReadAsync>d__5.MoveNext()
---> (Inner Exception #0) System.IO.IOException: Unable to transfer data on the transport connection: The I/O operation has been aborted because of either a thread exit or an application request. ---> System.Net.Sockets.SocketException: The I/O operation has been aborted because of either a thread exit or an application request
   --- End of inner exception stack trace ---
   at System.Net.Security.SslStreamInternal.<<FillBufferAsync>g__InternalFillBufferAsync|36_0>d`1.MoveNext()
   at System.Net.Security.SslStreamInternal.<ReadAsyncInternal>d__32`1.MoveNext()<---

Sorry for the whole rig-ma-role of an 'issue', I just need a pointer on how I'm supposed to manage the connections and streams.

Also, I noticed that the API does not accept CancellationTokens, is there a pattern that is expected for stopping connections and streams (though I do notice that IStream has a Cancel method).

Thank you for your help and guidance,
Matthew

Error: Expected SETTINGS frame as first frame

Hi,

I'm testing out this library in order to send requests to Apple's Push service (APNS). However, I'm having trouble setting up a connection properly and I'm receiving a protocol error.

trce: send ClientPreface
trce: send Settings flags=[] streamId=0x00000000 length=36
trce: recv Data flags=[EndOfStream,Padded] streamId=0x00000000 length=1508099
fail: Handling connection error Http2Error{streamId=0, code=ProtocolError, message="Expected SETTINGS frame as first frame"}

This is how I create the connection

 private async Task<Connection> CreateDirectConnection()
        {
            Uri uri = new Url(_settings.ApnsHost);
            X509Certificate2 clientCertificate = new X509Certificate2(Convert.FromBase64String(_settings.Cert), "test");
            var config =
                new ConnectionConfigurationBuilder(false)
                    .UseHuffmanStrategy(HuffmanStrategy.IfSmaller)
                    .UseSettings(Settings.Default)
                    .Build();

            // Create a TCP connection
            var tcpClient = new TcpClient();
            tcpClient.Client.NoDelay = true;
            await tcpClient.ConnectAsync(uri.Host, uri.Port);
            
            _logger.LogInformation("Connected to remote");
            var ssl = new SslStream(tcpClient.GetStream());
            
            await ssl.AuthenticateAsClientAsync(
                uri.Host,
                new X509CertificateCollection(new X509Certificate[] { clientCertificate }),
                SslProtocols.Tls12, 
                false
            );
    
            _logger.LogInformation("Authenticated against remote");
            var wrappedStreams = tcpClient.Client.CreateStreams();
            var conn = new Connection(
                config, wrappedStreams.ReadableStream, wrappedStreams.WriteableStream,
                options: new Connection.Options
                {
                    Logger = _logger
                });
  
            return conn;
        } 

and it's the first call to var stream = await connection.CreateStreamAsync(headers, false); that fails.

I'm I using this wrong or are there compability issues I'm running in to? The SSL connection seems to be working from what I can tell.

-Anders

How to handle multipart post from client?

I am working with a server that expects the client to perform a post with multipart/form-data. Each part has it's own headers that go along with it. I quickly skimmed the source and it looks like this may be a scenario that is not supported, can you point me in the right direction?

StreamImpl.ReadAsync blocks when State is Closed

It took me a while to figure this out.

I'm using http2dotnet to make gRPC calls to a server.

When there is no response content, the server closes the connection, so stream.State would be Closed.
I was calling stream.ReadAsync in the hope that it would return a 0-byte packet with readDataResult.EndOfStream set to true, but that was not the case.

Instead, the ReadAsync call would block forever.

In most .NET APIs, streams which are at the end of the stream return a 0-byte value when reading, and don't block.

How does http2dotnet relate to the upcoming http2 support in dotnet?

This library is clearly a huge amount of hard and impressive work, bravo! Even the documentation on the readme page, I could only wish more libraries were like this.

That said, before investing work into this, I am wondering how http2dotnet relates to the new (or upcoming) http2 support within the dotnet stack? And will some of this http2dotnet work be replaced by components now in the dotnet stack? (Or maybe even was some of this used as pull requests for that? I wouldn't be surprised).

I just need to know where I should invest my time, especially since I am a novice in networking / sockets / and other low-level networking stuff. My context is hoping to figure out how to send notifications to iOS using their newer http/2 api.

Thanks!

SslStream

Interested in using this in my server project, however I'm curious how SSL would be implemented. Can I just use an SslStream wrapping the Socket's stream or is there more to it?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.