reidmorrison / net_tcp_client Goto Github PK

i use this net_tcp_client for my JSON RPC client jrpc
but when i tried to use simple Logger.new($stdout) i faced with bug: block for logger.benchmark_debug in read() was called twice and return only result of last reading.
all works fine when i use github version, but i can't add github dependency to .gemspec

without logger all works fine.

should i wait new version release? or maybe i can fix this in a different way?

Hi Reid,

By the looks of it this gem isn't setup to use ssl.

I'm busy looking at a work around now.
How much of a mission would that be to implement ?

Possibly due to https://ruby-doc.org/core-2.6.2/Object.html#method-i-send having a name overlap...?

undefined method `socketwatcher.prod.ns-desktop-ub._var_run_docker_sock.active_conns.sample_count 1 1553738125' for #<Net::TCPClient:0x0000556ab000db90> (NoMethodError)

(the method name is actually the arg I send to client.send())

https://ruby-doc.org/stdlib-2.4.0/libdoc/openssl/rdoc/OpenSSL/SSL/SSLSocket.html#method-i-hostname-3D

When two SSL-enabled hostnames are served from the same IP, the SSL socket needs to use SNI (Server Name Indication) to tell the webserver which cert should be used for the conversation. This needs to be done before calling OpenSSL::SSL::SSLSocket#connect.

Add it right about here: https://github.com/rocketjob/net_tcp_client/blob/master/lib/net/tcp_client/tcp_client.rb#L683

Expect a PR shortly...

Hi,

I am looking into how to implement a Whois client using Net::TCPClient.
However I am a bit stuck, because I can't work out how to read an arbitrary number of bytes.

The Whois protocol involves:

1. Opening a TCP socket
2. Sending the request string
3. Reading all the bytes returned until the socket is closed

I guess it is quite similar to very basic HTTP.

If I try reading a large number of bytes, like this:

Net::TCPClient.connect(server: 'whois.nic.me:43') do |client|
  client.write("DOMAIN njh.me\r\n")
  response = client.read(4096)
  puts "Received: #{response}"
end

Then I get an error:

/Users/njh/.rbenv/versions/2.5.5/lib/ruby/gems/2.5.0/gems/net_tcp_client-2.2.0/lib/net/tcp_client/tcp_client.rb:651:in `socket_read': Connection lost while reading data (Net::TCPClient::ConnectionFailure)

Because the server closed the connection before the client received the full 4096 bytes. But if I set the read size, to a low number (for example 100), then I only get back those 100 bytes.

I have considered having a loop reading a small number of bytes at a time, but I can't see how to avoid the Net::TCPClient::ConnectionFailure error, given that I don't know how many bytes the server is going to send.

The reason for using Net::TCPClient at all and not just a plain TCPSocket, is because I want to try each of the IP addresses in DNS in turn and not just fail if the first chosen IP address is down.

Any help would be very appreciated.

The Net::TCPClient#socket_write calls socket.write_nonblock(data). However, it does not check the returned number of bytes written:

It returns the number of bytes written.

#write_nonblock just calls the write(2) system call. [...] The result may also be smaller than string.length (partial write). The caller should care such errors and partial write.

I believe that in the case of full socket write buffers, and calls to write with large buffers that the kernel does not accept in a single write syscall could result in lost data, as the write syscall only sends part of the buffer. This leads to corruption once that write returns, and the next write is called.

Here's a repro case demonstrating this issue: https://gist.github.com/SpComb/c8b857fc5bb575a1f859f7ea57603a29

The test-client.rb sends one ~1m lines of sequential integers formatted as '%07d\n' (00000001..0998001). It sends them 1k lines at a time, in 8kb write calls. It logs the return value from Net::TCPClient#write, which seems to be the value returned by socket.write_nonblocking.

The test-server.rb reads one line at a time, and prints . N + sleeps 1s/10k lines for the first 100k lines, and then 1s/100k lines for the rest. It parses each line as an integer, comparing it to the previous line, logging a ! message if the lines are not strictly sequential

During the first slow read period, the sending client gets blocked on a full send buffer, which shows up as short writes on the client after about ~400-500kb. It blocks for a short period, and then continues, as the server empties its recv buffer and makes room for more data:

connected: #<Net::TCPClient:0x00000000f92db0>
write 0...: 8000
write 1000...: 8000
write 2000...: 8000
write 3000...: 8000
...
write 440000...: 8000
write 441000...: 8000
write 442000...: 4218
write 443000...: 8000
write 444000...: 8000
....
write 638000...: 8000
write 639000...: 3592
write 640000...: 8000
...
write 826000...: 8000
write 827000...: 8000
write 828000...: 2109
write 829000...: 8000
write 830000...: 8000
...
write 998000...: 8000
write 999000...: 8000

Each of these short writes results in corruption on the server:

connect: 127.0.0.1:37232
. 0
. 10000
. 20000
. 30000
. 40000
. 50000
. 60000
. 70000
. 80000
. 90000
. 100000
. 200000
. 300000
. 400000
! 40443000 -40000474
! 0443001 +39999999
. 500000
. 600000
! 0640000 -552
. 700000
. 800000
! 82820829000 -82820000738
! 0829001 +82819999999
. 900000
eof

I'm using version 2.2.1 of the gem with Ruby 3.1.2p20 in Debian Bookworm.

I'm getting ArgumentError when calling Net::TCPClient.connect(connection_params) because the parameters are passed to new in a way not compatible with Ruby 3:

From: /var/lib/gems/3.1.0/gems/net_tcp_client-2.2.1/lib/net/tcp_client/tcp_client.rb:91 Net::TCPClient.connect:

    90: def self.connect(params = {})
 => 91:   require 'pry'; binding.pry
    92:   connection = new(params)
    93:   yield(connection)
    94: ensure
    95:   connection&.close
    96: end

[1] pry(Net::TCPClient)> params
=> {:server=>"foobar:8080", :connect_timeout=>1, :connect_retry_count=>4, :connect_retry_interval=>1, :write_timeout=>2, :read_timeout=>23}
[2] pry(Net::TCPClient)> new(params)
ArgumentError: wrong number of arguments (given 1, expected 0)

With Ruby 3 it appears to be necessary to use the double splat operator: new(**params)

See e.g. https://stackoverflow.com/a/68450312.

maybe we can use SOCKSSocket instead of Socket or implement it (looks like it is not so difficult)

such test

require 'socket'
require_relative 'test_helper'
require_relative 'simple_tcp_server'

class TCPClientTest < Minitest::Test
  describe Net::TCPClient do
    before do
      unless defined?(SemanticLogger)
        require 'logger'
        @logger       = Logger.new('test.log')
        @logger.level = Logger::DEBUG

        @server      = SimpleTCPServer.new(2000, @logger)
        @server_name = 'localhost:2000'
      end

      after do
        @server.stop if @server
      end

      it 'should be closed' do
        @client = Net::TCPClient.new(
              server:          @server_name,
              connect_timeout: -1,
              logger:          @logger
        )
        @client.close
        assert_equal true, @client.closed?
        assert_equal false, @client.alive?
      end
  end
end

raise error

Minitest::UnexpectedError: NoMethodError: undefined method `closed?' for nil:NilClass

because @socket is set to nil on close

module Net
  class TCPClient
    ...
    def_delegators :@socket, :closed?, :eof?, :setsockopt, :alive?
    ...
    def close
      @socket.close if @socket
      @socket = nil
      true
    end
  end
end

i think we should add methods closed? and alive? instead of delegate them
something like

def closed?
  @socket.nil? ? true : @socket.closed?
end

def alive?
  !closed?
end