zerowidth / rack-streaming-proxy Goto Github PK
View Code? Open in Web Editor NEWStreaming proxy for Rack, the rainbows to Rack::Proxy's unicorn
License: MIT License
Streaming proxy for Rack, the rainbows to Rack::Proxy's unicorn
License: MIT License
While creating a testcase for bug #6 I ended up finding an unreliable result with proxying non-chunked content. Here's the testcase:
Again this is runnable as "rackup puma_unreliable_proxying.ru".
Then asking for the same resource multiple times from the original server works fine:
$ curl http://localhost:9000
Hello!
pedrocr@wintermute:~$ curl http://localhost:9000
Hello!
pedrocr@wintermute:~$ curl http://localhost:9000
Hello!
pedrocr@wintermute:~$ curl http://localhost:9000
Hello!
pedrocr@wintermute:~$ curl http://localhost:9000
Hello!
But doing the same from the proxy server sometimes returns an empty result:
pedrocr@wintermute:~$ curl http://localhost:9292
Hello!
pedrocr@wintermute:~$ curl http://localhost:9292
pedrocr@wintermute:~$ curl http://localhost:9292
Hello!
I am running a Rails app in puma and using rack-streaming-proxy to proxy a MJPEG HTTP stream. The stream is a multipart message in chunked encoding. The proxy seems to break the stream by not proxying it as chunked encoding. All the code is here:
https://github.com/pedrocr/camerasink
Testing with telnet the original stream returns:
HTTP/1.0 200 OK
Server: camerasave
Date: Fri, 18 Apr 2014 11:13:06 GMT
Transfer-Encoding: chunked
Content-Type: multipart/x-mixed-replace;boundary=SurelyJPEGDoesntIncludeThis
51
--SurelyJPEGDoesntIncludeThis
Content-Type: image/jpeg
Content-Length: 9437
(the image contents go here, the headers after "51" are repeated on every new image)
The same stream after being proxied with rack-streaming-proxy and puma returns:
HTTP/1.0 200 OK
server: camerasave
date: Fri, 18 Apr 2014 11:12:16 GMT
content-type: multipart/x-mixed-replace;boundary=SurelyJPEGDoesntIncludeThis
Cache-Control: no-cache
X-Request-Id: c68247f6-e4ef-4507-b34c-c37330062289
X-Runtime: 0.256272
Connection: close
--SurelyJPEGDoesntIncludeThis
Content-Type: image/jpeg
Content-Length: 9427
(the image contents go here and the headers get repeated as well)
The difference seems to be that the proxied request doesn't have chunked encoding. At least in firefox this breaks the MJPEG streaming.
@zerowidth @fredngo Could you do me a favor?
We are using rack-streaming-proxy for our graph tool https://github.com/yohoushi/yohoushi.
We tested rack-streaming-proxy very well, and with deep understanding of codes, rack , and HTTP RFC, we fixed several issues such as fredngo#3, yohoushi@af0ef26, and yohoushi#2. We believe that we can be most active commiters of rack-streaming-proxy.
Could you do add me to owners of rack-streaming-proxy gem on rubygems so that we can maintain the gem? Like
gem owner rack-streaming-proxy -a [email protected]
I encountered an error when trying to proxy a POST without body, because in that case the Content-Length is not being set:
12:02:54 web.1 | [Rack::StreamingProxy] Parent process 62926 forked a child process 62927.
12:02:54 web.1 | [Rack::StreamingProxy] Child starting request to http://127.0.0.1:3600/messages/29/read.json
12:02:54 web.1 | [Rack::StreamingProxy] Child process 62927 passing on ArgumentError: Content-Length not given and Transfer-Encoding is not `chunked'
12:02:54 web.1 | [Rack::StreamingProxy] Child process 62927 closing connection.
12:02:54 web.1 | [Rack::StreamingProxy] Parent received unexpected nil status!
12:02:54 web.1 | [Rack::StreamingProxy] Child process 62927 exiting.
12:02:54 web.1 | [Rack::StreamingProxy] Parent process 62926 waiting for child process 62927 to exit.
12:02:54 web.1 | App 62909 stderr: Rack::StreamingProxy::UnknownError
Thank you for this middleware by the way. :)
Best regards,
Steven
I want to subclass Proxy to be able to do something like:
class ExternalProxy < Rack::StreamingProxy::Proxy
def initialize(app)
super app do | request |
if request.path.start_with?('/foo')
"/bar"
end
end
end
end
and plug it like
use ExternalProxy
Just as concept, I will be happy if there is another, simpler way
We are using the rack-streaming-proxy with a unicorn to proxy to a (only locally reachable) apache location, that is serving svn repositories via webdav. In some very special use cases, zombie processes are being spawned by the proxy (its the only part of the system that does spawn any child processes).
This happens when trying to commit a large number of files (~2000 or more) with a filesize > 10-15 kb each. In such a use case, every file being committed consistently spawns one zombie process.
I have came across this issue while using this gem.
Realized this is an issue caused by a change in rack which removed Rack::Utils.bytesize(chunk) in favor of chunk.bytesize usage.
I am using proxy to load dropbox URL. My purpose is to load URL and display image on canvas. It was working fine but since last few days it returns me 301 status.
streaming_proxy.rb
if request.path.start_with?('/proxy')
request.params['uri']
end
I am receiving,
[Rack::StreamingProxy] Parent process 11548 forked a child process 13396.
[Rack::StreamingProxy] Child starting request to https://www.dropbox.com/s/5l4vvb413squt2f/b4B86CN.jpg?dl=0&raw=1
[Rack::StreamingProxy] Child got response: Net::HTTPMovedPermanently
[Rack::StreamingProxy] Child process 13396 returning Status = 301.
[Rack::StreamingProxy] +-------------------------------------------------------------
[Rack::StreamingProxy] | Proxy Response Headers:
[Rack::StreamingProxy] Parent received: Status = 301.
[Rack::StreamingProxy] Parent received: Reponse has body? = true.
What wrong I am doing?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.