Crash when using Sinatra's stream method. about em_aws

[JoshMcKin]

hmmm. Try calling your code from within EM.synchrony block:

EM.synchrony do
  # some EM-aws stuff.
    EM.stop
end

[JoshMcKin]

I usually get that error when the em-http-request is not using em-synchony's version of em-http-request

[pwiebe]

Thanks. Tried this, but still am getting the error. If I switch out em_http_handler then it works fine. But that defeats the purpose.

I'll keep digging... Thanks again for your quick responses.

Sent from my iPad...

On May 22, 2012, at 21:32, "Joshua T. Mckinney" [email protected] wrote:

hmmm. Try calling your code from within EM.synchrony block:

EM.synchrony do
# some EM-aws stuff.
end

Let me know if that helps.

---

Reply to this email directly or view it on GitHub:
#5 (comment)

[JoshMcKin]

No problem. Whats the round trip request time, from start to fail. If it runs longer than your timeout set for EM in the app server (thin defaults to 30) you can sometimes get weird errors.

[JoshMcKin]

Try 0.1.5, it should resolve this issue.

[pwiebe]

Unfortunately, I still get the same thing happening.

Sent from my iPad

On Jun 16, 2012, at 13:03, "Joshua T. Mckinney" [email protected] wrote:

Try 0.1.5, it should resolve this issue.

---

Reply to this email directly or view it on GitHub:
#5 (comment)

[JoshMcKin]

hmm, wonder if it an EM or Synchrony thing. Have you tried file streaming from another source using EM::Synchrony directly?

[JoshMcKin]

Also, can you post an example of the code your using to stream the file.

Thanks

[pwiebe]

This is the general flow of the code: 

post '/get_file' do
 ...
  obj = $S3Client.buckets['files'].objects[key]
          stream do |out|
            obj_stream(obj) { |d| out << d }
          end
...
end

private

def obj_stream(obj)
    if obj.exists? 
size = obj.content_length 
byte_offset = 0 
while byte_offset < size 
 range = "bytes=#{byte_offset}-#{byte_offset + CHUNK_SIZE - 1}" 
 yield obj.read(:range => range)
 byte_offset += CHUNK_SIZE 
end 
    end 
end

---

From: Joshua T. Mckinney [email protected]
To: Philip Wiebe [email protected]
Sent: Tuesday, June 19, 2012 11:29 AM
Subject: Re: [em_aws] Crash when using Sinatra's stream method. (#5)

Also, can  you post an example of the code your using to stream the file.

Thanks

---

Reply to this email directly or view it on GitHub:
#5 (comment)

[pwiebe]

I have not tried any other form of streaming.

It seems that the 'yield' is getting interpreted at the wrong level?  Its strange.

---

From: Joshua T. Mckinney [email protected]
To: Philip Wiebe [email protected]
Sent: Tuesday, June 19, 2012 11:28 AM
Subject: Re: [em_aws] Crash when using Sinatra's stream method. (#5)

hmm, wonder if it an EM or Synchrony thing. Have you tried file streaming from another source using EM::Synchrony directly?

---

Reply to this email directly or view it on GitHub:
#5 (comment)

[JoshMcKin]

Ahhh. At this time the handler returns the body of the em-http-request, without streaming. We can look into adding a streaming option, not sure if is even possible but am willing to try when I get some free time.
Ref:
https://github.com/amazonwebservices/aws-sdk-for-ruby/blob/41dc5f953af70320c121db416964dab4e5c36166/lib/aws/s3/request.rb#L72
https://github.com/igrigorik/em-http-request/wiki/Streaming

[JoshMcKin]

@pwiebe if you have time can you please test the stream branch. I have no experience with streaming but going by em-http-request docs and the sdk's net_http_handler i think I should be close.

[pwiebe]

Sorry, I've been quite busy lately. I'll get to it this week and let you know.

Sent from my iPhone

On Jan 2, 2013, at 21:03, "Joshua T. Mckinney" [email protected] wrote:

@pwiebe if you have time can you please test the stream branch. I have no experience with streaming but going by em-http-request docs and the sdk's net_http_handler i think I should be close.

—
Reply to this email directly or view it on GitHub.