breaks webmock? about shrine-url

[jrochkind]

Trying to get a better stack trace, but having trouble, maybe down's use of fibers/threads is confusing things.

[jrochkind]

Appears to be unique to the http_rb backend. If I switch my storage to

 Shrine::Storage::Url.new(downloader: Down::NetHttp.new)

WebMock works fine.

But I'd rather not switch the downloader from default for no reason but to get WebMock to work; although to make the decision of whether to do so, it would be useful to know why shrine-url chooses httprb as a default, when down has net-http as a default.

[janko]

Yeah, the WebMock monkey patches for HTTP.rb still appears to be broken with later versions (httprb/http#212).

The main reason I switched shrine-url to HTTP.rb backend for the user is because HTTP.rb has lower memory usage, which is beneficial when uploading large files. Also, when you configure HTTP.rb with a timeout, that's the actual timeout you get (Net::HTTP will retry the request once automatically).

The reason for me was that I didn't want to deal with Net::HTTP anymore. It's ugly externally, it's ugly internally, and I hate it. And I think that HTTP.rb is a great library, so I think it's a better choice. But if you want to use the Net::HTTP backend because of its WebMock support, and you run into issues, I'll still help you out 😃

Net::HTTP will probably stay the default backend for Down, just because I don't want to force users to install more gems (because in that case I'd need to add http as a runtime dependency). Not really a great reason, but I'm trying to strike a balance.

[janko]

Note that this particular WebMock error is supposed to be fixed, I remembered now that I've made a patch myself bblimke/webmock@d6a7370 😄

[jrochkind]

Hmm. That commit is in webmock, odd it's not fixed. I haven't actually been able to reproduce the problem even with just down -- I can only reproduce with shrine-url, I can't figure out what shrine-url is doing to trigger the problem. But that commit and test give me a little bit of orientation, thanks. Any more orientation welcome!

I think shrine-url ends up using http_rb in an uncommon way, so other people aren't running into the bug in webmock, if that's what it is.

I'd really like to use http_rb, but testing my app reasonably without webmock is challenging. Hmm.

[jrochkind]

That fix and test are still in latest version of webmock... so it must be something else? I will try to investigate a bit more.

I'm confused about why I can't get any more of a stack trace than those three lines, to see exactly what shrine-url is doing, to be able to write an isolated reproduction of the problem. I guess it's got something to do with fibers, which shrine-url uses (?), that limits my backtrace?

If you had any hints for where in the code actually calls things, prior to the stack trace I'm able to get, that would also be helpful.

[janko]

That's weird, shrine-url is using Down in the simplest way. Do you think you could come up with self-contained example that reproduces the error? Something like

require "shrine/storage/url"
require "webmock"

storage = Shrine::Storage::Url.new

storage.download(some_url)
# or
storage.open(some_url).read

[jrochkind]

Aha! I was accidentally using WebMock 2.0, before your included fix. Everything is good with Webmock 3.0.

Not sure why I was having trouble reproducing, but no longer necessary to find out.

Sorry for the diversion, thanks for posting the commit that fixed, that led me to figure out why the fix wasn't in my app. Sorry thanks!