Waiting for network requests about ferrum

[route]

That's in fact already built into ferrum, for example when you call goto it will return to the main thread only when the whole page is loaded. If you click a link and there's network activity which will reload the frame it again will wait until it's over. Though for AJAX requests some wait_for method still have to be implemented.

[stoivo]

That interesting. I will spend some more time to inverstigate it tomorrow. Thank you very much for such a quick responds.

I make a user on our staging machine so you can try your self. I make a JWT token for you, it is valid for 7 days unless we rotate the key by then. When I run it I get the loader and when I pass PLEASE=true it wil halt/sleep of ±1 sec and then become loaded.

jwt = "eyJhbGciOiJIUzI1NiJ9.eyJ1c2VyX2lkIjoxOTc3LCJleHAiOjE1Njg2NzQ0NTB9.MwW0GrzyaRJUizMxs4KUAIEIbdzKqVlRARDd1WqDa_M"
browser = Ferrum::Browser.new(logger: STDOUT)
browser.goto("https://snotr2019.fakturabank.no")
browser.evaluate([
  "localStorage.setItem('jwt', '#{jwt}');",
  "localStorage.setItem('locale', 'en');",
].join)
puts browser.evaluate("localStorage.getItem('jwt')")
puts browser.evaluate("localStorage.getItem('locale')")

browser.evaluate([
  "localStorage.setItem('jwt', '#{jwt}');",
  "localStorage.setItem('locale', 'en');",
].join)

browser.goto("https://snotr2019.fakturabank.no/orders/34/pdf")
sleep 1 if ENV["PLEASE"] == "true"

browser.pdf(path: "invoice.pdf", printBackground: true)
browser.quit

[route]

You can simplify it to:

jwt = "eyJhbGciOiJIUzI1NiJ9.eyJ1c2VyX2lkIjoxOTc3LCJleHAiOjE1Njg2NzQ0NTB9.MwW0GrzyaRJUizMxs4KUAIEIbdzKqVlRARDd1WqDa_M"

browser = Ferrum::Browser.new(logger: STDOUT)
browser.goto("https://snotr2019.fakturabank.no")

browser.execute <<~JS
  localStorage.setItem('jwt', '#{jwt}');
  localStorage.setItem('locale', 'en');
JS

browser.goto("https://snotr2019.fakturabank.no/orders/34/pdf")

sleep 1 

browser.pdf(path: "invoice.pdf", printBackground: true)

So what happens is that on your second goto request server responds with 200 ok and Page.navigatedWithinDocument is triggered saying that the page is navigated which is correct. Though some JS is yet to do some work fetching some data and rendering it on the page. The thing is ferrum is fast and it executes faster then JS that is about to run. You caught the page in the intermediate state. Unfortunately there's no way to bypass it because there might always be some JS that is running in the background and we can't say for sure if it's going to fetch data. I'm gonna add a few methods in the future one for network activity another one is for waiting some selector like in puppeteer meanwhile you better stick with sleep. But if you are testing your application take a look at https://github.com/machinio/cuprite (it uses ferrum) and use waiting matchers.

[stoivo]

I will try not to bring in capybara jet. I did this and it works rather good

    Timeout.timeout(10) do
      until browser.evaluate <<~JS
          document.querySelector('.loading-content') === null &&
          document.querySelector('.paginated').children.length !== 0
        JS
        sleep 0.1
      end
    end

Which worked better than sleep. Thank you very much for the gem. You are awesome!

We used to have a express server using puppeteer. Now I have ported the code over to run in sidekiq instead and it is about 35% faster. So a lot of thanks to you.

Puppeter has a format option to pdf. Chrome dev protocoll only takes height and width. Would you be open to a PR implementing something similar based on https://github.com/GoogleChrome/puppeteer/blob/e0c8d46af1ce2c063778b66aa29c4c00ab8aeba5/lib/Page.js#L1180-L1193. If you would like to keep ferrum option as close to official guild i understand.

[route]

Yea these options look very convenient! Just some heads up on timeout.
I better hurry up and implement something like wait_for don't like how this evaluate looks lol.
Oh wow 35% that sounds great! Thanks :)