heroku / heroku-buildpack-xvfb-google-chrome

[DEPRECATED] Fork of heroku/heroku-buildpack-google-chrome that runs with Xvfb

heroku-buildpack-xvfb-google-chrome

This buildpack downloads and installs Xvfb and Google Chrome from your choice of release channels.

Note that Google Chrome now has a --headless flag, so for many applications, Xvfb may not be required. If so, consider this buildpack instead.

Only supported on Cedar-14

Note that this buildpack only works on the Cedar-14 stack, for which the end-of-life window began May 1, 2019.

For newer stacks such as Heroku-16 and Heroku-18 you must instead use the more modern Google Chrome Buildpack, which runs Chrome in headless mode without Xvfb. For more details see Browser and User Acceptance Testing.

Channels

You can choose your Chrome release channel by specifying GOOGLE_CHROME_CHANNEL as a config var for your app, in your app.json (for Heroku CI and Review Apps), or in your pipeline settings (for Heroku CI).

Valid values are stable, beta, and unstable. If unspecified, the stable channel will be used.

Shims and Command Line Flags

This buildpack installs shims that always add --disable-gpu and --no-sandbox to any google-chrome command as you'll have trouble running Chrome on a Heroku dyno otherwise.

You'll have two of these shims on your path: google-chrome and google-chrome-$GOOGLE_CHROME_CHANNEL. They both point to the binary of the selected channel.

Selenium

To use Selenium with this buildpack, you'll also need Chrome's webdriver. This buildpack does not install chromedriver, but there is a chromedriver buildpack also available.

Additionally, chromedriver expects Chrome to be installed at /usr/bin/google-chrome, but that's a read-only filesystem in a Heroku slug. You'll need to tell Selenium/chromedriver that the chrome binary is at /app/.apt/usr/bin/google-chrome instead.

To make that easier, this buildpack makes $GOOGLE_CHROME_BIN, and $GOOGLE_CHROME_SHIM available as environment variables. With them, you can use the standard location locally and the custom location on Heroku. An example configuration for Ruby's Capybara:

chrome_bin = ENV.fetch('GOOGLE_CHROME_SHIM', nil)

chrome_opts = chrome_bin ? { "chromeOptions" => { "binary" => chrome_bin } } : {}

Capybara.register_driver :chrome do |app|
  Capybara::Selenium::Driver.new(
     app,
     browser: :chrome,
     desired_capabilities: Selenium::WebDriver::Remote::Capabilities.chrome(chrome_opts)
  )
end

Releasing a new version

Make sure you publish this buildpack in the buildpack registry

heroku buildpacks:publish heroku/xvfb-google-chrome master

heroku-buildpack-xvfb-google-chrome's Issues

Need to support export file

In addition to installinb binaries we need to implement the export file so that other buildpacks can access the installed binaries.

Reference implementation from Ruby: https://github.com/heroku/heroku-buildpack-ruby/pull/319/files

Support ticket: https://support.heroku.com/tickets/506416

NSS version

I got this error recently when using this buildpack with the stable channel:

[1.018][SEVERE]: NSS_VersionCheck("3.26") failed. NSS >= 3.26 is required. Please upgrade to the latest NSS, and if you still get this error, contact your distribution maintainer.
#0 0x000000503fd7 <unknown>
#1 0x00000050be01 <unknown>
#2 0x0000007324c6 <unknown>
#3 0x000000672b25 <unknown>
#4 0x000000594de6 <unknown>
#5 0x00000057c084 <unknown>
#6 0x000000572f62 <unknown>
#7 0x00000046c066 <unknown>
#8 0x000000657b6a <unknown>
#9 0x000000655aea <unknown>
#10 0x000000544e7b <unknown>
#11 0x00000050da05 <unknown>
#12 0x00000050df42 <unknown>
#13 0x00000050e224 <unknown>
#14 0x000000510569 <unknown>
#15 0x00000051afc0 <unknown>
#16 0x000000530b27 <unknown>
#17 0x00000052c813 <unknown>
#18 0x7fe6bc386184 start_thread
#19 0x7fe6bc0b337d clone

It looks like the most recent NSS library is not installed.

Using SHIM cant find chrome binary and BIN causes ReadTimeout upon instantiating WebDriver

Selenium::WebDriver::Error::UnknownError: unknown error: no chrome binary at /tmp/build_ebdd71b82c2dd665242b0a925b088ada/.apt/usr/bin/google-chrome-stable
  (Driver info: chromedriver=2.30.477691 (6ee44a7247c639c0703f291d320bdf05c1531b57),platform=Linux 3.13.0-123-generic x86_64)
        from lib/scrap.rb:14:in `run'
        from (irb):2

If I use GOOGLE_CHROME_BIN it will work but immediately will fail when instantiating driver (line 14 below)

Net::ReadTimeout: Net::ReadTimeout
        from lib/scrap.rb:14:in `run'
        from (irb):2

driver = Selenium::WebDriver.for :chrome, desired_capabilities: Selenium::WebDriver::Remote::Capabilities.chrome(chrome_opts)

I had my script working on heroku with headless previously but now changing to xvfb as the headless version was losing connection half way through script when going through a page with infinite scroll.

Script runs fine locally in non-headless mode.

Wrong GOOGLE_CHROME_BIN path

Tried this buildpack on Heroku CI for UAT. GOOGLE_CHROME_BIN env var contained path to original chrome bin, though another path /app/.apt/usr/bin/google-chrome was required to find and exec Chrome without crash. Also, readme has err: it declares chrome_opts var, but uses chrome_options later.

For now my local fix is to use hardcoded /app/.apt/usr/bin/google-chrome in driver setup, if GOOGLE_CHROME_BIN is present.

org.openqa.selenium.remote.UnreachableBrowserException: Could not start a new session. Possible causes are invalid address of the remote server or browser start-up failure.

Hi! I'm getting this startup exception when I do a deploy on Heroku. I'm hoping someone can help me with it.

2018-07-26T17:01:00.929284+00:00 app[web.1]: org.openqa.selenium.remote.UnreachableBrowserException: Could not start a new session. Possible causes are invalid address of the remote server or browser start-up failure.
2018-07-26T17:01:00.929285+00:00 app[web.1]: Build info: version: 'unknown', revision: 'unknown', time: 'unknown'
2018-07-26T17:01:00.929287+00:00 app[web.1]: System info: host: 'ebd033c4-4bf1-41cd-8ce7-4d345aa7e6e2', ip: '172.17.207.246', os.name: 'Linux', os.arch: 'amd64', os.version: '4.4.0-1019-aws', java.version: '1.8.0_171-heroku'
2018-07-26T17:01:00.929289+00:00 app[web.1]: Driver info: driver.version: ChromeDriver
2018-07-26T17:01:00.929291+00:00 app[web.1]: at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:622) ~[selenium-remote-driver-3.9.1.jar!/:na]
2018-07-26T17:01:00.929292+00:00 app[web.1]: at org.openqa.selenium.remote.RemoteWebDriver.startSession(RemoteWebDriver.java:219) ~[selenium-remote-driver-3.9.1.jar!/:na]
2018-07-26T17:01:00.929304+00:00 app[web.1]: at org.openqa.selenium.remote.RemoteWebDriver.<init>(RemoteWebDriver.java:142) ~[selenium-remote-driver-3.9.1.jar!/:na]
2018-07-26T17:01:00.929305+00:00 app[web.1]: at org.openqa.selenium.chrome.ChromeDriver.<init>(ChromeDriver.java:181) ~[selenium-chrome-driver-3.9.1.jar!/:na]
2018-07-26T17:01:00.929307+00:00 app[web.1]: at org.openqa.selenium.chrome.ChromeDriver.<init>(ChromeDriver.java:168) ~[selenium-chrome-driver-3.9.1.jar!/:na]
2018-07-26T17:01:00.929308+00:00 app[web.1]: at org.openqa.selenium.chrome.ChromeDriver.<init>(ChromeDriver.java:157) ~[selenium-chrome-driver-3.9.1.jar!/:na]
2018-07-26T17:01:00.929310+00:00 app[web.1]: at com.stonesoupprogramming.automotivecms.cmsbackend.selenium.AbstractScrapeInstance.create(WebScraping.kt:27) ~[classes!/:0.0.1-SNAPSHOT]
2018-07-26T17:01:00.929312+00:00 app[web.1]: at com.stonesoupprogramming.automotivecms.cmsbackend.service.BaseScrapeService.runScrape(ScrapeService.kt:26) ~[classes!/:0.0.1-SNAPSHOT]
2018-07-26T17:01:00.929314+00:00 app[web.1]: at com.stonesoupprogramming.automotivecms.cmsbackend.service.BaseScrapeService$$FastClassBySpringCGLIB$$cc359a12.invoke(<generated>) [classes!/:0.0.1-SNAPSHOT]
2018-07-26T17:01:00.929315+00:00 app[web.1]: at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) [spring-core-5.0.7.RELEASE.jar!/:5.0.7.RELEASE]
2018-07-26T17:01:00.929317+00:00 app[web.1]: at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:746) [spring-aop-5.0.7.RELEASE.jar!/:5.0.7.RELEASE]
2018-07-26T17:01:00.929319+00:00 app[web.1]: at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) [spring-aop-5.0.7.RELEASE.jar!/:5.0.7.RELEASE]
2018-07-26T17:01:00.929321+00:00 app[web.1]: at org.springframework.aop.interceptor.AsyncExecutionInterceptor.lambda$invoke$0(AsyncExecutionInterceptor.java:115) [spring-aop-5.0.7.RELEASE.jar!/:5.0.7.RELEASE]
2018-07-26T17:01:00.929322+00:00 app[web.1]: at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_171-heroku]
2018-07-26T17:01:00.929324+00:00 app[web.1]: at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[na:1.8.0_171-heroku]
2018-07-26T17:01:00.929326+00:00 app[web.1]: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[na:1.8.0_171-heroku]
2018-07-26T17:01:00.929327+00:00 app[web.1]: at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_171-heroku]

This is the code that I am using to launch the selenium instance.

          with((ChromeOptions())){
                logger.info("Chrome binary at: ${System.getenv("GOOGLE_CHROME_BIN")}")
                setHeadless(true)
                setBinary(System.getenv("GOOGLE_CHROME_BIN"))
                addArguments(listOf("--disable-dev-shm-usage", "--no-sandbox", "--disable-gpu --remote-debugging-port=9222"))
                [email protected] = ChromeDriver(this@with)
            }

I also tried with with the GOOGLE_CHROME_SHIM variable but that doesn't seem to make a difference. My code is Kotlin running on Spring Boot 2. Thanks for the help!

Heroku 16

When will you support Heroku 16 stack?

Chrome crashes when started with ChromeDriver

This is a blocking issue to make proper E2E tests with Heroku CI.

It also happens on the headless version.

Here's what happens:

• Chrome crashes when its started through ChromeDriver (v2.9, latest version).
• This issue happens on regular dynos and on Heroku CI.
• It crashes on Heroku-16 AND Cedar-14 stacks.

How to reproduce:

• Deploy the Xvfb showcase.
• Run a one-off dyno with heroku run bash.
• Run ./run.sh.

You should observe some logs like those:

Starting ChromeDriver (v2.9.248304) on port 9515
[1.947][INFO]: COMMAND InitSession {
   "desiredCapabilities": {
      "browserName": "chrome",
      "chromeOptions": {
         "binary": "/app/.apt/opt/google/chrome/chrome"
      },
      "platform": "ANY"
   }
}
[1.947][INFO]: Populating Preferences file: {
   "alternate_error_pages": {
      "enabled": false
   },
   "autofill": {
      "enabled": false
   },
   "browser": {
      "check_default_browser": false
   },
   "distribution": {
      "import_bookmarks": false,
      "import_history": false,
      "import_search_engine": false,
      "make_chrome_default_for_user": false,
      "show_welcome_page": false,
      "skip_first_run_ui": true
   },
   "dns_prefetching": {
      "enabled": false
   },
   "profile": {
      "content_settings": {
         "pattern_pairs": {
            "https://*,*": {
               "media-stream": {
                  "audio": "Default",
                  "video": "Default"
               }
            }
         }
      },
      "default_content_settings": {
         "geolocation": 1,
         "mouselock": 1,
         "notifications": 1,
         "popups": 1,
         "ppapi-broker": 1
      },
      "password_manager_enabled": false
   },
   "safebrowsing": {
      "enabled": false
   },
   "search": {
      "suggest_enabled": false
   },
   "translate": {
      "enabled": false
   }
}
[1.948][INFO]: Populating Local State file: {
   "background_mode": {
      "enabled": false
   },
   "ssl": {
      "rev_checking": {
         "enabled": false
      }
   }
}
[1.952][INFO]: Launching chrome: /app/.apt/opt/google/chrome/chrome --disable-background-networking --disable-client-side-phishing-detection --disable-component-update --disable-default-apps --disable-hang-monitor --disable-prompt-on-repost --disable-sync --disable-web-resources --enable-logging --full-memory-crash-report --ignore-certificate-errors --load-extension=/tmp/.com.google.Chrome.pvYTke/internal --logging-level=1 --metrics-recording-only --no-first-run --password-store=basic --remote-debugging-port=12848 --safebrowsing-disable-auto-update --safebrowsing-disable-download-protection --use-mock-keychain --user-data-dir=/tmp/.com.google.Chrome.6F4Zir data:,
[1.952][DEBUG]: DevTools request: http://127.0.0.1:12848/json/version
[1.953][WARNING]: PAC support disabled because there is no system implementation
[2.263][DEBUG]: DevTools request failed
[23:23:0507/152545.736335:FATAL:setuid_sandbox_host.cc(157)] The SUID sandbox helper binary was found, but is not configured correctly. Rather than run without sandboxing I'm aborting now. You need to make sure that /app/.apt/opt/google/chrome/chrome-sandbox is owned by root and has mode 4755.
[2.317][DEBUG]: DevTools request: http://127.0.0.1:12848/json/version
[2.318][DEBUG]: DevTools request failed
[2.368][DEBUG]: DevTools request: http://127.0.0.1:12848/json/version
[2.369][DEBUG]: DevTools request failed
[2.421][DEBUG]: DevTools request: http://127.0.0.1:12848/json/version
[2.422][DEBUG]: DevTools request failed
...
...
...
[61.849][DEBUG]: DevTools request: http://127.0.0.1:12848/json/version
[61.850][DEBUG]: DevTools request failed
[61.901][DEBUG]: DevTools request: http://127.0.0.1:12848/json/version
[61.902][DEBUG]: DevTools request failed
[61.953][DEBUG]: DevTools request: http://127.0.0.1:12848/json/version
[61.954][DEBUG]: DevTools request failed
[61.954][INFO]: RESPONSE InitSession unknown error: Chrome failed to start: crashed
[61.954][DEBUG]: Log type 'driver' lost 0 entries on destruction
[61.954][DEBUG]: Log type 'browser' lost 0 entries on destruction

And the cURL response:

{
    "sessionId": "6dac5aa2b1513fa66d0e4267f59df42c",
    "status": 13,
    "value": {
        "message": "unknown error: Chrome failed to start: crashed\n  (Driver info: chromedriver=2.9.248304,platform=Linux 3.13.0-112-generic x86_64)"
    }
}

Overriding default_directory does not appear to work

The below works in a local instance, but as soon as I run on Heroku the files download into the default location (/app/Downloads/). Do you know for what reason we wouldn't be able to update the default download location?

I've confirmed the directory I'm attempting to download into is present on Heroku at the time
of download.

prefs = {
            'download' => {
                'default_directory' => download_dir,
                'prompt_for_download' => false,
            },
            'profile' => {
                'default_content_settings' => {'multiple-automatic-downloads' => 1}, #for chrome version olde ~42
                'default_content_setting_values' => {'automatic_downloads' => 1}, #for chrome newe 46
            }
        }

        caps = Selenium::WebDriver::Remote::Capabilities.chrome
        caps['chromeOptions'] = {:prefs => prefs}

        @browser = Watir::Browser.new :chrome, :desired_capabilities => caps

Missing libnss3

When I attempt to use the current master branch of this buildpack it selenium (via Capybara in ruby) fails to connect. It returns:

Selenium::WebDriver::Error::UnknownError: unknown error: Chrome failed to start: exited abnormally
  (Driver info: chromedriver=2.25.426924 (649f9b868f6783ec9de71c123212b908bf3b232e),platform=Linux 3.13.0-112-generic x86_64)

I found a fork that includes libnss3 and that seems to fix it. Would you consider merging that into master?