GithubHelp home page GithubHelp logo

watzon / marionette Goto Github PK

View Code? Open in Web Editor NEW
177.0 12.0 14.0 1.46 MB

Selenium alternative for Crystal. Browser manipulation without the Java overhead.

Home Page: https://watzon.github.io/marionette/

License: MIT License

Crystal 97.69% JavaScript 2.31%
puppeteer marionette headless firefox headless-browser testing devtools api browser crystal

marionette's Introduction

Marionette

Looking for a co-maintainer for Marionette. If interested just respond in the issue titled "Looking for a co-maintainer".

GitHub Workflow Status License Crystal Version

Marionette is a one-size-fits-all approach to WebDriver adapters. It works with most all web driver implementations, including:

  • Chrome
  • Chromium
  • Firefox
  • Safari
  • Edge
  • Internet Explorer
  • Opera
  • PhantomJS
  • Webkit GTK
  • WPE Webkit
  • Android

Table of Contents

Installation

  1. Make sure you have Crystal installed. This is a Crystal project and Crystal is required for usage. If you don't have it installed, see https://crystal-lang.org.

  2. Add Marionette to an existing project by adding the dependency to your shard.yml

    dependencies:
      marionette:
        github: watzon/marionette
        branch: master
  3. Run shards install to download and install Marionette as a dependency.

  4. Download and have installed at least one WebDriver. See the #webdriver section below for links to various downloads.

WebDriver

WebDriver is a protocol which allows browser implementations to be remote controlled via a common interface. It's because of this functionality that frameworks like Marionette are possible. To use the protocol you first have to have installed one of the many WebDriver implementations, here are some of those:

Firefox

GeckoDriver is implemented and supported by Mozilla directly.

Chrome

ChromeDriver is implemented and supported by the Chromium Project.

Opera

OperaChromiumDriver is implemented and supported by Opera Software.

Safari

SafariDriver is implemented and supported directy by Apple. It comes pre-installed with Safari and Safari Technology Preview.

Edge

Microsoft is implementing and maintaining the Microsoft Edge WebDriver.

Internet Explorer

Only version 11 is supported, and it requires additional configuration.

Note: Marionette specific configuration instructions coming soon.

Getting Started

The goal of Marionette is simplicity, which is why it's written in Crystal. Once you have a webdriver installed and sitting comfortably on your path, using it couldn't be easier:

require "marionette"

session = Marionette::WebDriver.create_session(:chrome)

# Navigate to crystal-lang.org
session.navigate("https://crystal-lang.org")

# Start an action chain and perform it
session.perform_actions do
  # Click the "INSTALL" link
  click ".main-actions a:nth-child(1)"
end

sleep 5
session.close

Driver Capabilities

Different drivers have different capabilities available to them. To make setting them a little easier, there's the DriverOptions module which is extended by Marionette itself. Take, for instance, Chrome:

# Make this instance headless
options = Marionette.chrome_options(args: ["headless"])

# Create a Chrome session
session = Marionette::WebDriver.create_session(:chrome, capabilities: options)

args in this case are arguments to be passed to the browser itself rather than the driver. If you wish to pass arguments to the driver you can use the args parameter in the create_session method.

Browser Manipulation

As shown above, you can initialize a new driver session for whatever driver you want using Marionette::WebDriver.create_session, the first and most important argument to which is :browser. Browser can be any of :chrome, :firefox, :opera, :safari, :edge, :internet_explorer, :webkit_gtk, :wpe_webkit, or :android.

If the driver for the chosen browser is installed under its usual name that should be all you need to do, if not you may need to provide the binary location via the :exe_path argument. Other notable arguments are:

  • :port - sets the port you want the driver to listen on
  • :env - a hash of environment variables for the driver to be aware of
  • :args - a string array of arguments to pass to the webdriver process
  • :options - a JSON compatible structure containing browser options. see here for some nice helpers.

Navigation

#navigate

The first thing you will want to do after launching a browser is to open your website. This can be achieved in a single line:

session.navigate("https://crystal-lang.org")

#current_url

You can read the current URL from the browser’s address bar using:

session.current_url
# => https://crystal-lang.org

#back

Pressing the browser’s back button:

session.back

#forward

Pressing the browser’s forward button:

session.forward

#refresh

Refresh the current page:

session.refresh

#title

You can read the current page title from the browser:

session.title
# => Crystal | The Crystal Programming Language

Windows and Tabs

WebDriver does not make the distinction between windows and tabs. If your site opens a new tab or window, Marionette will let you work with it using a window handle. Each window has a unique identifier which remains persistent in a single session.

#current_window

You can get the currently active window using:

session.current_window

This returns a Window instance containing a handle and allowing certain functions to be performed directly on the window instance.

#windows

You can get an array of all currently opened windows using:

session.windows

#new_window

You can create a new window or tab using:

session.new_window(:window) # default
session.new_window(:tab)

# Or using the Window object

Marionette::Window.new(:window) # default
Marionette::Window.new(:tab)

#switch_to_window

To interact with other windows you have to switch to them, this can be done with:

session.switch_to_window(window)

# Or using the Window object

window.switch

#close_window

When you are finished with a window or tab and it is not the last window or tab open in your browser, you should close it and switch back to the window you were using previously:

session.close_window(window)

# Or using the Window object

window.close

#close_current_window

Think of this is a shortcut to #close_window but for the currently active window:

session.close_current_window

#stop

When you are finished with the browser session you should call stop, instead of close:

session.stop

Stop will:

  • Close all the windows and tabs associated with that WebDriver session
  • Close the browser process
  • Close the background driver process

Stop will be automatically closed on process exit.

Frames and IFrames

Frames are a now deprecated means of building a site layout from multiple documents on the same domain. You are unlikely to work with them unless you are working with an pre HTML5 webapp. Iframes allow the insertion of a document from an entirely different domain, and are still commonly used.

If you need to work with frames or iframes, WebDriver allows you to work with them in the same way. Consider a button within an iframe. If we inspect the element using the browser development tools, we might see the following:

<div id="modal">
  <iframe id="buttonframe" name="myframe"  src="https://watzon.github.io">
   <button>Click here</button>
 </iframe>
</div>

If it was not for the iframe we would expect to click on the button using something like:

session.find_element!("button").click

However, if there are no buttons outside of the iframe, you might instead get a no such element error. This happens because Marionette is only aware of the elements in the top level document. To interact with the button, we will need to first switch to the frame, in a similar way to how we switch windows. WebDriver offers three ways of switching to a frame.

#switch_to_frame

The switch_to_frame session method allows us to tell the WebDriver that we want to switch the page context to the given frame/iframe:

# Find the element
iframe = session.find_element!("#modal>iframe")

# Switch to the frame
session.switch_to_frame(iframe)

# Now we can click the button
session.find_element!("button").click

#switch_to_parent_frame

If you're in a nested set of frames you can switch back to the parent frame using:

session.switch_to_parent_frame

#leave_frame

When you're done inside a frame and want to get back to the normal document context you can use:

session.leave_frame

Window Management

Screen resolution can impact how your web application renders, so WebDriver provides mechanisms for moving and resizing the browser window.

#size

You can fetch window dimensions either collectively or individually:

# First get the current window
window = session.current_window

# Access dimensions individually
width = window.width
height = window.height

# Or collectively as a Size object
window.size
# => Size(width: 800.0, height: 600.0)

#resize_to

To resize the window:

window.resize_to(600, 800)

# Or using an existing Size object `size`

window.size = size

#position

You can fetch the coordinates of the top left corner of the browser window:

window.position
# => Location(x: 0.0, y: 0.0)

#move_to

You can also easily set the position of the window:

window.move_to(250, 250)

# Or using a Location object `location`

window.position = location

#maximize

To maximize the given window:

window.maximize

#minimize

To minimize the given window:

window.minimize

#fullscreen

To make the given window full screen:

window.fullscreen

Working with Elements

Element represents a DOM element. WebElements can be found by searching from the document root using a WebDriver instance, or by searching under another Element.

WebDriver API provides built-in methods to find the WebElements which are based on different properties like ID, Name, Class, XPath, CSS Selectors, link Text, etc.

#active_element

Get and return the active element:

session.active_element

#find_element

find_element and find_element! are used to find and store references to page elements. The only difference between the two is that find_element! will throw an exception if the element is not found and will not be nil, whereas find_element will return nil if the element is not found.

# Get the search box element by name
search_bar = session.find_element!("q", :name)

# Peform an action
search_bar.send_keys("Crystal programming language")

# Click the submit button (using the default :css option this time)
session.find_element!("button.submit").click

#find_elements

Used to find an array of elements matching the given selector:

# Get all elements with the tag name "p"
elements = session.find_elements("p", :tag_name)

elements.each do |el|
  puts el.text
end

#find_child

Used to find a child element within the context of parent element. Like find_element it has raising and non-raising varients:

# Find the search form
form = session.find_element!("form", :tag_name)

# Find the form's child search box
search_bar = form.find_child!("q", :name)

search_bar.send_keys("Crystal programming language")

#find_children

Just like find_elements and find_child:

element = session.find_element!("div", :tag_name)

paragraphs = element.find_children("p", :tag_name)

paragraphs.each do |para|
  puts para.text
end

Contributing

  1. Fork it ( https://github.com/watzon/marionette/fork )
  2. Create your feature branch (git checkout -b my-new-feature)
  3. Commit your changes (git commit -am 'Add some feature')
  4. Push to the branch (git push origin my-new-feature)
  5. Create a new Pull Request

Contributors

This project exists thanks to all the people who contribute.

marionette's People

Contributors

bararchy avatar blobcodes avatar chocolateboy avatar diegogub avatar eliasjpr avatar ikaru5 avatar lucasintel avatar sija avatar watzon avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

marionette's Issues

How to remove everything but the web page content area?

I'd like to make the page full screen, disable F11, and not show the Chrome is being controlled by automated test software notification.

  • I can get the fullscreen part via:
      options = Marionette.chrome_options(args: ["start-fullscreen"])
      session = Marionette::WebDriver.create_session(:chrome, exe_path: webdriver_path, capabilities: options)
  • I'm not sure how to disable F11.

  • This link looks helpful for removing the Chrome is being ... part, but I'm not sure how it translates to marionette:

https://help.applitools.com/hc/en-us/articles/360007189411--Chrome-is-being-controlled-by-automated-test-software-notification

Export to PDF

Puppeteer has the ability to export a page to a PDF, and that feature was recently added to Firefox as well.

Is it possible to support this in Marionette? As far as I can tell, the WebDriver protocol does not support this feature.

Examples

A complete example would be useful.

Undefined constant LocatorStrategy on HTMLElement.find_element

There seem to be something broken with the instance method find_element of HTMLElement.

    # Returns an `HTMLElement` instance that matches the specified
    # method and target, relative to the current element.
    def find_element(method, target)
      @browser.find_element(by : LocatorStrategy, target, @id)
    end

Also I assume that the next method starting on line 22, should be named find_elements.

I fixed both of them like this for me:

    # Returns an `HTMLElement` instance that matches the specified
    # method and target, relative to the current element.
    def find_element(by, target)
      @browser.find_element(by, target, @id)
    end

    # Returns an array of all `HTMLElement`s that matches the specified
    # method and target, relative to the current element.
    def find_elements(by, target)
      @browser.find_elements(by, target, @id)
    end

Error in execute async

# Execute JS script asynchronously. See `#execute_script`.
    def execute_script_async(script, args = nil, timeout = @timeout, new_sandbox = true)
      params = {
        scriptTimeout: timeout,
        script:        script,
        args:          args || [] of String,
        newSandbox:    sandbox,
      }

      debug("Executing async script")

      response = @transport.request("WebDriver:ExecuteScriptAsync", params)
      response["value"]?
    end

param name is new_sandbox and arg is sandbox

Launcher does not provide enough time to start Firefox

On launch I get an Unhandled exception: Error connecting to '127.0.0.1:2828': Connection refused (Socket::ConnectError).

I'm running on an ssd and a fast Ryzen CPU, so firefox starts pretty fast.

To quickfix this for me I added a sleep 3 in launcher.cr on line 81 like this:

        # ...
        stderr ||= Process::ORIGINAL_STDERR
        Process.new(executable.to_s, args, env, output: stdout, error: stderr)
        sleep 3
      end

      browser = Browser.new(address, port, extended, timeout)
      # ...

external cookie file

I have a cookie file I want to use so I dont have to auth everything. Is there a way to use an external cookie file?

Failed to install shard because of http_proxy_server

I've created a new app. Added the dependency and typed in shards install:

[renich@introdesk ehdown]$ cat shard.yml 
name: ehdown
version: 0.1.0

authors:
  - Rene Bon Ciric <[email protected]>

targets:
  ehdown:
    main: src/ehdown.cr

crystal: 0.33.0

license: MIT

dependencies:
  marionette:
    github: watzon/marionette

[renich@introdesk ehdown]$ shards install
Resolving dependencies
Fetching https://github.com/watzon/marionette.git
Fetching https://github.com/hydecr/strange.git
Fetching https://github.com/mamantoha/http_proxy.git
Error shard name (http_proxy_server) doesn't match dependency name (http_proxy)

Clearing cookies on close_session

Hi
is there any option to clean browser cookies/session after finishing work with marionette? I’m currently using marionette to log in my account on some site. But after I close firefox browser and open it again, I’m still logged in. I’ve tried to manipulate with cookies and set-cookies headers when I log in, but no luck. Also it would be great have ability to do not close browser, but just some kind of re-session marionette to start test again?
Probably marionette has some kind of private or incognito mode, when I close the session it cleans all the cookies?
Any advice is really appreciated.
Thanks in advance.

Support unicode for filling forms

Hello. I tried to fill a value with keyboard input.
English is OK. but it seems it doesn't accept other languages.
I think it is encoding problem.
I hope I can fill it with various languages.

Thank you.

Screenshot returning nil?

Putting this here so I can show progress. cc: @bararchy

Apparently there is a situation where screenshot might return nil

Error in Cross-Site Scripting entry_point http://vulnerable-bank.com/app_v3_banking.php: Expected Hash for #[](key : String), not Nil (Exception)

  from /usr/share/crystal/src/json/any.cr:104:7 in '[]'

  from lib/marionette/src/marionette/browser.cr:594:14 in 'screenshot'

  from /usr/share/crystal/src/fiber.cr:255:3 in 'run'

  from ???

Find elements is broken

find_elements crashes with


In lib/marionette/src/marionette/session.cr:456:10

 456 | if parent
          ^-----
Error: undefined local variable or method 'parent' for Marionette::Session

Fixing that (by adding parent to parameters) raises a new error because ids are not hooked up right. I think the method needs a look, looks like the "Huge update" in november broke some stuff

I'm sure this is user error, but I don't know what the problem is

If I run Marionette.launch I get the following error:

[DEBUG  ] - Using firefox executable at /usr/bin/firefox
[DEBUG  ] - Launching browser
Unhandled exception: Error connecting to '127.0.0.1:2828': Connection refused (Errno)
  from /usr/lib/crystal/socket/tcp_socket.cr:73:15 in 'initialize'
  from /usr/lib/crystal/socket/tcp_socket.cr:27:3 in 'new:dns_timeout:connect_timeout'
  from lib/marionette/src/marionette/transport.cr:19:17 in 'initialize'
  from lib/marionette/src/marionette/transport.cr:18:5 in 'new'
  from lib/marionette/src/marionette/browser.cr:119:20 in 'initialize'
  from lib/marionette/src/marionette/browser.cr:118:5 in 'new'
  from lib/marionette/src/marionette/launcher.cr:83:7 in 'launch'
  from lib/marionette/src/marionette/launcher.cr:36:5 in 'launch:stdout:stderr:args:default_viewport'
  from src/bot.cr:16:5 in 'main_loop'
  from src/bot.cr:65:1 in '__crystal_main'
  from /usr/lib/crystal/crystal/main.cr:106:5 in 'main_user_code'
  from /usr/lib/crystal/crystal/main.cr:92:7 in 'main'
  from /usr/lib/crystal/crystal/main.cr:115:3 in 'main'
  from __libc_start_main
  from _start
  from ???

TypeCastError from #page_source

Hi,

I suspect I'm doing something wrong here but I can't see it. What am I missing?

Test case

foo.cr:

require "marionette"

session = Marionette::WebDriver.create_session(:chrome)
session.navigate("https://example.com")
puts session.page_source

shard.yml:

name: foo
version: 0.0.1

targets:
  foo:
    main: foo.cr

dependencies:
  marionette:
    github: watzon/marionette
    branch: master

crystal: 1.0.0

license: MIT

Error

martin:/tmp$ shards install
Resolving dependencies
Fetching https://github.com/watzon/marionette.git
Installing marionette (0.3.0 at a962bb3)
Writing shard.lock
martin:/tmp$
martin:/tmp$ crystal --version
Crystal 1.0.0 [dd40a2442] (2021-03-22)

LLVM: 10.0.0
Default target: x86_64-unknown-linux-gnu
martin:/tmp$
martin:/tmp$ crystal foo.cr
Unhandled exception: cast from Hash(String, JSON::Any) to String failed, at /usr/share/crystal/src/json/any.cr:215:5:215 (TypeCastError)
  from /usr/share/crystal/src/json/any.cr:214:3 in 'as_s'
  from lib/marionette/src/marionette/session.cr:356:7 in 'page_source'
  from foo.cr:5:1 in '__crystal_main'
  from /usr/share/crystal/src/crystal/main.cr:110:5 in 'main_user_code'
  from /usr/share/crystal/src/crystal/main.cr:96:7 in 'main'
  from /usr/share/crystal/src/crystal/main.cr:119:3 in 'main'
  from __libc_start_main
  from _start
  from ???
martin:/tmp$ 

Looking for a co-maintainer

I don't have a ton of time for open source projects like this lately, so things have been going by the wayside. That being said I know that several of my projects including Marionette are used in a few other projects, so if anyone would be interested in co-maintaining this project let me know.

Pageloadstrategy not supported

require "marionette"

options = Marionette.chrome_options(args: ["headless"],  page_load_strategy: Marionette::PageLoadStrategy::Eager)
session = Marionette::WebDriver.create_session(:chrome, exe_path: "bin/chromedriver", capabilities: options)
Unhandled exception: invalid argument: cannot parse capability: goog:chromeOptions
from invalid argument: unrecognized chrome option: pageLoadStrategy (Marionette::Error::InvalidArgument)
  from ................/crystal/src/class.cr:148:0 in 'execute'
  from lib/marionette/src/marionette/session.cr:49:18 in 'start'
  from lib/marionette/src/marionette/session.cr:34:5 in 'start:capabilities'
  from lib/marionette/src/marionette/web_driver.cr:46:7 in 'get_session:capabilities'
  from lib/marionette/src/marionette/web_driver.cr:40:7 in 'create_session:exe_path:capabilities'
  from src/scripts/marionette.cr:4:1 in '__crystal_main'

Error running at_exit handler

When I ran the basic example with Firefox, it gives me this error:

Error running at_exit handler: Tried to run command without establishing a connection

Here's the source code:

require "marionette"

session = Marionette::WebDriver.create_session(:firefox)

# Navigate to crystal-lang.org
session.navigate("https://crystal-lang.org")

# Start an action chain and perform it
session.perform_actions do
  # Click the "INSTALL" link
  click ".main-actions a:nth-child(1)"
end

sleep 5
session.close

Also, the geckodriver version:

$ geckodriver --version
geckodriver 0.26.0 (e9783a644016 2019-10-10 13:38 +0000)

The source code of this program is available from
testing/geckodriver in https://hg.mozilla.org/mozilla-central.

This program is subject to the terms of the Mozilla Public License 2.0.
You can obtain a copy of the license at https://mozilla.org/MPL/2.0/.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.