pyqa / prodigyqa

Unified Test Automation Framework in Python

Unified QA automation framework!

prodigyqa is a unified test automation framework based on python. This has come up after thoroughly study made on existing methodologies used in majority of projects for UI/Rest API testing, This is expected to solve a list of problem statements readily. The framework aims to be a constructive blend of various guidelines, coding standards, concepts, processes, practices, project hierarchies, modularity, reporting mechanism, test data injections etc., to pillar automation testing.

Key Features

• Selenium functions: This module contains a good number of Selenium wrapped functions which are commonly used for UI Testing. These wrapped functions provide a shield of exception handling over the regular selenium methods, handling all different kinds of exception that can occur. If due to some reason, any of the provided method doesnot work, then there is a provision of fall back mechanism, provided in each method, by leveraging additional technologies/javascript

• Rest API functions: This module contains Rest API wrapped functions which are commonly used for Rest API Testing, by leveraging python 'requests' library. The methods are generalized, to handle any kind of request like GET, PUT, POST, DELETE etc. The module has provison to manage the authentication token. Additionally, there are generic methods for commonly used assertion types and for validating the parameters, that come along with a request like headers, payload etc. in a most possible simplified manner

• Utilities: This module aims to provide community backed utility libraries built with a focus on reusability. Following are the utilities, currently supported by the framework:

  • Image Comparison: This module provides provision for image comparison through openCV, SSIM(Structured similarity index) as opposed to pixel by pixel comparison. The methods are generic and need just 2 images to compute the difference.

  • Captcha Reading: This utility performs the captcha reading from an image, by leveraging the 'pytesseract' module. It takes an image as input, containing captcha, and returns a string, mentioning the captcha.

  • Spell Check, Accessibility Check in web application: This module is in WIP mode, where we are enabling spell checks and accessibility checks in web applications, by leveraging the web spider concept, which browses the World Wide Web in a methodical, automated manner, takes out all the links from a web page so that the process could be repeated.

• Reporting: The framework is flexible enough to work with multiple reporting platforms like reportportal.io, allure-report etc. A regular, self-contained and customizable HTML report can also be generated through the use of pytest-html module.(which is bundled as a dependency for this package)

• Continuous Testing: The framework provides the facility for CT(Continuous Testing) by leveraging Docker. The docker file provided in the framework can be used to setup the necessary prerequisites/environment, needed to run the framework, on any server, from scratch.

Prerequisites

The framework works with

• python 3.5+
• pip
• pytest to be installed in the machine.

Installing

Valid/Tested Version is suggested to be installed directly from PyPi as it will solve the issue of dependencies automatically.

pip install prodigyqa

In case of any custom updates done to the current setup,we will need to clone the current repository and run python setup.py develop so that the local changes are reflected in the install version.

Running the tests

The sample test cases for all the features are listed under prodigyqa --> Examples folder. To run the sample tests, open command prompt/terminal, go to prodigyqa --> Examples folder and run the following command:

pytest {filename}.py -s (-s indicates the standard output, please refer here for a detailed understanding around pytest framework and its features/plugins/options etc.)

Browser Actions

Browser Actions Module method Summary

This module wraps around selenium web driver functions. Below are the major areas handled in this module:

• Most frequently, a DOM refresh will cause an exception (StaleElementReferenceException) if the object is changing state. we are handling this by checking Web Page Expected to be in the ready state and Catch the exception to prevent a failure if the object is not present in web page after web page is in the expected state.
• Most of the functions of this module will Catch the exception to prevent a failure.

Method Name	Description	Args	Usage
page_readiness_wait	Web Page Expected to be in ready state.		self.page_readiness_wait()
locator_check	Local Method to classify the type of locator.	(a)locator_dict: dictionary of locator value, locator by and value	self.locator_check( locator_dict)
open	Open the passed 'url'.		self.open(url)
reload_page	Method to refresh the page by selenium or java script.		self.reload_page()
get_page_source	Return the entire HTML source of the current page or frame.		self.get_page_source()
get_title	Return the title of current page.		self.get_title()
get_location	Return the current browser URL using Selenium/Java Script.		self.get_location()
get_attribute	fetch attribute from provided locator/element/parent element with child locator.	(a) locator: dictionary of identifier type and value ({'by':'id', 'value':'start-of-content.'}). (b) attribute_name: attribute name to get it's value . (c) element: it is a webelement . (d) type : the type value should be 'locator' or 'element' or 'mixed'	self.get_attribute(locator=None, element=None, attribute_name=None, type='locator')
click	Click an element.	(a) locator: dictionary of identifier type and value ({'by':'id', 'value':'start-of-content.'}).	self.click(locator)
send_keys	Send text but does not clear the existing text.	(a) locator: dictionary of identifier type and value ({'by':'id', 'value':'start-of-content.'}). (b) string to send.	self.send_keys(locator)
get_text	Get text from provided Locator.	(a) locator: dictionary of identifier type and value ({'by':'id', 'value':'start-of-content.'}).	self.get_text(locator)
go_back	Simulate back button on browser using selenium or js.		self.go_back()
go_forward	Simulate forward button on browser using selenium or js.		self.go_forward()
set_window_size	Set width and height of the current window. (window.resizeTo)	(a) width: the width in pixels to set the window to. (b) height: the height in pixels to set the window to.	self.set_window_size(800,600)
maximize	Maximize the current window.		self.maximize()
get_driver_name	Return the name of webdriver instance.		self.get_driver_name()
get_domain_url	Method to extract domain url from webdriver itself.		self.get_domain_url()
clear_text	Clear the text if it's a text entry element	(a) locator: dictionary of identifier type and value ({'by':'id', 'value':'start-of-content.'}).	self.clear_text(locator)
capture_screenshot	Save screenshot to the directory(existing or new one).	(a) filepath: file name with directory path(C:/images/image.png).	self.capture_screenshot(self, filepath)
switch_to_active_element	Return the element with focus, or BODY if nothing has focus.		self.switch_to_active_element()
switch_to_window	Switch focus to the specified window using selenium/javascript.	(a) name of the window to switch	self.switch_to_window(window)
switch_to_frame	Switch focus to the specified frame using selenium/javascript.	(a) framename: name of the frame to switch.	self.switch_to_frame(framename)
switch_to_default_content	Switch focus to the default frame.		self.switch_to_default_content()
switch_to_alert	Switch focus to an alert on the page.		self.switch_to_alert()
hover_on_element	Hover on a particular element.	(a) locator: dictionary of identifier type and value ({'by':'id', 'value':'start-of-content.'}).	self.hover_on_element(locator)
hover_on_click	Hover & click a particular element.	(a) locator: dictionary of identifier type and value ({'by':'id', 'value':'start-of-content.'}).	self.hover_on_click(locator)
wait_for_element	Wait for an element to exist in UI.	(a) locator: dictionary of identifier type and value ({'by':'id', 'value':'start-of-content.'}).	self.wait_for_element(locator)
wait_and_accept_alert	Wait and accept alert present on the page.		self.wait_and_accept_alert()
wait_and_reject_alert	Wait for alert and rejects.		self.wait_and_reject_alert()
select_option_by_index	Select the option by index.	(a) locator: dictionary of identifier type and value ({'by':'id', 'value':'start-of-content.'}). (b) index: integer value for index.	self.select_option_by_index(locator, index)
select_option_by_value	Select the option by using value.	(a) locator: dictionary of identifier type and value ({'by':'id', 'value':'start-of-content.'}). (b) value: string value to select option.	self.select_option_by_value(locator, value)
select_option_by_text	Select the value by using text.	(a) locator: dictionary of identifier type and value ({'by':'id', 'value':'start-of-content.'}). (b) text: string value to select option.	self.select_option_by_text(locator, text)
scroll_to_footer	Scroll till end of the page.		self.scroll_to_footer()
scroll_to_element	Scroll to a particular element on the page.	(a) locator: dictionary of identifier type and value ({'by':'id', 'value':'start-of-content.'}).	self.scroll_to_element(locator)
javascript_click	Perfom a click using javscript	(a) locator: dictionary of identifier type and value ({'by':'id', 'value':'start-of-content.'}).	self.java(locator)
is_element_displayed	Return boolean value based on the element is displayed with locator.	(a) locator: dictionary of identifier type and value ({'by':'id', 'value':'start-of-content.'}).	self.is_element_displayed(locator)
is_element_enabled	Return boolean value based on the element is enabled with locator.	(a) locator: dictionary of identifier type and value ({'by':'id', 'value':'start-of-content.'}).	self.is_element_enabled(locator)
is_element_selected	Return boolean value based on the element is selected with locator.	(a) locator: dictionary of identifier type and value ({'by':'id', 'value':'start-of-content.'}).	self.is_element_selected(locator)
switch_to_active_window	Switch focus to new window	Switch focus to new window from current window	self.switch_to_active_window()
switch_to_frame_by_index	Switch focus to the specified frame based on index	Index of the frame	self.switch_to_frame_by_index(index)

API Test Module

REST API Module method Summary

Method Name	Description	Args	Usage
apirequest	triggers rest api request based on the input method and kwargs	(a).method: GET/POST/PUT/PATCH/DELETE (b).kwargs: Refer below REST API kwarg section table	self.apirequest(method='GET')
assert_in_resp	Check whether response data contain input member.	(a)resp: response to validate. (b)member: value to check in response. (c)container: response key path in dot format which should starts with 'resp.'. example: resp.data.0.name	self.assert_in_resp(resp, member, container)
assert_not_in_resp	Check whether response data contain input member.	(a)resp: response to validate. (b)member: value to check in response. (c)container: response key path in dot format which should starts with 'resp.'. example: resp.data.0.name	self.assert_not_in_resp(resp, member, container)
assert_equal_resp	Check whether response data contain input member.	(a)resp: response to validate. (b)member: value to check in response. (c)container: response key path in dot format which should starts with 'resp.'. example: resp.data.0.name	self.assert_equal_resp(resp, member, container)
assert_not_equal_resp	Check whether response data contain input member.	(a)resp: response to validate. (b)member: value to check in response. (c)container: response key path in dot format which should starts with 'resp.'. example: resp.data.0.name	self.assert_not_equal_resp(resp, member, container)

REST API kwarg section

Arg Name	Arg type	Description
url	standard	API request url
params	optional	Dictionary, list of tuples or bytes to send in the body of the Request.
data	optional	Dictionary, list of tuples, bytes, or file-like object to send in the body of the Request.
json	optional	A JSON serializable Python object to send in the body of the Request.
headers	optional	Dictionary of HTTP Headers to send with the Request.
cookies	optional	Dict or CookieJar object to send with the Request.
files	optional	Dictionary of 'name': file-like-objects(or {'name': file-tuple}) for multipart encoding upload. file-tuple can be a 2-tuple ('filename', fileobj), 3-tuple ('filename', fileobj, 'content_type') or a 4-tuple ('filename', fileobj, 'content_type', custom_headers), where 'content-type' is a string defining the content type of the given file and custom_headers a dict-like object containing additional headers to add for the file.
auth	optional	Auth tuple to enable Basic/Digest/Custom HTTP Auth.
timeout (float or tuple)	optional	How many seconds to wait for the server to send data before giving up, as a float, or a (connect timeout, read timeout) tuple.
allow_redirects (bool)	optional	Boolean. Enable/disable GET/OPTIONS/POST/PUT/PATCH/DELETE/HEAD redirection. Defaults to True.
proxies	optional	Dictionary mapping protocol to the URL of the proxy.
verify	optional	Either a boolean, in which case it controls whether we verify the servers TLS certificate, or a string, in which case it must be a path to a CA bundle to use. Defaults to True.
stream	optional	if False, the response content will be immediately downloaded.
cert	optional	if String, path to ssl client cert file (.pem). If Tuple, (cert, key) pair.

Utils

Spell checker of utils Module method Summary

This Spell checker method of utils module will capture web page content and Validate it against the standard dictionary to ensure spell is correct.

Method Name	Description	Args	Usage
spell_checker	Spell checker.	(a) url: web page url (b) words: list of application specific words	spell_checker(self, url, words=[])
l

Comparison Module

Module has a utility methods for comparing various file type

Method Name	Description	Args	Usage
compare_images	Compare images and returns structural similarity over the image. Measure of SSIM is returned between 0-1.0 where 1.0 is the most identical and 0 being completely different.	a) source image path. b)target image path.	self.compare_images(source, target)
compare_json	Compare json files and returns dictionary of difference of target compared to source.	a) source json. b)target json	self.compare_json(source, target)
compare_files	Compare two files and return xl of difference(if any). SupportedfFile Types are xls or xlsx csv	a) Source file Path. b)target file Path.	self.compare_files(source, target)

prodigyqa's Issues

ENH: Saucelab integration

Is your feature request related to a problem? Please describe.
Need to integrate framework with sauce lab to run test cases on cloud platform.

Describe the solution you'd like

• Create Sauce lab trial version account.
• Sauce-python client to report job information to the Sauce Labs dashboard.
• Configure desired capabilities to set the type of browser and OS that we will automate.
• Configure command executor to point to the Sauce cloud and use Sauce Labs authentication to log in.
• Sample test script which will make use of all above mentioned points for demo purpose.

ENH: Logging Feature to be enabled on imgqa

Lets use "Loguru" for logging and to be kept in the most simplest way possible.

ENH - Comparison module

Is your feature request related to a problem? Please describe.
Need to create a comparison module which supports various kinds of comparisons

• PDF

• JSON

• Excel/CSV

• Need to Integrate Image comparison utility as well into this module

Describe the solution you'd like

• Method to accept two entities and compare them in an automated way

• Use the extension of file to automatically ensure that it is a supported format

• Basic checks which are needed to be checked for Excel/CSV

• File type

• Number of Rows

• Number of Columns

• Compare Column Headers of both files

• Perform a quick rearrangement of data dimensions on fly

• Compare it row wise and let the difference details be captured as part of log/result/traceback

• Basic checks which are needed to be checked for PDFs

• File type

• Number of pages

• Compare the pdfs(text only) and let the difference details be detailed as part of an image file like png.

• Basic checks which are needed to be checked for json

• File type

• Structure of json objects.

• JSON objects are sorted.

• Compare the json objects and let the difference details be captured as part of log/result/traceback.

ENH: Spell Checker Module on Web Pages

Is your feature request related to a problem? Please describe.
Spell Checker Module/Utility to be built and to be integrated with Web Spider Module

Describe the solution you'd like

• Visit a webpage (using requests module in case of non-login pages, using selenium in case of login scenario to application)
• Parse through the page and tokenize the English words present on the page
• Validate it against the standard dictionary to ensure spell is correct
• There must be a way of how additional words could be specified e.g: proper nouns using a data file or a custom list of dictionary
• Output shall be the list of pages vs words correct vs words

TST: test cases for API Tester module

Test cases are to be added for apitester module

ENH: Addition of Mobile Testing Module

Post a clear analysis of Appium vs POCO and which we could pick for necessary verifications of mobile testing

ENH: Selenium page actions improvement

Is your feature request related to a problem? Please describe.

• Documentation to be improved

• Sleep to be removed

• Few WIP methods are to be closed

• Coding Standards to be improved

• Restructure or beautify module with decorators

• Add up Logging Feature for the necessary set of actions

ENH: Addition of Performance Testing Module

Performance Testing Module to be added

PYPI relase to be automated

currently manual push of sdist and bdist is happening, it must be automated through Travis Build itself

P.S: please speak to @surevs in case you are interested to work around this

ENH - Implement selenium grid to achieve remote execution.

Is your feature request related to a problem? Please describe.
Need to implement selenium grid, to achieve remote execution of test cases across different browsers, operating systems and machines in parallel.

Describe the solution you'd like

• Use selenium standalone jar for creating hub and nodes.
• Configure desired capabilities to set the type of browser and OS that we will automate.
• Configure remote web driver to set which node (or machine) that our test will run against.
• Sample test script which will make use of all above mentioned points for demo purpose.

Fix the Errors on Methods in Browser Actions

Describe the bug
Browser Actions Method to be corrected/updated

Methods list

• get_current_url
• switch_to_alert
• clear_text

Expected behavior
Methods to work while test is happening

Screenshots
NA
Desktop (please complete the following information):

• Across OS
• All Browser [e.g. chrome, safari]
• Version [e.g. 0.3.0]

ENH: Automated Accessibility Tests as per standard W3C

Is your feature request related to a problem? Please describe.
Accessibility Tests are to be conducted in an automated way

Describe the solution you'd like

• Visit a webpage (using requests module in case of non-login pages, using selenium in case of login scenario to application)
• Parse through the HTML page source and conduct Accessibility tests on the page itself
• Validate it against the standard W3C standards
• There must be a way of how additional words could be specified e.g: proper nouns using a data file or a custom list of dictionary

BUG: Repository should be compatible with all versions of python

Current scenario limits the possibilities as it is currently limited to python 2.7 (This issue stands as a placeholder for putting up the consolidated details)

This is a bug to be solved for across versions of python 2.7+ and 3.5+

ENH:Web Spider Enhancement

Is your feature request related to a problem? Please describe.

• Documentation to be improved

• Sleep to be removed

• Few WIP methods are to be closed

• Coding Standards to be improved

Documentation of Repository would need a clear directions with examples

There are a lot of readme files which have been a little scattered and unlinked.
Either we wish to have them linked or would need a clear and concise way of going through the needed documentation which shall be pushed to PyPi and also will be available as a readme.

Program to lint the Object Repository

Objective of this is to Link Object Repository for

1. JSON
2. Yaml
3. Excel
   to handle/fix duplicates in object repository.
   For eg., When two objects one with locator value,value and another with only locator value then the approach or program to pick the right object

TST: Test cases for Browser Actions

This is more of an issue placeholder for Browser Actions Module with the necessary test to be running through Travis

ENH:CLI to be created for ease of environment maintenance

Is your feature request related to a problem? Please describe.
It is very tough to write up a guideline and perform a necessary set of activities like to setup browsers/install a set of pre-required packages/software to ensure various items are present before a user picks it up.

Describe the solution you'd like
modules/utilities are to be driven through this CLI with proper on-screen help as needed.
something like imgqa setup browsers or imgqa setup comparison or imgqa setup web spider or passing a value like imgqa test web app ${url} so that it is easy and makes the integration of this framework

Describe alternatives you've considered
write a command line utility in go and integrate it with imgqa. but not a probable solution. Let's consider this doing by python itself

Restructure Examples in a neat way

Current Examples have been dumped into one folder, while it could be done in a better way of grouping things and tests (though duplicated can be part of different folder).

as per feedback recieved

ENH: Addition of Security Testing Module

Security Tests that could be used in an automated fashion, please feel free to explore and rely on pytest existing plugins that are in place

MAINT: Repository Maintenance activities

• Setup Travis CI for Python 2.7 + and python 3+

• Setup Automated Build and Release Mechanism for Docker

• Setup PYPA access to enable binary and source distribution factors

• Update ReadMe to enable proper linking of activities

BUG: Browser Actions to handle the import error

==================================== ERRORS ====================================
_______________ ERROR collecting Examples/sampleseleniumtest.py ________________
sampleseleniumtest.py:3: in <module>
    from imgqa import BrowserActions
../imgqa/__init__.py:3: in <module>
    from imgqa.browseractions import BrowserActions  # noqa
../imgqa/browseractions.py:32: in <module>
    class BrowserActions(unittest.TestCase):
../imgqa/browseractions.py:107: in BrowserActions
    @page_readiness_wait
E   TypeError: page_readiness_wait() takes exactly 2 arguments (1 given)
=========================== 1 error in 2.87 seconds ============================ERROR: not found: /Users/revanthmvs/Desktop/new_github/imgqa/Examples/sampleseleniumtest.py::TestClass
(no name '/Users/revanthmvs/Desktop/new_github/imgqa/Examples/sampleseleniumtest.py::TestClass' in any of [<Module 'Examples/sampleseleniumtest.py'>])

Dependabot couldn't authenticate with https://pypi.python.org/simple/

Dependabot couldn't authenticate with https://pypi.python.org/simple/.

You can provide authentication details in your Dependabot dashboard by clicking into the account menu (in the top right) and selecting 'Config variables'.

View the update logs.

Add more functions to Browser Actions

ENH: Email Module to be created

Interact with Email and extract the needed text

• read email
• fetch email
• send email (which could be enhanced with sharing of pytest-html report by default but optional)
• Should be able to pick elements from email or should be able to validate the content or context of email

BUG:Imports to be improved

The current way of imports is very rudimentary like from imgqa.core.page_action import PageActions which should be moved to from imgqa import PageActions (example) so that any future restructuring of the repository must not take a hit and removes the posterior complexity

Web Spider to be enhanced using scrapy

Web spider has to be enhanced for non login screens, where we can automated the url scraping from scrapy when compared to selenium as it is fast and can be implemented on multi threading too ( speeds up the activity)

Image Comparison Overhaul- as per feedback recieved

Describe the bug

1. Image Comparison outcome has been confusing and needs an overhaul.(feedback recieved)
2. Simplify the output of Image Comparison with 2 methods

• compare_source_in_target(sourceimage, targetimage) - which should ideally return a booleanvalue (True/False)
• Compare_images(sourceimage,targetimage) - which should ideally return only the Structural similarity value which should be used for the tester/user to maintain conditions depending upon value recieved in their test scripts