This is a boilerplate for testing web & mobile hybrid apps on both iOS & Android.
Node
, XCode
, Android Studio
In order to get the tests to run locally, you need to install the following pieces of software:
-
homebrew
- install usingruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
-
pyenv
- install usingbrew install pyenv
This is a python version manager -
python 2.7.6
- install usingpyenv install 2.7.6
-
pyenv global 2.7.6
- Set python version 2.7.6 to be used globallyAdd the following to ~/.bash_profile
export PYENV_ROOT="$HOME/.pyenv" export PATH="$PYENV_ROOT/bin:$PATH" export PATH="$PYENV_ROOT/shims:$PATH" export PATH="$PYENV_ROOT/completions/pyenv.bash:$PATH"
-
pytest
- install usingpip install pytest
-
pytest_bdd
- install usingpip install pytest-bdd
-
pylint
- install usingpip install pylint
-
Appium-Python-Client
- install usingpip install Appium-Python-Client
-
carthage
- install usingbrew install carthage
-
This setup assumes that node is already installed on your system, if not please download the precompiled package from https://nodejs.org/en/download/ (select
Latest LTS Version
) -
appium
- install usingnpm install -g [email protected]
(Using v1.7.2 as this is the latest version ADF supports at this moment) -
SKIPPED. Not working on ADF. requests[security] - install using
pip install requests[security]
-
pip install Faker future gherkin-official python-dateutil pyyaml requests retry
- TODOs
In order to start the tests, you have to start Appium in another terminal window. To do that run appium
The mobile application needs to be built (follow the steps from your project)
Note
- for iOS builds take the .app folder, and zip it, then enter the path to the zipped file in conf_driver.py
- for Android builds, you should have the .apk file available after running the ionic cordova build android command
Then simply go to the tests_root folder and run - py.test -vv --gherkin-terminal-reporter --not_publish_results True
Flag --not_publish_results True
will make it sure that test runned locally will not be published to TestRail
Difference here is in listing .feature file, which will afterwards run only tests of those feature files listed.
-
Go to tests_root/tests/constants.json
-
Edit suites with feature file(s) you would like to run. eg:
-
"project": { "id": 1, "name": "MC", "test_plan": "MC_Sprint_1", "env": "Apple iPhone 8Plus_iOS-11.0", "suites": { "calculator": "Calculator", "eula_welcome_screen": "EULA - Welcome Screen" }, "tags":"", "language": "en", "market": "us" }
-
-
After setting this up, you can run:
py.test -vv --gherkin-terminal-reporter --not_publish_results True
. But before that, read all of the below explained. -
suites can contain a list of Test Suites (.feature files) to be added to the run and results publish
-
Info:
- suites = if empty all tests will be executed.
- tags = further filtering
- language = mandatory, taking string from i18n.json for selected language
- market = mandatory, in order to know for which market to trigger tests
brew install libimobiledevice
npm install -g ios-deploy
Linting = the process of analyzing the source code to flag programming errors, bugs, stylistic errors, and suspicious constructs.
IMPORTANT: Lint your code before any commit
- Go to tests_root folder
- Run
pylint ./tests
- There should be only one Error:
E: 4, 0: invalid syntax (<string>, line 4) (syntax-error)
- This is due to a pylint issue: root files or folders cannot be ignored from linting. Will follow the fix
- A rating above 9.00 should be kept for the code
Prerequisites for ADF: constants.json
filled (See Publish test results to TestRail
section and see TestRail-Integration, Dependencies
section)
- Install Docker Community Edition from https://docker.com
- Go to ubuntu_docker folder
- Create Docker from ubuntu_docker folder
docker build -t ubuntu_01:v0.1 ./
- builds the docker image with name ubuntu_01docker image ls -a
- lists all docker images- this should output 2 images one being ubuntu_01
- to save the image
docker save ubuntu_01 > ubuntu_01.tar
- to load a saved image
docker load --input ubuntu_01.tar
docker create -v [ABSOLUTE_PATH]/tests_root:/media/tests_root --name ubuntu_01 [DOCKER_IMAGE_ID]
- create docker container based on imagedocker container ls -a
- lists all docker containersdocker start [DOCKER_CONTAINER_ID]
- starts docker container with iddocker exec -it [DOCKER_CONTAINER_ID] sh
- opens sh terminal within running docker containercd media/tests_root
sh package_tests.sh
- run pytest with parameters - tags
This is HOW TO guide for TestRail integration of this project
Add TestRail API credentials to ./tests_root/tests/constants.json
file
"testrail": {
"user_email": "",
"user_key": "",
"url": "https://[SUBDOMAIN].testrail.io",
"verify_ssl": "true"
}
Note: Please follow instructions for generating user_key: http://docs.gurock.com/testrail-api2/accessing
!DO NOT PUSH your credentials into git repo
- Go to tests_root/
- Run:
To import/update test Scenarios for ALL feature files
py.test -vv --not_publish_results True --publish True --object_path "tests/features"
To import/update test Scenarios for INDIVIDUAL .feature file
py.test -vv --not_publish_results True --publish True --object_path "tests/features/[FILE_NAME].feature"
- Each .feature file is a product functionality
- Unique key is the pair of feature name + feature description
Feature: End User License Agreement (EULA) As a Quitter I open the app for the first time I want to be able to view and accept the terms and conditions and privacy policy.
- It will create a new test suite for each .feature file published
- If test suite was imported it will update all the tests within
- Each Scenario is a TestRail case
- Unique key is pair of scenario name + data set (The Examples line in json format)
Scenario: Add two numbers Given I have powered calculator on When I enter <50> into the calculator When I enter <70> into the calculator When I press add Then The result should be <120> on the screen Examples: | number_1 | number_2 | result | | 10 | 20 | 30 | | 50 | 60 | 120 |
- It will create a new case for each Scenario published
- If case was imported it will update it with latest changes
- Scenario tags:
- @automation = case is automated
- @LWH-1 = case ref to Jira ticket (the feature ticket)
- @smoke / @sanity / @regression / None = case priority Critical / High / Medium / Low
- @market_us = case is for USA market
- @not_market_ca = case is not for Canada market
- Steps are imported as separate ones with empty Expected Results
- Do NOT use And and But keys as it will fail the match of test cases during results publishing
- You have to manually create the test plan in TestRail
- Naming convention: [JIRA_PROJECT_NAME][SPRINT_NAME][MARKET]
- eg: LWH_Sprint-1_us or LWH_Regression_us
- Test Plan can be empty
- Automated tests will create Test Runs for each Test Suite that exists and is in scope of testing: See: project.suites
- Test Run name: [TEST_SUITE_NAME] [ENV]
- eg: EULA - Welcome Screen iPhone8_iOS-11.4
- If Test Run is present it will only add a new set of results with the current timestamp
- Test results are published to TestRail at the end of testing
- The reason of failure is also added to the test step
- Naming convention: [JIRA_PROJECT_NAME][SPRINT_NAME][MARKET]
- Go to tests_root/tests/constants.json
-
Edit project with corresponding data. eg:
-
"project": { "id": 1, "name": "LWH", "test_plan": "LWH_Sprint_1", "env": "Apple iPhone 8Plus_iOS-11.0", "suites": { "calculator": "Calculator", "eula_welcome_screen": "EULA - Welcome Screen" }, "tags":"", "language": "en", "market": "us" }
-
suites can contain a list of Test Suites (.feature files) to be added to the run and results publish
-
If suites is left empty then all the available suites (.feature files) will be ran and results published for that environment
-
Info
- id = mandatory, taken from TestRail, is the id of the project. Can be picked up from url in TestRail. Make sure id is correct.
- name = mandatory, name of the project you will publish to.
- test_plan = mandatory, title of the test plan created manually in TestRail
- env = mandatory, device name that will be displayed upon published test run result
- suites = if empty all tests will be executed.
- tags = further filtering
- language = mandatory, taking string from i18n.json for selected language
- market = mandatory, in order to know for which market to trigger tests
-
- Go to tests_root/
- Run:
py.test -vv --gherkin-terminal-reporter
- this will run and publish tests results- NOTE: To avoid publishing results (for local scope)
py.test -vv --gherkin-terminal-reporter --not_publish_results True
To benefit from autocomplete please set tests_root folder as Sources Root
- Right click on tests_root_
- Click on Mark Directory As
- Click on Sources Root
In case you run in any issues with the above steps below are a set of tips for common problems:
- chromedriver errors while installing appium use:
npm install -g [email protected] --unsafe-perm=true --allow-root
- If installation fails and it asks for a sudo, you can type following (sudo is not recommended):
- 1st reinstall homebrew
brew cleanup brew uninstall --force
- 2nd reinstall appium
npm uninstall -g appium npm install -g appium-doctor npm install -g [email protected] appium-doctor //checks your appium installation
- 3rd if none of the above work then you can try the following
sudo chown -R $USER ~/.npm sudo chown -R $USER /usr/lib/node_modules sudo chown -R $USER /usr/local/lib/node_modules
- 1st reinstall homebrew
- Install latest Java 8
- use your preferred text editor (nano, vim, etc) and edit .bash_profile with the following
- paste the following two lines:
export JAVA_HOME=$(/usr/libexec/java_home) export PATH=$JAVA_HOME/bin:$PATH
- save the change
- type
source .bash_profile
to make the changes available in the current terminal session - To verify in console type,
echo $JAVA_HOME
. It should display/System/Library/Java/JavaVirtualMachines/1.X.X.jdk/Contents/Home
. Second way to verify is to typeappium-doctor
- Install Android Studio
- Open Android Studio and install all dependencies
- use your preferred text editor (nano, vim, etc) and edit .bash_profile with the following
- paste the following three lines (change username on first line and Android install location if it is not the default one):
export ANDROID_HOME=/Users/[USERNAME]/Library/Android/sdk export PATH=$ANDROID_HOME/platform-tools:$PATH export PATH=$ANDROID_HOME/tools:$PATH
- save the change
- type
source .bash_profile
to make the changes available in the current terminal session - run
appium-doctor
to verify it looks good
Modus Create is a digital product consultancy. We use a distributed team of the best talent in the world to offer a full suite of digital product design-build services; ranging from consumer facing apps, to digital migration, to agile development training, and business transformation.
This project is part of Modus Labs.
This project is MIT licensed.