mozilla / mozillians-tests

DEPRECATED - Mozillians Tests

Tests for mozillians.org

Thank you for checking out our Mozillians test suite! This repository contains tests for Mozillians - a community phonebook for core contributors.

Table of contents:

• Getting involved
• How to run the tests
• Writing tests

Getting involved

We love working with contributors to improve test coverage our projects, but it does require a few skills. By contributing to our test suite you will have an opportunity to learn and/or improve your skills with Python, Selenium WebDriver, GitHub, virtual environments, the Page Object Model, and more.

Our new contributor guide should help you to get started, and will also point you in the right direction if you need to ask questions.

How to run the tests

Clone the repository

If you have cloned this project already, then you can skip this; otherwise you'll need to clone this repo using Git. If you do not know how to clone a GitHub repository, check out this help page from GitHub.

If you think you would like to contribute to the tests by writing or maintaining them in the future, it would be a good idea to create a fork of this repository first, and then clone that. GitHub also has great instructions for forking a repository.

Create test variables files

Some of the tests require credentials associated with account with specific access levels. Create at least three users on staging. Vouch at least two of these users by adding '/vouch' to the end of the profile URL for each user. In one of the vouched users' profiles, join at least one group and mark groups as private.

Create a file outside of the project (to avoid accidentally exposing the credentials) with the following format. You will reference this file when running the tests using the --variables command line option.

Note that the vouched key is a list. This is so that multiple vouched users can be used when running the tests in parallel. It's recommended that you have as many vouched users as you intend to have tests running in parallel.

{
  "web-mozillians-staging.production.paas.mozilla.community": {
    "users": {
      "vouched": [
        {
          "username": "vouched",
          "email": "[email protected]",
          "name": "Vouched User"
        }
      ],
      "unvouched": {
        "username": "unvouched",
        "email": "[email protected]",
        "name": "Unvouched User"
      },
      "private": {
        "username": "private",
        "email": "[email protected]",
        "name": "Private User"
      }
    }
  }
}

Then you can run the tests using Docker:

  $ docker build -t mozillians-tests .
  $ docker run -it \
    --mount type=bind,source=/path/to/variables.json,destination=/variables.json,readonly \
    mozillians-tests

Run the tests using Sauce Labs

You will need a Sauce Labs account, with a .saucelabs file in your home directory containing your username and API key, as follows:

[credentials]
username = username
key = secret

Then you can run the tests against Sauce Labs using Docker by passing the --driver SauceLabs argument as shown below. The --mount argument is important, as it allows your .saucelabs file to be accessed by the Docker container:

$ docker build -t mozillians-tests .
$ docker run -it \
  --mount type=bind,source=$HOME/.saucelabs,destination=/src/.saucelabs,readonly \
  --mount type=bind,source=/path/to/variables.json,destination=/variables.json,readonly \
  mozillians-tests pytest --variables /variables.json \
  --driver SauceLabs --capability browserName Firefox

See the documentation on specifying capabilities and the Sauce Labs platform configurator for selecting the target platform.

Writing tests

If you want to get involved and add more tests, then there are just a few things we'd like to ask you to do:

1. Follow our simple style guide.
2. Fork this project with your own GitHub account.
3. Make sure all tests are passing, and submit a pull request.
4. Always feel free to reach out to us and ask questions.

mozillians-tests's Issues

Investigate production failure of all test_that_filter_by_country_works

At least one test appear to be failing on prod due to an insufficient wait test_that_filter_by_country_works(). http://selenium.qa.mtv2.mozilla.com:8080/job/mozillians.prod.saucelabs/

• please investigate this failure and ensure that this is the reason that the test is failing
• if it is due to an insufficient wait, please update test_that_filter_by_country_works, as well as test_that_filter_by_region_works and test_that_filter_by_city_works. They all share the same logic
• if it is a true regression, please file a new bug and apply an xfail to the test

Fix test_non_ascii_characters_are_allowed_in_profile_information

Currently test_non_ascii_characters_are_allowed_in_profile_information() is failing for the wrong reason, a missing locator.

Task:
0. fix it :)

Skills

profile.add_skill(u'\u0394\u03D4\u03D5\u03D7\u03C7\u03C9\u03CA\u03E2')
profile.add_language(u'\u0394\u03D4\u03D5\u03D7\u03C7\u03C9\u03CA\u03E2')
E AttributeError: 'Register' object has no attribute 'add_language'

Verify that groups can be added an removed from a profile

Via the profile edit page (/user/edit/) ensure groups can be added and removed from a users profile.

See bug 934518 for additional details.

In /user/edit, verify the links in the Services section return 200 OK

Using the verify_status_code_is_ok(self, url) [0], verify the links in the Services section of /user/edit return 200 OK.

Verify the links in the /about page return 200's

Consider using the requests api

Mozillians test review

1. Assign this Issue to yourself
2. Go through the Mozillians-tests folder: https://github.com/mozilla/mozillians-tests/tree/master/tests
3. Open each .py file
4. Check each file for: typos in function or object names, misspellings, references to pivotal tracker, references to litmus
5. If you find one, remove it and submit a pull request for each one
6. If you review all test files, and find no errors, then comment on this Issue saying you completed the test and no errors were found

Upgrade py.test dependency to 2.7

We should update to the latest version of py.test. This can be done by updating the pinned versions of pytest, py, and any pytest plugins from requirements.txt to the latest versions. See mozilla/Addon-Tests@bfa2336 as an example.

We should also remove any dependencies that are not required, such as pytest-xdist, which is useful for running tests in parallel, but is not a hard requirement.

Before submitting a pull request please test the changes locally. If you need any assistance, either comment here or ask in #mozwebqa on irc.mozilla.org. Details of how to get onto IRC can be found here.

Change Travis-CI integration to just flake8 checking

We have decided to stop running our tests on Travis-CI at least in the short term, and to just use Travis-CI for basic code checking on our repositories.

In order to do this, we need to set up .travis.yml as follows:

language: python
python: 2.7
install: "pip install flake8"
script: "flake8 . --ignore=E501"

We should also remove any dependencies in requirements.txt that are related to running tests on Travis-CI. In this case, removing the dependency on pytest-xdist.

"test_that_user_can_create_and_delete_group" needs some modifications

test_that_user_can_create_and_delete_group is not properly deleting the group and there are a lot of groups already created.
This is a false positive.

Profile page: non-ascii characters are should be allowed in skills section

Verify non-ascii characters are allowed in the skills section (/user/edit#skills) of a profile.
Specifically the following sections (see bug 835318):

• Groups
• Skills
• Languages

Add robustness to the search tests

Currently the tests in test_search.py are fairly ambiguous and weak. They simply assert results > zero - Assert.true(search_page.results_count > 0).

Update the tests to verify that at minimum one result per test contains a valid result.

Website url has been modified

This change was done on dev. The website url has the "/" at the end removed, as seen in the screenshot below. Probably this change will be made for stage soon. This issue is open for your information and also because the test that uses the website url fails on dev because of this change. I will open a pull with this update, once it will land in stage.
Dev: http://postimg.org/image/mggwd4305/
Stage: http://postimg.org/image/5h7xxursl/

Test external accounts to ensure each option exists

Create high-level smoke tests to make sure that all of the external account options are able to be selected.

Add robustness to the current location search test

d6c43ca added a search test -- let's add some robustness by verifying the results do indeed pertain to the search ("Mountain View") and not simply Assert.true(search_page.results_count > 0).

At this time @rbillings noticed that the results returned for a location search do not include the location that the mozillians are associated with thus there doesn't appear to be a way to assert that the results pertain to that location. We need to reach out to the developers in #commtools to find out how to expose this.

Verify the links in the footer return 200's

Issue #55 laid a good groundwork for testing links. Consider refactoring the code and moving this into the base template #59 (comment).

Look into and fix the disparity between Sauce Labs and in-house prod and stage runs

Please look into why, and fix, the disparity between the Sauce Labs staging + prod jobs, and their in-house Grid counterparts -- they're showing different results, and shouldn't be.

Update tests once changes from dev are pushed to stage/prod

Tests started failing on mozillians.dev due to some changes in the flow on the site.
Some of the changes are related to extra fields that are required in the functionality flow.

@m8ttyB @justinpotts Do you know if these changes are intended? If yes, when are they going to land on stage/prod?

Tests failing on dev environment

Two tests started failing in the Jenkins test run because the locator for the biography in the users profile has changed.

The tests are failing with the following error:

self = <tests.test_profile.TestProfile instance at 0x10dc382d8>
mozwebqa = <pytest_mozwebqa.pytest_mozwebqa.TestSetup instance at 0x10db16440>

@pytest.mark.credentials
def test_edit_profile_information(self, mozwebqa):
home_page = Home(mozwebqa)

home_page.login()

profile_page = home_page.header.click_view_profile_menu_item()
edit_profile_page = home_page.header.click_edit_profile_menu_item()
Assert.true(edit_profile_page.is_csrf_token_present)
current_time = str(time.time()).split('.')[0]

# New profile data
new_full_name = "Updated Mozillians User %s" % current_time
new_biography = "Hello, I'm new here and trying stuff out. Oh, and by the way: I'm a robot, run in a cronjob, most likely, run at %s" % current_time
new_website = "http://%s.com/" % current_time

# Update the profile fields
edit_profile_page.set_full_name(new_full_name)
edit_profile_page.set_website(new_website)
edit_profile_page.set_bio(new_biography)
edit_profile_page.click_update_button()

# Get the current data of profile fields
name = profile_page.name
> biography = profile_page.biography

tests/test_profile.py:55:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <pages.profile.Profile object at 0x10dceb710>

@property
def biography(self):
> return self.selenium.find_element(*self._biography_locator).text

pages/profile.py:49:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <selenium.webdriver.remote.webdriver.WebDriver object at 0x10dc3b590>
by = 'css selector', value = '#bio > p:nth-child(3)'

def find_element(self, by=By.ID, value=None):
"""
'Private' method used by the find_element_by_* methods.

:Usage:
Use the corresponding find_element_by_* instead of this.

:rtype: WebElement
"""
if not By.is_valid(by) or not isinstance(value, str):
raise InvalidSelectorException("Invalid locator values passed in")

return self.execute(Command.FIND_ELEMENT,
> {'using': by, 'value': value})['value']

def find_elements(self, by=By.ID, value=None):

.env/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py:655:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <selenium.webdriver.remote.webdriver.WebDriver object at 0x10dc3b590>
driver_command = 'findElement'
params = {'sessionId': u'd79b8a3c94b7473898805c125bfe93ce', 'using': 'css selector', 'value': '#bio > p:nth-child(3)'}

def execute(self, driver_command, params=None):
"""
Sends a command to be executed by a command.CommandExecutor.

:Args:
- driver_command: The name of the command to execute as a string.
- params: A dictionary of named parameters to send with the command.

:Returns:
The command's JSON response loaded into a dictionary object.
"""
if not params:
params = {'sessionId': self.session_id}
elif 'sessionId' not in params:
params['sessionId'] = self.session_id

params = self._wrap_value(params)
response = self.command_executor.execute(driver_command, params)
if response:
> self.error_handler.check_response(response)

.env/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py:166:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <selenium.webdriver.remote.errorhandler.ErrorHandler object at 0x10dc3b990>
response = {u'class': u'org.openqa.selenium.remote.Response', u'hCode': 1853232813, u'sessionId': u'd79b8a3c94b7473898805c125bfe93ce', u'state': u'no such element', ...}

def check_response(self, response):
"""
Checks that a JSON response from the WebDriver does not have an error.

:Args:
- response - The JSON response from the WebDriver server as a dictionary
object.

:Raises: If the response contains an error message.
"""
status = response['status']
if status == ErrorCode.SUCCESS:
return
exception_class = ErrorInResponseException
if status in ErrorCode.NO_SUCH_ELEMENT:
exception_class = NoSuchElementException
elif status in ErrorCode.NO_SUCH_FRAME:
exception_class = NoSuchFrameException
elif status in ErrorCode.NO_SUCH_WINDOW:
exception_class = NoSuchWindowException
elif status in ErrorCode.STALE_ELEMENT_REFERENCE:
exception_class = StaleElementReferenceException
elif status in ErrorCode.ELEMENT_NOT_VISIBLE:
exception_class = ElementNotVisibleException
elif status in ErrorCode.INVALID_ELEMENT_STATE:
exception_class = InvalidElementStateException
elif status in ErrorCode.INVALID_SELECTOR \
or status in ErrorCode.INVALID_XPATH_SELECTOR \
or status in ErrorCode.INVALID_XPATH_SELECTOR_RETURN_TYPER:
exception_class = InvalidSelectorException
elif status in ErrorCode.ELEMENT_IS_NOT_SELECTABLE:
exception_class = ElementNotSelectableException
elif status in ErrorCode.INVALID_COOKIE_DOMAIN:
exception_class = WebDriverException
elif status in ErrorCode.UNABLE_TO_SET_COOKIE:
exception_class = WebDriverException
elif status in ErrorCode.TIMEOUT:
exception_class = TimeoutException
elif status in ErrorCode.SCRIPT_TIMEOUT:
exception_class = TimeoutException
elif status in ErrorCode.UNKNOWN_ERROR:
exception_class = WebDriverException
elif status in ErrorCode.UNEXPECTED_ALERT_OPEN:
exception_class = UnexpectedAlertPresentException
elif status in ErrorCode.NO_ALERT_OPEN:
exception_class = NoAlertPresentException
elif status in ErrorCode.IME_NOT_AVAILABLE:
exception_class = ImeNotAvailableException
elif status in ErrorCode.IME_ENGINE_ACTIVATION_FAILED:
exception_class = ImeActivationFailedException
elif status in ErrorCode.MOVE_TARGET_OUT_OF_BOUNDS:
exception_class = MoveTargetOutOfBoundsException
else:
exception_class = WebDriverException
value = response['value']
if isinstance(value, basestring):
if exception_class == ErrorInResponseException:
raise exception_class(response, value)
raise exception_class(value)
message = ''
if 'message' in value:
message = value['message']

screen = None
if 'screen' in value:
screen = value['screen']

stacktrace = None
if 'stackTrace' in value and value['stackTrace']:
stacktrace = []
try:
for frame in value['stackTrace']:
line = self._value_or_default(frame, 'lineNumber', '')
file = self._value_or_default(frame, 'fileName', '<anonymous>')
if line:
file = "%s:%s" % (file, line)
meth = self._value_or_default(frame, 'methodName', '<anonymous>')
if 'className' in frame:
meth = "%s.%s" % (frame['className'], meth)
msg = " at %s (%s)"
msg = msg % (meth, file)
stacktrace.append(msg)
except TypeError:
pass
if exception_class == ErrorInResponseException:
raise exception_class(response, message)
> raise exception_class(message, screen, stacktrace)
E NoSuchElementException: Message: u'Unable to locate element: {"method":"css selector","selector":"#bio > p:nth-child(3)"}\nCommand duration or timeout: 10.08 seconds\nFor documentation on this error, please visit: http://seleniumhq.org/exceptions/no_such_element.html\nBuild info: version: \'2.39.0\', revision: \'ff23eac\', time: \'2013-12-16 16:11:15\'\nSystem info: host: \'osxchef.miso\', ip: \'172.20.20.40\', os.name: \'Mac OS X\', os.arch: \'x86_64\', os.version: \'10.6.8\', java.version: \'1.6.0_65\'\nSession ID: 8e834fe1-aa9b-2544-8e00-e734431c615e\nDriver info: org.openqa.selenium.firefox.FirefoxDriver\nCapabilities [{platform=MAC, acceptSslCerts=true, javascriptEnabled=true, browserName=firefox, rotatable=false, locationContextEnabled=true, version=27.0, cssSelectorsEnabled=true, databaseEnabled=true, handlesAlerts=true, browserConnectionEnabled=true, nativeEvents=false, webStorageEnabled=true, applicationCacheEnabled=true, takesScreenshot=true}]' ; Stacktrace:
E at sun.reflect.NativeConstructorAccessorImpl.newInstance0 (NativeConstructorAccessorImpl.java:-2)
E at sun.reflect.NativeConstructorAccessorImpl.newInstance (NativeConstructorAccessorImpl.java:39)
E at sun.reflect.DelegatingConstructorAccessorImpl.newInstance (DelegatingConstructorAccessorImpl.java:27)
E at java.lang.reflect.Constructor.newInstance (Constructor.java:513)
E at org.openqa.selenium.remote.ErrorHandler.createThrowable (ErrorHandler.java:193)
E at org.openqa.selenium.remote.ErrorHandler.throwIfResponseFailed (ErrorHandler.java:145)
E at org.openqa.selenium.remote.RemoteWebDriver.execute (RemoteWebDriver.java:554)
E at org.openqa.selenium.remote.RemoteWebDriver.findElement (RemoteWebDriver.java:307)
E at org.openqa.selenium.remote.RemoteWebDriver.findElementByCssSelector (RemoteWebDriver.java:396)
E at org.openqa.selenium.By$ByCssSelector.findElement (By.java:432)
E at org.openqa.selenium.remote.RemoteWebDriver.findElement (RemoteWebDriver.java:299)
E at sun.reflect.GeneratedMethodAccessor24.invoke (None:-1)
E at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:25)
E at java.lang.reflect.Method.invoke (Method.java:597)
E at org.openqa.selenium.support.events.EventFiringWebDriver$2.invoke (EventFiringWebDriver.java:101)
E at com.sun.proxy.$Proxy2.findElement (None:-1)
E at org.openqa.selenium.support.events.EventFiringWebDriver.findElement (EventFiringWebDriver.java:184)
E at org.openqa.selenium.remote.server.handler.FindElement.call (FindElement.java:47)
E at org.openqa.selenium.remote.server.handler.FindElement.call (FindElement.java:1)
E at java.util.concurrent.FutureTask$Sync.innerRun (FutureTask.java:303)
E at java.util.concurrent.FutureTask.run (FutureTask.java:138)
E at org.openqa.selenium.remote.server.DefaultSession$1.run (DefaultSession.java:170)
E at java.util.concurrent.ThreadPoolExecutor$Worker.runTask (ThreadPoolExecutor.java:895)
E at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:918)
E at java.lang.Thread.run (Thread.java:695)

.env/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py:164: NoSuchElementException

The tests are using the following css locator to identify the element: "#bio > p:nth-child(3)". The new locator is: "#bio > .note > p"

@m8ttyB Will this change also be included in on staging and prod?

Verify on logout; Persona (BID) session is also terminated

See https://bugzilla.mozilla.org/show_bug.cgi?id=909333 for more details

Xfail test_creating_profile_without_checking_privacy_policy_checkbox

Within test_profile.py, test_creating_profile_without_checking_privacy_policy_checkbox is now failing due to a timeout. There have been no code changes that impact this area. This test should be xFailed.

tests/test_profile.py:143: in test_creating_profile_without_checking_privacy_policy_checkbox
profile.set_location('Durango, Colorado')
pages/register.py:40: in set_location
lambda s: self._selenium_root.find_element(*self._country_locator).text != "")
.env/lib/python2.7/site-packages/selenium/webdriver/support/wait.py:78: in until
raise TimeoutException(message, screen, stacktrace)
E TimeoutException: Message:

Decid what to do with base_test.py

I'm not sure what the intent of base_test.py is. It doesn't appear to be used. Had we planned on using this or can we remove this file?

Need test for creating profile without checking PP

Per bug 797790 we need to have a test for the flow of trying to create a profile without selecting the privacy policy checkbox.

Fix test_vouched_user_count

https://mbrandt.pastebin.mozilla.org/5431262

Update location filter tests to use the MozilliansUser profile

The location filter tests rely on an account that has these fields set in it's profile - currently the tests use the mbrandt profile. The owner of this profile is notorious for being unreliable; deleting his account at inconvenient times, changing and removing profile data, and generally being a disagreeable person ;-)

We should update the tests to use our in-house test account - MozilliansUser

• def test_that_filter_by_city_works() here
• def test_that_filter_by_region_works()here
• def test_that_filter_by_county_works()here

test_that_search_returns_results_for_irc_nickname failing on stage

test_that_search_returns_results_for_irc_nickname is failing on stage because searching for "mbrandt" returns no results.

I was able to reproduce this issue manually, but only for this user. Searching after IRC name for other users worked fine. Also, the issue is only reproduced while logged in.

@m8ttyB Do you know why this issue is only reproduced for your user?
I tested using the IRC name of other 5 users without problems.

Migrate from browserid to bidpom

Due to conflicts with another package, the browserid package was renamed to bidpom. This suite should be updated to use the new package. This involves changing the dependency name in requirements.txt and updating the use of the package to the new package structure. Documentation with examples for bidpom can be found here.

This issue is a suitable Outreachy application contribution.

Verify that a user can update their email address

Using persona tester, create a test that:

• creates a new account
• edit the account's email to a new one [pulled from persona tester]
• logout, login
• revert account to previous email address

Update profile_edit tests

... to reflect recent UI changes as described in https://bugzilla.mozilla.org/show_bug.cgi?id=1129848

Also tracked here: https://bugzilla.mozilla.org/show_bug.cgi?id=1192832

Add ID to 'div.well > p:nth-of-type(2)'

This element would do well with an ID rather than this particular locator. It;s the no_results_message_body in Search.py.

remove test_creating_profile_with_non_matching_passwords

The test is not valid any more because of browser id

Update test_creating_profile_with_invalid_email_address for browser id

The new browser id interface has made the test invalid .
Update the test to work with http://personatestuser.org/

Tests are failing intermittently trying to communicate with http://personatestuser.org/email/

We need to create a new persona test user for a couple of the tests, and sometimes that is failing. I have pinged #identity about this and hope that someone will help fix this soon.

Xfail test_profile_creation

Test profile creation is now failing due to a timeout. There have been no code changes that impact this area. This test should be xFailed.

tests/test_profile.py:164: in test_profile_creation
profile.set_location('Mountain View, 94041, California')
pages/register.py:40: in set_location
lambda s: self._selenium_root.find_element(*self._country_locator).text != "")
.env/lib/python2.7/site-packages/selenium/webdriver/support/wait.py:78: in until
raise TimeoutException(message, screen, stacktrace)
E TimeoutException: Message:

Fix broken tests

Additional ui enhancements have landed for user's profile pages - in some cases additional workflow steps have been added and in other cases simply updating the locators will suffice.

Please fix the broken tests :)

Verify languages can be added and removed from a profile

Extend test_profile_creation() to also include adding a language from the /user/edit#language drop-down.

https://github.com/mozilla/mozillians-tests/blob/master/tests/test_profile.py#L149

Update test_group_deletion()

Refactor test_group_deletion() to follow a new user workflow

• create a new group, verify group was created
• delete the group, verify successful deletion

Verify ES has indexed properly

The last several Mozillians released have met with Elastic Search (ES) failing to fully re-index itself. Sancus (webdev) has postulated that we can write a test that verifies ES is returning the proper results/has successfully indexed.

Note this will take some experimentation. It is possible that this is a poor route to test for this failure case. That said this should be a fun exercise given that you'll be using the Mozillians API. Ping #commtools with questions on it's usage.

Using the Mozillians API, write a test that compares the number of indexed people to the number of vouched/complete profile people in the database. These two number should be concisitent

Update tests which check that filter by city and by region works to select a specific user

I've seen test_that_filter_by_region_works.py failing when visiting the profile of a user from a specific location, because the city field was not among user's details.
http://selenium.qa.mtv2.mozilla.com:8080/view/Mozillians/job/mozillians.dev.saucelabs/
http://selenium.qa.mtv2.mozilla.com:8080/view/Mozillians/job/mozillians.dev/785/HTML_Report/
I checked and while we create a profile, the only required field in the location area is country.
In our tests we select a random profile and we check that it has country/region set in the location details. (E.g: https://github.com/mozilla/mozillians-tests/blob/master/tests/test_profile.py#L230)
As some users don't have this details, the tests will fails. Should we update our tests to always select a specific user who has all this details in the profile page, instead of selecting the user randomly?
@m8ttyB, what do you think?

fix test_that_search_returns_results_for_irc_nickname

The search is returning a profile page

Investigate and fix or submit bug

File a bug and then Xfail redirect test

@justinpotts I think this would be a great 1st test to work on for Mozillians.

https://mozillians.allizom.org/zh-CN/group/258-l10n/toggle/ is currently returning a 404 on our dev and staging instances (not prod).

This may be expected behavior but the devs have not noted it so this appears to be a regression.

1. test_302_redirect_for_anonymous_users is failing
2. file a bug noting that the behavior changed - https://bugzilla.mozilla.org/enter_bug.cgi?product=Community%20Tools&component=Phonebook
3. xfail the test like issue #120
4. run test against dev/stage/prod to ensure the xfail works correctly

Update test_group_addition()

Update the test_group_addition() to follow a slightly different workflow -

• verify that an existing group can be joined
• verify that the group that was joined can be left

Un-pin pytest-mozwebqa, so we always pick up the latest

In today's Web QA team meeting, we discussed and decided to un-pin our pytest-mozwebqa dependency, so we'll always be running the latest version (and can move with it, address issues, etc.).

Make it so :-)

Switch to using pytest-variables for credentials

We should move the handling of credentials to pytest-variables. This means changing the example credentials.yml file to variables.json with suitable top level keys and migrating the tests to use the injected variables. We should also add pytest-variables as a dependency.

The variables file would probably look something like this:

{
  "users": {
    "default": {
      "email": "",
      "password": "",
      "name": ""},
    "unvouched": {
      "email": "",
      "password": "",
      "name": ""},
    "unconfirmed": {
      "email": "",
      "password": "",
      "name": ""}
  }
}

An example of a migrated test would look something like this:

def test_login_logout(self, mozwebqa, variables):
    user = variables['users']['default']
    home_page = Home(mozwebqa)
    home_page.login(user['email'], user['password'])
    Assert.true(home_page.header.is_logout_menu_item_present)
    home_page.header.click_logout_menu_item()
    Assert.true(home_page.is_browserid_link_present)

You can also see mozilla/sumo-tests@df597f0 for an example of migrating to pytest-variables.

Add test to ensure groups and functional areas are only able to be viewd by vouched users

Write a test that checks to make sure groups and functional areas cannot be viewed with an unvouched account, and verify they can still be viewed by a vouched account.

https://bugzilla.mozilla.org/show_bug.cgi?id=1089894

update test_profile_creation

update test to use http://personatestuser.org/

Refactor group tests

The team is working on a new feature called curated groups (bug 936569) which changes the way that Mozillians interact with the groups feature as well as how groups are created.

Until the new feature becomes stable on stage DO NOT work on refactoring the failing tests. Not only are the features still in flux as we work on the UX but it is likely the current set of groups tests will no longer be valid given the new site workflows.

Update test_creating_profile_without_checking_privacy_policy_checkbox

update test to use http://personatestuser.org/

Are the private groups tests working as intended?

I've recently looked into creating new test accounts, and the 'private' account has me concerned that we're not testing what we mean to. The tests test_private_groups_field_as_public_when_logged_in and test_private_groups_field_when_not_logged_in use the 'private' user to check that the groups are not displayed to the public. As far as I can tell, this user is not vouched on staging or production, and does not belong to any groups. I can't see a way that the groups would ever be displayed for this user, regardless of the privacy settings. I think these tests are giving false positives and should either be deleted (along with the private account) or should be fixed.

/cc @johngian @akatsoulas

Verify that a non en-US locale can set a date they became involved with Mozilla

Borrowing from bug 914448, ensure that the /es/ locale can set the date they become involved with Mozilla - /es/user/edit/

https://mozillians.allizom.org/es/user/edit/

Migrate mozillians to the new design

Module 'tests/test_about_page.py'

• Function 'test_about_page'
• Function 'test_that_links_in_the_about_page_return_200_code'

Module 'tests/test_account.py'

• Function 'test_login_logout'
• Function 'test_that_links_in_footer_return_200_code'

Module 'tests/test_invite.py'

• Function 'test_inviting_an_invalid_email_address'
• Function 'test_invite'

Module 'tests/test_profile.py'

• Function 'test_profile_deletion_confirmation'
• Function 'test_edit_profile_information
• Function 'test_browserid_link_present'
• Function 'test_creating_profile_without_checking_privacy_policy_checkbox'
• Function 'test_profile_creation'
• Function 'test_non_ascii_characters_are_allowed_in_profile_information'
• Function 'test_that_filter_by_city_works'
• Function 'test_that_filter_by_region_works'
• Function 'test_that_filter_by_county_works'

Module 'tests/test_redirects.py'

• Function 'test_302_redirect_for_anonymous_users'
• Function 'test_200_for_anonymous_users'

Module 'tests/test_search.py'

• Function 'test_that_search_returns_results_for_email_substring'
• Function 'test_that_search_returns_results_for_first_name'
• Function 'test_that_search_returns_results_for_irc_nickname'