mozilla / stubattribution-tests

DEPRECATED

Tests for Stub Attribution

This repository contains tests for Firefox Stub Attribution - A service which accepts an attribution code and returns a modified stub installer.

Table of contents:

• Getting involved
• How to run the tests
• Writing tests
• Test plan

Getting involved

We love working with contributors to improve test coverage on our projects, but it does require a few skills. By contributing to our test suite you will have an opportunity to learn and/or improve your skills with Python, Selenium WebDriver, GitHub, virtual environments, the Page Object Model, and more.

Our new contributor guide should help you to get started, and will also point you in the right direction if you need to ask questions.

How to run the tests

This suite of tests uses a real browser (Google Chrome) on Windows 10, with real end-user actions, and gets us all the way up to, but not including downloading/running/verifying the stub-installer build.

Clone the repository

If you have cloned this project already, then you can skip this; otherwise you'll need to clone this repo using Git. If you do not know how to clone a GitHub repository, check out this help page from GitHub.

If you think you would like to contribute to the tests by writing or maintaining them in the future, it would be a good idea to create a fork of this repository first, and then clone that. GitHub also has great instructions for forking a repository.

Run the tests using Sauce Labs

You will need a Sauce Labs account, with a .saucelabs file in your home directory containing your username and API key, as follows:

[credentials]
username = username
key = secret

Then you can run the tests using Docker. The --mount argument is important, as it allows your .saucelabs file to be accessed by the Docker container:

$ docker build -t stubattribution-tests .
$ docker run -it --mount type=bind,source=$HOME/.saucelabs,destination=/src/.saucelabs,readonly stubattribution-tests

Run the tests using Google Chrome on Windows

Whilst we recommend using Sauce Labs to run the tests, it's also possible to run them locally if you have Windows 10, Google Chrome, and chromedriver.

Install Pipenv, and then run the following command:

$ pipenv run pytest

Writing tests

If you want to get involved and add more tests, then there are just a few things we'd like to ask you to do:

1. Follow our simple style guide.
2. Fork this project with your own GitHub account.
3. Make sure all tests are passing, and submit a pull request.
4. Always feel free to reach out to us and ask questions.

Test plan

The current test plan can be found on the Mozilla wiki. Test suites that require manual execution are stored in our Testrail instance.

stubattribution-tests's Issues

Initial Update

Hi 👊

This is my first visit to this fine repo, but it seems you have been working hard to keep all dependencies updated so far.

Once you have closed this issue, I'll create separate pull requests for every update as soon as I find one.

That's it for now!

Happy merging! 🤖

Write a negative test to ensure that non-whitelisted values for source= param are changed to "(other)"

Look into writing and/or write an end-to-end negative test to cover the case where, if the value for source=[value] doesn't match one in our whitelist, it should be changed/reset to "(other)" (we'll have to be careful to parse/encode/decode/format that string, in Python).

/cc @chrismore and @oremj so they are both aware that I'm aware and going to (try to) cover this from my end too :-)

Remove timestamp assertions

Due to a character limit bug, campaign information was being stripped in some instances. Timestamps have been removed from the stub attr service. Tests need to be updated to reflect this.

• mozilla-services/stubattribution#82 (comment)
• https://docs.google.com/document/d/1bvCDUDdxiPkCAtqsqtqxclO9VSv6WkhYJdAvCn5cNIg/edit?usp=sharing

[docs] Links to TestRail test suite in README

In the README, we need to link to the (few) manual tests in TestRail, for convenience/better visibility (at least, within Mozilla): https://testrail.stage.mozaws.net/index.php?/suites/view/406&group_by=cases:section_id&group_order=asc

Fix failing test_stub_attribution_organic test

Need to fix test_stub_attribution_organic:

15:40:53 FAIL tests/test_stub_attribution_organic.py::test_organic_flow_param_values
15:40:53 =================================== FAILURES ===================================
15:40:53 ________________________ test_organic_flow_param_values ________________________
15:40:53 [gw8] linux2 -- Python 2.7.5 /home/jenkins/workspace/stubattribution.adhoc@2/.tox/py27/bin/python2.7
15:40:53 tests/test_stub_attribution_organic.py:55: in test_organic_flow_param_values
15:40:53     derived_url = derive_url(selenium, '{0}/en-US/'.format(base_url))
15:40:53 tests/test_stub_attribution_organic.py:15: in derive_url
15:40:53     EC.element_to_be_clickable((By.ID, "fx-download-link")))
15:40:53 .tox/py27/lib/python2.7/site-packages/selenium/webdriver/support/wait.py:80: in until
15:40:53     raise TimeoutException(message, screen, stacktrace)
15:40:53 E   TimeoutException: Message:
15:40:53 ------------------------------- pytest-selenium --------------------------------
15:40:53 URL: https://bedrock-demo-bouncer-stage.us-west.moz.works/en-US/
15:40:53 Sauce Labs Job: http://saucelabs.com/jobs/826768a345f948549e710639cbbd772f
15:40:53 ===================== 1 failed, 2 passed in 35.07 seconds ======================

Add a print(derived_url) statement (to all tests) to see if we've actually got an attribution_code

Occasionally, lately, we've been seeing sporadic failures which look like this:

1 tests failed.
FAILED:  tests.test_campaign_flow.test_campaign_flow_param_values[google-paidsearch-Fake%20campaign-test term]

Error Message:
KeyError: 'attribution_code'

Stack Trace:
tests/test_campaign_flow.py:63: in test_campaign_flow_param_values
    new_dict = breakout_utm_param_values(derived_url)
tests/test_campaign_flow.py:23: in breakout_utm_param_values
    attribution_code = key_value_dict['attribution_code'][0].encode('utf-8')
E   KeyError: 'attribution_code'

I think we could probably help ourselves out by doing something like:

print(derived_url) on failure, at the least, to know if we have the stub attribution's hashed string/code, like so:

?attribution_code=c291cmNlPXd3dy5nb29nbGUuY29tJm1lZGl1bT1yZWZlcnJhbCZjYW1wYWlnbj0obm90IHNldCkmY29udGVudD0obm90IHNldCkmdGltZXN0YW1wPTE0ODY1OTc4MDI.&

Element is not clickable at point error after being upgraded to Chrome 61

Since Sauce Labs upgraded to Chrome 61 for "latest," we've been getting this failure consistently:

10:12:31 FAIL tests/test_stub_attribution_organic.py::test_organic_flow_param_values
10:12:31 =================================== FAILURES ===================================
10:12:31 ________________________ test_organic_flow_param_values ________________________
10:12:31 [gw6] linux2 -- Python 2.7.5 /home/jenkins/workspace/stubattribution.adhoc/.tox/py27/bin/python2.7
10:12:31 tests/test_stub_attribution_organic.py:55: in test_organic_flow_param_values
10:12:31     derived_url = derive_url(selenium, '{0}/en-US/'.format(base_url))
10:12:31 tests/test_stub_attribution_organic.py:16: in derive_url
10:12:31     getFirefoxTodayLink.click()
10:12:31 .tox/py27/lib/python2.7/site-packages/selenium/webdriver/remote/webelement.py:78: in click
10:12:31     self._execute(Command.CLICK_ELEMENT)
10:12:31 .tox/py27/lib/python2.7/site-packages/selenium/webdriver/remote/webelement.py:499: in _execute
10:12:31     return self._parent.execute(command, params)
10:12:31 .tox/py27/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py:297: in execute
10:12:31     self.error_handler.check_response(response)
10:12:31 .tox/py27/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py:194: in check_response
10:12:31     raise exception_class(message, screen, stacktrace)
10:12:31 E   WebDriverException: Message: unknown error: Element is not clickable at point (503, 1892)
10:12:31 E     (Session info: chrome=61.0.3163.79)
10:12:31 E     (Driver info: chromedriver=2.29.461591 (62ebf098771772160f391d75e589dc567915b233),platform=Windows NT 6.1.7601 SP1 x86_64)
10:12:31 ------------------------------- pytest-selenium --------------------------------
10:12:31 URL: https://bedrock-demo-bouncer-stage.us-west.moz.works/en-US/
10:12:31 Sauce Labs Job: http://saucelabs.com/jobs/620109a6d66840d2956a8cf280d9a985
10:12:31 ===================== 1 failed, 2 passed in 27.35 seconds ======================

I'll get a PR to downgrade us to Chrome 60, temporarily.

Assertion does not follow the pattern defined by style guide

In the style guide link, the recommended is:
assert 'expected' == 'actual' # good
assert 'actual' == 'expected' #bad
it's possible check more info in: mozilla style guide

stubattribution-tests/tests/test_organic_flow.py

Line 47 in 5738616

assert actual == expected

test_organic_flow_param_values is failing; need to use "(none)" as medium, instead of "referral"

stubattribution.stage[0] is failing in one test, test_organic_flow_param_values, using PYTEST_BASE_URL=https://www.allizom.org:

22:17:01 [gw2] [100%] FAILED tests/test_organic_flow.py::test_organic_flow_param_values 
22:17:01 
22:17:01 =================================== FAILURES ===================================
22:17:01 ________________________ test_organic_flow_param_values ________________________
22:17:01 [gw2] linux2 -- Python 2.7.13 /usr/local/bin/python
22:17:01 tests/test_organic_flow.py:47: in test_organic_flow_param_values
22:17:01     assert actual == expected
22:17:01 E   AssertionError: assert {'campaign': ....mozilla.org'} == {'campaign': '....allizom.org'}
22:17:01 E     Common items:
22:17:01 E     {'campaign': '(not set)', 'content': '(not set)'}
22:17:01 E     Differing items:
22:17:01 E     {'source': 'www.mozilla.org'} != {'source': 'www.allizom.org'}
22:17:01 E     {'medium': '(none)'} != {'medium': 'referral'}
22:17:01 E     Full diff:
22:17:01 E     {'campaign': '(not set)',
22:17:01 E     'content': '(not set)',
22:17:01 E     -  'medium': '(none)',
22:17:01 E     +  'medium': 'referral',
22:17:01 E     -  'source': 'www.mozilla.org'}
22:17:01 E     ?                 ^^^^  ^
22:17:01 E     +  'source': 'www.allizom.org'}
22:17:01 E     ?                 ^  ^^^^
22:17:01 ------------------------------- pytest-selenium --------------------------------
22:17:01 URL: https://www.allizom.org/en-US/firefox/download/thanks/
22:17:01 Sauce Labs Job: http://saucelabs.com/jobs/5f8ed4c0e71e4d3586350c69215f7f5a
22:17:01  generated xml file: /home/jenkins/slave/workspace/stubattribution.stage@2/results/junit.xml 
22:17:01  generated html file: /home/jenkins/slave/workspace/stubattribution.stage@2/results/index.html 
22:17:01 =========================== short test summary info ============================
22:17:01 FAIL tests/test_organic_flow.py::test_organic_flow_param_values
22:17:01 ===================== 1 failed, 2 passed in 22.26 seconds ======================

[0] https://qa-master.fxtest.jenkins.stage.mozaws.net/job/stubattribution.stage/4415/

Failing tests: TypeError: character mapping must return integer, None or unicode

Need to fix:

19:04:40 ------------------------------- pytest-selenium --------------------------------
19:04:40 URL: https://bedrock-demo-bouncer-stage.us-west.moz.works/en-US/firefox/new/?scene=2
19:04:40 Sauce Labs Job: http://saucelabs.com/jobs/5e9e330375164107beec690a907ac3c5
19:04:40 ---------------------------- Captured log teardown -----------------------------
19:04:40 remote_connection.py       480 DEBUG    DELETE http://****@162.222.75.45:80/wd/hub/session/5e9e330375164107beec690a907ac3c5 {"sessionId": "5e9e330375164107beec690a907ac3c5"}
19:04:40 remote_connection.py       567 DEBUG    Finished Request {u'status': 0, u'sessionId': u'5e9e330375164107beec690a907ac3c5', u'value': u''}
19:04:40 _ test_campaign_flow_param_values[google-paidsearch-Fake%20campaign-test term] _
19:04:40 [gw5] linux2 -- Python 2.7.14 /usr/local/bin/python
19:04:40 tests/test_campaign_flow.py:63: in test_campaign_flow_param_values
19:04:40     new_dict = breakout_utm_param_values(derived_url)
19:04:40 tests/test_campaign_flow.py:24: in breakout_utm_param_values
19:04:40     attribution_code = querystringsafe_base64.decode(attribution_code)
19:04:40 /usr/local/lib/python2.7/site-packages/querystringsafe_base64.py:79: in decode
19:04:40     return urlsafe_b64decode(padded_string.replace(b'.', b'='))
19:04:40 /usr/local/lib/python2.7/base64.py:119: in urlsafe_b64decode
19:04:40     return b64decode(s.translate(_urlsafe_decode_translation))
19:04:40 E   TypeError: character mapping must return integer, None or unicode

Look at adding macOS/OSX desktop-testing coverage

Need to come back to this coverage/testing/scoping Issue for https://bugzilla.mozilla.org/show_bug.cgi?id=1344771 - Desktop Attribution on macOS/OSX.

test_organic_flow_param_values fails in staging and prod due to site-UX change on Mozilla.org

test_organic_flow_param_values is failing on both staging and production, due to a change in Mozilla.org's header-navigation UX.

Namely, it's no longer just a link, it's now what looks to be a CSS :hover selector + display:none toggle, which pops a fly-out sub-menu out.

Note also that we need to retain the page.clickDownload() call, here:

stubattribution-tests/tests/test_organic_flow.py

Line 39 in 8276750

derived_url = page.click_download().download_link_location

, as this test relies on, functionality & security-wise, an "actual" click with referer [sic] headers -- possibly more.

Error Message

NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":"a[data-link-type=nav][data-link-name=Firefox]"}   (Session info: chrome=70.0.3538.67)   (Driver info: chromedriver=2.43.600210 (68dcf5eebde37173d4027fa8635e332711d2874a),platform=Windows NT 10.0.10586 x86_64)

Stacktrace

base_url = 'http://www.mozilla.org'
selenium = <selenium.webdriver.remote.webdriver.WebDriver (session="373ea991275c47c380e4659695f520b2")>

    @pytest.mark.nondestructive
    def test_organic_flow_param_values(base_url, selenium):
        # we:
        # 1. compare the values we expect from breaking out downloadLink in derive_url()
        # 2. ...to the utm_param_values we expect to see for source, medium, campaign, and content
>       page = Home(selenium, base_url).open().open_firefox()

base_url   = 'http://www.mozilla.org'
selenium   = <selenium.webdriver.remote.webdriver.WebDriver (session="373ea991275c47c380e4659695f520b2")>

tests/test_organic_flow.py:38: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
pages/home.py:10: in open_firefox
    self.find_element(*self._firefox_locator).click()
/usr/local/lib/python2.7/site-packages/pypom/view.py:34: in find_element
    return self.driver_adapter.find_element(strategy, locator)
/usr/local/lib/python2.7/site-packages/pypom/selenium_driver.py:65: in find_element
    return self.driver.find_element(strategy, locator)
/usr/local/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py:978: in find_element
    'value': value})['value']
/usr/local/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py:321: in execute
    self.error_handler.check_response(response)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <selenium.webdriver.remote.errorhandler.ErrorHandler object at 0x7f7cbcd63490>
response = {'sessionId': '373ea991275c47c380e4659695f520b2', 'status': 7, 'value': {'message': 'no such element: Unable to locate...ver info: chromedriver=2.43.600210 (68dcf5eebde37173d4027fa8635e332711d2874a),platform=Windows NT 10.0.10586 x86_64)'}}

    def check_response(self, response):
        """
        Checks that a JSON response from the WebDriver does not have an error.
    
        :Args:
         - response - The JSON response from the WebDriver server as a dictionary
           object.
    
        :Raises: If the response contains an error message.
        """
        status = response.get('status', None)
        if status is None or status == ErrorCode.SUCCESS:
            return
        value = None
        message = response.get("message", "")
        screen = response.get("screen", "")
        stacktrace = None
        if isinstance(status, int):
            value_json = response.get('value', None)
            if value_json and isinstance(value_json, basestring):
                import json
                try:
                    value = json.loads(value_json)
                    if len(value.keys()) == 1:
                        value = value['value']
                    status = value.get('error', None)
                    if status is None:
                        status = value["status"]
                        message = value["value"]
                        if not isinstance(message, basestring):
                            value = message
                            message = message.get('message')
                    else:
                        message = value.get('message', None)
                except ValueError:
                    pass
    
        exception_class = ErrorInResponseException
        if status in ErrorCode.NO_SUCH_ELEMENT:
            exception_class = NoSuchElementException
        elif status in ErrorCode.NO_SUCH_FRAME:
            exception_class = NoSuchFrameException
        elif status in ErrorCode.NO_SUCH_WINDOW:
            exception_class = NoSuchWindowException
        elif status in ErrorCode.STALE_ELEMENT_REFERENCE:
            exception_class = StaleElementReferenceException
        elif status in ErrorCode.ELEMENT_NOT_VISIBLE:
            exception_class = ElementNotVisibleException
        elif status in ErrorCode.INVALID_ELEMENT_STATE:
            exception_class = InvalidElementStateException
        elif status in ErrorCode.INVALID_SELECTOR \
                or status in ErrorCode.INVALID_XPATH_SELECTOR \
                or status in ErrorCode.INVALID_XPATH_SELECTOR_RETURN_TYPER:
            exception_class = InvalidSelectorException
        elif status in ErrorCode.ELEMENT_IS_NOT_SELECTABLE:
            exception_class = ElementNotSelectableException
        elif status in ErrorCode.ELEMENT_NOT_INTERACTABLE:
            exception_class = ElementNotInteractableException
        elif status in ErrorCode.INVALID_COOKIE_DOMAIN:
            exception_class = InvalidCookieDomainException
        elif status in ErrorCode.UNABLE_TO_SET_COOKIE:
            exception_class = UnableToSetCookieException
        elif status in ErrorCode.TIMEOUT:
            exception_class = TimeoutException
        elif status in ErrorCode.SCRIPT_TIMEOUT:
            exception_class = TimeoutException
        elif status in ErrorCode.UNKNOWN_ERROR:
            exception_class = WebDriverException
        elif status in ErrorCode.UNEXPECTED_ALERT_OPEN:
            exception_class = UnexpectedAlertPresentException
        elif status in ErrorCode.NO_ALERT_OPEN:
            exception_class = NoAlertPresentException
        elif status in ErrorCode.IME_NOT_AVAILABLE:
            exception_class = ImeNotAvailableException
        elif status in ErrorCode.IME_ENGINE_ACTIVATION_FAILED:
            exception_class = ImeActivationFailedException
        elif status in ErrorCode.MOVE_TARGET_OUT_OF_BOUNDS:
            exception_class = MoveTargetOutOfBoundsException
        elif status in ErrorCode.JAVASCRIPT_ERROR:
            exception_class = JavascriptException
        elif status in ErrorCode.SESSION_NOT_CREATED:
            exception_class = SessionNotCreatedException
        elif status in ErrorCode.INVALID_ARGUMENT:
            exception_class = InvalidArgumentException
        elif status in ErrorCode.NO_SUCH_COOKIE:
            exception_class = NoSuchCookieException
        elif status in ErrorCode.UNABLE_TO_CAPTURE_SCREEN:
            exception_class = ScreenshotException
        elif status in ErrorCode.ELEMENT_CLICK_INTERCEPTED:
            exception_class = ElementClickInterceptedException
        elif status in ErrorCode.INSECURE_CERTIFICATE:
            exception_class = InsecureCertificateException
        elif status in ErrorCode.INVALID_COORDINATES:
            exception_class = InvalidCoordinatesException
        elif status in ErrorCode.INVALID_SESSION_ID:
            exception_class = InvalidSessionIdException
        elif status in ErrorCode.UNKNOWN_METHOD:
            exception_class = UnknownMethodException
        else:
            exception_class = WebDriverException
        if value == '' or value is None:
            value = response['value']
        if isinstance(value, basestring):
            if exception_class == ErrorInResponseException:
                raise exception_class(response, value)
            raise exception_class(value)
        if message == "" and 'message' in value:
            message = value['message']
    
        screen = None
        if 'screen' in value:
            screen = value['screen']
    
        stacktrace = None
        if 'stackTrace' in value and value['stackTrace']:
            stacktrace = []
            try:
                for frame in value['stackTrace']:
                    line = self._value_or_default(frame, 'lineNumber', '')
                    file = self._value_or_default(frame, 'fileName', '<anonymous>')
                    if line:
                        file = "%s:%s" % (file, line)
                    meth = self._value_or_default(frame, 'methodName', '<anonymous>')
                    if 'className' in frame:
                        meth = "%s.%s" % (frame['className'], meth)
                    msg = "    at %s (%s)"
                    msg = msg % (meth, file)
                    stacktrace.append(msg)
            except TypeError:
                pass
        if exception_class == ErrorInResponseException:
            raise exception_class(response, message)
        elif exception_class == UnexpectedAlertPresentException:
            alert_text = None
            if 'data' in value:
                alert_text = value['data'].get('text')
            elif 'alert' in value:
                alert_text = value['alert'].get('text')
            raise exception_class(message, screen, stacktrace, alert_text)
>       raise exception_class(message, screen, stacktrace)
E       NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":"a[data-link-type=nav][data-link-name=Firefox]"}
E         (Session info: chrome=70.0.3538.67)
E         (Driver info: chromedriver=2.43.600210 (68dcf5eebde37173d4027fa8635e332711d2874a),platform=Windows NT 10.0.10586 x86_64)

exception_class = <class 'selenium.common.exceptions.NoSuchElementException'>
message    = u'no such element: Unable to locate element: {"method":"css selector","selector":"a[data-link-type=nav][data-link-name=Firefox]"}\n  (Session info: chrome=70.0.3538.67)\n  (Driver info: chromedriver=2.43.600210 (68dcf5eebde37173d4027fa8635e332711d2874a),platform=Windows NT 10.0.10586 x86_64)'
response   = {u'sessionId': u'373ea991275c47c380e4659695f520b2',
 u'status': 7,
 u'value': {u'message': u'no such element: Unable to locate element: {"method":"css selector","selector":"a[data-link-type=nav][data-link-name=Firefox]"}\n  (Session info: chrome=70.0.3538.67)\n  (Driver info: chromedriver=2.43.600210 (68dcf5eebde37173d4027fa8635e332711d2874a),platform=Windows NT 10.0.10586 x86_64)'}}
screen     = None
self       = <selenium.webdriver.remote.errorhandler.ErrorHandler object at 0x7f7cbcd63490>
stacktrace = None
status     = 7
value      = {u'message': u'no such element: Unable to locate element: {"method":"css selector","selector":"a[data-link-type=nav][data-link-name=Firefox]"}\n  (Session info: chrome=70.0.3538.67)\n  (Driver info: chromedriver=2.43.600210 (68dcf5eebde37173d4027fa8635e332711d2874a),platform=Windows NT 10.0.10586 x86_64)'}
value_json = {u'message': u'no such element: Unable to locate element: {"method":"css selector","selector":"a[data-link-type=nav][data-link-name=Firefox]"}\n  (Session info: chrome=70.0.3538.67)\n  (Driver info: chromedriver=2.43.600210 (68dcf5eebde37173d4027fa8635e332711d2874a),platform=Windows NT 10.0.10586 x86_64)'}

/usr/local/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py:242: NoSuchElementException

/cc @davehunt for good measure

InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised"

In any of the most recent Stub Attribution test runs on Jenkins, we see the following in the console output:

19:41:04 SKIP [1] tests/test_organic_flow.py:33: Issue 227; need nav-flow changes
19:41:04 =============================== warnings summary ===============================
19:41:04 /usr/local/lib/python2.7/site-packages/urllib3/connectionpool.py:847
19:41:04   /usr/local/lib/python2.7/site-packages/urllib3/connectionpool.py:847: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
19:41:04     InsecureRequestWarning)
19:41:04   /usr/local/lib/python2.7/site-packages/urllib3/connectionpool.py:847: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
19:41:04     InsecureRequestWarning)

Dependabot can't evaluate your Python dependency files

Dependabot can't evaluate your Python dependency files.

As a result, Dependabot couldn't check whether any of your dependencies are out-of-date.

The error Dependabot encountered was:

InstallationError("Invalid requirement: 'Unless otherwise stated, this license applies to all files contained'\n",)

You can mention @dependabot in the comments below to contact the Dependabot team.

CODE_OF_CONDUCT.md file missing

As of January 1 2019, Mozilla requires that all GitHub projects include this CODE_OF_CONDUCT.md file in the project root. The file has two parts:

1. Required Text - All text under the headings Community Participation Guidelines and How to Report, are required, and should not be altered.
2. Optional Text - The Project Specific Etiquette heading provides a space to speak more specifically about ways people can work effectively and inclusively together. Some examples of those can be found on the Firefox Debugger project, and Common Voice. (The optional part is commented out in the raw template file, and will not be visible until you modify and uncomment that part.)

If you have any questions about this file, or Code of Conduct policies and procedures, please see Mozilla-GitHub-Standards or email [email protected].

(Message COC001)

test_organic_flow_param_values fails when survey appears on bottom of homepage

https://github.com/mozilla/stubattribution-tests/blob/67c572aac2a5ddafd04349ea3c1060cc44074070/tests/test_stub_attribution_organic.py fails on both prod and staging, when a small survey appears at the bottom of the viewport, on the homepage.

See screenshot.

Here's the exception we get:

Error Message

tests/test_stub_attribution_organic.py:55: in test_organic_flow_param_values     derived_url = derive_url(selenium, '{0}/en-US/'.format(base_url)) tests/test_stub_attribution_organic.py:16: in derive_url     getFirefoxTodayLink.click() .tox/py27/lib/python2.7/site-packages/selenium/webdriver/remote/webelement.py:77: in click     self._execute(Command.CLICK_ELEMENT) .tox/py27/lib/python2.7/site-packages/selenium/webdriver/remote/webelement.py:493: in _execute     return self._parent.execute(command, params) .tox/py27/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py:249: in execute     self.error_handler.check_response(response) .tox/py27/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py:193: in check_response     raise exception_class(message, screen, stacktrace) E   WebDriverException: Message: unknown error: Element <a id="fx-download-link" href="/en-US/firefox/new/">...</a> is not clickable at point (502, 683). Other element would receive the click: <span>...</span> E     (Session info: chrome=57.0.2987.98) E     (Driver info: chromedriver=2.28.455520 (cc17746adff54984afff480136733114c6b3704b),platform=Windows NT 10.0.10586 x86_64)

Stacktrace

tests/test_stub_attribution_organic.py:55: in test_organic_flow_param_values
    derived_url = derive_url(selenium, '{0}/en-US/'.format(base_url))
tests/test_stub_attribution_organic.py:16: in derive_url
    getFirefoxTodayLink.click()
.tox/py27/lib/python2.7/site-packages/selenium/webdriver/remote/webelement.py:77: in click
    self._execute(Command.CLICK_ELEMENT)
.tox/py27/lib/python2.7/site-packages/selenium/webdriver/remote/webelement.py:493: in _execute
    return self._parent.execute(command, params)
.tox/py27/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py:249: in execute
    self.error_handler.check_response(response)
.tox/py27/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py:193: in check_response
    raise exception_class(message, screen, stacktrace)
E   WebDriverException: Message: unknown error: Element <a id="fx-download-link" href="/en-US/firefox/new/">...</a> is not clickable at point (502, 683). Other element would receive the click: <span>...</span>
E     (Session info: chrome=57.0.2987.98)
E     (Driver info: chromedriver=2.28.455520 (cc17746adff54984afff480136733114c6b3704b),platform=Windows NT 10.0.10586 x86_64)

Bump shared-library version to 1.4

Bump to shared pipeline library version 1.5

Investigate/add debug info for KeyError: attribution_code error/test failure

From time to time, we see the following KeyError: attribution_code failure/error:

From https://qa-preprod-master.fxtest.jenkins.stage.mozaws.net/job/stub-attribution/job/master/1084/console

15:34:41 
15:34:41 =================================== FAILURES ===================================
15:34:41 _ test_campaign_flow_param_values[google-paidsearch-Fake%20campaign-test term] _
15:34:41 [gw0] linux2 -- Python 2.7.13 /usr/local/bin/python
15:34:41 tests/test_campaign_flow.py:61: in test_campaign_flow_param_values
15:34:41     new_dict = breakout_utm_param_values(derived_url)
15:34:41 tests/test_campaign_flow.py:23: in breakout_utm_param_values
15:34:41     attribution_code = key_value_dict['attribution_code'][0].encode('utf-8')
15:34:41 E   KeyError: 'attribution_code'
15:34:41 ------------------------------- pytest-selenium --------------------------------
15:34:41 URL: https://www.allizom.org/en-US/firefox/download/thanks/
15:34:41 Sauce Labs Job: https://saucelabs.com/jobs/b1bedb528e5543a5b4e65e3976013899
15:34:41  generated xml file: /home/jenkins/workspace/stub-attribution_master-HSZMGT5RUDURFLDGHEIDP6CDEDC6SV7YGHL2LWY6PRFZ5CSAMPWA@2/results/junit.xml 
15:34:41  generated html file: /home/jenkins/workspace/stub-attribution_master-HSZMGT5RUDURFLDGHEIDP6CDEDC6SV7YGHL2LWY6PRFZ5CSAMPWA@2/results/index.html 
15:34:41 =========================== short test summary info============================
15:34:41 FAIL tests/test_campaign_flow.py::test_campaign_flow_param_values[google-paidsearch-Fake%20campaign-test term]
15:34:41 ===================== 1 failed, 2 passed in 17.19 seconds ======================

Bedrock-demo-bouncer-stage.us-west.moz.works is down (tests on stage are failing)

The us-west deis 1 cluster was decomissioned today, which likely was what took down the somewhat ephemeral https://bedrock-demo-bouncer-stage.us-west.moz.works/ instance.

@pmac - not urgent, but if it's easy, mind recreating/reprovisioning this? I can easily change the BASE_URL we pass to pytest, too.

Thanks!