csmcallister / fed-a11y

Automated accessibility testing of U.S. Federal Government websites.

Fed A11y

Automated accessibility testing of most U.S. Federal Government websites.

Getting Started

If you're wanting to develop against this project, follow these steps to get started.

Install Python

This project uses Python 3.11, although other versions should be fine. You can install Python from here, although using another utility (e.g. pyenv or homebrew, for OSX) is fine as well.

Next, activate your python virtual environment:

python -m venv env
source env/bin/activate
pip install -r requirements.txt

Get the Data

Download a snapshot of the data as JSON and place it at feda11y/static/data.json:

wget -O feda11y/static/data.json https://www.feda11y.com/data

Now make a dummy file for the historical data called feda11y/static/hist.json and put something minimal like this in it:

[{"2020-03-09": 0.2}, {"2020-03-16": 0.3}, {"2020-03-23": 0.4}, {"2020-03-30": 0.5}]

These files are normally pulled from cloud storage with each deploy, so it's imperative you recreate them locally before continuing.

Run the Tests

This will verify that everything is working as expected:

coverage run -m pytest

It will also let you see the test coverage with:

coverage report

Start the App

To start the app in development mode, set the following environment variables or create a .env file with the following contents:

FLASK_APP=main.py
FLASK_ENV=development
FLASK_DEBUG=1
SECRET_KEY="somethingsupersecret"

Now you can start the app with:

flask --app main.py run

Alternatively, you can use a gunicorn with gunicorn main:app

Go to http://127.0.0.1:5000/ and check it out (use port 8000 if you used gunicorn).

Deploy

We use Heroku, which makes it as simple as:

git push -f https://heroku:$[e-mail]/$HEROKU_APP_NAME.git master

where you've made a Heroku account and set the HEROKU_API_KEY and HEROKU_APP_NAME environment variables beforehand.

You can test things out locally with:

heroku local web

Accessibility

Just like the backend to this site, we use pa11y to smoke test our site's accessibility. You can run these tests yourself downloading the pa11y or pa11y-ci tools and then starting the app with flask run to give yourself a live endpoint to test. Our configuration file for the a11y tests is located at .pa11yci.

If you notice any issues that aren't caught by these automated tests, please let us know by opening an issue.

Contributing

If you'd like to contribute, hop on over to our contributing docs.

If you've got questions, open an issue.

LICENSE

GNU General Public License. See it here.

fed-a11y's Issues

BUG: Weekly deploy is failing

Describe the bug
The weekly deploy in circleci is failing with what seems to be an expired access token.

To Reproduce
Steps to reproduce the behavior:

Rerun the failed job.

Expected behavior
The git push deploy succeeds.

TST: Deploy circleci job fails to find last test report

Describe the bug
When the deploy job is run and the last job wasn't the lint-and-test job, which stores a test report artifact, but rather the weekly-deploy job, the wget command fails to download the last test report because it doesn't exist.

To Reproduce
Steps to reproduce the behavior:

1. Open a PR and let the tests run
2. Wait for the weekly-deploy job to be triggered
3. Merge the PR
4. Watch the download test report step of the deploy job fail.

Expected behavior
Handle edge cases like this when the last CI Job didn't produce a test report. A while loop that iterates through past CI builds should do the trick.

SEC: Upgrade Heroku Stack

Describe the bug
https://devcenter.heroku.com/articles/upgrading-to-the-latest-stack

BUG: 500 response instead of 404

Describe the bug
A GET to https://www.feda11y.com/static/images/ results in 500 instead of a 404

To Reproduce
Go to the url.

Additional Info

NotFound: 404 Not Found: The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.
  File "flask/app.py", line 1949, in full_dispatch_request
    rv = self.dispatch_request()
  File "flask/app.py", line 1935, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "flask/helpers.py", line 1082, in send_static_file
    self.static_folder, filename, cache_timeout=cache_timeout
  File "flask/helpers.py", line 767, in send_from_directory
    raise NotFound()

BuildError: Could not build url for endpoint 'static'. Did you forget to specify values ['filename']?
(12 additional frame(s) were not displayed)
...
  File "flask/helpers.py", line 370, in url_for
    return appctx.app.handle_url_build_error(error, endpoint, values)
  File "flask/app.py", line 2215, in handle_url_build_error
    reraise(exc_type, exc_value, tb)
  File "flask/_compat.py", line 39, in reraise
    raise value
  File "flask/helpers.py", line 358, in url_for
    endpoint, values, method=method, force_external=external
  File "werkzeug/routing.py", line 2020, in build
    raise BuildError(endpoint, values, method, self)

BuildError: Could not build url for endpoint 'static'. Did you forget to specify values ['filename']?
(11 additional frame(s) were not displayed)
...
  File "flask/helpers.py", line 370, in url_for
    return appctx.app.handle_url_build_error(error, endpoint, values)
  File "flask/app.py", line 2215, in handle_url_build_error
    reraise(exc_type, exc_value, tb)
  File "flask/_compat.py", line 39, in reraise
    raise value
  File "flask/helpers.py", line 358, in url_for
    endpoint, values, method=method, force_external=external
  File "werkzeug/routing.py", line 2020, in build
    raise BuildError(endpoint, values, method, self)

ENH: Explain a11y standard switch

As a visitor to the website, I want to know why there was a sharp drop in accessibility scan results on 6/8/2021 so that I can have trust in the process underlying that data.

Acceptance Criteria

• When I see the sharp drop, there is a link to the explanation
• When I visit the FAQ page, the standard that's in place is adequately described.

Additional Context
pa11y/pa11y#528
csmcallister/fed-a11y-scan#10