Metro Board Reports helps the Los Angeles community understand the activities of the Los Angeles County Metropolitan Transportation Authority (Metro) – a government agency that consists of several Board Members, who set policy, coordinate, plan, fund, build, and operate transit services and transportation programs throughout LA County.
The Metro Board Reports site monitors all things related to the Metro Board of Directors:
- the board reports introduced and passed
- its various committees and the meetings they hold
- the board members themselves
This site ultimately encourages greater public dialogue and increased awareness about transportation issues in LA County.
Metro Board Reports is a member of the Councilmatic family. Learn how to build your own Councilmatic site!
These days, we run apps in containers for local development. More on that here.
We use the pre-commit framework to use Git pre-commit hooks that keep our codebase clean.
To set up Pre-Commit, install the Python package on your local machine using
python -m pip install pre-commitIf you'd rather not install pre-commit globally, create and activate a virtual environment in this repo before running the above command.
Then, run
pre-commit installto set up the git hooks.
Since hooks are run locally, you can modify which scripts are run before each commit by modifying .pre-commit-config.yaml.
There should be an entry in the DataMade LastPass account called 'LA Metro - secrets.py.' Copy its contents into a file called secrets.py and place it in lametro/.
The Metro app ingests updated data from the Legistar API several times an hour.
To import data into your local instance, copy DataMade's Legistar API token from
Bitwarden (search LA Metro Legistar API Token). Next, create a copy of the secrets
file and paste in the token:
cp lametro/secrets.py.example lametro/secrets.pyThen, simply run:
docker-compose run --rm scrapersThis may take a few minutes to an hour, depending on the volume of recent updates.
First, create your own local env file:
cp .env.local.example .env.localNext, bring up the app:
docker-compose upWhen your logs indicate that your app is up and running, visit http://localhost:8001 to visit your shiny, new local application!
If you wish to use search in your local install, you need a SmartLogic API
key. Initiated DataMade staff may retrieve values for the SMART_LOGIC_ENVIRONMENT
and SMART_LOGIC_KEY environment variables from Heroku:
heroku config:get SMART_LOGIC_ENVIRONMENT SMART_LOGIC_KEY -a la-metro-councilmatic-stagingPaste these values into your .env.local file.
Then, run the refresh_guid management command to grab the appropriate
classifications for topics in the database.
docker-compose exec app python manage.py refresh_guidFinally, add data to your search index with the update_index command from
Haystack.
docker-compose run --rm app python manage.py update_indexWhen the command exits, your search index has been filled.
Occasionally, after a while without running an event scrape, you may find that your local app is broken. If this happens, make sure you have events in your database that are scheduled for the future, as the app queries for upcoming events in order to render the landing page.
-
Make sure there are future events scheduled in Legistar. Go to the LA Metro Legistar page and open up the time filter for "All Years".
-
If you notice that there are future events in Legistar, run a small windowed event scrape:
docker-compose run --rm scrapers pupa update lametro events window=0.05 --rpm=0This will bring in a few upcoming events, and your app will be able to load the landing page.
It's sometimes helpful to make sure you have a specific bill in your database for debugging. Here's how you can scrape a bill you need:
-
Go to the Legistar Web API at the following URL: http://webapi.legistar.com/v1/metro/matters/?$filter=MatterFile%20eq%20%27<bill_identifier>%27 and find the
<MatterId>of the bill. The identifier should be in XXXX-XXXX format, and theMatterIdis a 4 digit number. -
Run the following command in your shell:
docker-compose run --rm scrapers pupa update lametro bills matter_ids=<bill_matter_id> --rpm=0If you want to use the S3 bucket, you’ll need the AWS S3 API keys. This can be found by running:
heroku config:get AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY -a la-metro-councilmatic-stagingGrab the values for the AWS_ACCESS_KEY_ID and the AWS_SECRET_ACCESS_KEY and add them to your .env.local file.
Now you should be able to start uploading some files!
Hooray! A new member has been elected or appointed to the Board of Directors. There are a few changes you need to make so they appear correctly on the site.
- Add the new member to the
VOTING_POSTSobject in the Metro person scraper. Once your PR is reviewed, merge your changes, pull them into our scrapers fork, and follow the steps to deploy your change.- Example: opencivicdata/scrapers-us-municipal#337
- Tip: Once your scraper change is deployed. Run
docker-compose run --rm scrapers pupa import lametro people --rpm=0to capture the change locally.
- After the revised person scrape runs, remove any board memberships for the
new member that were created without a post.
from lametro.models import Person Person.objects.get(family_name='<MEMBER LAST NAME>').memberships.filter(organization__name='Board of Directors', post__isnull=True).delete()
To confirm your changes worked, run the app locally and confirm the following:
- View the Board of Directors
listing and confirm the new member is listed with the correct post, e.g.,
Los Angeles Country Board Supervisor, District 1.- If you only see
Board Member, the new member's post has not been added. Double check that you updated theVOTING_POSTSobject in the person scraper (e.g., does the member's name as it appears in the API match the key you added?), that your changes to the scraper have been deployed, and that a person scrape has been run since the deployment.
- If you only see
LA Metro Councilmatic has a basic test suite. If you need to run it, simply run:
docker-compose -f docker-compose.yml -f tests/docker-compose.yml run --rm appLA Metro Councilmatic uses Locust for load
testing. There is a starter script in locustfile.py that visits the homepage,
event listing, and an event detail page at random intervals between 60 and 90
seconds. This script was derived from user behavior in Google Analytics.
(If needed, request analytics access from Metro.)
You can run the load tests using the locust service in docker-compose.locust.yml:
docker-compose -f docker-compose.yml -f docker-compose.locust.yml run --service-ports --rm locustThis will start the Locust web server on http://localhost:8089. For more details, see the Locust documentation.
This repo is set up to deploy review apps on Heroku, and those pull from the staging database to match the experience of deploying as closely as possible! However, note that in order to prevent unapproved model changes from effecting the staging database, migrations are prevented from running on review apps. So those will still have to be reviewed locally.
If something is not behaving intuitively, it is a bug, and should be reported. Report it here: https://github.com/datamade/la-metro-councilmatic/issues
- Fork the project.
- Make your feature addition or bug fix.
- Commit, do not mess with rakefile, version, or history.
- Send a pull request. Bonus points for topic branches.
Copyright (c) 2023 DataMade. Released under the MIT License.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent
