city-bureau / city-scrapers-cle Goto Github PK

URL: https://www.clevelandohio.gov/CityofCleveland/Home/Government/CityAgencies/OPS

This one is a bit weird because there is no schedule. Basically whenever a meeting is scheduled, that info and the link to the meeting is posted at the bottom of this page.

double check with the agency to verify this new url is correct

The new link for the design review is at https://planning.clevelandohio.gov/designreview/schedule.php

scraper is at
https://github.com/City-Bureau/city-scrapers-cle/blob/main/city_scrapers/spiders/cle_design_review.py

The scraper for CMSD needs to be updated.

Upcoming board minutes and agendas are at https://go.boarddocs.com/oh/cmsd/Board.nsf/Public

same issues as bug as noted in issue #52. PR#0053

Meetings are still going on, however, due to covid the meetings are online. I'm unsure how City-Bureau would handle
remote meetings.

A link to the meetings postings can be found here: http://bc.cuyahogacounty.us/en-US/Board-of-Control.aspx
The meeting links currently all seem like they are the same: http://council.cuyahogacounty.us/en-US/Streaming-Video-CMB.aspx

Reported by Kate W. in the documenters Open House

on the documenters.org ; there are no meetings listed for Cuyahoga County Council.

The documenters url for county council is
https://cleveland.documenters.org/agencies/cuyahoga-county-council-209/
and upcoming County Council meetings should be listed there... but they are not. .

The scraper is specifically using an API from county council's website - http://council.cuyahogacounty.us/api/items/GetItemsByType?itemTypeCode=EVENT;NEWS;EVENTREG&languageCd=en-US&siteKey=141&returnEventsAfterDate=06/05/2020&returnEventsBeforeDate=12/02/2020
to obtain results and all of the meetings that returned happened this summer.

Looking at the county council website, there's no meetings listed ?! http://council.cuyahogacounty.us/en-US/events-calendar.aspx

There is a new Board of Elections Website with a new calendar page, so we will need to change the scraper to reflect the current page here: https://boe.cuyahogacounty.gov/calendar

This is the old scraper: https://github.com/City-Bureau/city-scrapers-cle/blob/main/city_scrapers/spiders/cuya_elections.py

The URL for the Board of Building Standards has changed; and the scraper needs to be updated as a result.

The new url is:
https://clevelandohio.gov/CityofCleveland/Home/Government/CityAgencies/CityPlanningCommission/BuildingStandards

Its scraper is at:
https://github.com/City-Bureau/city-scrapers-cle/blob/main/city_scrapers/spiders/cle_building_standards.py

hello. @skorasaurus

the cle_city_scrapers spider currently failing due to an empty meeting entry (see screenshot). specifically, the start time is empty because the meeting is cancelled, and therefore the start time is returning as a NoneType object as opposed to a datatime object.

PR here: #56

I'm Lawrence Caswell, field coordinator for Cleveland Documenters. A few things:

THE MAIN THING: The Cle Docs team has decided to focus primarily on Cleveland City Council for this pilot year.
Capacity and impact were the main factors in the decision. Of note: all of Council and the mayor are up for reelection next year. This doesn't mean we won't be documenting other meetings. It does mean that we need to make sure the docs.org pages for Cleveland City Council and its committees have the information our documenters will need. We’re reviewing those pages now.

2 questions:

• What can I do to help?
• We have been added to a mailing list for info from Council. Would access to that email be useful for City Scrapers?

The other updates:

• We plan to hold our first Cleveland Documenters orientation and “Covering Your Local Government” training in late October. We should begin assigning meetings the following week.

• Beginning this week, we’re holding Cleveland Documenters virtual “office hours” every Friday from 2-3pm via Zoom. Drop in during the hour to ask questions, meet others in the network, and to talk about how we can build this community. City Scrapers included!. Here’s the FB event page: https://www.facebook.com/events/800955484057069/

create a github/gitlab notification so that when the bi-daily cron job that scrapes fails; we're notified in the open-cleveland slack https://github.com/City-Bureau/city-scrapers-cle/blob/main/.github/workflows/cron.yml

According to
https://cleveland.documenters.org/agencies/greater-cleveland-regional-transit-authority-community-advisory-committee-184/
there are no upcoming meetings.

But the meetings are listed at https://www.documenters.org/agencies/greater-cleveland-regional-transit-authority-211/

URL:

I noticed that the County Veterans Service Commission's Board board does not list any HTML elements of upcoming meetings; just that they will be held (without listing the dates).

How should these meetings be created to show up in the calendar since there's nothing to exactly scrape?

They do have a calendar but no meetings are listed there.

create a slack notification OR create a github issue when a particular scraper fails

minor issue; but with the landmarks commission, the youtube url is being stripped of special characters, so the url looks as follows on a meeting page

Cuyahoga Homeless Services

There appear to be related events on this calendar from cuyahoga county health services site

https://hhs.cuyahogacounty.us/news/calendar/2020/09/10/family-and-children-first-council-meeting/family-and-children-first-council-meeting .

There were no events on the page linked in the scrapers. Maybe that's something to use?

verify if it works; they did a redesign;

the new url to scrape from is: https://planning.clevelandohio.gov/designreview/schedule.php
(this is correct, despite the URL; this also has the planning committee there

@skorasaurus

hey. Paul mentioned this scraper broke yesterday (08/22/2022). i think i fixed it, making a PR now.

bug: lines#83-85 .split() the text for each select item, and assumes the return of two items max. however, some options have additional verbiage on their item so returns more than two items.

fix: lines#83-88 (on PR#53) now extract the month and day separately instead of assuming the return of two items.

the url for its meetings has changed:
The new URL is https://www.adamhscc.org/about-us/board-members/board-meetings

The scraper is at: https://github.com/City-Bureau/city-scrapers-cle/blob/main/city_scrapers/spiders/cuya_adamhs.py

This is a lower priority.

There is a newly formed committee organized by the ADAMHS board (Alcohol, Drug Addiction, Mental Health Services) that is a new form of another meeting body we have been covering at Documenters. This committee now has its own page and calendar so it would be great to create a scraper for it.

Here is the page URL: https://www.adamhscc.org/about-us/current-initiatives/cuyahoga-county-diversion-center

Documenters page: https://www.documenters.org/agencies/cuyahoga-county-arts-culture-193/

scraper is currently at https://github.com/City-Bureau/city-scrapers-cle/blob/main/city_scrapers/spiders/cuya_arts_culture.py

The url for the scraping is still at https://www.cacgrants.org/about-us/board/board-materials

making a note for myself so I don't forget. As a web developer of Cleveland Public Library, I modified the page where the Library's Board Agendas are at https://cpl.org ; so changes will need to be made for the scraper.

I'll try to tackle this over the weekend.

Recommended new link to scrape is https://cpl.org/aboutthelibrary/board-of-trustees/board-agenda-archive/

At https://cleveland.documenters.org/scrapers/ ; it's reported the landmarks committee doesn't have any upcoming meetings

But at https://cleveland.documenters.org/agencies/cleveland-landmarks-commission-158/ upcoming meetings are listed.

and then if you go to the individual meeting's page within documenters, (e.g.
https://cleveland.documenters.org/meetings/landmarks-commission-32679/) it says the meeting is canceled. !??!?

This appears to be the old page that was scraped: http://city.cleveland.oh.us/CityofCleveland/Home/Government/CityAgencies/CityPlanningCommission/LandmarksCommission

This is the new page: https://planning.clevelandohio.gov/landmark/AGENDALIST.html

This would be a new scraper; for:
Cuyahoga County Citizens' Advisory Council on Equity (CACE)

This does a have a corresponding page at the Documenters already made for it:
https://cleveland.documenters.org/agencies/cuyahoga-county-citizens-advisory-council-on-equity-cace-250/

The meeting information is at: http://bc.cuyahogacounty.us/en-US/Citizens-Advisory-Council-Equity.aspx?Year=2021

While trying to learn how to parse PDFs from other examples, I learned that the county board of Developmental Disabilities
has updated their website; the url in the scraper redirects to their homepage
the new url is https://cuyahogabdd.org/about-us/meet-the-board/

The URL for the MHRAC committee calendar is here: https://www.adamhscc.org/about-us/current-initiatives/task-forces-and-coalitions/mental-health-response-advisory-committee-mhrac

Needs to be created from scratch. All the General MHRAC meetings and subcommittee meetings are on this calendar.

Agency is ADAMHS Board of Cuyahoga County.