codeforpdx / opentransit-collector Goto Github PK

Currently the S3 keys created by Orion do not have leading zeros in the components with 2 digit numbers like dates and months, so an alphabetic sort of the S3 keys doesn't necessarily put them in the correct order. It would be better to add leading zeros to all path components. This would also require changing opentransit-state-api accordingly.

Issue

Last week, when the streetcar provider got an error it blocked the other provider, trimet, from getting data and putting it in s3.

Proposed fix

I propose two improvements:

1. better error handling / logging in the nextbus provider script
2. separating the provider calls so that they can fail independently

1. better error handling / logging in the nextbus provider script

See code here for Portland's streetcar provider nextbus - https://github.com/codeforpdx/opentransit-collector/blob/master/src/providers/nextbus.js. We could build in some error handling to try/except flow and log errors. Logging errors would be helpful to aid in bug fixing. Right now, the errors go to the console. If you miss it, it can be hard to recreate.

2. separating the provider calls so that they can fail independently

We could add better error handling to the index.js file https://github.com/codeforpdx/opentransit-collector/blob/master/src/index.js. When the streetcar (nextbus) provider was broken, it was blocking the trimet gtfs-realtime collector from working. Ideally, each provider could fail independently.

The git repository was renamed from orion to opentransit-collector. The code (e.g. environment variables and config files) and documentation should be updated accordingly.

Problem

Trimet's API response can have "bad json" (i.e. json-like string that breaks a parser). This will break the line

Goals

• Primary - Make sure that the collector-dev docker stays alive when it encounters "bad json".
• Nice to have - Capture timestamp and bad json string if possible for future reporting to Trimet?

There was an issue with the gtfs-rt feed for the street car so I removed it from the opentransit-collector process. I removed it from the process so that we would still get data from the trimet feed.

Someone can try to recreate the error locally.