Comments (2)
It would be possible to limit error reporting on the worker side. But the workers are intended to report at most one batch (4*n_cores
) of errored tasks before the backend stops delivering tasks to them. The fact that worker error reporting became a problem revealed an underlying problem in job cancellation (see #887 and #921).
Workers do not currently associate tasks with the batch they were received in, so the first suggestion would add a bit of complexity. The second suggestion would only involve keeping a set of errored jobs, which could be global to the worker or just one set local to each polling operation (I'm now realizing maybe this is what you meant by your first suggestion). This would be relatively simple, but it does add one more step to the reporting process (thus one more thing that can go wrong).
I'm inclined to just get the job cancellation working properly, in which case we should only ever get one batch of errors before an errored job stops. We could then reinforce it with the second suggestion (perhaps limited to the scope of a single polling operation) if we feel there's still risk to be managed.
from r5.
Closing this because we merged #918 to the default branch. In the future I'll try to more regularly set the associated issues on pull requests, which should auto-close them upon merge.
from r5.
Related Issues (20)
- r5 analysis problem HOT 1
- Make TransferAllowance an interface
- Readme in docs/README.r5.md still relevant?
- Send IDs for origin and destination pointsets in HTTP API
- Improve error messages on small/large GIS features
- Support uploading Shapefiles as a single .zip HOT 1
- Extra minute of wait time?
- Unsupported Operation on single point with decay function HOT 2
- Routes added by modifications are not recognizable in paths output HOT 1
- Return warning when decay function is above 0 at maximum cutoff
- Selected-link analysis for assignment HOT 1
- Incorrect display of regional results referencing deleted destination grids
- Some fields of CSV writers are not initialized while headers are written
- User chooses to take a long boomerang transfer at a far away station rather than walk across the street
- A feed_id for regional analysis path results HOT 6
- Check number of opportunities in destination layers used for regional analysis
- Human-readable names for destination layers in CSV output HOT 1
- Transit layer loaded but numerous "ERROR c.c.r.s.StreetLayer" errors (r5r)
- Snapped distance not being calculated as crow flies "walking" distance (r5r) HOT 6
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from r5.