Comments (10)
Yes, I deleted it after I saw your earlier comment.
from civis-r.
Hi @HenrikBengtsson,
Unfortunately, the "succeeded" state does not come from CivisFuture()
itself, but from the Civis API. The possible states for a job in the Civis API are: "idle", "queued", "running", "succeeded", "failed", "cancelled", and "sleeping". To fully replace "succeeded" with "finished", we would not only need to replace it here in the civis-r
repo (for the R Client), but also in the Civis API, and in the other Clients (for Python and JS).
Also, this change is highly likely to break user pipelines: a child job may be waiting for a "succeeded" response from a parent job before executing.
We would greatly appreciate it if you could keep the exception in place! Would that be possible? If not, there are a few things we could do:
- Upper bound the version of the
future
package in thecivis
DESCRIPTION file. We currently Importfuture (>= 1.8.0)
, so we could add something likefuture (<= 1.31.0)
to make sure our users only use thecivis
package with afuture
package that still has the Civis exception. - We could try to add a copy of the
future::SignalConditions()
function with the Civis exception intocivis
and redirect ourfuture::Future()
to use thecivis::SignalConditions()
function instead?
Do you have a preference? Or maybe another suggestion for how to resolve this issue?
from civis-r.
Update
future$state <- scripts_get_containers_runs(id = future$job$containerId,
so that the correct string in stored.
from civis-r.
I've created HenrikBengtsson/future#667 for improving on the documentation around the internal state
object. I'll have to think more about exactly which possible values should be supported, but at least it'll help you go get the gist.
Unfortunately, the "succeeded" state does not come from CivisFuture() itself, but from the Civis API. The possible states for a job in the Civis API are: "idle", "queued", "running", "succeeded", "failed", "cancelled", and "sleeping". To fully replace "succeeded" with "finished",
To further clarify my previous comment. If you'd like to record the state returned by the Civis API, you are free to store those in your own custom field, say, civis_state
. That will be ignored by the future framework. The state
field is used by the Future
class and has special purpose there. So, you must not use that for your own purposes.
Without knowing or having access to the Civis API, my best guess would be to map:
- Civis states
"queued"
,"running"
to future state"running"
- Civis state
"succeeded"
to future state"finished"
- Civis state
"failed"
to future state"failed"
- Civis state
"cancelled"
to future state"interrupted"
I don't know what Civis states "idle"
and "sleeping"
represent in the life cycle of a future object.
As I mention in HenrikBengtsson/future#667, I might revisit the need for future states "failed"
and "interrupted"
later on, so right now, I'm mainly concerned about you "succeeded"
mapping to "finished"
.
from civis-r.
from civis-r.
For now, I set an upper bound for the future
package in our DESCRIPTION file (Imports: future <= 1.31.0)).
This is a temporary solution, as we are discussing internally on how to make this change without breaking user pipelines. Defining a new "civis_state" field would be a breaking change, that would require a major version update (to v4.0.0), so we are exploring possible alternative approaches. I don't want this to hold up your work on the future
package for longer than it already has though.
from civis-r.
For now, I set an upper bound for the future package in our DESCRIPTION file (Imports: future <= 1.31.0)).
I suspect this might cause more problems for users. I'm not sure, but I think it'll also break my revdep checks when I update future. I'm also not sure CRAN supports that - I'm not aware of any other packages that done that.
Defining a new "civis_state" field would be a breaking change, that would require a major version update (to v4.0.0), so we are exploring possible alternative approaches. I don't want this to hold up your work on the future package for longer than it already has though.
Thanks for moving forward. However, I honestly don't understand the problem. Standard encapsulation techniques should allow you to use whatever name is needed. If I can do:
## FIXME: civis::CivisFuture uses 'succeeded' /HB 2019-06-18
if (!future$state %in% c("finished", "failed", "succeeded")) {
...
}
in the future package, I don't see why you do similar thinkgs in the civis package instead.
Or, are you saying you have server code that rely on the field state
and that it takes certain values?
from civis-r.
v3.1.0 was approved and is now available on CRAN (here). This version deprecates the "local" argument in CivisFuture and adds new API endpoints. It does still have the upper bound of future <= 1.31.0
in the Imports field of the DESCRIPTION, which is only meant as a stop-gap solution.
For a more permanent solution, I made (and merged) a new PR here to map the "succeeded" status we get from the Civis API to a "finished" future$state
, in both resolved.CivisFuture (here) and result.CivisFuture (here). I was initially concerned about users depending on the state of the future in their pipeline logic, but that does not appear to be the case. Instead, once a future is resolved, users get the state of the API endpoint itself, using the await()
function (which should still return a "succeeded" string). With this change, I also reverted the previous change in the DESCRIPTION file: Imports: future (>= 1.8.0)
. If we do encounter any unexpected issues, we will still have the new v3.1.0 to fall back on.
from civis-r.
@HenrikBengtsson, civis v3.1.1
is now available on CRAN (here). This versions removes the upper bound on future
(Reverting back to requiring future >= 1.8.0
, and maps the "succeeded" output from the Civis API to a "finished" future$state
.
Please let me know if there is anything else that may be preventing you from making the desired changes on your end!
from civis-r.
I ran it through my revdep checks, and it looks good now. Thanks.
from civis-r.
Related Issues (20)
- ENH helper function to list models
- ENH helper function to share models
- ENH automatically deploy docs on release
- ENH provide option to override default polling interval HOT 1
- ENH: Automatically retry on all verbs when rate limit is exeeded
- ENH: Point civis_ml script templates to aliases HOT 1
- Add `patch` option to `get_script_fun` HOT 4
- Generalize `write_job_output` to an arbitrary object instead of "File" HOT 2
- Generalized `patch` function (plus a motivating example) HOT 3
- ENH: retry more than 3 times
- Is there any use in writing a file locally first before calling `write_civis_file`? HOT 3
- Adding project/report sharing utility code HOT 3
- BUG: error handling on 429
- Switch to CSVImports for Redshift upload
- DOC: Documentation on CASS/NCOA endpoint not complete HOT 1
- BUG: `fetch_all` throws error on `aliases_list` API function HOT 3
- Argument 'local' of Future() to become defunct [breaks your package] HOT 4
- Is this repo and the 'civis' package maintained? HOT 2
- New SQL reports will need API functions
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from civis-r.