Comments (8)
from determined.
Hi,
Thanks for trying out this project!
There is an experimental API called Native API that has idiomatic design as the native frameworks, like Tensorflow Keras and Estimator. It requires much less effort to re-code your current model than Trial API. Please check out this tutorial for using it with Tensorflow Keras (https://docs.determined.ai/latest/tutorials/native-tf-keras/index.html). Full documentation is on its way in the next release!
Please join our slack community for any questions or support: https://join.slack.com/t/determined-community/shared_invite/zt-cnj7802v-KcVbaUrIzQOwmkmY7gP0Ew
from determined.
Exactly my same comment of @arita37 The moment I saw
from determined.pytorch import DataLoader, PyTorchTrial
or
from determined.keras import TFKerasTrial, TFKerasTrialContext
class FashionMNISTTrial(TFKerasTrial):
I closed the tutorial
the new approach is definitely more interesting
from determined.experimental.keras import init
# When running from this code from a notebook, add a `command` argument to
# init() specifying the notebook file name.
context = init(config, mode=experimental.Mode.CLUSTER, context_dir=".")
...
model = context.wrap_model(model)
which I would see it evolved into a single decorator line
import determined as dd
@dd.config
def build_model():
return tf.keras.models.Sequential(
[
tf.keras.layers.Flatten(input_shape=(28, 28)),
tf.keras.layers.Dense(128, activation="relu"),
tf.keras.layers.Dropout(0.2),
tf.keras.layers.Dense(10, activation="softmax"),
]
all the configs and other details can be configured at run time
dd.parse_args
which clearly puts almost zero efforts for the user to try out the library.
Worst case scenario the software does the same thing as before
best case scenario it does much more.
Just my 2cents :)
from determined.
from determined.
from determined.
You could simply check if the object is a subclass of one or the other project:
from determined.
@arita37 Right now, we don't have Native API with Pytorch support. For Pytorch, the answer to your question is yes we need to re-code the code with the trial class.
from determined.
@Mistobaan Thanks for the idea of the decorator and the feedback on the new API. The decorator approach definitely looks more flexible. The reason why we don't support decorator is that given how the current system loads user code we want to ensure the model is built after calling init
. That's why we only support .wrap_model
for the time being.
from determined.
Related Issues (20)
- 🐛[bug] Error Starting Up Cluster using det deploy HOT 4
- 🐛[bug] Bad ref on requirements.rst in Docs HOT 1
- 🐛[bug] Resources failed with non-zero exit code: container failed with non-zero exit code: 80 HOT 5
- 🐛[bug] Master refuses to accept agents connection HOT 4
- 🤔[question] Changing the default config path for the determined-agent.service HOT 5
- 🤔[question] Updating the default Determined-Pytorch container to 2.1/2.2 HOT 1
- 🐛[bug] Running Mnist Tutorial distributed causes Runtime Errors and Hanging behavior HOT 12
- 🤔[question] dialing to http://172.22.0.1:32862: dial tcp 172.22.0.1:32862: connect: connection refused HOT 2
- 🐛[bug] Kernel status: pending HOT 11
- 🤔[question] Where can I find the source code of the CLI? HOT 1
- 🤔[question] Can not connect to master node HOT 6
- 🤔[question] Open to updates to EKS deployment? HOT 6
- 🤔[question] How to get pod address by experiment HOT 1
- Integrated with VTable HOT 1
- 🐛[bug] pulling container image: error parsing image name HOT 3
- 💡[feat] local cluster to use offline docker images HOT 1
- 💡[feat] the request to add a feature that releases resources automatically in case of a timeout or if the GPU utilization falls below a certain threshold HOT 4
- 💡[feat] delete the task logs HOT 1
- 🤔[question] I want to callback the interface when the resource is released. HOT 2
- 🤔[question] where to set the `find_unused_parameters=True` HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from determined.