Comments (7)
Thanks @cantwbr -- I like examples too, maybe as a first step outlining all the things we can do in a MD doc can be super-useful, i.e. the training commands that @lukaszkaiser wrote above, but more extensive.
The next thing would be good colabs that show training etc.
But, yes, please let us know where we can be better and more importantly what would help you more.
But note that we don't do datasets at all, those are still sourced from TFDS or T2T
from trax.
Thanks for the suggestion @cantwbr !
We have a number of working examples in Trax -- you can see under configs/ a bunch of more advanced models that work. For example, you can train Transformer to translate with
python -m trax.trainer --config_file=$PWD/trax/configs/transformer_wmt_ende_8gb_adafactor.gin
Or an image-generation newer model Reformer like this:
python -m trax.trainer --config_file=$PWD/trax/configs/reformer_imagenet64.gin
There are a few things though that we don't know and it'd be very helpful to hear from you:
- are bash commands ok or is it better to provide colabs (ipython notebooks)?
- should we focus on providing pre-trained models too? (it slows us down, but if it's important...)
- what's the best way to group or present these models?
Please let us know what you think and what would be most helpful!
from trax.
I don't really care about bash vs colabs, but it would be great to have the examples integrated into some API documentation showing the main entry points for different models and layers.
from trax.
Thanks for the suggestion @cantwbr !
We have a number of working examples in Trax -- you can see under configs/ a bunch of more advanced models that work. For example, you can train Transformer to translate with
python -m trax.trainer --config_file=$PWD/trax/configs/transformer_wmt_ende_8gb_adafactor.gin
Or an image-generation newer model Reformer like this:
python -m trax.trainer --config_file=$PWD/trax/configs/reformer_imagenet64.gin
There are a few things though that we don't know and it'd be very helpful to hear from you:
- are bash commands ok or is it better to provide colabs (ipython notebooks)?
- should we focus on providing pre-trained models too? (it slows us down, but if it's important...)
- what's the best way to group or present these models?
Please let us know what you think and what would be most helpful!
Personally, I wouldn't bother having a pre-trained model (but might be helpful for others). I would assume
python -m trax.trainer --config_file=$PWD/trax/configs/reformer_wmt_ende.gin
can be used to train reformer for translation. A guide about how to plug reformer for NMT task would be very helpful
from trax.
Having pretrained models would be extremely useful.
I have a small hobby project for which I would like to host a translation API.
This would be no problem, but I don't have the GPU capacity to train the models myself.
from trax.
to run python -m trax.trainer --config_file=$PWD/trax/configs/mlp_mnist.gin
I get the error :
File "/home/elham/gin-config/trax/models/research/bert.py", line 20, in <module>
from tensorflow.train import load_checkpoint
ModuleNotFoundError: No module named 'tensorflow.train
any ideas?
I already have tensorflow installed
update:
resolved by editing bert.py
I changed two lines:
from tensorflow.train import load_checkpoint
>>> import tensorflow
ckpt = load_checkpoint(self.init_checkpoint)
>> ckpt = tensorflow.train.load_checkpoint(self.init_checkpoint)
Still don't understand why this should be a problem
from trax.
These lines are already changed at head, I hope it works now :). Feel free to reopen or make a new issue of course if there are more problems.
from trax.
Related Issues (20)
- The colab button on Knowledge_Tracing_Transformer.ipynb is not open
- TypeError: float() argument must be a string or a number, not 'jaxlib.tpu_client_extension.PyTpuBuffer'
- Machine Translation Refromer model.pkl for trax 1.4.1?
- ImportError: cannot import name 'MergeHeads' from 'trax.layers.attention'
- how to use trax to translate other languages
- Limit the dataset from TFDS
- TypeError: unsupported operand type(s) for ==: 'Array' and 'tuple' HOT 2
- SelfAttention - problem with tensorflow 2.11.0
- AttributeError: module 'jax.ops' has no attribute 'index_add' HOT 1
- Unable to import trax HOT 1
- Cannot import Trax HOT 8
- Could not normally run trax using GPU in local computer
- Issue when running training_loop.run(2000) - message StopIteration in next_batch(self)
- Is possible Linformer algorithm ?
- Can I do simple tokenization?
- Can't run `bert_vocab_from_dataset` without `TypeError: Tensor is unhashable` when import `trax` with `tensorflow`
- Are any easy ways to use something like `train_test_split` from `sklearn`?
- AttributeError: 'function' object has no attribute 'n_steps_per_checkpoint' for NLP Machine translation model HOT 1
- Error loading loop from a checkpoint HOT 1
- Inconsistency in function's doc-string HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from trax.