Comments (5)
Hey @rruizdeaustri
I will have a more detail look, but in general here are some comments:
- Use a simpler model, forget about the wide component and use simply a deeptabular component with defaults. (review the code in your example since the
optimizers
andschedures
are not correctly defined. TheTrainer
not throwing an error is intentional, I might change it, but just define your Trainer as
trainer = Trainer(
model,
objective="binary",
callbacks=[ModelCheckpoint(filepath="model_weights/wd_out")],
metrics=[Accuracy],
)
- The results with Transformer based models depend A LOT on the parameters, far more than in GBMs, where all, XGBoost, LightGBM and CatBoost perform almost to their best performance out of the box. You could have a look to this relatively old post see if it helps
I hope this helps and let me know how you get on with this, see if I can help more
from pytorch-widedeep.
Hi @jrzaurin,
I have made the modifications you suggested and results
make more sense now. I'm optimising hyper-parameters
with optima in resnet and transformer models but the results are
far from the one got with LightGMB: AUC ~0.93 versus ~ 0.98 for lgqbm
Thanks !
Rbt
from pytorch-widedeep.
Hey @rruizdeaustri
Thanks for sharing the results :)
0.05 is perhaps a bit too much, maybe I can look at some examples if you would be willing to share them. However, I am afraid that this is the "brutal" reality for most (true) real world cases when it comes to DL vs GBMs.
You could try some other libraries see if their implementations are better or you get better results (?)
In my experience I have used DL for tabular data in a few occasions, but never aimed to beat GBMs, since I knew was a lost battle.
from pytorch-widedeep.
Hi @jrzaurin,
Yes, these are too much differences !
I could share with you the files I'm using to train as well as the data if you like. Let me know !
Thanks !
from pytorch-widedeep.
Hey @rruizdeaustri !
I am traveling at the moment, but if you join the slack channel we can move the conversation there and we can share the files. See if I have the time to give it a go myself! :)
Thanks!
from pytorch-widedeep.
Related Issues (20)
- Not Being able to reproduce Bert results HOT 5
- pytorch vision module error HOT 1
- save_best_only error and NaN during training HOT 9
- CyclicLR throws ZeroDivisionError when finetuning with a single batch. HOT 2
- EarlyStopping does not store and restore the model HOT 5
- Can I use time series data HOT 6
- CUDA error: device-side assert triggered HOT 5
- Wrong paper links on ContrastiveDenoisingTrainer HOT 2
- how to save the best Epoch HOT 11
- Dropout layer being created on forward pass (in MultiHeadedAttention) HOT 1
- about Wide's input dim HOT 5
- ImportError: cannot import name 'LRScheduler' from 'torch.optim.lr_scheduler' HOT 8
- OSError when importing the package HOT 4
- AttributeError: 'TabMlp' object has no attribute 'with_fds' HOT 3
- Colab session crash on .fit HOT 3
- IndexError: index out of range in self HOT 4
- how to use lr warmup in traing stage? HOT 3
- 'TextPreprocessor' object has no attribute 'embedding_matrix' HOT 6
- How to install the previous version, the current 1.5 version has been working problems? HOT 14
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pytorch-widedeep.