Comments (15)
Yes
from resdsql.
Hi!
Have you fine-tuned RESDSQL on your dataset? Or did you only use the checkpoints we provided to perform inference on your dataset?
from resdsql.
I have been utilizing the provided checkpoints in RESDSQL to enhance my work. However, I am uncertain about the process of fine-tuning RESDSQL on my own dataset. Could you kindly provide guidance on how to proceed with fine-tuning RESDSQL using my specific dataset?
from resdsql.
RESDSQL has been fine-tuned on Spider. Therefore, you should prepare your dataset in the same format as it (its home page https://yale-lily.github.io/spider).
In fact, most Text-to-SQL datasets organize their data in Spider's format (e.g., Dr. Spider, CSpider, BIRD, Kaggle-DBQA, etc.).
from resdsql.
I had already set my dataset in the format of Spider (i.e. tables.json)
from resdsql.
Just tables.json
is not enough.
To train RESDSQL on your dataset, you have to prepare at least three files (Take Spider's file as an example):
database
, a folder where the sqlite databases are saved.train_spider.json
, a json file that contains pairs of training data, each of them should contain three fields:db_id
,query
, andquestion
.tables.json
, a json file that describes the schema of all databases.
To run inference and evaluation, you should prepare a separate dev_gold.sql
file containing the gold SQL query and its corresponding db_id.
from resdsql.
Okay, I'll try this solution.
BTW I have a question do I need to train the model every time whenever I change my dataset?
Is there any way RESDSQL will generate an SQL query on the hidden test set?
from resdsql.
No, if your training set and test set have the same (or similar) distribution, it can be naturally generalized to the hidden test set without additional training.
from resdsql.
Okay.
I followed your training steps and noticed that both train_spider.json
and dev.json
were required. However, I am a bit confused about their differences. Are they essentially the same file with different names, or do they serve distinct purposes in the training process?
from resdsql.
They are different files. train_spider.json
is the training set, and dev.json
is the development set.
from resdsql.
We use dev.json
to select the best checkpoint during fine-tuning.
from resdsql.
So can I use the same dev.json file? or Do I need to create it separately as per my dataset?
from resdsql.
My suggestion would be to create a separate dev.json
so that you can evaluate the performance of the model on unseen data.
from resdsql.
If you're training and evaluating your model on the training set, I don't think it makes sense because the model will memorize your training data to quickly reach (close to) 100% accuracy.
from resdsql.
You mean, train_spider.json and dev.json are the same as we are splitting our data into two sets i.e. train set and test set
from resdsql.
Related Issues (20)
- Execuse me. What happened to paper CodeS? Isn't this article open source before? HOT 9
- Low training metrics HOT 14
- Support for Historical Conversation in RESDSQL HOT 4
- Question about evaluation scripts HOT 2
- 请问推理方法 HOT 2
- 最低支持的GPU内存是多少,我怎么跑不起来。
- Dev result file?
- 部分带有别名的sql在经过normalization处理后出现错误 HOT 2
- Inference script not working HOT 5
- CoSQL HOT 1
- 训练Cross-Encoder的时候为什么24G的显存还不够用? HOT 1
- 关于RESDSQL在BIRD上的运行时间 HOT 2
- Training cross-coder error HOT 1
- xlm_roberta_text2natsql_schema_item_classifier HOT 3
- Evaluation detail on CSpider HOT 1
- 你好,请问如何将自己的数据集处理成CSpider的形式? HOT 3
- 你好,请问如何SQL2NatSQL?我想用自己的数据集跑text2NatSQL的方法。 HOT 2
- 请问模型训练有多gpu并行支持吗 HOT 1
- Can the ranking-filter successfully choose all the right schema items? HOT 1
- 为什么我使用对bird训练的classifier时出现了truncated_dataset.json文件,而且陷入了循环无法结束运行 HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from resdsql.