Comments (4)
Note that d = d:shuffle()
is syntactic sugar for doing d = tnt.ShuffleDataset{dataset = d}
. So of course select()
is gone in d
.
Can't you do e = d:shuffle()
and get sample
s from e:iterator()
?
from torchnet.
It works now, thanks 👍
local tnt = require 'torchnet'
local d = tnt.TableDataset{data = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10}}
d = d:split{
train = 0.7,
val = 0.3
}
d:select('val')
print('val')
for sample in d:iterator()() do
print(sample)
end
d:select('train')
print('train')
for sample in d:iterator()() do
print(sample)
end
trainset=d:shuffle()
print('train2')
for sample in trainset:iterator()() do
print(sample)
end
d:select('val')
testset=d:shuffle()
print('val2')
for sample in testset:iterator()() do
print(sample)
end
from torchnet.
Be careful with this though! Your call to d:select('val')
also changes trainset
because SplitDataset.select
is not actually cloning. So both trainset
and testset
have the same underlying dataset d
, with the same partition select
ed at any point in time.
We may have to consider changing SplitDataset
to actually return a table of Dataset
s to prevent these kinds of bugs from occurring.
from torchnet.
Oh so it didn't solve the problem. That's what I thought in the first place. I really think returning a table of Dataset
is a good idea.
I found this issue thinking about a possible BootstrapDataset
which could split a Dataset
in two partitions :
- the first one (train) would be made with a
shuffle{remplacement=true}
call, - the second one (val) would be made from all the non picked samples from the first partition.
from torchnet.
Related Issues (20)
- Document uncorrect about "transform.perm"
- for ListDataset, add an onComplete argument HOT 2
- OptimEngine.test not implemented HOT 2
- fatal thread panic on parallelDatasetIterator HOT 1
- Improve ParallelDatasetIterator documentation HOT 13
- How can i use MSE criterion? HOT 5
- IndexedDataset using string as index for large dataset HOT 4
- returning vector in ListDataset problem. HOT 2
- This error is unclear - what is the problem with my code that is causing this? HOT 3
- Segmentation fault (core dumped) HOT 8
- Bug report: not entering into iterator until thorough depth. HOT 2
- ClassErrorMeter throwing size mismatch error HOT 1
- meter.MultilabelConfusionMeter invalid argument error HOT 1
- some bugs of transform.merge() HOT 1
- bug in transform.tablemergekeys() HOT 1
- Unable to install qlua, and therefore: qlua: module 'torchnet' not found HOT 1
- Hi, I doubt that whether I can use torchnet in win10 64x. Could anybody tell me? HOT 2
- "not enough memory" with small dataset when nthread>1 for ParallelDatasetIterator HOT 4
- How to add torchnet to a custom package on kaggle HOT 1
- RecursionError with meter.ConfusionMeter
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from torchnet.