Behaviour with small dataset about char-rnn

[karpathy]

Hello, I'm guessing you have to use a much smaller batch size, maybe try 10, or even down to one. The issue is that you are having too small number of batches, for this small data, I think. Also try making seq_length 50.

I'll add checks in the code to give more appropriate error messages.

[hnykda]

I guess that the problem is that there are no batches in test part. Negative number is nonsence...

 ~/char-rnn (remakes) $ th train.lua -data_dir data/sample -checkpoint_dir cvs/sample -gpuid 0 -eval_val_every 10 -num_layers 1 -rnn_size 50 -val_frac 0.15 -batch_size 100                                                                                         
using CUDA on GPU 0...  
loading data files...   
cutting off end of data so that the batches/sequences divide evenly     
reshaping tensor...     
data load done. Number of batches in train: 18, val: 2, test: -1
vocab size: 7   
creating an LSTM with 1 layers  
number of parameters in the model: 12157        
cloning rnn     
cloning criterion       
1/540 (epoch 0.056), train_loss = 1.91737684, grad/param norm = 2.1561e+00, time/batch = 0.07s  
2/540 (epoch 0.111), train_loss = 1.89370414, grad/param norm = 2.2051e+00, time/batch = 0.04s  
3/540 (epoch 0.167), train_loss = 1.85842120, grad/param norm = 2.4545e+00, time/batch = 0.04s  
4/540 (epoch 0.222), train_loss = 1.78653944, grad/param norm = 3.3634e+00, time/batch = 0.04s  
5/540 (epoch 0.278), train_loss = 1.66507999, grad/param norm = 2.8590e+00, time/batch = 0.04s  
6/540 (epoch 0.333), train_loss = 1.61959934, grad/param norm = 1.6471e+00, time/batch = 0.04s  
7/540 (epoch 0.389), train_loss = 1.61162854, grad/param norm = 1.7974e+00, time/batch = 0.04s  
8/540 (epoch 0.444), train_loss = 1.58403392, grad/param norm = 1.0281e+00, time/batch = 0.04s  
9/540 (epoch 0.500), train_loss = 1.60601129, grad/param norm = 1.7299e+00, time/batch = 0.04s  
evaluating loss over split index 2      
1/2...  
/home/kotrfa/torch/install/bin/luajit: /home/kotrfa/char-rnn/train.lua:131: attempt to index local 'x' (a nil value)
stack traceback:
        /home/kotrfa/char-rnn/train.lua:131: in function 'eval_split'
        /home/kotrfa/char-rnn/train.lua:234: in main chunk
        [C]: in function 'dofile'
        ...trfa/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:131: in main chunk
        [C]: at 0x00406670

I don't really understand what is the test batch for? I thought that data are sliced to two parts - train and evaluation. The train loss is being printed every step, while evaluation is printed on every eval_val_every. But what is the role of test?

[amangupta199334]

What code do we use for testing purpose?