indiejoseph / chinese-char-rnn Goto Github PK

I am using around 1GB Chinese data for training.
My server has around 30GB RAM, but it is still running out of memory.
May I know what is the proper hardware setting for such a corpus?
Thanks~

andy1028@andy1028-Envy:/media/andy1028/data1t/os_prj/github/chinese-char-rnn$ python train.py
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcublas.so locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcudnn.so locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcufft.so locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcurand.so locally
{'batch_size': 50,
'checkpoint_dir': 'checkpoint',
'data_dir': 'data',
'dataset_name': 'news',
'decay_rate': 0.95,
'export': False,
'grad_clip': 5.0,
'keep_prob': 0.5,
'l2_reg_lambda': 0.0,
'layer_depth': 2,
'learning_rate': 1,
'log_dir': 'log',
'nce_samples': 25,
'num_epochs': 25,
'rnn_size': 128,
'sample': '',
'save_every': 1000,
'seq_length': 25,
'valid_every': 500}
reading text file
Traceback (most recent call last):
File "train.py", line 220, in
tf.app.run()
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/app.py", line 30, in run
sys.exit(main(sys.argv))
File "train.py", line 91, in main
FLAGS.batch_size, FLAGS.seq_length)
File "/media/andy1028/data1t/os_prj/github/chinese-char-rnn/utils.py", line 91, in init
self.preprocess(input_file, vocab_file, tensor_file, vdata_file, valid_file)
File "/media/andy1028/data1t/os_prj/github/chinese-char-rnn/utils.py", line 99, in preprocess
with codecs.open(input_file, "r", encoding=self.encoding) as f:
File "/usr/lib/python2.7/codecs.py", line 884, in open
file = builtin.open(filename, mode, buffering)
IOError: [Errno 2] No such file or directory: 'data/news/input.txt'
andy1028@andy1028-Envy:/media/andy1028/data1t/os_prj/github/chinese-char-rnn$