fabiencro / knmt Goto Github PK

I am sorry, it must be obvious for you all. But I'm new at this and can't understand/execute the knmt commands. To prepare the data, your command is:
knmt make_data train.src train.tgt data_prefix --dev_src valid.src --dev_tgt valid.tgt --src_voc_size 100000 --tgt_voc_size 30000

my files' extensions are en and bn

should I replace train.src as train.en and train.tgt as train.bn? I did that, but it says, (after showing some other errors): "No such file or directory"

Also, what should I write in the place of data_prefix?

Please help me out with an actual command with actual file names.

Sir,

Please find a solution for this. The code is not running after this...

ubuntu 16.04 Nvidia V100
knmt version
/usr/local/lib/python2.7/dist-packages/cupy-4.0.0b2-py2.7-linux-x86_64.egg/cupy/core/fusion.py:659: FutureWarning: cupy.core.fusion is experimental. The interface can change in the future.
util.experimental('cupy.core.fusion')
/usr/local/lib/python2.7/dist-packages/h5py/init.py:36: FutureWarning: Conversion of the second argument of issubdtype from float to np.floating is deprecated. In future, it will be treated as np.float64 == np.dtype(float).type.
from ._conv import register_converters as _register_converters
package version: 0.10.0
installed in: /home/levinth/knmt/nmt_chainer/utilities

*********** chainer version ***********
version 4.0.0b2
cuda True
cudnn True
cuda_version 9000
cudnn_version unavailable

but
levinth@zt-gpu-lin-1:/knmt$ echo $CUDA_PATH
/usr/local/cuda
levinth@zt-gpu-lin-1:/knmt$ echo $LD_LIBRARY_PATH
/opt/lib:/usr/local/cuda-9.0/lib64:/usr/local/cuda/extras/CUPTI/lib64:/usr/local/nccl:/usr/local/lib:/usr/local/mpi/lib:/usr/lib/nx/X11/Xinerama:/usr/lib/nx/X11:/home/levinth/torch/install/lib:/home/levinth/torch/install/lib:
levinth@zt-gpu-lin-1:/knmt$ ls -l /usr/local/cuda
lrwxrwxrwx 1 root root 8 Nov 8 14:33 /usr/local/cuda -> cuda-9.0
levinth@zt-gpu-lin-1:/knmt$ ls -l /usr/local/cuda/lib64/libcudnn*
lrwxrwxrwx 1 root root 43 Nov 8 15:21 /usr/local/cuda/lib64/libcudnn.so.7 -> /usr/lib/x86_64-linux-gnu/libcudnn.so.7.0.3
lrwxrwxrwx 1 root root 43 Nov 8 15:14 /usr/local/cuda/lib64/libcudnn.so.7.0 -> /usr/lib/x86_64-linux-gnu/libcudnn.so.7.0.3
levinth@zt-gpu-lin-1:~/knmt$ ls -l /usr/lib/x86_64-linux-gnu/libcudnn.so.7.0.3
-rw-r--r-- 1 root root 285372592 Sep 15 20:02 /usr/lib/x86_64-linux-gnu/libcudnn.so.7.0.3

does this have anything to do with chainer/chainer#3922
where is ws_size?