Comments (18)
jonathanasdf
commented on July 24, 2024
Is this when going through the codelab? Can you post a stack trace?
from lingvo.
nim456
commented on July 24, 2024
no its when i am running it on the jupyter notebook
from lingvo.
nim456
commented on July 24, 2024
I0301 06:12:24.380959 140645137897216 download_brown_corpus.py:45]
Download completed. Preprocessing...
I0301 06:12:58.832870 140645137897216 download_brown_corpus.py:139] All done.
Overwriting lingvo/tasks/punctuator/input_generator.py
Overwriting lingvo/tasks/punctuator/params/codelab.py
741
I0301 06:13:05.301585 140595434813184 model_imports.py:46] Importing lingvo.tasks.asr.params
I0301 06:13:05.337105 140595434813184 model_registry.py:124] Registering models from module: lingvo.tasks.asr.params.librispeech
I0301 06:13:05.343930 140595434813184 model_imports.py:46] Importing lingvo.tasks.image.params
I0301 06:13:05.347310 140595434813184 model_registry.py:124] Registering models from module: lingvo.tasks.image.params.mnist
I0301 06:13:05.347580 140595434813184 model_imports.py:46] Importing lingvo.tasks.lm.params
I0301 06:13:05.350440 140595434813184 model_registry.py:124] Registering models from module: lingvo.tasks.lm.params.one_billion_wds
I0301 06:13:05.351483 140595434813184 model_imports.py:46] Importing lingvo.tasks.mt.params
I0301 06:13:05.358597 140595434813184 model_registry.py:124] Registering models from module: lingvo.tasks.mt.params.wmt14_en_de
I0301 06:13:05.367434 140595434813184 model_registry.py:124] Registering models from module: lingvo.tasks.mt.params.wmtm16_en_de
I0301 06:13:05.367786 140595434813184 model_imports.py:46] Importing lingvo.tasks.punctuator.params
I0301 06:13:05.373699 140595434813184 model_registry.py:124] Registering models from module: lingvo.tasks.punctuator.params.codelab
2019-03-01 06:13:05.374471: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2019-03-01 06:13:05.412597: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Could not dlopen library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory
2019-03-01 06:13:05.412668: E tensorflow/stream_executor/cuda/cuda_driver.cc:318] failed call to cuInit: UNKNOWN ERROR (303)
2019-03-01 06:13:05.412727: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:160] no NVIDIA GPU device is present: /dev/nvidia0 does not exist
2019-03-01 06:13:05.418375: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2200000000 Hz
2019-03-01 06:13:05.422339: I tensorflow/compiler/xla/service/service.cc:162] XLA service 0x6dfb690 executing computations on platform Host. Devices:
2019-03-01 06:13:05.422418: I tensorflow/compiler/xla/service/service.cc:169] StreamExecutor device (0): <undefined>, <undefined>
2019-03-01 06:13:05.450027: I tensorflow/core/distributed_runtime/rpc/grpc_channel.cc:250] Initialize GrpcChannelCache for job local -> {0 -> localhost:44249}
2019-03-01 06:13:05.554020: I tensorflow/core/distributed_runtime/rpc/grpc_server_lib.cc:359] Started server with target: grpc://localhost:44249
I0301 06:13:05.554538 140595434813184 trainer.py:1258] Job controller start
I0301 06:13:05.915537 140595434813184 base_runner.py:67] ============================================================
I0301 06:13:05.958966 140595434813184 base_runner.py:69] add_summary : True
I0301 06:13:05.959337 140595434813184 base_runner.py:69] allow_implicit_capture : NoneType
I0301 06:13:05.959486 140595434813184 base_runner.py:69] cls : type/lingvo.core.base_model/SingleTaskModel
I0301 06:13:05.959630 140595434813184 base_runner.py:69] cluster.add_summary : NoneType
I0301 06:13:05.959784 140595434813184 base_runner.py:69] cluster.cls : type/lingvo.core.cluster/_Cluster
I0301 06:13:05.959971 140595434813184 base_runner.py:69] cluster.controller.devices_per_split : 1
I0301 06:13:05.960146 140595434813184 base_runner.py:69] cluster.controller.gpus_per_replica : 0
I0301 06:13:05.960326 140595434813184 base_runner.py:69] cluster.controller.name : '/job:local'
I0301 06:13:05.960522 140595434813184 base_runner.py:69] cluster.controller.num_tpu_hosts : 0
I0301 06:13:05.960676 140595434813184 base_runner.py:69] cluster.controller.replicas : 1
I0301 06:13:05.960855 140595434813184 base_runner.py:69] cluster.controller.tpus_per_replica : 0
I0301 06:13:05.961020 140595434813184 base_runner.py:69] cluster.decoder.devices_per_split : 1
I0301 06:13:05.961184 140595434813184 base_runner.py:69] cluster.decoder.gpus_per_replica : 0
I0301 06:13:05.961360 140595434813184 base_runner.py:69] cluster.decoder.name : '/job:local'
I0301 06:13:05.961535 140595434813184 base_runner.py:69] cluster.decoder.num_tpu_hosts : 0
I0301 06:13:05.961663 140595434813184 base_runner.py:69] cluster.decoder.replicas : 1
I0301 06:13:05.961806 140595434813184 base_runner.py:69] cluster.decoder.tpus_per_replica : 0
I0301 06:13:05.961980 140595434813184 base_runner.py:69] cluster.evaler.devices_per_split : 1
I0301 06:13:05.962181 140595434813184 base_runner.py:69] cluster.evaler.gpus_per_replica : 0
I0301 06:13:05.962340 140595434813184 base_runner.py:69] cluster.evaler.name : '/job:local'
I0301 06:13:05.962493 140595434813184 base_runner.py:69] cluster.evaler.num_tpu_hosts : 0
I0301 06:13:05.962634 140595434813184 base_runner.py:69] cluster.evaler.replicas : 1
I0301 06:13:05.962773 140595434813184 base_runner.py:69] cluster.evaler.tpus_per_replica : 0
I0301 06:13:05.962923 140595434813184 base_runner.py:69] cluster.input.devices_per_split : 1
I0301 06:13:05.963064 140595434813184 base_runner.py:69] cluster.input.gpus_per_replica : 0
I0301 06:13:05.963224 140595434813184 base_runner.py:69] cluster.input.name : '/job:local'
I0301 06:13:05.963380 140595434813184 base_runner.py:69] cluster.input.num_tpu_hosts : 0
I0301 06:13:05.963506 140595434813184 base_runner.py:69] cluster.input.replicas : 0
I0301 06:13:05.963650 140595434813184 base_runner.py:69] cluster.input.tpus_per_replica : 0
I0301 06:13:05.963824 140595434813184 base_runner.py:69] cluster.job : 'controller'
I0301 06:13:05.963960 140595434813184 base_runner.py:69] cluster.mode : 'sync'
I0301 06:13:05.964093 140595434813184 base_runner.py:69] cluster.ps.devices_per_split : 1
I0301 06:13:05.964215 140595434813184 base_runner.py:69] cluster.ps.gpus_per_replica : 0
I0301 06:13:05.964359 140595434813184 base_runner.py:69] cluster.ps.name : '/job:local'
I0301 06:13:05.964497 140595434813184 base_runner.py:69] cluster.ps.num_tpu_hosts : 0
I0301 06:13:05.964637 140595434813184 base_runner.py:69] cluster.ps.replicas : 1
I0301 06:13:05.964778 140595434813184 base_runner.py:69] cluster.ps.tpus_per_replica : 0
I0301 06:13:05.964932 140595434813184 base_runner.py:69] cluster.task : 0
I0301 06:13:05.965056 140595434813184 base_runner.py:69] cluster.worker.devices_per_split : 1
I0301 06:13:05.965213 140595434813184 base_runner.py:69] cluster.worker.gpus_per_replica : 0
I0301 06:13:05.965361 140595434813184 base_runner.py:69] cluster.worker.name : '/job:local'
I0301 06:13:05.965488 140595434813184 base_runner.py:69] cluster.worker.num_tpu_hosts : 0
I0301 06:13:05.965631 140595434813184 base_runner.py:69] cluster.worker.replicas : 1
I0301 06:13:05.965754 140595434813184 base_runner.py:69] cluster.worker.tpus_per_replica : 0
I0301 06:13:05.965909 140595434813184 base_runner.py:69] dtype : float32
I0301 06:13:05.966068 140595434813184 base_runner.py:69] fprop_dtype : NoneType
I0301 06:13:05.966187 140595434813184 base_runner.py:69] inference_driver_name : NoneType
I0301 06:13:05.966320 140595434813184 base_runner.py:69] input.add_summary : True
I0301 06:13:05.966439 140595434813184 base_runner.py:69] input.allow_implicit_capture : NoneType
I0301 06:13:05.966576 140595434813184 base_runner.py:69] input.bucket_batch_limit : [512, 256, 160, 80, 40]
I0301 06:13:05.966702 140595434813184 base_runner.py:69] input.bucket_upper_bound : [10, 20, 30, 60, 120]
I0301 06:13:05.966856 140595434813184 base_runner.py:69] input.cls : type/lingvo.tasks.punctuator.input_generator/PunctuatorInput
I0301 06:13:05.966989 140595434813184 base_runner.py:69] input.dtype : float32
I0301 06:13:05.967117 140595434813184 base_runner.py:69] input.file_buffer_size : 10000
I0301 06:13:05.967252 140595434813184 base_runner.py:69] input.file_parallelism : 1
I0301 06:13:05.967394 140595434813184 base_runner.py:69] input.file_pattern : 'text:/tmp/punctuator_data/train.txt'
I0301 06:13:05.967530 140595434813184 base_runner.py:69] input.file_random_seed : 0
I0301 06:13:05.967648 140595434813184 base_runner.py:69] input.flush_every_n : 0
I0301 06:13:05.967772 140595434813184 base_runner.py:69] input.fprop_dtype : NoneType
I0301 06:13:05.967917 140595434813184 base_runner.py:69] input.inference_driver_name : NoneType
I0301 06:13:05.968044 140595434813184 base_runner.py:69] input.is_eval : NoneType
I0301 06:13:05.968183 140595434813184 base_runner.py:69] input.is_inference : NoneType
I0301 06:13:05.968324 140595434813184 base_runner.py:69] input.name : 'input'
I0301 06:13:05.968445 140595434813184 base_runner.py:69] input.num_batcher_threads : 1
I0301 06:13:05.968581 140595434813184 base_runner.py:69] input.num_samples : 0
I0301 06:13:05.968725 140595434813184 base_runner.py:69] input.pad_to_max_seq_length : False
I0301 06:13:05.968867 140595434813184 base_runner.py:69] input.params_init.method : 'xavier'
I0301 06:13:05.969003 140595434813184 base_runner.py:69] input.params_init.scale : 1.000001
I0301 06:13:05.969146 140595434813184 base_runner.py:69] input.params_init.seed : NoneType
I0301 06:13:05.969293 140595434813184 base_runner.py:69] input.per_step_infer : False
I0301 06:13:05.969413 140595434813184 base_runner.py:69] input.random_seed : NoneType
I0301 06:13:05.969531 140595434813184 base_runner.py:69] input.source_max_length : 122
I0301 06:13:05.969670 140595434813184 base_runner.py:69] input.target_max_length : 122
I0301 06:13:05.969804 140595434813184 base_runner.py:69] input.tokenizer.add_summary : True
I0301 06:13:05.969964 140595434813184 base_runner.py:69] input.tokenizer.allow_implicit_capture : NoneType
I0301 06:13:05.970103 140595434813184 base_runner.py:69] input.tokenizer.append_eos : True
I0301 06:13:05.970228 140595434813184 base_runner.py:69] input.tokenizer.cls : type/lingvo.core.tokenizers/WpmTokenizer
I0301 06:13:05.970375 140595434813184 base_runner.py:69] input.tokenizer.dtype : float32
I0301 06:13:05.970510 140595434813184 base_runner.py:69] input.tokenizer.fprop_dtype : NoneType
I0301 06:13:05.970633 140595434813184 base_runner.py:69] input.tokenizer.inference_driver_name : NoneType
I0301 06:13:05.970761 140595434813184 base_runner.py:69] input.tokenizer.is_eval : NoneType
I0301 06:13:05.970916 140595434813184 base_runner.py:69] input.tokenizer.is_inference : NoneType
I0301 06:13:05.971045 140595434813184 base_runner.py:69] input.tokenizer.merge_prob : 1.0
I0301 06:13:05.971183 140595434813184 base_runner.py:69] input.tokenizer.name : 'tokenizer'
I0301 06:13:05.971328 140595434813184 base_runner.py:69] input.tokenizer.pad_to_max_length : False
I0301 06:13:05.971465 140595434813184 base_runner.py:69] input.tokenizer.params_init.method : 'xavier'
I0301 06:13:05.971599 140595434813184 base_runner.py:69] input.tokenizer.params_init.scale : 1.000001
I0301 06:13:05.971734 140595434813184 base_runner.py:69] input.tokenizer.params_init.seed : NoneType
I0301 06:13:05.971878 140595434813184 base_runner.py:69] input.tokenizer.per_step_infer : False
I0301 06:13:05.972003 140595434813184 base_runner.py:69] input.tokenizer.random_seed : NoneType
I0301 06:13:05.972134 140595434813184 base_runner.py:69] input.tokenizer.target_eos_id : 2
I0301 06:13:05.972286 140595434813184 base_runner.py:69] input.tokenizer.target_sos_id : 1
I0301 06:13:05.972421 140595434813184 base_runner.py:69] input.tokenizer.target_unk_id : 0
I0301 06:13:05.972563 140595434813184 base_runner.py:69] input.tokenizer.vn.global_vn : False
I0301 06:13:05.972695 140595434813184 base_runner.py:69] input.tokenizer.vn.per_step_vn : False
I0301 06:13:05.972836 140595434813184 base_runner.py:69] input.tokenizer.vn.scale : NoneType
I0301 06:13:05.972975 140595434813184 base_runner.py:69] input.tokenizer.vn.seed : NoneType
I0301 06:13:05.973109 140595434813184 base_runner.py:69] input.tokenizer.vocab_filepath : '/root/.cache/bazel/_bazel_root/5b16c1cfa7fded7181f7e172eb7ad8a9/execroot/__main__/bazel-out/k8-opt/bin/lingvo/ipython_kernel.runfiles/__main__/lingvo/tasks/punctuator/params/brown_corpus_wpm.16000.vocab'
I0301 06:13:05.973243 140595434813184 base_runner.py:69] input.tokenizer.vocab_size : 16000
I0301 06:13:05.973397 140595434813184 base_runner.py:69] input.tokenizer_dict : {}
I0301 06:13:05.973540 140595434813184 base_runner.py:69] input.tpu_infeed_parallism : 1
I0301 06:13:05.973681 140595434813184 base_runner.py:69] input.use_per_host_infeed : False
I0301 06:13:05.973882 140595434813184 base_runner.py:69] input.use_within_batch_mixing : False
I0301 06:13:05.974111 140595434813184 base_runner.py:69] input.vn.global_vn : False
I0301 06:13:05.974256 140595434813184 base_runner.py:69] input.vn.per_step_vn : False
I0301 06:13:05.974404 140595434813184 base_runner.py:69] input.vn.scale : NoneType
I0301 06:13:05.974524 140595434813184 base_runner.py:69] input.vn.seed : NoneType
I0301 06:13:05.974658 140595434813184 base_runner.py:69] is_eval : NoneType
I0301 06:13:05.974801 140595434813184 base_runner.py:69] is_inference : NoneType
I0301 06:13:05.974955 140595434813184 base_runner.py:69] model : 'punctuator.codelab.RNMTModel@/root/.cache/bazel/_bazel_root/5b16c1cfa7fded7181f7e172eb7ad8a9/execroot/__main__/bazel-out/k8-opt/bin/lingvo/ipython_kernel.runfiles/__main__/lingvo/tasks/punctuator/params/codelab.py:90'
I0301 06:13:05.975100 140595434813184 base_runner.py:69] name : ''
I0301 06:13:05.975244 140595434813184 base_runner.py:69] params_init.method : 'xavier'
I0301 06:13:05.975404 140595434813184 base_runner.py:69] params_init.scale : 1.000001
I0301 06:13:05.975537 140595434813184 base_runner.py:69] params_init.seed : NoneType
I0301 06:13:05.975672 140595434813184 base_runner.py:69] per_step_infer : False
I0301 06:13:05.975826 140595434813184 base_runner.py:69] random_seed : NoneType
I0301 06:13:05.975970 140595434813184 base_runner.py:69] task.add_summary : True
I0301 06:13:05.976092 140595434813184 base_runner.py:69] task.allow_implicit_capture : NoneType
I0301 06:13:05.976218 140595434813184 base_runner.py:69] task.cls : type/lingvo.tasks.punctuator.model/RNMTModel
I0301 06:13:05.976370 140595434813184 base_runner.py:69] task.decoder.add_summary : True
I0301 06:13:05.976502 140595434813184 base_runner.py:69] task.decoder.allow_implicit_capture : NoneType
I0301 06:13:05.976630 140595434813184 base_runner.py:69] task.decoder.atten_rnn_cell_tpl.add_summary : True
I0301 06:13:05.976766 140595434813184 base_runner.py:69] task.decoder.atten_rnn_cell_tpl.allow_implicit_capture : NoneType
I0301 06:13:05.976922 140595434813184 base_runner.py:69] task.decoder.atten_rnn_cell_tpl.apply_pruning : False
I0301 06:13:05.977097 140595434813184 base_runner.py:69] task.decoder.atten_rnn_cell_tpl.bias_init.method : 'constant'
I0301 06:13:05.977243 140595434813184 base_runner.py:69] task.decoder.atten_rnn_cell_tpl.bias_init.scale : 0.0
I0301 06:13:05.977392 140595434813184 base_runner.py:69] task.decoder.atten_rnn_cell_tpl.bias_init.seed : 0
I0301 06:13:05.977519 140595434813184 base_runner.py:69] task.decoder.atten_rnn_cell_tpl.cell_value_cap : 10.0
I0301 06:13:05.977662 140595434813184 base_runner.py:69] task.decoder.atten_rnn_cell_tpl.cls : type/lingvo.core.rnn_cell/LayerNormalizedLSTMCellSimple
I0301 06:13:05.977824 140595434813184 base_runner.py:69] task.decoder.atten_rnn_cell_tpl.couple_input_forget_gates : False
I0301 06:13:05.978001 140595434813184 base_runner.py:69] task.decoder.atten_rnn_cell_tpl.dtype : float32
I0301 06:13:05.978127 140595434813184 base_runner.py:69] task.decoder.atten_rnn_cell_tpl.enable_lstm_bias : True
I0301 06:13:05.978279 140595434813184 base_runner.py:69] task.decoder.atten_rnn_cell_tpl.forget_gate_bias : 0.0
I0301 06:13:05.978413 140595434813184 base_runner.py:69] task.decoder.atten_rnn_cell_tpl.fprop_dtype : NoneType
I0301 06:13:05.978549 140595434813184 base_runner.py:69] task.decoder.atten_rnn_cell_tpl.inference_driver_name : NoneType
I0301 06:13:05.978681 140595434813184 base_runner.py:69] task.decoder.atten_rnn_cell_tpl.inputs_arity : 1
I0301 06:13:05.978849 140595434813184 base_runner.py:69] task.decoder.atten_rnn_cell_tpl.is_eval : NoneType
I0301 06:13:05.979000 140595434813184 base_runner.py:69] task.decoder.atten_rnn_cell_tpl.is_inference : NoneType
I0301 06:13:05.979171 140595434813184 base_runner.py:69] task.decoder.atten_rnn_cell_tpl.layer_norm_epsilon : 1e-08
I0301 06:13:05.979331 140595434813184 base_runner.py:69] task.decoder.atten_rnn_cell_tpl.name : ''
I0301 06:13:05.979465 140595434813184 base_runner.py:69] task.decoder.atten_rnn_cell_tpl.num_hidden_nodes : 0
I0301 06:13:05.979594 140595434813184 base_runner.py:69] task.decoder.atten_rnn_cell_tpl.num_input_nodes : 0
I0301 06:13:05.979734 140595434813184 base_runner.py:69] task.decoder.atten_rnn_cell_tpl.num_output_nodes : 1024
I0301 06:13:05.979904 140595434813184 base_runner.py:69] task.decoder.atten_rnn_cell_tpl.output_nonlinearity : False
I0301 06:13:05.980088 140595434813184 base_runner.py:69] task.decoder.atten_rnn_cell_tpl.params_init.method : 'uniform'
I0301 06:13:05.980228 140595434813184 base_runner.py:69] task.decoder.atten_rnn_cell_tpl.params_init.scale : 0.04
I0301 06:13:05.980396 140595434813184 base_runner.py:69] task.decoder.atten_rnn_cell_tpl.params_init.seed : NoneType
I0301 06:13:05.980545 140595434813184 base_runner.py:69] task.decoder.atten_rnn_cell_tpl.per_step_infer : False
I0301 06:13:05.980696 140595434813184 base_runner.py:69] task.decoder.atten_rnn_cell_tpl.qdomain.c_state : NoneType
I0301 06:13:05.980849 140595434813184 base_runner.py:69] task.decoder.atten_rnn_cell_tpl.qdomain.default : NoneType
I0301 06:13:05.980992 140595434813184 base_runner.py:69] task.decoder.atten_rnn_cell_tpl.qdomain.fullyconnected : NoneType
I0301 06:13:05.981118 140595434813184 base_runner.py:69] task.decoder.atten_rnn_cell_tpl.qdomain.m_state : NoneType
I0301 06:13:05.981236 140595434813184 base_runner.py:69] task.decoder.atten_rnn_cell_tpl.qdomain.weight : NoneType
I0301 06:13:05.981391 140595434813184 base_runner.py:69] task.decoder.atten_rnn_cell_tpl.random_seed : NoneType
I0301 06:13:05.981542 140595434813184 base_runner.py:69] task.decoder.atten_rnn_cell_tpl.reset_cell_state : False
I0301 06:13:05.981725 140595434813184 base_runner.py:69] task.decoder.atten_rnn_cell_tpl.vn.global_vn : False
I0301 06:13:05.981944 140595434813184 base_runner.py:69] task.decoder.atten_rnn_cell_tpl.vn.per_step_vn : False
I0301 06:13:05.982095 140595434813184 base_runner.py:69] task.decoder.atten_rnn_cell_tpl.vn.scale : NoneType
I0301 06:13:05.982280 140595434813184 base_runner.py:69] task.decoder.atten_rnn_cell_tpl.vn.seed : NoneType
I0301 06:13:05.982456 140595434813184 base_runner.py:69] task.decoder.atten_rnn_cell_tpl.zo_prob : 0.0
I0301 06:13:05.982633 140595434813184 base_runner.py:69] task.decoder.atten_rnn_cls : type/lingvo.core.rnn_layers/FRNNWithAttention
I0301 06:13:05.982836 140595434813184 base_runner.py:69] task.decoder.attention.add_summary : True
I0301 06:13:05.983020 140595434813184 base_runner.py:69] task.decoder.attention.allow_implicit_capture : NoneType
I0301 06:13:05.983175 140595434813184 base_runner.py:69] task.decoder.attention.atten_dropout_deterministic : True
I0301 06:13:05.983377 140595434813184 base_runner.py:69] task.decoder.attention.atten_dropout_prob : 0.3
I0301 06:13:05.983556 140595434813184 base_runner.py:69] task.decoder.attention.cls : type/lingvo.core.attention/MultiHeadedAttention
I0301 06:13:05.983746 140595434813184 base_runner.py:69] task.decoder.attention.context_dim : 1024
I0301 06:13:05.983903 140595434813184 base_runner.py:69] task.decoder.attention.ctx_post_proj_dim : 0
I0301 06:13:05.984030 140595434813184 base_runner.py:69] task.decoder.attention.dtype : float32
I0301 06:13:05.984188 140595434813184 base_runner.py:69] task.decoder.attention.enable_ctx_post_proj : False
I0301 06:13:05.984384 140595434813184 base_runner.py:69] task.decoder.attention.enable_ctx_pre_proj : False
I0301 06:13:05.984533 140595434813184 base_runner.py:69] task.decoder.attention.enable_query_proj : True
I0301 06:13:05.984677 140595434813184 base_runner.py:69] task.decoder.attention.enable_source_proj : True
I0301 06:13:05.984821 140595434813184 base_runner.py:69] task.decoder.attention.fprop_dtype : NoneType
I0301 06:13:05.984956 140595434813184 base_runner.py:69] task.decoder.attention.hidden_dim : 1024
I0301 06:13:05.985122 140595434813184 base_runner.py:69] task.decoder.attention.inference_driver_name : NoneType
I0301 06:13:05.985316 140595434813184 base_runner.py:69] task.decoder.attention.inner_atten_params.add_summary : True
I0301 06:13:05.985455 140595434813184 base_runner.py:69] task.decoder.attention.inner_atten_params.allow_implicit_capture : NoneType
I0301 06:13:05.985588 140595434813184 base_runner.py:69] task.decoder.attention.inner_atten_params.atten_dropout_deterministic : False
I0301 06:13:05.985732 140595434813184 base_runner.py:69] task.decoder.attention.inner_atten_params.atten_dropout_prob : 0.0
I0301 06:13:05.985939 140595434813184 base_runner.py:69] task.decoder.attention.inner_atten_params.cls : type/lingvo.core.attention/AdditiveAttention
I0301 06:13:05.986126 140595434813184 base_runner.py:69] task.decoder.attention.inner_atten_params.dtype : float32
I0301 06:13:05.986283 140595434813184 base_runner.py:69] task.decoder.attention.inner_atten_params.fprop_dtype : NoneType
I0301 06:13:05.986443 140595434813184 base_runner.py:69] task.decoder.attention.inner_atten_params.hidden_dim : 0
I0301 06:13:05.986619 140595434813184 base_runner.py:69] task.decoder.attention.inner_atten_params.inference_driver_name : NoneType
I0301 06:13:05.986768 140595434813184 base_runner.py:69] task.decoder.attention.inner_atten_params.is_eval : NoneType
I0301 06:13:05.986917 140595434813184 base_runner.py:69] task.decoder.attention.inner_atten_params.is_inference : NoneType
I0301 06:13:05.987086 140595434813184 base_runner.py:69] task.decoder.attention.inner_atten_params.name : ''
I0301 06:13:05.987340 140595434813184 base_runner.py:69] task.decoder.attention.inner_atten_params.packed_input : False
I0301 06:13:05.987550 140595434813184 base_runner.py:69] task.decoder.attention.inner_atten_params.params_init.method : 'gaussian_sqrt_dim'
I0301 06:13:05.987708 140595434813184 base_runner.py:69] task.decoder.attention.inner_atten_params.params_init.scale : 1.0
I0301 06:13:05.987853 140595434813184 base_runner.py:69] task.decoder.attention.inner_atten_params.params_init.seed : NoneType
I0301 06:13:05.987986 140595434813184 base_runner.py:69] task.decoder.attention.inner_atten_params.per_step_infer : False
I0301 06:13:05.988115 140595434813184 base_runner.py:69] task.decoder.attention.inner_atten_params.qdomain.default : NoneType
I0301 06:13:05.988259 140595434813184 base_runner.py:69] task.decoder.attention.inner_atten_params.qdomain.fullyconnected : NoneType
I0301 06:13:05.988408 140595434813184 base_runner.py:69] task.decoder.attention.inner_atten_params.qdomain.softmax : NoneType
I0301 06:13:05.988537 140595434813184 base_runner.py:69] task.decoder.attention.inner_atten_params.query_dim : 0
I0301 06:13:05.988676 140595434813184 base_runner.py:69] task.decoder.attention.inner_atten_params.random_seed : NoneType
I0301 06:13:05.988832 140595434813184 base_runner.py:69] task.decoder.attention.inner_atten_params.same_batch_size : False
I0301 06:13:05.988970 140595434813184 base_runner.py:69] task.decoder.attention.inner_atten_params.source_dim : 0
I0301 06:13:05.989110 140595434813184 base_runner.py:69] task.decoder.attention.inner_atten_params.vn.global_vn : False
I0301 06:13:05.989259 140595434813184 base_runner.py:69] task.decoder.attention.inner_atten_params.vn.per_step_vn : False
I0301 06:13:05.989412 140595434813184 base_runner.py:69] task.decoder.attention.inner_atten_params.vn.scale : NoneType
I0301 06:13:05.989532 140595434813184 base_runner.py:69] task.decoder.attention.inner_atten_params.vn.seed : NoneType
I0301 06:13:05.989661 140595434813184 base_runner.py:69] task.decoder.attention.is_eval : NoneType
I0301 06:13:05.989799 140595434813184 base_runner.py:69] task.decoder.attention.is_inference : NoneType
I0301 06:13:05.989973 140595434813184 base_runner.py:69] task.decoder.attention.name : ''
I0301 06:13:05.990133 140595434813184 base_runner.py:69] task.decoder.attention.num_attention_heads : 4
I0301 06:13:05.990293 140595434813184 base_runner.py:69] task.decoder.attention.packed_input : False
I0301 06:13:05.990466 140595434813184 base_runner.py:69] task.decoder.attention.params_init.method : 'xavier'
I0301 06:13:05.990643 140595434813184 base_runner.py:69] task.decoder.attention.params_init.scale : 1.0
I0301 06:13:05.990833 140595434813184 base_runner.py:69] task.decoder.attention.params_init.seed : NoneType
I0301 06:13:05.990989 140595434813184 base_runner.py:69] task.decoder.attention.per_step_infer : False
I0301 06:13:05.991127 140595434813184 base_runner.py:69] task.decoder.attention.qdomain.atten_context : NoneType
I0301 06:13:05.991276 140595434813184 base_runner.py:69] task.decoder.attention.qdomain.default : NoneType
I0301 06:13:05.991424 140595434813184 base_runner.py:69] task.decoder.attention.qdomain.fullyconnected : NoneType
I0301 06:13:05.991580 140595434813184 base_runner.py:69] task.decoder.attention.qdomain.softmax : NoneType
I0301 06:13:05.991717 140595434813184 base_runner.py:69] task.decoder.attention.query_dim : 1024
I0301 06:13:05.991854 140595434813184 base_runner.py:69] task.decoder.attention.random_seed : NoneType
I0301 06:13:05.992013 140595434813184 base_runner.py:69] task.decoder.attention.source_dim : 1024
I0301 06:13:05.992173 140595434813184 base_runner.py:69] task.decoder.attention.use_source_vec_as_attention_value : True
I0301 06:13:05.992341 140595434813184 base_runner.py:69] task.decoder.attention.vn.global_vn : False
I0301 06:13:05.992482 140595434813184 base_runner.py:69] task.decoder.attention.vn.per_step_vn : False
I0301 06:13:05.992624 140595434813184 base_runner.py:69] task.decoder.attention.vn.scale : NoneType
I0301 06:13:05.992763 140595434813184 base_runner.py:69] task.decoder.attention.vn.seed : NoneType
I0301 06:13:05.992923 140595434813184 base_runner.py:69] task.decoder.beam_search.add_summary : True
I0301 06:13:05.993061 140595434813184 base_runner.py:69] task.decoder.beam_search.allow_empty_terminated_hyp : True
I0301 06:13:05.993191 140595434813184 base_runner.py:69] task.decoder.beam_search.allow_implicit_capture : NoneType
I0301 06:13:05.993336 140595434813184 base_runner.py:69] task.decoder.beam_search.batch_major_state : True
I0301 06:13:05.993515 140595434813184 base_runner.py:69] task.decoder.beam_search.beam_size : 3.0
I0301 06:13:05.993685 140595434813184 base_runner.py:69] task.decoder.beam_search.cls : type/lingvo.core.beam_search_helper/BeamSearchHelper
I0301 06:13:05.993844 140595434813184 base_runner.py:69] task.decoder.beam_search.coverage_penalty : 0.2
I0301 06:13:05.994066 140595434813184 base_runner.py:69] task.decoder.beam_search.dtype : float32
I0301 06:13:05.994339 140595434813184 base_runner.py:69] task.decoder.beam_search.ensure_full_beam : False
I0301 06:13:05.994563 140595434813184 base_runner.py:69] task.decoder.beam_search.force_eos_in_last_step : False
I0301 06:13:05.994770 140595434813184 base_runner.py:69] task.decoder.beam_search.fprop_dtype : NoneType
I0301 06:13:05.995048 140595434813184 base_runner.py:69] task.decoder.beam_search.inference_driver_name : NoneType
I0301 06:13:05.995246 140595434813184 base_runner.py:69] task.decoder.beam_search.is_eval : NoneType
I0301 06:13:05.995558 140595434813184 base_runner.py:69] task.decoder.beam_search.is_inference : NoneType
I0301 06:13:05.995759 140595434813184 base_runner.py:69] task.decoder.beam_search.length_normalization : 0.2
I0301 06:13:05.996085 140595434813184 base_runner.py:69] task.decoder.beam_search.merge_paths : False
I0301 06:13:05.996366 140595434813184 base_runner.py:69] task.decoder.beam_search.name : 'beam_search'
I0301 06:13:05.996561 140595434813184 base_runner.py:69] task.decoder.beam_search.num_hyps_per_beam : 16
I0301 06:13:05.996864 140595434813184 base_runner.py:69] task.decoder.beam_search.params_init.method : 'xavier'
I0301 06:13:05.997056 140595434813184 base_runner.py:69] task.decoder.beam_search.params_init.scale : 1.000001
I0301 06:13:05.997396 140595434813184 base_runner.py:69] task.decoder.beam_search.params_init.seed : NoneType
I0301 06:13:05.997633 140595434813184 base_runner.py:69] task.decoder.beam_search.per_step_infer : False
I0301 06:13:05.997844 140595434813184 base_runner.py:69] task.decoder.beam_search.random_seed : NoneType
I0301 06:13:05.998069 140595434813184 base_runner.py:69] task.decoder.beam_search.target_eoc_id : -1
I0301 06:13:05.998286 140595434813184 base_runner.py:69] task.decoder.beam_search.target_eos_id : 2
I0301 06:13:05.998491 140595434813184 base_runner.py:69] task.decoder.beam_search.target_seq_len : 0
I0301 06:13:05.998698 140595434813184 base_runner.py:69] task.decoder.beam_search.target_seq_length_ratio : 1.0
I0301 06:13:05.998915 140595434813184 base_runner.py:69] task.decoder.beam_search.target_sos_id : 1
I0301 06:13:05.999119 140595434813184 base_runner.py:69] task.decoder.beam_search.valid_eos_max_logit_delta : 5.0
I0301 06:13:05.999332 140595434813184 base_runner.py:69] task.decoder.beam_search.vn.global_vn : False
I0301 06:13:05.999538 140595434813184 base_runner.py:69] task.decoder.beam_search.vn.per_step_vn : False
I0301 06:13:05.999751 140595434813184 base_runner.py:69] task.decoder.beam_search.vn.scale : NoneType
I0301 06:13:05.999968 140595434813184 base_runner.py:69] task.decoder.beam_search.vn.seed : NoneType
I0301 06:13:06.000170 140595434813184 base_runner.py:69] task.decoder.cc_schedule : NoneType
I0301 06:13:06.000479 140595434813184 base_runner.py:69] task.decoder.cls : type/lingvo.tasks.mt.decoder/MTDecoderV1
I0301 06:13:06.000678 140595434813184 base_runner.py:69] task.decoder.dropout_prob : 0.3
I0301 06:13:06.000890 140595434813184 base_runner.py:69] task.decoder.dtype : float32
I0301 06:13:06.001094 140595434813184 base_runner.py:69] task.decoder.emb.add_summary : True
I0301 06:13:06.001315 140595434813184 base_runner.py:69] task.decoder.emb.allow_implicit_capture : NoneType
I0301 06:13:06.001518 140595434813184 base_runner.py:69] task.decoder.emb.cls : type/lingvo.core.layers/EmbeddingLayer
I0301 06:13:06.001724 140595434813184 base_runner.py:69] task.decoder.emb.dtype : float32
I0301 06:13:06.001961 140595434813184 base_runner.py:69] task.decoder.emb.embedding_dim : 1024
I0301 06:13:06.002167 140595434813184 base_runner.py:69] task.decoder.emb.fprop_dtype : NoneType
I0301 06:13:06.002429 140595434813184 base_runner.py:69] task.decoder.emb.inference_driver_name : NoneType
I0301 06:13:06.002676 140595434813184 base_runner.py:69] task.decoder.emb.is_eval : NoneType
I0301 06:13:06.002885 140595434813184 base_runner.py:69] task.decoder.emb.is_inference : NoneType
I0301 06:13:06.003093 140595434813184 base_runner.py:69] task.decoder.emb.max_num_shards : 16
I0301 06:13:06.003361 140595434813184 base_runner.py:69] task.decoder.emb.name : ''
I0301 06:13:06.003573 140595434813184 base_runner.py:69] task.decoder.emb.on_ps : True
I0301 06:13:06.003781 140595434813184 base_runner.py:69] task.decoder.emb.params_init.method : 'uniform'
I0301 06:13:06.004060 140595434813184 base_runner.py:69] task.decoder.emb.params_init.scale : 0.04
I0301 06:13:06.004319 140595434813184 base_runner.py:69] task.decoder.emb.params_init.seed : NoneType
I0301 06:13:06.004532 140595434813184 base_runner.py:69] task.decoder.emb.per_step_infer : False
I0301 06:13:06.004751 140595434813184 base_runner.py:69] task.decoder.emb.random_seed : NoneType
I0301 06:13:06.005011 140595434813184 base_runner.py:69] task.decoder.emb.scale_sqrt_depth : False
I0301 06:13:06.005215 140595434813184 base_runner.py:69] task.decoder.emb.vn.global_vn : False
I0301 06:13:06.005486 140595434813184 base_runner.py:69] task.decoder.emb.vn.per_step_vn : False
I0301 06:13:06.005729 140595434813184 base_runner.py:69] task.decoder.emb.vn.scale : 1.0
I0301 06:13:06.005955 140595434813184 base_runner.py:69] task.decoder.emb.vn.seed : NoneType
I0301 06:13:06.006166 140595434813184 base_runner.py:69] task.decoder.emb.vocab_size : 16000
I0301 06:13:06.006417 140595434813184 base_runner.py:69] task.decoder.feed_attention_context_vec_to_softmax : True
I0301 06:13:06.006663 140595434813184 base_runner.py:69] task.decoder.fprop_dtype : NoneType
I0301 06:13:06.006903 140595434813184 base_runner.py:69] task.decoder.inference_driver_name : NoneType
I0301 06:13:06.007185 140595434813184 base_runner.py:69] task.decoder.is_eval : NoneType
I0301 06:13:06.007468 140595434813184 base_runner.py:69] task.decoder.is_inference : NoneType
I0301 06:13:06.007765 140595434813184 base_runner.py:69] task.decoder.label_smoothing.add_summary : True
I0301 06:13:06.008094 140595434813184 base_runner.py:69] task.decoder.label_smoothing.allow_implicit_capture : NoneType
I0301 06:13:06.008377 140595434813184 base_runner.py:69] task.decoder.label_smoothing.cls : type/lingvo.core.layers/UniformLabelSmoother
I0301 06:13:06.008646 140595434813184 base_runner.py:69] task.decoder.label_smoothing.dtype : float32
I0301 06:13:06.008919 140595434813184 base_runner.py:69] task.decoder.label_smoothing.fprop_dtype : NoneType
I0301 06:13:06.009181 140595434813184 base_runner.py:69] task.decoder.label_smoothing.inference_driver_name : NoneType
I0301 06:13:06.009469 140595434813184 base_runner.py:69] task.decoder.label_smoothing.is_eval : NoneType
I0301 06:13:06.009670 140595434813184 base_runner.py:69] task.decoder.label_smoothing.is_inference : NoneType
I0301 06:13:06.009973 140595434813184 base_runner.py:69] task.decoder.label_smoothing.name : ''
I0301 06:13:06.010238 140595434813184 base_runner.py:69] task.decoder.label_smoothing.num_classes : 16000
I0301 06:13:06.010531 140595434813184 base_runner.py:69] task.decoder.label_smoothing.params_init.method : 'xavier'
I0301 06:13:06.010806 140595434813184 base_runner.py:69] task.decoder.label_smoothing.params_init.scale : 1.000001
I0301 06:13:06.011084 140595434813184 base_runner.py:69] task.decoder.label_smoothing.params_init.seed : NoneType
I0301 06:13:06.011363 140595434813184 base_runner.py:69] task.decoder.label_smoothing.per_step_infer : False
I0301 06:13:06.011637 140595434813184 base_runner.py:69] task.decoder.label_smoothing.random_seed : NoneType
I0301 06:13:06.011914 140595434813184 base_runner.py:69] task.decoder.label_smoothing.token_id_uncertainty_larger : NoneType
I0301 06:13:06.012197 140595434813184 base_runner.py:69] task.decoder.label_smoothing.uncertainty : 0.1
I0301 06:13:06.012475 140595434813184 base_runner.py:69] task.decoder.label_smoothing.uncertainty_larger : 0.1
I0301 06:13:06.012762 140595434813184 base_runner.py:69] task.decoder.label_smoothing.vn.global_vn : False
I0301 06:13:06.013045 140595434813184 base_runner.py:69] task.decoder.label_smoothing.vn.per_step_vn : False
I0301 06:13:06.013328 140595434813184 base_runner.py:69] task.decoder.label_smoothing.vn.scale : NoneType
I0301 06:13:06.013593 140595434813184 base_runner.py:69] task.decoder.label_smoothing.vn.seed : NoneType
I0301 06:13:06.013879 140595434813184 base_runner.py:69] task.decoder.name : ''
I0301 06:13:06.014163 140595434813184 base_runner.py:69] task.decoder.packed_input : False
I0301 06:13:06.014444 140595434813184 base_runner.py:69] task.decoder.params_init.method : 'xavier'
I0301 06:13:06.014707 140595434813184 base_runner.py:69] task.decoder.params_init.scale : 1.000001
I0301 06:13:06.014981 140595434813184 base_runner.py:69] task.decoder.params_init.seed : NoneType
I0301 06:13:06.015242 140595434813184 base_runner.py:69] task.decoder.per_step_infer : False
I0301 06:13:06.015537 140595434813184 base_runner.py:69] task.decoder.per_word_avg_loss : False
I0301 06:13:06.015794 140595434813184 base_runner.py:69] task.decoder.qdomain.default : NoneType
I0301 06:13:06.016060 140595434813184 base_runner.py:69] task.decoder.qlogsoftmax_range_min : -10.0
I0301 06:13:06.016325 140595434813184 base_runner.py:69] task.decoder.random_seed : NoneType
I0301 06:13:06.016573 140595434813184 base_runner.py:69] task.decoder.residual_start : 2
I0301 06:13:06.016835 140595434813184 base_runner.py:69] task.decoder.rnn_cell_dim : 1024
I0301 06:13:06.017076 140595434813184 base_runner.py:69] task.decoder.rnn_cell_tpl.add_summary : True
I0301 06:13:06.017303 140595434813184 base_runner.py:69] task.decoder.rnn_cell_tpl.allow_implicit_capture : NoneType
I0301 06:13:06.017559 140595434813184 base_runner.py:69] task.decoder.rnn_cell_tpl.apply_pruning : False
I0301 06:13:06.017807 140595434813184 base_runner.py:69] task.decoder.rnn_cell_tpl.bias_init.method : 'constant'
I0301 06:13:06.018094 140595434813184 base_runner.py:69] task.decoder.rnn_cell_tpl.bias_init.scale : 0.0
I0301 06:13:06.018366 140595434813184 base_runner.py:69] task.decoder.rnn_cell_tpl.bias_init.seed : 0
I0301 06:13:06.018614 140595434813184 base_runner.py:69] task.decoder.rnn_cell_tpl.cell_value_cap : 10.0
I0301 06:13:06.018876 140595434813184 base_runner.py:69] task.decoder.rnn_cell_tpl.cls : type/lingvo.core.rnn_cell/LayerNormalizedLSTMCellSimple
I0301 06:13:06.019115 140595434813184 base_runner.py:69] task.decoder.rnn_cell_tpl.couple_input_forget_gates : False
I0301 06:13:06.019380 140595434813184 base_runner.py:69] task.decoder.rnn_cell_tpl.dtype : float32
I0301 06:13:06.019639 140595434813184 base_runner.py:69] task.decoder.rnn_cell_tpl.enable_lstm_bias : True
I0301 06:13:06.019896 140595434813184 base_runner.py:69] task.decoder.rnn_cell_tpl.forget_gate_bias : 0.0
I0301 06:13:06.020142 140595434813184 base_runner.py:69] task.decoder.rnn_cell_tpl.fprop_dtype : NoneType
I0301 06:13:06.020401 140595434813184 base_runner.py:69] task.decoder.rnn_cell_tpl.inference_driver_name : NoneType
I0301 06:13:06.020647 140595434813184 base_runner.py:69] task.decoder.rnn_cell_tpl.inputs_arity : 1
I0301 06:13:06.020903 140595434813184 base_runner.py:69] task.decoder.rnn_cell_tpl.is_eval : NoneType
I0301 06:13:06.021150 140595434813184 base_runner.py:69] task.decoder.rnn_cell_tpl.is_inference : NoneType
I0301 06:13:06.021413 140595434813184 base_runner.py:69] task.decoder.rnn_cell_tpl.layer_norm_epsilon : 1e-08
I0301 06:13:06.021672 140595434813184 base_runner.py:69] task.decoder.rnn_cell_tpl.name : ''
I0301 06:13:06.021960 140595434813184 base_runner.py:69] task.decoder.rnn_cell_tpl.num_hidden_nodes : 0
I0301 06:13:06.022207 140595434813184 base_runner.py:69] task.decoder.rnn_cell_tpl.num_input_nodes : 0
I0301 06:13:06.022471 140595434813184 base_runner.py:69] task.decoder.rnn_cell_tpl.num_output_nodes : 1024
I0301 06:13:06.022720 140595434813184 base_runner.py:69] task.decoder.rnn_cell_tpl.output_nonlinearity : False
I0301 06:13:06.022974 140595434813184 base_runner.py:69] task.decoder.rnn_cell_tpl.params_init.method : 'uniform'
I0301 06:13:06.023217 140595434813184 base_runner.py:69] task.decoder.rnn_cell_tpl.params_init.scale : 0.04
I0301 06:13:06.023482 140595434813184 base_runner.py:69] task.decoder.rnn_cell_tpl.params_init.seed : NoneType
I0301 06:13:06.023727 140595434813184 base_runner.py:69] task.decoder.rnn_cell_tpl.per_step_infer : False
I0301 06:13:06.023983 140595434813184 base_runner.py:69] task.decoder.rnn_cell_tpl.qdomain.c_state : NoneType
I0301 06:13:06.024233 140595434813184 base_runner.py:69] task.decoder.rnn_cell_tpl.qdomain.default : NoneType
I0301 06:13:06.024522 140595434813184 base_runner.py:69] task.decoder.rnn_cell_tpl.qdomain.fullyconnected : NoneType
I0301 06:13:06.024786 140595434813184 base_runner.py:69] task.decoder.rnn_cell_tpl.qdomain.m_state : NoneType
I0301 06:13:06.025063 140595434813184 base_runner.py:69] task.decoder.rnn_cell_tpl.qdomain.weight : NoneType
I0301 06:13:06.025333 140595434813184 base_runner.py:69] task.decoder.rnn_cell_tpl.random_seed : NoneType
I0301 06:13:06.025604 140595434813184 base_runner.py:69] task.decoder.rnn_cell_tpl.reset_cell_state : False
I0301 06:13:06.025882 140595434813184 base_runner.py:69] task.decoder.rnn_cell_tpl.vn.global_vn : False
I0301 06:13:06.026201 140595434813184 base_runner.py:69] task.decoder.rnn_cell_tpl.vn.per_step_vn : False
I0301 06:13:06.026520 140595434813184 base_runner.py:69] task.decoder.rnn_cell_tpl.vn.scale : NoneType
I0301 06:13:06.026837 140595434813184 base_runner.py:69] task.decoder.rnn_cell_tpl.vn.seed : NoneType
I0301 06:13:06.027210 140595434813184 base_runner.py:69] task.decoder.rnn_cell_tpl.zo_prob : 0.0
I0301 06:13:06.027491 140595434813184 base_runner.py:69] task.decoder.rnn_layers : 8
I0301 06:13:06.027694 140595434813184 base_runner.py:69] task.decoder.softmax.add_summary : True
I0301 06:13:06.027916 140595434813184 base_runner.py:69] task.decoder.softmax.allow_implicit_capture : NoneType
I0301 06:13:06.028125 140595434813184 base_runner.py:69] task.decoder.softmax.apply_pruning : False
I0301 06:13:06.028367 140595434813184 base_runner.py:69] task.decoder.softmax.chunk_size : 0
I0301 06:13:06.028577 140595434813184 base_runner.py:69] task.decoder.softmax.cls : type/lingvo.core.layers/SimpleFullSoftmax
I0301 06:13:06.028788 140595434813184 base_runner.py:69] task.decoder.softmax.dtype : float32
I0301 06:13:06.029078 140595434813184 base_runner.py:69] task.decoder.softmax.fprop_dtype : NoneType
I0301 06:13:06.029464 140595434813184 base_runner.py:69] task.decoder.softmax.inference_driver_name : NoneType
I0301 06:13:06.029717 140595434813184 base_runner.py:69] task.decoder.softmax.input_dim : 0
I0301 06:13:06.030011 140595434813184 base_runner.py:69] task.decoder.softmax.is_eval : NoneType
I0301 06:13:06.030318 140595434813184 base_runner.py:69] task.decoder.softmax.is_inference : NoneType
I0301 06:13:06.030589 140595434813184 base_runner.py:69] task.decoder.softmax.logits_abs_max : NoneType
I0301 06:13:06.030896 140595434813184 base_runner.py:69] task.decoder.softmax.name : ''
I0301 06:13:06.031191 140595434813184 base_runner.py:69] task.decoder.softmax.num_classes : 16000
I0301 06:13:06.031495 140595434813184 base_runner.py:69] task.decoder.softmax.num_sampled : 0
I0301 06:13:06.031785 140595434813184 base_runner.py:69] task.decoder.softmax.num_shards : 16
I0301 06:13:06.032097 140595434813184 base_runner.py:69] task.decoder.softmax.params_init.method : 'uniform'
I0301 06:13:06.032344 140595434813184 base_runner.py:69] task.decoder.softmax.params_init.scale : 0.04
I0301 06:13:06.032699 140595434813184 base_runner.py:69] task.decoder.softmax.params_init.seed : NoneType
I0301 06:13:06.032946 140595434813184 base_runner.py:69] task.decoder.softmax.per_step_infer : False
I0301 06:13:06.033252 140595434813184 base_runner.py:69] task.decoder.softmax.qdomain.default : NoneType
I0301 06:13:06.033582 140595434813184 base_runner.py:69] task.decoder.softmax.random_seed : NoneType
I0301 06:13:06.033900 140595434813184 base_runner.py:69] task.decoder.softmax.vn.global_vn : False
I0301 06:13:06.034216 140595434813184 base_runner.py:69] task.decoder.softmax.vn.per_step_vn : False
I0301 06:13:06.034532 140595434813184 base_runner.py:69] task.decoder.softmax.vn.scale : 1.0
I0301 06:13:06.034840 140595434813184 base_runner.py:69] task.decoder.softmax.vn.seed : NoneType
I0301 06:13:06.035139 140595434813184 base_runner.py:69] task.decoder.source_dim : 1024
I0301 06:13:06.035458 140595434813184 base_runner.py:69] task.decoder.target_eos_id : 2
I0301 06:13:06.035762 140595434813184 base_runner.py:69] task.decoder.target_seq_len : 300
I0301 06:13:06.036045 140595434813184 base_runner.py:69] task.decoder.target_sequence_sampler.add_summary : True
I0301 06:13:06.036299 140595434813184 base_runner.py:69] task.decoder.target_sequence_sampler.allow_implicit_capture : NoneType
I0301 06:13:06.036547 140595434813184 base_runner.py:69] task.decoder.target_sequence_sampler.cls : type/lingvo.core.target_sequence_sampler/TargetSequenceSampler
I0301 06:13:06.036798 140595434813184 base_runner.py:69] task.decoder.target_sequence_sampler.dtype : float32
I0301 06:13:06.037056 140595434813184 base_runner.py:69] task.decoder.target_sequence_sampler.fprop_dtype : NoneType
I0301 06:13:06.037334 140595434813184 base_runner.py:69] task.decoder.target_sequence_sampler.inference_driver_name : NoneType
I0301 06:13:06.037592 140595434813184 base_runner.py:69] task.decoder.target_sequence_sampler.is_eval : NoneType
I0301 06:13:06.037859 140595434813184 base_runner.py:69] task.decoder.target_sequence_sampler.is_inference : NoneType
I0301 06:13:06.038129 140595434813184 base_runner.py:69] task.decoder.target_sequence_sampler.name : 'target_sequence_sampler'
I0301 06:13:06.038381 140595434813184 base_runner.py:69] task.decoder.target_sequence_sampler.params_init.method : 'xavier'
I0301 06:13:06.038626 140595434813184 base_runner.py:69] task.decoder.target_sequence_sampler.params_init.scale : 1.000001
I0301 06:13:06.038882 140595434813184 base_runner.py:69] task.decoder.target_sequence_sampler.params_init.seed : NoneType
I0301 06:13:06.039129 140595434813184 base_runner.py:69] task.decoder.target_sequence_sampler.per_step_infer : False
I0301 06:13:06.039388 140595434813184 base_runner.py:69] task.decoder.target_sequence_sampler.random_seed : NoneType
I0301 06:13:06.039634 140595434813184 base_runner.py:69] task.decoder.target_sequence_sampler.target_eoc_id : -1
I0301 06:13:06.039905 140595434813184 base_runner.py:69] task.decoder.target_sequence_sampler.target_eos_id : 2
I0301 06:13:06.040218 140595434813184 base_runner.py:69] task.decoder.target_sequence_sampler.target_seq_len : 0
I0301 06:13:06.040486 140595434813184 base_runner.py:69] task.decoder.target_sequence_sampler.target_sos_id : 1
I0301 06:13:06.040746 140595434813184 base_runner.py:69] task.decoder.target_sequence_sampler.temperature : 1.0
I0301 06:13:06.041019 140595434813184 base_runner.py:69] task.decoder.target_sequence_sampler.vn.global_vn : False
I0301 06:13:06.041311 140595434813184 base_runner.py:69] task.decoder.target_sequence_sampler.vn.per_step_vn : False
I0301 06:13:06.041573 140595434813184 base_runner.py:69] task.decoder.target_sequence_sampler.vn.scale : NoneType
I0301 06:13:06.041853 140595434813184 base_runner.py:69] task.decoder.target_sequence_sampler.vn.seed : NoneType
I0301 06:13:06.042138 140595434813184 base_runner.py:69] task.decoder.target_sos_id : 1
I0301 06:13:06.042417 140595434813184 base_runner.py:69] task.decoder.unidi_rnn_type : 'func'
I0301 06:13:06.042680 140595434813184 base_runner.py:69] task.decoder.use_prev_atten_ctx : False
I0301 06:13:06.042952 140595434813184 base_runner.py:69] task.decoder.use_zero_atten_state : False
I0301 06:13:06.043215 140595434813184 base_runner.py:69] task.decoder.vn.global_vn : False
I0301 06:13:06.043494 140595434813184 base_runner.py:69] task.decoder.vn.per_step_vn : False
I0301 06:13:06.043756 140595434813184 base_runner.py:69] task.decoder.vn.scale : NoneType
I0301 06:13:06.044033 140595434813184 base_runner.py:69] task.decoder.vn.seed : NoneType
I0301 06:13:06.044311 140595434813184 base_runner.py:69] task.dtype : float32
I0301 06:13:06.044575 140595434813184 base_runner.py:69] task.encoder.add_summary : True
I0301 06:13:06.044848 140595434813184 base_runner.py:69] task.encoder.allow_implicit_capture : NoneType
I0301 06:13:06.045119 140595434813184 base_runner.py:69] task.encoder.bidi_rnn_type : 'func'
I0301 06:13:06.045402 140595434813184 base_runner.py:69] task.encoder.cc_schedule : NoneType
I0301 06:13:06.045665 140595434813184 base_runner.py:69] task.encoder.cls : type/lingvo.tasks.mt.encoder/MTEncoderBiRNN
I0301 06:13:06.045972 140595434813184 base_runner.py:69] task.encoder.dropout_prob : 0.3
I0301 06:13:06.046240 140595434813184 base_runner.py:69] task.encoder.dtype : float32
I0301 06:13:06.046514 140595434813184 base_runner.py:69] task.encoder.emb.add_summary : True
I0301 06:13:06.046792 140595434813184 base_runner.py:69] task.encoder.emb.allow_implicit_capture : NoneType
I0301 06:13:06.047075 140595434813184 base_runner.py:69] task.encoder.emb.cls : type/lingvo.core.layers/EmbeddingLayer
I0301 06:13:06.047352 140595434813184 base_runner.py:69] task.encoder.emb.dtype : float32
I0301 06:13:06.047619 140595434813184 base_runner.py:69] task.encoder.emb.embedding_dim : 1024
I0301 06:13:06.047895 140595434813184 base_runner.py:69] task.encoder.emb.fprop_dtype : NoneType
I0301 06:13:06.048156 140595434813184 base_runner.py:69] task.encoder.emb.inference_driver_name : NoneType
I0301 06:13:06.048441 140595434813184 base_runner.py:69] task.encoder.emb.is_eval : NoneType
I0301 06:13:06.048707 140595434813184 base_runner.py:69] task.encoder.emb.is_inference : NoneType
I0301 06:13:06.048975 140595434813184 base_runner.py:69] task.encoder.emb.max_num_shards : 16
I0301 06:13:06.049236 140595434813184 base_runner.py:69] task.encoder.emb.name : ''
I0301 06:13:06.049518 140595434813184 base_runner.py:69] task.encoder.emb.on_ps : True
I0301 06:13:06.049781 140595434813184 base_runner.py:69] task.encoder.emb.params_init.method : 'uniform'
I0301 06:13:06.050079 140595434813184 base_runner.py:69] task.encoder.emb.params_init.scale : 0.04
I0301 06:13:06.050373 140595434813184 base_runner.py:69] task.encoder.emb.params_init.seed : NoneType
I0301 06:13:06.050739 140595434813184 base_runner.py:69] task.encoder.emb.per_step_infer : False
I0301 06:13:06.051057 140595434813184 base_runner.py:69] task.encoder.emb.random_seed : NoneType
I0301 06:13:06.051330 140595434813184 base_runner.py:69] task.encoder.emb.scale_sqrt_depth : False
I0301 06:13:06.051603 140595434813184 base_runner.py:69] task.encoder.emb.vn.global_vn : False
I0301 06:13:06.051978 140595434813184 base_runner.py:69] task.encoder.emb.vn.per_step_vn : False
I0301 06:13:06.052407 140595434813184 base_runner.py:69] task.encoder.emb.vn.scale : 1.0
I0301 06:13:06.052680 140595434813184 base_runner.py:69] task.encoder.emb.vn.seed : NoneType
I0301 06:13:06.052994 140595434813184 base_runner.py:69] task.encoder.emb.vocab_size : 16000
I0301 06:13:06.053299 140595434813184 base_runner.py:69] task.encoder.encoder_out_dim : 1024
I0301 06:13:06.053565 140595434813184 base_runner.py:69] task.encoder.fprop_dtype : NoneType
I0301 06:13:06.053833 140595434813184 base_runner.py:69] task.encoder.inference_driver_name : NoneType
I0301 06:13:06.054156 140595434813184 base_runner.py:69] task.encoder.is_eval : NoneType
I0301 06:13:06.054385 140595434813184 base_runner.py:69] task.encoder.is_inference : NoneType
I0301 06:13:06.054758 140595434813184 base_runner.py:69] task.encoder.is_transparent : False
I0301 06:13:06.055054 140595434813184 base_runner.py:69] task.encoder.lstm_cell_size : 1024
I0301 06:13:06.055299 140595434813184 base_runner.py:69] task.encoder.lstm_tpl.add_summary : True
I0301 06:13:06.055592 140595434813184 base_runner.py:69] task.encoder.lstm_tpl.allow_implicit_capture : NoneType
I0301 06:13:06.055991 140595434813184 base_runner.py:69] task.encoder.lstm_tpl.apply_pruning : False
I0301 06:13:06.056504 140595434813184 base_runner.py:69] task.encoder.lstm_tpl.bias_init.method : 'constant'
I0301 06:13:06.056950 140595434813184 base_runner.py:69] task.encoder.lstm_tpl.bias_init.scale : 0.0
I0301 06:13:06.057245 140595434813184 base_runner.py:69] task.encoder.lstm_tpl.bias_init.seed : 0
I0301 06:13:06.057504 140595434813184 base_runner.py:69] task.encoder.lstm_tpl.cell_value_cap : 10.0
I0301 06:13:06.057718 140595434813184 base_runner.py:69] task.encoder.lstm_tpl.cls : type/lingvo.core.rnn_cell/LayerNormalizedLSTMCellSimple
I0301 06:13:06.058048 140595434813184 base_runner.py:69] task.encoder.lstm_tpl.couple_input_forget_gates : False
I0301 06:13:06.058283 140595434813184 base_runner.py:69] task.encoder.lstm_tpl.dtype : float32
I0301 06:13:06.058530 140595434813184 base_runner.py:69] task.encoder.lstm_tpl.enable_lstm_bias : True
I0301 06:13:06.058823 140595434813184 base_runner.py:69] task.encoder.lstm_tpl.forget_gate_bias : 0.0
I0301 06:13:06.059089 140595434813184 base_runner.py:69] task.encoder.lstm_tpl.fprop_dtype : NoneType
I0301 06:13:06.059367 140595434813184 base_runner.py:69] task.encoder.lstm_tpl.inference_driver_name : NoneType
I0301 06:13:06.059638 140595434813184 base_runner.py:69] task.encoder.lstm_tpl.inputs_arity : 1
I0301 06:13:06.059912 140595434813184 base_runner.py:69] task.encoder.lstm_tpl.is_eval : NoneType
I0301 06:13:06.060184 140595434813184 base_runner.py:69] task.encoder.lstm_tpl.is_inference : NoneType
I0301 06:13:06.060472 140595434813184 base_runner.py:69] task.encoder.lstm_tpl.layer_norm_epsilon : 1e-08
I0301 06:13:06.060749 140595434813184 base_runner.py:69] task.encoder.lstm_tpl.name : ''
I0301 06:13:06.061022 140595434813184 base_runner.py:69] task.encoder.lstm_tpl.num_hidden_nodes : 0
I0301 06:13:06.061309 140595434813184 base_runner.py:69] task.encoder.lstm_tpl.num_input_nodes : 0
I0301 06:13:06.061570 140595434813184 base_runner.py:69] task.encoder.lstm_tpl.num_output_nodes : 1024
I0301 06:13:06.061851 140595434813184 base_runner.py:69] task.encoder.lstm_tpl.output_nonlinearity : False
I0301 06:13:06.062148 140595434813184 base_runner.py:69] task.encoder.lstm_tpl.params_init.method : 'uniform'
I0301 06:13:06.062427 140595434813184 base_runner.py:69] task.encoder.lstm_tpl.params_init.scale : 0.04
I0301 06:13:06.062711 140595434813184 base_runner.py:69] task.encoder.lstm_tpl.params_init.seed : NoneType
I0301 06:13:06.063002 140595434813184 base_runner.py:69] task.encoder.lstm_tpl.per_step_infer : False
I0301 06:13:06.063255 140595434813184 base_runner.py:69] task.encoder.lstm_tpl.qdomain.c_state : NoneType
I0301 06:13:06.063538 140595434813184 base_runner.py:69] task.encoder.lstm_tpl.qdomain.default : NoneType
I0301 06:13:06.063787 140595434813184 base_runner.py:69] task.encoder.lstm_tpl.qdomain.fullyconnected : NoneType
I0301 06:13:06.064065 140595434813184 base_runner.py:69] task.encoder.lstm_tpl.qdomain.m_state : NoneType
I0301 06:13:06.064347 140595434813184 base_runner.py:69] task.encoder.lstm_tpl.qdomain.weight : NoneType
I0301 06:13:06.064606 140595434813184 base_runner.py:69] task.encoder.lstm_tpl.random_seed : NoneType
I0301 06:13:06.064872 140595434813184 base_runner.py:69] task.encoder.lstm_tpl.reset_cell_state : False
I0301 06:13:06.065131 140595434813184 base_runner.py:69] task.encoder.lstm_tpl.vn.global_vn : False
I0301 06:13:06.065402 140595434813184 base_runner.py:69] task.encoder.lstm_tpl.vn.per_step_vn : False
I0301 06:13:06.065679 140595434813184 base_runner.py:69] task.encoder.lstm_tpl.vn.scale : NoneType
I0301 06:13:06.065959 140595434813184 base_runner.py:69] task.encoder.lstm_tpl.vn.seed : NoneType
I0301 06:13:06.066221 140595434813184 base_runner.py:69] task.encoder.lstm_tpl.zo_prob : 0.0
I0301 06:13:06.066498 140595434813184 base_runner.py:69] task.encoder.name : ''
I0301 06:13:06.066700 140595434813184 base_runner.py:69] task.encoder.num_lstm_layers : 6
I0301 06:13:06.066917 140595434813184 base_runner.py:69] task.encoder.packed_input : False
I0301 06:13:06.067132 140595434813184 base_runner.py:69] task.encoder.params_init.method : 'xavier'
I0301 06:13:06.067357 140595434813184 base_runner.py:69] task.encoder.params_init.scale : 1.000001
I0301 06:13:06.067565 140595434813184 base_runner.py:69] task.encoder.params_init.seed : NoneType
I0301 06:13:06.067774 140595434813184 base_runner.py:69] task.encoder.per_step_infer : False
I0301 06:13:06.067996 140595434813184 base_runner.py:69] task.encoder.proj_tpl.activation : 'RELU'
I0301 06:13:06.068193 140595434813184 base_runner.py:69] task.encoder.proj_tpl.add_summary : True
I0301 06:13:06.068406 140595434813184 base_runner.py:69] task.encoder.proj_tpl.affine_last : False
I0301 06:13:06.068604 140595434813184 base_runner.py:69] task.encoder.proj_tpl.allow_implicit_capture : NoneType
I0301 06:13:06.068800 140595434813184 base_runner.py:69] task.encoder.proj_tpl.batch_norm : True
I0301 06:13:06.069009 140595434813184 base_runner.py:69] task.encoder.proj_tpl.bias_init : 0.0
I0301 06:13:06.069205 140595434813184 base_runner.py:69] task.encoder.proj_tpl.bn_fold_weights : NoneType
I0301 06:13:06.069416 140595434813184 base_runner.py:69] task.encoder.proj_tpl.cls : type/lingvo.core.layers/ProjectionLayer
I0301 06:13:06.069616 140595434813184 base_runner.py:69] task.encoder.proj_tpl.dtype : float32
I0301 06:13:06.069824 140595434813184 base_runner.py:69] task.encoder.proj_tpl.fprop_dtype : NoneType
I0301 06:13:06.070044 140595434813184 base_runner.py:69] task.encoder.proj_tpl.has_bias : False
I0301 06:13:06.070250 140595434813184 base_runner.py:69] task.encoder.proj_tpl.inference_driver_name : NoneType
I0301 06:13:06.070465 140595434813184 base_runner.py:69] task.encoder.proj_tpl.input_dim : 0
I0301 06:13:06.070664 140595434813184 base_runner.py:69] task.encoder.proj_tpl.is_eval : NoneType
I0301 06:13:06.070878 140595434813184 base_runner.py:69] task.encoder.proj_tpl.is_inference : NoneType
I0301 06:13:06.071083 140595434813184 base_runner.py:69] task.encoder.proj_tpl.name : ''
I0301 06:13:06.071291 140595434813184 base_runner.py:69] task.encoder.proj_tpl.output_dim : 0
I0301 06:13:06.071491 140595434813184 base_runner.py:69] task.encoder.proj_tpl.params_init.method : 'xavier'
I0301 06:13:06.071688 140595434813184 base_runner.py:69] task.encoder.proj_tpl.params_init.scale : 1.000001
I0301 06:13:06.071893 140595434813184 base_runner.py:69] task.encoder.proj_tpl.params_init.seed : NoneType
I0301 06:13:06.072089 140595434813184 base_runner.py:69] task.encoder.proj_tpl.per_step_infer : False
I0301 06:13:06.072299 140595434813184 base_runner.py:69] task.encoder.proj_tpl.qdomain.default : NoneType
I0301 06:13:06.072498 140595434813184 base_runner.py:69] task.encoder.proj_tpl.random_seed : NoneType
I0301 06:13:06.072694 140595434813184 base_runner.py:69] task.encoder.proj_tpl.vn.global_vn : False
I0301 06:13:06.072901 140595434813184 base_runner.py:69] task.encoder.proj_tpl.vn.per_step_vn : False
I0301 06:13:06.073096 140595434813184 base_runner.py:69] task.encoder.proj_tpl.vn.scale : NoneType
I0301 06:13:06.073309 140595434813184 base_runner.py:69] task.encoder.proj_tpl.vn.seed : NoneType
I0301 06:13:06.073508 140595434813184 base_runner.py:69] task.encoder.proj_tpl.weight_norm : False
I0301 06:13:06.073704 140595434813184 base_runner.py:69] task.encoder.random_seed : NoneType
I0301 06:13:06.073913 140595434813184 base_runner.py:69] task.encoder.residual_start : 2
I0301 06:13:06.074131 140595434813184 base_runner.py:69] task.encoder.transparent_merger_tpl.add_summary : True
I0301 06:13:06.074348 140595434813184 base_runner.py:69] task.encoder.transparent_merger_tpl.add_weight_summaries : True
I0301 06:13:06.074549 140595434813184 base_runner.py:69] task.encoder.transparent_merger_tpl.allow_implicit_capture : NoneType
I0301 06:13:06.074748 140595434813184 base_runner.py:69] task.encoder.transparent_merger_tpl.cls : type/lingvo.core.layers/WeightedSumLayer
I0301 06:13:06.074959 140595434813184 base_runner.py:69] task.encoder.transparent_merger_tpl.dtype : float32
I0301 06:13:06.075156 140595434813184 base_runner.py:69] task.encoder.transparent_merger_tpl.fprop_dtype : NoneType
I0301 06:13:06.075360 140595434813184 base_runner.py:69] task.encoder.transparent_merger_tpl.global_weight_scale : 1.0
I0301 06:13:06.075557 140595434813184 base_runner.py:69] task.encoder.transparent_merger_tpl.inference_driver_name : NoneType
I0301 06:13:06.075758 140595434813184 base_runner.py:69] task.encoder.transparent_merger_tpl.is_eval : NoneType
I0301 06:13:06.075963 140595434813184 base_runner.py:69] task.encoder.transparent_merger_tpl.is_inference : NoneType
I0301 06:13:06.076162 140595434813184 base_runner.py:69] task.encoder.transparent_merger_tpl.minimal_prob : 0.0
I0301 06:13:06.076371 140595434813184 base_runner.py:69] task.encoder.transparent_merger_tpl.name : ''
I0301 06:13:06.076567 140595434813184 base_runner.py:69] task.encoder.transparent_merger_tpl.num_sources : 0
I0301 06:13:06.076764 140595434813184 base_runner.py:69] task.encoder.transparent_merger_tpl.params_init.method : 'xavier'
I0301 06:13:06.076968 140595434813184 base_runner.py:69] task.encoder.transparent_merger_tpl.params_init.scale : 1.000001
I0301 06:13:06.077162 140595434813184 base_runner.py:69] task.encoder.transparent_merger_tpl.params_init.seed : NoneType
I0301 06:13:06.077377 140595434813184 base_runner.py:69] task.encoder.transparent_merger_tpl.per_step_infer : False
I0301 06:13:06.077574 140595434813184 base_runner.py:69] task.encoder.transparent_merger_tpl.random_seed : NoneType
I0301 06:13:06.077770 140595434813184 base_runner.py:69] task.encoder.transparent_merger_tpl.vn.global_vn : False
I0301 06:13:06.078001 140595434813184 base_runner.py:69] task.encoder.transparent_merger_tpl.vn.per_step_vn : False
I0301 06:13:06.078196 140595434813184 base_runner.py:69] task.encoder.transparent_merger_tpl.vn.scale : NoneType
I0301 06:13:06.078418 140595434813184 base_runner.py:69] task.encoder.transparent_merger_tpl.vn.seed : NoneType
I0301 06:13:06.078618 140595434813184 base_runner.py:69] task.encoder.transparent_merger_tpl.weighted_merger_dropout_prob : 0.1
I0301 06:13:06.078824 140595434813184 base_runner.py:69] task.encoder.transparent_merger_tpl.weighted_merger_softmax : True
I0301 06:13:06.079022 140595434813184 base_runner.py:69] task.encoder.vn.global_vn : False
I0301 06:13:06.079220 140595434813184 base_runner.py:69] task.encoder.vn.per_step_vn : False
I0301 06:13:06.079436 140595434813184 base_runner.py:69] task.encoder.vn.scale : NoneType
I0301 06:13:06.079627 140595434813184 base_runner.py:69] task.encoder.vn.seed : NoneType
I0301 06:13:06.079830 140595434813184 base_runner.py:69] task.eval.decoder_samples_per_summary : 0
I0301 06:13:06.080029 140595434813184 base_runner.py:69] task.eval.samples_per_summary : 2466
I0301 06:13:06.080226 140595434813184 base_runner.py:69] task.fprop_dtype : NoneType
I0301 06:13:06.080442 140595434813184 base_runner.py:69] task.inference_driver_name : NoneType
I0301 06:13:06.080641 140595434813184 base_runner.py:69] task.input : NoneType
I0301 06:13:06.080848 140595434813184 base_runner.py:69] task.is_eval : NoneType
I0301 06:13:06.081046 140595434813184 base_runner.py:69] task.is_inference : NoneType
I0301 06:13:06.081243 140595434813184 base_runner.py:69] task.name : 'punctuator_rnmt'
I0301 06:13:06.081455 140595434813184 base_runner.py:69] task.online_encoder : NoneType
I0301 06:13:06.081653 140595434813184 base_runner.py:69] task.params_init.method : 'xavier'
I0301 06:13:06.081860 140595434813184 base_runner.py:69] task.params_init.scale : 1.000001
I0301 06:13:06.082076 140595434813184 base_runner.py:69] task.params_init.seed : NoneType
I0301 06:13:06.082285 140595434813184 base_runner.py:69] task.per_step_infer : False
I0301 06:13:06.082484 140595434813184 base_runner.py:69] task.random_seed : NoneType
I0301 06:13:06.082681 140595434813184 base_runner.py:69] task.train.bprop_variable_filter : NoneType
I0301 06:13:06.082885 140595434813184 base_runner.py:69] task.train.clip_gradient_norm_to_value : 0.0
I0301 06:13:06.083086 140595434813184 base_runner.py:69] task.train.early_stop.metric_history.jobname : 'eval_dev'
I0301 06:13:06.083312 140595434813184 base_runner.py:69] task.train.early_stop.metric_history.local_filesystem : False
I0301 06:13:06.083513 140595434813184 base_runner.py:69] task.train.early_stop.metric_history.logdir : ''
I0301 06:13:06.083712 140595434813184 base_runner.py:69] task.train.early_stop.metric_history.metric : 'log_pplx'
I0301 06:13:06.083919 140595434813184 base_runner.py:69] task.train.early_stop.metric_history.minimize : True
I0301 06:13:06.084116 140595434813184 base_runner.py:69] task.train.early_stop.metric_history.name : 'MetricHistory'
I0301 06:13:06.084326 140595434813184 base_runner.py:69] task.train.early_stop.metric_history.tfevent_file : False
I0301 06:13:06.084525 140595434813184 base_runner.py:69] task.train.early_stop.name : 'EarlyStop'
I0301 06:13:06.084732 140595434813184 base_runner.py:69] task.train.early_stop.tolerance : 0.0
I0301 06:13:06.084939 140595434813184 base_runner.py:69] task.train.early_stop.verbose : True
I0301 06:13:06.085135 140595434813184 base_runner.py:69] task.train.early_stop.window : 0
I0301 06:13:06.085340 140595434813184 base_runner.py:69] task.train.ema_decay : 0.0
I0301 06:13:06.085539 140595434813184 base_runner.py:69] task.train.grad_norm_to_clip_to_zero : 100000.0
I0301 06:13:06.085736 140595434813184 base_runner.py:69] task.train.grad_norm_tracker.add_summary : True
I0301 06:13:06.085964 140595434813184 base_runner.py:69] task.train.grad_norm_tracker.allow_implicit_capture : NoneType
I0301 06:13:06.086172 140595434813184 base_runner.py:69] task.train.grad_norm_tracker.clip_threshold : 4.0
I0301 06:13:06.086364 140595434813184 base_runner.py:69] task.train.grad_norm_tracker.cls : type/lingvo.core.layers/GradNormTracker
I0301 06:13:06.086556 140595434813184 base_runner.py:69] task.train.grad_norm_tracker.decay : 0.995
I0301 06:13:06.086780 140595434813184 base_runner.py:69] task.train.grad_norm_tracker.dtype : float32
I0301 06:13:06.087018 140595434813184 base_runner.py:69] task.train.grad_norm_tracker.fprop_dtype : NoneType
I0301 06:13:06.087220 140595434813184 base_runner.py:69] task.train.grad_norm_tracker.grad_norm_clip_cap_min : 0.0
I0301 06:13:06.087445 140595434813184 base_runner.py:69] task.train.grad_norm_tracker.grad_norm_lower_cap : 0.01
I0301 06:13:06.087650 140595434813184 base_runner.py:69] task.train.grad_norm_tracker.inference_driver_name : NoneType
I0301 06:13:06.087825 140595434813184 base_runner.py:69] task.train.grad_norm_tracker.is_eval : NoneType
I0301 06:13:06.088021 140595434813184 base_runner.py:69] task.train.grad_norm_tracker.is_inference : NoneType
I0301 06:13:06.088223 140595434813184 base_runner.py:69] task.train.grad_norm_tracker.name : 'gradient_norm_tracker'
I0301 06:13:06.088437 140595434813184 base_runner.py:69] task.train.grad_norm_tracker.params_init.method : 'xavier'
I0301 06:13:06.088645 140595434813184 base_runner.py:69] task.train.grad_norm_tracker.params_init.scale : 1.000001
I0301 06:13:06.088852 140595434813184 base_runner.py:69] task.train.grad_norm_tracker.params_init.seed : NoneType
I0301 06:13:06.089059 140595434813184 base_runner.py:69] task.train.grad_norm_tracker.per_step_infer : False
I0301 06:13:06.089258 140595434813184 base_runner.py:69] task.train.grad_norm_tracker.random_seed : NoneType
I0301 06:13:06.089471 140595434813184 base_runner.py:69] task.train.grad_norm_tracker.vn.global_vn : False
I0301 06:13:06.089670 140595434813184 base_runner.py:69] task.train.grad_norm_tracker.vn.per_step_vn : False
I0301 06:13:06.089885 140595434813184 base_runner.py:69] task.train.grad_norm_tracker.vn.scale : NoneType
I0301 06:13:06.090115 140595434813184 base_runner.py:69] task.train.grad_norm_tracker.vn.seed : NoneType
I0301 06:13:06.090337 140595434813184 base_runner.py:69] task.train.init_from_checkpoint_rules : {}
I0301 06:13:06.090559 140595434813184 base_runner.py:69] task.train.l1_regularizer_weight : NoneType
I0301 06:13:06.090765 140595434813184 base_runner.py:69] task.train.l2_regularizer_weight : 1e-05
I0301 06:13:06.090977 140595434813184 base_runner.py:69] task.train.learning_rate : 0.0001
I0301 06:13:06.091181 140595434813184 base_runner.py:69] task.train.lr_schedule.add_summary : True
I0301 06:13:06.091398 140595434813184 base_runner.py:69] task.train.lr_schedule.allow_implicit_capture : NoneType
I0301 06:13:06.091609 140595434813184 base_runner.py:69] task.train.lr_schedule.cls : type/lingvo.core.lr_schedule/LinearRampupExponentialDecayScaledByNumSplitSchedule
I0301 06:13:06.091825 140595434813184 base_runner.py:69] task.train.lr_schedule.decay_end : 1200000
I0301 06:13:06.092029 140595434813184 base_runner.py:69] task.train.lr_schedule.decay_start : 400000
I0301 06:13:06.092231 140595434813184 base_runner.py:69] task.train.lr_schedule.dtype : float32
I0301 06:13:06.092453 140595434813184 base_runner.py:69] task.train.lr_schedule.fprop_dtype : NoneType
I0301 06:13:06.092662 140595434813184 base_runner.py:69] task.train.lr_schedule.inference_driver_name : NoneType
I0301 06:13:06.092873 140595434813184 base_runner.py:69] task.train.lr_schedule.is_eval : NoneType
I0301 06:13:06.093070 140595434813184 base_runner.py:69] task.train.lr_schedule.is_inference : NoneType
I0301 06:13:06.093288 140595434813184 base_runner.py:69] task.train.lr_schedule.max : 100000000.0
I0301 06:13:06.093497 140595434813184 base_runner.py:69] task.train.lr_schedule.min : 0.5
I0301 06:13:06.093708 140595434813184 base_runner.py:69] task.train.lr_schedule.name : 'LRSched'
I0301 06:13:06.093915 140595434813184 base_runner.py:69] task.train.lr_schedule.num_splits : 0
I0301 06:13:06.094137 140595434813184 base_runner.py:69] task.train.lr_schedule.params_init.method : 'xavier'
I0301 06:13:06.094353 140595434813184 base_runner.py:69] task.train.lr_schedule.params_init.scale : 1.000001
I0301 06:13:06.094563 140595434813184 base_runner.py:69] task.train.lr_schedule.params_init.seed : NoneType
I0301 06:13:06.094773 140595434813184 base_runner.py:69] task.train.lr_schedule.per_step_infer : False
I0301 06:13:06.094990 140595434813184 base_runner.py:69] task.train.lr_schedule.random_seed : NoneType
I0301 06:13:06.095206 140595434813184 base_runner.py:69] task.train.lr_schedule.vn.global_vn : False
I0301 06:13:06.095423 140595434813184 base_runner.py:69] task.train.lr_schedule.vn.per_step_vn : False
I0301 06:13:06.095630 140595434813184 base_runner.py:69] task.train.lr_schedule.vn.scale : NoneType
I0301 06:13:06.095849 140595434813184 base_runner.py:69] task.train.lr_schedule.vn.seed : NoneType
I0301 06:13:06.096048 140595434813184 base_runner.py:69] task.train.lr_schedule.warmup : 500
I0301 06:13:06.096251 140595434813184 base_runner.py:69] task.train.lr_schedule.warmup_init : 1.0
I0301 06:13:06.096473 140595434813184 base_runner.py:69] task.train.max_steps : 4000000
I0301 06:13:06.096681 140595434813184 base_runner.py:69] task.train.optimizer.add_summary : True
I0301 06:13:06.096899 140595434813184 base_runner.py:69] task.train.optimizer.allow_implicit_capture : NoneType
I0301 06:13:06.097095 140595434813184 base_runner.py:69] task.train.optimizer.beta1 : 0.9
I0301 06:13:06.097313 140595434813184 base_runner.py:69] task.train.optimizer.beta2 : 0.98
I0301 06:13:06.097521 140595434813184 base_runner.py:69] task.train.optimizer.cls : type/lingvo.core.optimizer/Adam
I0301 06:13:06.097733 140595434813184 base_runner.py:69] task.train.optimizer.dtype : float32
I0301 06:13:06.097961 140595434813184 base_runner.py:69] task.train.optimizer.epsilon : 1e-06
I0301 06:13:06.098159 140595434813184 base_runner.py:69] task.train.optimizer.fprop_dtype : NoneType
I0301 06:13:06.098377 140595434813184 base_runner.py:69] task.train.optimizer.inference_driver_name : NoneType
I0301 06:13:06.098582 140595434813184 base_runner.py:69] task.train.optimizer.is_eval : NoneType
I0301 06:13:06.098788 140595434813184 base_runner.py:69] task.train.optimizer.is_inference : NoneType
I0301 06:13:06.099014 140595434813184 base_runner.py:69] task.train.optimizer.name : 'Adam'
I0301 06:13:06.099212 140595434813184 base_runner.py:69] task.train.optimizer.params_init.method : 'xavier'
I0301 06:13:06.099430 140595434813184 base_runner.py:69] task.train.optimizer.params_init.scale : 1.000001
I0301 06:13:06.099636 140595434813184 base_runner.py:69] task.train.optimizer.params_init.seed : NoneType
I0301 06:13:06.099850 140595434813184 base_runner.py:69] task.train.optimizer.per_step_infer : False
I0301 06:13:06.100053 140595434813184 base_runner.py:69] task.train.optimizer.random_seed : NoneType
I0301 06:13:06.100255 140595434813184 base_runner.py:69] task.train.optimizer.vn.global_vn : False
I0301 06:13:06.100466 140595434813184 base_runner.py:69] task.train.optimizer.vn.per_step_vn : False
I0301 06:13:06.100716 140595434813184 base_runner.py:69] task.train.optimizer.vn.scale : NoneType
I0301 06:13:06.100924 140595434813184 base_runner.py:69] task.train.optimizer.vn.seed : NoneType
I0301 06:13:06.101128 140595434813184 base_runner.py:69] task.train.pruning_hparams_dict : NoneType
I0301 06:13:06.101344 140595434813184 base_runner.py:69] task.train.save_interval_seconds : 600
I0301 06:13:06.101552 140595434813184 base_runner.py:69] task.train.start_up_delay_steps : 200
I0301 06:13:06.101758 140595434813184 base_runner.py:69] task.train.summary_interval_steps : 100
I0301 06:13:06.101986 140595434813184 base_runner.py:69] task.train.task_global_step : False
I0301 06:13:06.102196 140595434813184 base_runner.py:69] task.train.tpu_steps_per_loop : 100
I0301 06:13:06.102418 140595434813184 base_runner.py:69] task.train.vn_start_step : 200000000
I0301 06:13:06.102627 140595434813184 base_runner.py:69] task.train.vn_std : 0.0
I0301 06:13:06.102843 140595434813184 base_runner.py:69] task.vn.global_vn : False
I0301 06:13:06.103044 140595434813184 base_runner.py:69] task.vn.per_step_vn : False
I0301 06:13:06.103261 140595434813184 base_runner.py:69] task.vn.scale : NoneType
I0301 06:13:06.103481 140595434813184 base_runner.py:69] task.vn.seed : NoneType
I0301 06:13:06.103688 140595434813184 base_runner.py:69] train.early_stop.metric_history.jobname : 'eval_dev'
I0301 06:13:06.103905 140595434813184 base_runner.py:69] train.early_stop.metric_history.local_filesystem : False
I0301 06:13:06.104104 140595434813184 base_runner.py:69] train.early_stop.metric_history.logdir : ''
I0301 06:13:06.104334 140595434813184 base_runner.py:69] train.early_stop.metric_history.metric : 'log_pplx'
I0301 06:13:06.104542 140595434813184 base_runner.py:69] train.early_stop.metric_history.minimize : True
I0301 06:13:06.104749 140595434813184 base_runner.py:69] train.early_stop.metric_history.name : 'MetricHistory'
I0301 06:13:06.104959 140595434813184 base_runner.py:69] train.early_stop.metric_history.tfevent_file : False
I0301 06:13:06.105161 140595434813184 base_runner.py:69] train.early_stop.name : 'EarlyStop'
I0301 06:13:06.105398 140595434813184 base_runner.py:69] train.early_stop.tolerance : 0.0
I0301 06:13:06.105612 140595434813184 base_runner.py:69] train.early_stop.verbose : True
I0301 06:13:06.105827 140595434813184 base_runner.py:69] train.early_stop.window : 0
I0301 06:13:06.106065 140595434813184 base_runner.py:69] train.ema_decay : 0.0
I0301 06:13:06.106285 140595434813184 base_runner.py:69] train.init_from_checkpoint_rules : {}
I0301 06:13:06.106497 140595434813184 base_runner.py:69] train.max_steps : 4000000
I0301 06:13:06.106708 140595434813184 base_runner.py:69] train.save_interval_seconds : 600
I0301 06:13:06.106936 140595434813184 base_runner.py:69] train.start_up_delay_steps : 200
I0301 06:13:06.107146 140595434813184 base_runner.py:69] train.summary_interval_steps : 100
I0301 06:13:06.107369 140595434813184 base_runner.py:69] train.tpu_steps_per_loop : 100
I0301 06:13:06.107582 140595434813184 base_runner.py:69] vn.global_vn : False
I0301 06:13:06.107793 140595434813184 base_runner.py:69] vn.per_step_vn : False
I0301 06:13:06.108017 140595434813184 base_runner.py:69] vn.scale : NoneType
I0301 06:13:06.108227 140595434813184 base_runner.py:69] vn.seed : NoneType
I0301 06:13:06.108457 140595434813184 base_runner.py:69]
I0301 06:13:06.108716 140595434813184 base_runner.py:70] ============================================================
I0301 06:13:06.116491 140595434813184 base_runner.py:115] Starting ...
I0301 06:13:06.182372 140595434813184 cluster.py:419] _LeastLoadedPlacer : ['/job:local/replica:0/task:0/device:CPU:0']
I0301 06:13:06.234208 140595434813184 cluster.py:437] Place variable global_step on /job:local/replica:0/task:0/device:CPU:0 8
I0301 06:13:06.239692 140595434813184 base_model.py:1054] Training parameters for <class 'lingvo.core.base_model.SingleTaskModel'>: {
early_stop: {
metric_history: {
"eval_dev"
local_filesystem: False
"/tmp/punctuator"
"log_pplx"
minimize: True
"MetricHistory"
tfevent_file: False
}
"EarlyStop"
tolerance: 0.0
verbose: True
window: 0
}
ema_decay: 0.0
init_from_checkpoint_rules: {}
max_steps: 4000000
save_interval_seconds: 600
start_up_delay_steps: 200
summary_interval_steps: 100
tpu_steps_per_loop: 100
}
I0301 06:13:06.391488 140595434813184 base_input_generator.py:475] bucket_batch_limit [512, 256, 160, 80, 40]
IndexError Traceback (most recent call last)
/root/.cache/bazel/_bazel_root/5b16c1cfa7fded7181f7e172eb7ad8a9/execroot/__main__/bazel-out/k8-opt/bin/lingvo/ipython_kernel.runfiles/__main__/test.py in <module>()
55 "--run_locally=cpu", # or cpu.
56 ]
---> 57 tf.app.run(trainer.main, argv=argv)
58
59
/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/app.pyc in run(main, argv)
38 main = main or _sys.modules['__main__'].main
39
---> 40 _run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)
/usr/local/lib/python2.7/dist-packages/absl/app.pyc in run(main, argv, flags_parser)
298 callback()
299 try:
--> 300 _run_main(main, args)
301 except UsageError as error:
302 usage(shorthelp=True, detailed_error=error, exitcode=error.exitcode)
/usr/local/lib/python2.7/dist-packages/absl/app.pyc in _run_main(main, argv)
249 sys.exit(retval)
250 else:
--> 251 sys.exit(main(argv))
252
253
/root/.cache/bazel/_bazel_root/5b16c1cfa7fded7181f7e172eb7ad8a9/execroot/__main__/bazel-out/k8-opt/bin/lingvo/ipython_kernel.runfiles/__main__/lingvo/trainer.pyc in main(unused_argv)
1541 # pylint: disable=unused-variable
1542 from lingvo import model_imports
-> 1543 RunnerManager(FLAGS.model).Start()
1544
1545
/root/.cache/bazel/_bazel_root/5b16c1cfa7fded7181f7e172eb7ad8a9/execroot/__main__/bazel-out/k8-opt/bin/lingvo/ipython_kernel.runfiles/__main__/lingvo/trainer.pyc in Start(self)
1534 self.MaybeConfigRunDistributed()
1535 self.MaybeLaunchTensorFlow()
-> 1536 self.StartRunners(self.CreateRunners(FLAGS.job.split(','), FLAGS.logdir))
1537
1538
/root/.cache/bazel/_bazel_root/5b16c1cfa7fded7181f7e172eb7ad8a9/execroot/__main__/bazel-out/k8-opt/bin/lingvo/ipython_kernel.runfiles/__main__/lingvo/trainer.pyc in CreateRunners(self, jobs, logdir, trial)
1306
1307 runner = self._CreateRunner(j, FLAGS.model_task_name, logdir, tf_master,
-> 1308 trial)
1309 runners.append(runner)
1310 return runners
/root/.cache/bazel/_bazel_root/5b16c1cfa7fded7181f7e172eb7ad8a9/execroot/__main__/bazel-out/k8-opt/bin/lingvo/ipython_kernel.runfiles/__main__/lingvo/trainer.pyc in _CreateRunner(self, job, model_task_name, logdir, tf_master, trial)
1260 if job == 'controller':
1261 cfg = self.GetParamsForDataset('controller', 'Train')
-> 1262 return self.Controller(cfg, *common_args)
1263 elif job == 'trainer':
1264 cfg = self.GetParamsForDataset('trainer', 'Train')
/root/.cache/bazel/_bazel_root/5b16c1cfa7fded7181f7e172eb7ad8a9/execroot/__main__/bazel-out/k8-opt/bin/lingvo/ipython_kernel.runfiles/__main__/lingvo/trainer.pyc in __init__(self, *args, **kwargs)
190 with self._graph.as_default(), tf.container(self._container_id):
191 with self._cluster, tf.device(self._cluster.GetPlacer()):
--> 192 self._model = self.params.cls(self.params)
193 self._params = self._model.params
194 self._model.ConstructFPropBPropGraph()
/root/.cache/bazel/_bazel_root/5b16c1cfa7fded7181f7e172eb7ad8a9/execroot/__main__/bazel-out/k8-opt/bin/lingvo/ipython_kernel.runfiles/__main__/lingvo/core/base_layer.pyc in wrapper(self, *args, **kwargs)
116 try:
117 # Calls the layer's real __init__ method.
--> 118 func(self, *args, **kwargs)
119 # pylint: disable=protected-access
120 self._CheckInvariants()
/root/.cache/bazel/_bazel_root/5b16c1cfa7fded7181f7e172eb7ad8a9/execroot/__main__/bazel-out/k8-opt/bin/lingvo/ipython_kernel.runfiles/__main__/lingvo/core/base_model.pyc in __init__(self, params)
1156
1157 p = self.params
-> 1158 self.CreateChild('_task', p.task)
1159
1160 @property
/root/.cache/bazel/_bazel_root/5b16c1cfa7fded7181f7e172eb7ad8a9/execroot/__main__/bazel-out/k8-opt/bin/lingvo/ipython_kernel.runfiles/__main__/lingvo/core/base_layer.pyc in CreateChild(self, name, params)
589 params.name = name
590 p = self.CopyBaseParams(self.params, params.Copy())
--> 591 child = p.cls(p)
592 self._private_children[name] = child
593
/root/.cache/bazel/_bazel_root/5b16c1cfa7fded7181f7e172eb7ad8a9/execroot/__main__/bazel-out/k8-opt/bin/lingvo/ipython_kernel.runfiles/__main__/lingvo/core/base_layer.pyc in wrapper(self, *args, **kwargs)
116 try:
117 # Calls the layer's real __init__ method.
--> 118 func(self, *args, **kwargs)
119 # pylint: disable=protected-access
120 self._CheckInvariants()
/root/.cache/bazel/_bazel_root/5b16c1cfa7fded7181f7e172eb7ad8a9/execroot/__main__/bazel-out/k8-opt/bin/lingvo/ipython_kernel.runfiles/__main__/lingvo/tasks/mt/model.pyc in __init__(self, params)
51 @base_layer.initializer
52 def __init__(self, params):
---> 53 super(MTBaseModel, self).__init__(params)
54 p = self.params
55
/root/.cache/bazel/_bazel_root/5b16c1cfa7fded7181f7e172eb7ad8a9/execroot/__main__/bazel-out/k8-opt/bin/lingvo/ipython_kernel.runfiles/__main__/lingvo/core/base_layer.pyc in wrapper(self, *args, **kwargs)
116 try:
117 # Calls the layer's real __init__ method.
--> 118 func(self, *args, **kwargs)
119 # pylint: disable=protected-access
120 self._CheckInvariants()
/root/.cache/bazel/_bazel_root/5b16c1cfa7fded7181f7e172eb7ad8a9/execroot/__main__/bazel-out/k8-opt/bin/lingvo/ipython_kernel.runfiles/__main__/lingvo/core/base_model.pyc in __init__(self, params)
223 with tf.device(
224 self.cluster.input_device), py_utils.outside_all_rewrites():
--> 225 self.CreateChild('input', p.input)
226
227 self._var_grads = None
/root/.cache/bazel/_bazel_root/5b16c1cfa7fded7181f7e172eb7ad8a9/execroot/__main__/bazel-out/k8-opt/bin/lingvo/ipython_kernel.runfiles/__main__/lingvo/core/base_layer.pyc in CreateChild(self, name, params)
589 params.name = name
590 p = self.CopyBaseParams(self.params, params.Copy())
--> 591 child = p.cls(p)
592 self._private_children[name] = child
593
/root/.cache/bazel/_bazel_root/5b16c1cfa7fded7181f7e172eb7ad8a9/execroot/__main__/bazel-out/k8-opt/bin/lingvo/ipython_kernel.runfiles/__main__/lingvo/core/base_layer.pyc in wrapper(self, *args, **kwargs)
116 try:
117 # Calls the layer's real __init__ method.
--> 118 func(self, *args, **kwargs)
119 # pylint: disable=protected-access
120 self._CheckInvariants()
/root/.cache/bazel/_bazel_root/5b16c1cfa7fded7181f7e172eb7ad8a9/execroot/__main__/bazel-out/k8-opt/bin/lingvo/ipython_kernel.runfiles/__main__/lingvo/tasks/punctuator/input_generator.py in __init__(self, params)
89 # Build the input processing graph.
90 (self._src_ids, self._src_paddings, self._tgt_ids, self._tgt_paddings,
---> 91 self._tgt_labels, self._tgt_weights) = self._BuildDataSource()
92
93 self._input_batch_size = tf.shape(self._src_ids)
/root/.cache/bazel/_bazel_root/5b16c1cfa7fded7181f7e172eb7ad8a9/execroot/__main__/bazel-out/k8-opt/bin/lingvo/ipython_kernel.runfiles/__main__/lingvo/core/base_input_generator.pyc in _BuildDataSource(self)
387 input_file_pattern = p.file_pattern
388 if isinstance(input_file_pattern, six.string_types):
--> 389 return self._DataSourceFromFilePattern(input_file_pattern)
390 elif isinstance(input_file_pattern, list):
391 if p.use_within_batch_mixing:
/root/.cache/bazel/_bazel_root/5b16c1cfa7fded7181f7e172eb7ad8a9/execroot/__main__/bazel-out/k8-opt/bin/lingvo/ipython_kernel.runfiles/__main__/lingvo/tasks/punctuator/input_generator.py in _DataSourceFromFilePattern(self, file_pattern)
81 # The constant values to use for padding each of the outputs.
82 dynamic_padding_constants=[0, 1, 0, 1, 0, 0],
---> 83 **self.CommonInputOpArgs())
84
85 @base_layer.initializer
/root/.cache/bazel/_bazel_root/5b16c1cfa7fded7181f7e172eb7ad8a9/execroot/__main__/bazel-out/k8-opt/bin/lingvo/ipython_kernel.runfiles/__main__/lingvo/core/ops/py_x_ops.pyc in generic_input(processor, *args, **kwargs)
71 tf.DType(a.type) for a in processor.definition.signature.output_arg
72 ]
---> 73 assert out_types[-1] == tf.int32, ('%s is not expected.' % out_types[-1])
74 return gen_x_ops.generic_input(
75 processor=processor, out_types=out_types[:-1], *args, **kwargs)
IndexError: list index out of range
this is my whole trace
from lingvo.
jonathanasdf
commented on July 24, 2024
Can you please run this script
https://github.com/tensorflow/lingvo/blob/master/tf_env_collect.sh
from lingvo.
nim456
commented on July 24, 2024
== cat /etc/issue ===============================================
Linux bdb04a5e3ddf 4.15.0-1027-gcp #28~16.04.1-Ubuntu SMP Fri Jan 18 10:10:51 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
VERSION="16.04.5 LTS (Xenial Xerus)"
VERSION_ID="16.04"
VERSION_CODENAME=xenial
== are we in docker =============================================
Yes
== compiler =====================================================
c++ (Ubuntu 4.8.5-4ubuntu2) 4.8.5
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
== bazel =====================================================
Build label: 0.19.0
Build time: Mon Oct 29 14:35:30 2018 (1540823730)
Build timestamp: 1540823730
Build timestamp as int: 1540823730
== uname -a =====================================================
Linux bdb04a5e3ddf 4.15.0-1027-gcp #28~16.04.1-Ubuntu SMP Fri Jan 18 10:10:51 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
== check pips ===================================================
numpy 1.16.2
protobuf 3.6.1
== check for virtualenv =========================================
False
== tensorflow import ============================================
tf.VERSION = 1.13.0-dev20190227
tf.GIT_VERSION = v1.12.0-9066-g571e140e13
tf.COMPILER_VERSION = v1.12.0-9066-g571e140e13
Sanity check: array([1], dtype=int32)
== env ==========================================================
LD_LIBRARY_PATH is unset
DYLD_LIBRARY_PATH is unset
== nvidia-smi ===================================================
./tf_env_collect.sh: line 109: nvidia-smi: command not found
== cuda libs ===================================================
/usr/local/cuda-10.0/targets/x86_64-linux/lib/libcudart.so.10.0.130
these are my settings .....what next
from lingvo.
drpngx
commented on July 24, 2024
Strange. Could you go to py_x_ops.py in generic_input, and print processor.definition, processor.definition.signature before the assert?
from lingvo.
nim456
commented on July 24, 2024
when I try to print processor.definition and processor.definition.signature....... it is showing like this
IOPub data rate exceeded.
The notebook server will temporarily stop sending output
to the client in order to avoid crashing it.
To change this limit, set the config variable
--NotebookApp.iopub_data_rate_limit.
Current values:
NotebookApp.iopub_data_rate_limit=1000000.0 (bytes/sec)
NotebookApp.rate_limit_window=3.0 (secs)
from lingvo.
drpngx
commented on July 24, 2024
OK, I'll try to take a look tomorrow to repro.
from lingvo.
nim456
commented on July 24, 2024
yes pls let me know whether you are getting the same error when you are running
tf.app.run(trainer.main, argv=argv)
from lingvo.
nim456
commented on July 24, 2024
this is what I am getting when I try to print (processor.definition.signature) only ....
name: "PunctuatorInput._ProcessLine_lwV6oJwUKNw"
input_arg {
name: "line"
type: DT_STRING
}
description: "A single-text-line processor.\n Gets a string tensor representing a line of text that have been read from\n the input file, and splits it to graphemes (characters).\n We use original characters as the target labels, and the lowercased and\n punctuation-removed characters as the source labels.\n Args:\n line: a 1D string tensor.\n Returns:\n A list of tensors, in the expected order by init.\n "
is_stateful: true
from lingvo.
drpngx
commented on July 24, 2024
Yes, this is definitely wrong. It got the input arguments, but not the proper return arguments.
@zffchen78 maybe there's something obvious I'm missing.
from lingvo.
nim456
commented on July 24, 2024
yes pls do tell me someone how can I solve that list index out of range error .....
this is exactly what i am getting...
@drpngx @jonathanasdf @zffchen78 have look at this
IndexError Traceback (most recent call last)
in ()
23 "--run_locally=cpu", # or cpu.
24 ]
---> 25 tf.app.run(trainer.main, argv=argv)
/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/app.pyc in run(main, argv)
38 main = main or _sys.modules['main'].main
39
---> 40 _run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)
/usr/local/lib/python2.7/dist-packages/absl/app.pyc in run(main, argv, flags_parser)
298 callback()
299 try:
--> 300 _run_main(main, args)
301 except UsageError as error:
302 usage(shorthelp=True, detailed_error=error, exitcode=error.exitcode)
/usr/local/lib/python2.7/dist-packages/absl/app.pyc in _run_main(main, argv)
249 sys.exit(retval)
250 else:
--> 251 sys.exit(main(argv))
252
253
/root/.cache/bazel/_bazel_root/5b16c1cfa7fded7181f7e172eb7ad8a9/execroot/main/bazel-out/k8-opt/bin/lingvo/ipython_kernel.runfiles/main/lingvo/trainer.pyc in main(unused_argv)
1541 # pylint: disable=unused-variable
1542 from lingvo import model_imports
-> 1543 RunnerManager(FLAGS.model).Start()
1544
1545
/root/.cache/bazel/_bazel_root/5b16c1cfa7fded7181f7e172eb7ad8a9/execroot/main/bazel-out/k8-opt/bin/lingvo/ipython_kernel.runfiles/main/lingvo/trainer.pyc in Start(self)
1534 self.MaybeConfigRunDistributed()
1535 self.MaybeLaunchTensorFlow()
-> 1536 self.StartRunners(self.CreateRunners(FLAGS.job.split(','), FLAGS.logdir))
1537
1538
/root/.cache/bazel/_bazel_root/5b16c1cfa7fded7181f7e172eb7ad8a9/execroot/main/bazel-out/k8-opt/bin/lingvo/ipython_kernel.runfiles/main/lingvo/trainer.pyc in CreateRunners(self, jobs, logdir, trial)
1306
1307 runner = self._CreateRunner(j, FLAGS.model_task_name, logdir, tf_master,
-> 1308 trial)
1309 runners.append(runner)
1310 return runners
/root/.cache/bazel/_bazel_root/5b16c1cfa7fded7181f7e172eb7ad8a9/execroot/main/bazel-out/k8-opt/bin/lingvo/ipython_kernel.runfiles/main/lingvo/trainer.pyc in _CreateRunner(self, job, model_task_name, logdir, tf_master, trial)
1260 if job == 'controller':
1261 cfg = self.GetParamsForDataset('controller', 'Train')
-> 1262 return self.Controller(cfg, *common_args)
1263 elif job == 'trainer':
1264 cfg = self.GetParamsForDataset('trainer', 'Train')
/root/.cache/bazel/_bazel_root/5b16c1cfa7fded7181f7e172eb7ad8a9/execroot/main/bazel-out/k8-opt/bin/lingvo/ipython_kernel.runfiles/main/lingvo/trainer.pyc in init(self, *args, **kwargs)
190 with self._graph.as_default(), tf.container(self._container_id):
191 with self._cluster, tf.device(self._cluster.GetPlacer()):
--> 192 self._model = self.params.cls(self.params)
193 self._params = self._model.params
194 self._model.ConstructFPropBPropGraph()
/root/.cache/bazel/_bazel_root/5b16c1cfa7fded7181f7e172eb7ad8a9/execroot/main/bazel-out/k8-opt/bin/lingvo/ipython_kernel.runfiles/main/lingvo/core/base_layer.pyc in wrapper(self, *args, **kwargs)
116 try:
117 # Calls the layer's real init method.
--> 118 func(self, *args, **kwargs)
119 # pylint: disable=protected-access
120 self._CheckInvariants()
/root/.cache/bazel/_bazel_root/5b16c1cfa7fded7181f7e172eb7ad8a9/execroot/main/bazel-out/k8-opt/bin/lingvo/ipython_kernel.runfiles/main/lingvo/core/base_model.pyc in init(self, params)
1156
1157 p = self.params
-> 1158 self.CreateChild('_task', p.task)
1159
1160 @Property
/root/.cache/bazel/_bazel_root/5b16c1cfa7fded7181f7e172eb7ad8a9/execroot/main/bazel-out/k8-opt/bin/lingvo/ipython_kernel.runfiles/main/lingvo/core/base_layer.pyc in CreateChild(self, name, params)
589 params.name = name
590 p = self.CopyBaseParams(self.params, params.Copy())
--> 591 child = p.cls(p)
592 self._private_children[name] = child
593
/root/.cache/bazel/_bazel_root/5b16c1cfa7fded7181f7e172eb7ad8a9/execroot/main/bazel-out/k8-opt/bin/lingvo/ipython_kernel.runfiles/main/lingvo/core/base_layer.pyc in wrapper(self, *args, **kwargs)
116 try:
117 # Calls the layer's real init method.
--> 118 func(self, *args, **kwargs)
119 # pylint: disable=protected-access
120 self._CheckInvariants()
/root/.cache/bazel/_bazel_root/5b16c1cfa7fded7181f7e172eb7ad8a9/execroot/main/bazel-out/k8-opt/bin/lingvo/ipython_kernel.runfiles/main/lingvo/tasks/mt/model.pyc in init(self, params)
51 @base_layer.initializer
52 def init(self, params):
---> 53 super(MTBaseModel, self).init(params)
54 p = self.params
55
/root/.cache/bazel/_bazel_root/5b16c1cfa7fded7181f7e172eb7ad8a9/execroot/main/bazel-out/k8-opt/bin/lingvo/ipython_kernel.runfiles/main/lingvo/core/base_layer.pyc in wrapper(self, *args, **kwargs)
116 try:
117 # Calls the layer's real init method.
--> 118 func(self, *args, **kwargs)
119 # pylint: disable=protected-access
120 self._CheckInvariants()
/root/.cache/bazel/_bazel_root/5b16c1cfa7fded7181f7e172eb7ad8a9/execroot/main/bazel-out/k8-opt/bin/lingvo/ipython_kernel.runfiles/main/lingvo/core/base_model.pyc in init(self, params)
223 with tf.device(
224 self.cluster.input_device), py_utils.outside_all_rewrites():
--> 225 self.CreateChild('input', p.input)
226
227 self._var_grads = None
/root/.cache/bazel/_bazel_root/5b16c1cfa7fded7181f7e172eb7ad8a9/execroot/main/bazel-out/k8-opt/bin/lingvo/ipython_kernel.runfiles/main/lingvo/core/base_layer.pyc in CreateChild(self, name, params)
589 params.name = name
590 p = self.CopyBaseParams(self.params, params.Copy())
--> 591 child = p.cls(p)
592 self._private_children[name] = child
593
/root/.cache/bazel/_bazel_root/5b16c1cfa7fded7181f7e172eb7ad8a9/execroot/main/bazel-out/k8-opt/bin/lingvo/ipython_kernel.runfiles/main/lingvo/core/base_layer.pyc in wrapper(self, *args, **kwargs)
116 try:
117 # Calls the layer's real init method.
--> 118 func(self, *args, **kwargs)
119 # pylint: disable=protected-access
120 self._CheckInvariants()
/root/.cache/bazel/_bazel_root/5b16c1cfa7fded7181f7e172eb7ad8a9/execroot/main/bazel-out/k8-opt/bin/lingvo/ipython_kernel.runfiles/main/lingvo/tasks/punctuator/input_generator.py in init(self, params)
89 # Build the input processing graph.
90 (self._src_ids, self._src_paddings, self._tgt_ids, self._tgt_paddings,
---> 91 self._tgt_labels, self._tgt_weights) = self._BuildDataSource()
92
93 self._input_batch_size = tf.shape(self._src_ids)[0]
/root/.cache/bazel/_bazel_root/5b16c1cfa7fded7181f7e172eb7ad8a9/execroot/main/bazel-out/k8-opt/bin/lingvo/ipython_kernel.runfiles/main/lingvo/core/base_input_generator.pyc in _BuildDataSource(self)
387 input_file_pattern = p.file_pattern
388 if isinstance(input_file_pattern, six.string_types):
--> 389 return self._DataSourceFromFilePattern(input_file_pattern)
390 elif isinstance(input_file_pattern, list):
391 if p.use_within_batch_mixing:
/root/.cache/bazel/_bazel_root/5b16c1cfa7fded7181f7e172eb7ad8a9/execroot/main/bazel-out/k8-opt/bin/lingvo/ipython_kernel.runfiles/main/lingvo/tasks/punctuator/input_generator.py in _DataSourceFromFilePattern(self, file_pattern)
81 # The constant values to use for padding each of the outputs.
82 dynamic_padding_constants=[0, 1, 0, 1, 0, 0],
---> 83 **self.CommonInputOpArgs())
84
85 @base_layer.initializer
/root/.cache/bazel/_bazel_root/5b16c1cfa7fded7181f7e172eb7ad8a9/execroot/main/bazel-out/k8-opt/bin/lingvo/ipython_kernel.runfiles/main/lingvo/core/ops/py_x_ops.py in generic_input(processor, args, **kwargs)
75 print(""*50)
76 print(processor.definition.signature)
---> 77 assert out_types[-1] == tf.int32, ('%s is not expected.' % out_types[-1])
78 return gen_x_ops.generic_input(
79 processor=processor, out_types=out_types[:-1], *args, **kwargs)
IndexError: list index out of range
from lingvo.
nim456
commented on July 24, 2024
@drpngx I have solved that jupyter notebook issue which is coming while printing process.definition now its showing
s: "\342\226\201conversion"
s: "\342\226\201Without"
s: "va"
s: "ugene"
s: "\342\226\201inhib"
s: "\342\226\201Helva"
s: "\342\226\201parlor"
s: "\342\226\201parade"
s: "\342\226\201Gar"
s: "\342\226\201wart"
s: "\342\226\201smart"
s: "ieve"
s: "\342\226\201cellar"
s: "\342\226\201encouragement"
s: "10"
s: "60"
and it is continuing like this and after that this is what i am getting
node_def {
name: "while/concat/values_1"
op: "Const"
input: "^while/Identity"
attr {
key: "dtype"
value {
type: DT_INT32
}
}
attr {
key: "value"
value {
tensor {
dtype: DT_INT32
tensor_shape {
dim {
size: 1
}
}
int_val: 2
}
}
}
}
and the same pattern is what I am getting and after that defintion.signature is getting printed which i have shown above but indexing issue still remains ...
from lingvo.
nim456
commented on July 24, 2024
can anyone here tell me how to solve this problem ?
from lingvo.
drpngx
commented on July 24, 2024
It's not clear what the problem is. Can you go to tasks/punctuator/input_generator.py in ProcessLine and print out the return values?
from lingvo.
nim456
commented on July 24, 2024
how much time it will take to run the whole model ?
from lingvo.
drpngx
commented on July 24, 2024
Maybe 2-3 days on a single GPU, should give you decent results.
from lingvo.
nim456
commented on July 24, 2024
ok thanks
from lingvo.
Related Issues (20)
- DeepFusion network structure HOT 1
- DeepFusion Learnable Align Impl Details HOT 1
- How can I get the logits for one whole sequence in the asr task? HOT 3
- when will the deepfusion code be released? HOT 1
- Car models seem to be disabled for now
- Bazel build failure
- Learnable Align Attention Implementation HOT 1
- DeepFusion Readme HOT 4
- DeepFusion reproduce HOT 11
- Cannot run trainer.py with --model=car.waymo_deepfusion.DeepFusionCenterPointPed, undefined symbol: _ZNK10tensorflow8OpKernel11TraceStringERKNS_15OpKernelContextEb HOT 4
- Cannot import py_camera_model_ops from waymo_open_dataset.camera.ops HOT 3
- images
- Question about DeepFusion
- question about RandomVectorQuantizer
- cannot import name "hyperparams_pb2' from lingvo.core' how to deal with HOT 1
- Switch from prebuilt protoc to build from source
- Raw dependency on "//third_party/py/flax/training:checkpoints"
- unreplicate_metrics=True fails on my training
- Feature request: lingvo.jax.asserts.HasShape HOT 1
- RFC: lingvo.jax exception flag mechanism
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from lingvo.