els-rd / transformer-deploy Goto Github PK

[01/14/2022-11:18:37] [TRT] [W] Calibrator is not being used. Users must provide dynamic range for all tensors that are not Int32 or Bool.
[01/14/2022-11:18:37] [TRT] [E] 4: [standardEngineBuilder.cpp::initCalibrationParams::1402] Error Code 4: Internal Error (Calibration failure occurred with no scaling factors detected. This could be due to no int8 calibrator or insufficient custom scales for network layers. Please see int8 sample to setup calibration correctly.)
[01/14/2022-11:18:37] [TRT] [E] 2: [builder.cpp::buildSerializedNetwork::609] Error Code 2: Internal Error (Assertion enginePtr != nullptr failed. )

Traceback (most recent call last):
  File "/data/repos/transformer-deploy/src/transformer_deploy/convert.py", line 326, in <module>
    entrypoint()
  File "/data/repos/transformer-deploy/src/transformer_deploy/convert.py", line 322, in entrypoint
    main(commands=args)
  File "/data/repos/transformer-deploy/src/transformer_deploy/convert.py", line 216, in main
    engine: ICudaEngine = build_engine(
  File "/data/repos/transformer-deploy/src/transformer_deploy/backends/trt_utils.py", line 181, in build_engine
    engine: ICudaEngine = runtime.deserialize_cuda_engine(trt_engine)
TypeError: deserialize_cuda_engine(): incompatible function arguments. The following argument types are supported:
    1. (self: tensorrt.tensorrt.Runtime, serialized_engine: buffer) -> tensorrt.tensorrt.ICudaEngine

Invoked with: <tensorrt.tensorrt.Runtime object at 0x7feb14128e30>, None

AssertionError                            Traceback (most recent call last)
<timed eval> in <module>

/opt/conda/lib/python3.7/site-packages/transformer_deploy/convert.py in main(input_args)
    245             onnx_path=onnx_model_path,
    246             onnx_optim_fp16_path=onnx_optim_fp16_path,
--> 247             use_cuda=True,
    248         )
    249         onnx_model = create_model_for_provider(path=onnx_optim_fp16_path, provider_to_use="CUDAExecutionProvider")

/opt/conda/lib/python3.7/site-packages/transformer_deploy/backends/ort_utils.py in optimize_onnx(onnx_path, onnx_optim_fp16_path, use_cuda)
     72         num_heads=0,  # automatic detection don't work with opset 13
     73         hidden_size=0,  # automatic detection
---> 74         optimization_options=optimization_options,
     75     )
     76 

/opt/conda/lib/python3.7/site-packages/onnxruntime/transformers/optimizer.py in optimize_model(input, model_type, num_heads, hidden_size, optimization_options, opt_level, use_gpu, only_onnxruntime)
    289 
    290     if not only_onnxruntime:
--> 291         optimizer.optimize(optimization_options)
    292 
    293     # Remove the temporary model.

/opt/conda/lib/python3.7/site-packages/onnxruntime/transformers/onnx_model_bert.py in optimize(self, options, add_dynamic_axes)
    317             if options is not None:
    318                 self.attention_mask.set_mask_format(options.attention_mask_format)
--> 319             self.fuse_attention()
    320 
    321         self.fuse_shape()

/opt/conda/lib/python3.7/site-packages/onnxruntime/transformers/onnx_model_bert.py in fuse_attention(self)
     52 
     53     def fuse_attention(self):
---> 54         self.attention_fusion.apply()
     55 
     56     def fuse_gelu(self):

/opt/conda/lib/python3.7/site-packages/onnxruntime/transformers/fusion_base.py in apply(self)
     41                     raise Exception("Can not find node in any graphs")
     42                 self.this_graph_name = graph.name
---> 43                 self.fuse(node, input_name_to_nodes, output_name_to_node)
     44 
     45         op_list = [node.op_type for node in self.nodes_to_add]

/opt/conda/lib/python3.7/site-packages/onnxruntime/transformers/fusion_attention.py in fuse(self, normalize_node, input_name_to_nodes, output_name_to_node)
    444             new_node = self.create_attention_node(mask_index, matmul_q, matmul_k, matmul_v, add_q, add_k, add_v,
    445                                                   q_num_heads, self.hidden_size, root_input,
--> 446                                                   attention_last_node.output[0], add_qk_str)
    447             if new_node is None:
    448                 return

/opt/conda/lib/python3.7/site-packages/onnxruntime/transformers/fusion_attention.py in create_attention_node(self, mask_index, q_matmul, k_matmul, v_matmul, q_add, k_add, v_add, num_heads, hidden_size, input, output, add_qk_str)
    161             Union[NodeProto, None]: the node created or None if failed.
    162         """
--> 163         assert num_heads > 0
    164 
    165         if hidden_size > 0 and (hidden_size % num_heads) != 0:

AssertionError: 

name: "tensorrt_model"
platform: "tensorrt_plan"
max_batch_size: 0
input [
 {
    name: "input_ids"
    data_type: TYPE_INT32
    dims: [ -1, -1  ]
  },

{
    name: "attention_mask"
    data_type: TYPE_INT32
    dims: [-1, -1 ]
},

{
    name: "decoder_input_ids"
    data_type: TYPE_INT32
    dims: [ -1, -1]
},

{
   name: "decoder_attention_mask"
   data_type: TYPE_INT32
   dims: [ -1, -1 ]
}

]
output [
{
    name: "last_hidden_state"
    data_type: TYPE_FP32
    dims: [ -1, -1, 768 ]
  },

{
    name: "input.151"
    data_type: TYPE_FP32
    dims: [ -1, -1, -1 ]
  }

]

instance_group [
    {
        count: 1
        kind: KIND_GPU
    }
]

def forward(self, ids, masks, token_type_ids):        
    last_hidden_state, _ = self.l1(ids, attention_mask=masks, token_type_ids=token_type_ids)
    avg_pooling = self.l2(last_hidden_state, dim=1)
    output = self.l3(avg_pooling)
    
    return output`

checkpoint = {
    'epoch': args.epochs,
    'num_labels': args.num_labels,
    'max_text_length': MAX_TEXT_LENGTH,
    'state_dict': model_2_save.state_dict()
}

torch.save(checkpoint, args.model_dir + "/pt_model.pt")`

batch_size=8
        with QATCalibrate(method="histogram", percentile=99.999) as qat:
            model_q = self.model.cuda()
            qat.setup_model_qat(model_q)  # prepare quantizer to any model

            with torch.no_grad():
                for start_index in range(0, 650, batch_size):
                    end_index = start_index + batch_size
                    data = self.data[start_index:end_index]
                    data = self.tokenizer(data, return_tensors='pt', padding=True, truncation=True, max_length=512)
                    input_torch = {
                        k: torch.tensor(v, dtype=torch.long, device="cuda")
                        for k, v in data.items()
                        if k in ["input_ids", "attention_mask", "token_type_ids"]
                    }
                    model_q(**input_torch)

{ "question": "Did the stock come down?", "context": "Some text about stocks, with reference if stocks went up or down..." }

{"score": 0.02739531360566616, "start": 103, "end": 173, "answer": "sent stocks sliding to their worst performance in months on Wednesday."}

"inputs": [{"name": "TEXT", "shape": [2], "datatype": "BYTES", "data": ["Did the stock come down?", "\Some text about stocks, with reference if stocks went up or down..."]}]}

{"model_name":"roberta_onnx_inference","model_version":"1","parameters":{"sequence_id":0,"sequence_start":false,"sequence_end":false},"outputs":[{"name":"output","datatype":"FP32","shape":[1,2],"data":[-0.2091064453125,-0.06671142578125]}]}

/usr/local/lib/python3.8/dist-packages/transformers/models/gpt2/modeling_gpt2.py:196: TracerWarning: Converting a tensor to a Python float might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  attn_weights = attn_weights / (float(value.size(-1)) ** 0.5)
[03/14/2022-09:03:29] [TRT] [E] onnx2trt_utils.cpp:1571: Failed to open file: transformer.wte.weight
[03/14/2022-09:03:29] [TRT] [E] 4: [network.cpp::validate::2633] Error Code 4: Internal Error (Network must have at least one output)
[03/14/2022-09:03:29] [TRT] [E] 2: [builder.cpp::buildSerializedNetwork::609] Error Code 2: Internal Error (Assertion enginePtr != nullptr failed. )
Traceback (most recent call last):
  File "/usr/local/bin/convert_model", line 8, in <module>
    sys.exit(entrypoint())
  File "/usr/local/lib/python3.8/dist-packages/transformer_deploy/convert.py", line 357, in entrypoint
    main(commands=args)
  File "/usr/local/lib/python3.8/dist-packages/transformer_deploy/convert.py", line 262, in main
    engine: ICudaEngine = build_engine(
  File "/usr/local/lib/python3.8/dist-packages/transformer_deploy/backends/trt_utils.py", line 126, in build_engine
    engine: ICudaEngine = runtime.deserialize_cuda_engine(trt_engine)
TypeError: deserialize_cuda_engine(): incompatible function arguments. The following argument types are supported:
    1. (self: tensorrt.tensorrt.Runtime, serialized_engine: buffer) -> tensorrt.tensorrt.ICudaEngine

Invoked with: <tensorrt.tensorrt.Runtime object at 0x7fe431b73a30>, None

docker run -it --rm --gpus device=6 \
    --shm-size=1g --ulimit memlock=-1 --ulimit stack=67108864 \
    -v $PWD:/project ghcr.io/els-rd/transformer-deploy:0.4.0 \
    bash -c "cd /project && \
        convert_model -m distilgpt2 \
        --backend tensorrt \
        --seq-len 1 128 128 \
        --task text-generation"

docker run -it --rm --gpus device=6 \
    --shm-size=1g --ulimit memlock=-1 --ulimit stack=67108864 \
    -v $PWD:/project ghcr.io/els-rd/transformer-deploy:0.4.0 \
    bash -c "cd /project && \
        convert_model -m gpt2-large \
        --backend tensorrt \
        --seq-len 1 128 128 \
        --task text-generation"

docker run -it --rm --gpus all \
  -v $PWD:/project ghcr.io/els-rd/transformer-deploy:0.4.0 \
  bash -c "cd /project && \
    convert_model -m \"philschmid/MiniLM-L6-H384-uncased-sst2\" \
    --backend tensorrt onnx \
    --seq-len 16 128 128"

Inference done on Tesla M60
latencies:
[Pytorch (FP32)] mean=6.31ms, sd=1.32ms, min=4.48ms, max=10.75ms, median=6.39ms, 95p=8.63ms, 99p=9.33ms
[Pytorch (FP16)] mean=8.81ms, sd=2.02ms, min=6.59ms, max=55.42ms, median=8.70ms, 95p=11.20ms, 99p=12.16ms
**### [TensorRT (FP16)] mean=4.59ms, sd=1.97ms, min=2.27ms, max=10.38ms, median=4.47ms, 95p=8.02ms, 99p=8.86ms**
[ONNX Runtime (FP32)] mean=5.03ms, sd=2.00ms, min=2.64ms, max=10.45ms, median=5.16ms, 95p=8.37ms, 99p=9.17ms
[ONNX Runtime (optimized)] mean=5.19ms, sd=2.04ms, min=2.80ms, max=10.59ms, median=5.25ms, 95p=8.67ms, 99p=9.40ms

# @ means no data conversion (curl feature)
curl -X POST  http://localhost:8000/v2/models/transformer_onnx_inference/versions/1/infer \
  --data-binary "@demo/query_body.bin" \
  --header "Inference-Header-Content-Length: 160"

Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting pytorch-quantization
  Downloading pytorch-quantization-0.0.1.dev5.tar.gz (7.9 kB)
  Preparing metadata (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /opt/conda/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-kmmchqi2/pytorch-quantization_5a45d9f46d524a108b6ec28eb5238500/setup.py'"'"'; __file__='"'"'/tmp/pip-install-kmmchqi2/pytorch-quantization_5a45d9f46d524a108b6ec28eb5238500/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-_58u5ior
       cwd: /tmp/pip-install-kmmchqi2/pytorch-quantization_5a45d9f46d524a108b6ec28eb5238500/
  Complete output (16 lines):
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/tmp/pip-install-kmmchqi2/pytorch-quantization_5a45d9f46d524a108b6ec28eb5238500/setup.py", line 150, in <module>
      raise RuntimeError(open("ERROR.txt", "r").read())
  RuntimeError:
  ###########################################################################################
  The package you are trying to install is only a placeholder project on PyPI.org repository.
  This package is hosted on NVIDIA Python Package Index.
  
  This package can be installed as:
  ```
  $ pip install nvidia-pyindex
  $ pip install pytorch-quantization
  ```
  ###########################################################################################
  
  ----------------------------------------
WARNING: Discarding https://files.pythonhosted.org/packages/35/ea/c6c4ab73da4e36b9eddea7ff687b98e1bccb59bfb3bd0c24459914fb17f2/pytorch-quantization-0.0.1.dev5.tar.gz#sha256=4702207b088af5a1e58ee31d5ceee14aaa21bc3ef36b39ca996a6ee4d0ffb4dd (from https://pypi.org/simple/pytorch-quantization/). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
  Downloading pytorch-quantization-0.0.1.dev4.tar.gz (4.1 kB)
  Preparing metadata (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /opt/conda/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-kmmchqi2/pytorch-quantization_b0afa379038d4cbc8e3d8b401f5d74f4/setup.py'"'"'; __file__='"'"'/tmp/pip-install-kmmchqi2/pytorch-quantization_b0afa379038d4cbc8e3d8b401f5d74f4/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-s1sedxj7
       cwd: /tmp/pip-install-kmmchqi2/pytorch-quantization_b0afa379038d4cbc8e3d8b401f5d74f4/
  Complete output (15 lines):
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/tmp/pip-install-kmmchqi2/pytorch-quantization_b0afa379038d4cbc8e3d8b401f5d74f4/setup.py", line 150, in <module>
      raise RuntimeError(open("ERROR.txt", "r").read())
  RuntimeError:
  ###########################################################################################
  The package you are trying to install is only a placeholder project on PyPI.org repository.
  This package is hosted on NVIDIA Python Package Index.
  
  This package can be installed as:
  ```
  $ pip install nvidia-pyindex
  $ pip install pytorch-quantization
 ```
  ###########################################################################################
  ----------------------------------------
WARNING: Discarding https://files.pythonhosted.org/packages/d4/3e/e891628c040badc4d18ca48a28bf5a991654161fb32ee5f54ec2317e2664/pytorch-quantization-0.0.1.dev4.tar.gz#sha256=6fea1f1ba851353d65f08098fe19041cd045ca9239e98e5f7058cb1872b6ea57 (from https://pypi.org/simple/pytorch-quantization/). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
ERROR: Could not find a version that satisfies the requirement pytorch-quantization (from versions: 0.0.1.dev4, 0.0.1.dev5)
ERROR: No matching distribution found for pytorch-quantization

$ convert_model -m roberta-large-mnli --backend tensorrt onnx pytorch --seq-len 16 128 128 --batch-size 1 32 32

...
    engine = build_engine(
  File "/home/sam_havens/transformer-deploy/venv/lib/python3.8/site-packages/transformer_deploy/backends/trt_utils.py", line 181, in build_engine
    engine: ICudaEngine = runtime.deserialize_cuda_engine(trt_engine)
TypeError: deserialize_cuda_engine(): incompatible function arguments. The following argument types are supported:
    1. (self: tensorrt.tensorrt.Runtime, serialized_engine: buffer) -> tensorrt.tensorrt.ICudaEngine

Invoked with: <tensorrt.tensorrt.Runtime object at 0x7f5df2f18bb0>, None

[03/18/2022-12:51:55] [TRT] [E] 2: [virtualMemoryBuffer.cpp::resizePhysical::161] Error Code 2: OutOfMemory (no further information)
[03/18/2022-12:51:55] [TRT] [E] 2: [virtualMemoryBuffer.cpp::resizePhysical::161] Error Code 2: OutOfMemory (no further information)
[03/18/2022-12:51:55] [TRT] [E] 10: [optimizer.cpp::computeCosts::2011] Error Code 10: Internal Error (Could not find any implementation for node {ForeignNode[encoder.embed_tokens.weight...Mul_406]}.)
[03/18/2022-12:51:55] [TRT] [E] 2: [builder.cpp::buildSerializedNetwork::609] Error Code 2: Internal Error (Assertion enginePtr != nullptr failed. )
Traceback (most recent call last):
  File "export_onnx_to_trt.py", line 100, in <module>
    build_t5_engine(onnx_encoder_path, trt_encoder_path, [input_id_shape])
  File "export_onnx_to_trt.py", line 86, in build_t5_engine
    engine: ICudaEngine = build_engine(
  File "/app/utils.py", line 209, in build_engine
    engine: ICudaEngine = runtime.deserialize_cuda_engine(trt_engine)
TypeError: deserialize_cuda_engine(): incompatible function arguments. The following argument types are supported:
    1. (self: tensorrt.tensorrt.Runtime, serialized_engine: buffer) -> tensorrt.tensorrt.ICudaEngine

Invoked with: <tensorrt.tensorrt.Runtime object at 0x7f380bbf8930>, None

docker run -it --rm --gpus all \
  -v $PWD:/project ghcr.io/els-rd/transformer-deploy:0.4.0 \
  bash -c "cd /project && \
    convert_model -m \"distilbert-base-uncased-finetuned-sst-2-english\" \
    --backend onnx \
    --seq-len 16 128 128"

def get_random_input_encoder() -> Dict[str, torch.Tensor]:
    max_seq = 512
    seq_len = random.randint(a=1, b=max_seq)
    batch = max_seq // seq_len
    random_input_ids = torch.randint(
        low=0, high=tokenizer.vocab_size, size=(batch, seq_len), dtype=torch.int32, device="cuda"
    )
    inputs = {"input_ids": random_input_ids}
    return inputs


keep_fp32_encoder = get_keep_fp32_nodes(onnx_model_path=encoder_model_path, get_input=get_random_input_encoder)
assert len(keep_fp32_encoder) > 0
enc_model_onnx = convert_fp16(onnx_model=encoder_model_path, nodes_to_exclude=keep_fp32_encoder)
save_onnx(proto=enc_model_onnx, model_path=encoder_fp16_model_path)

del enc_model_onnx
torch.cuda.empty_cache()
gc.collect()

RuntimeError                              Traceback (most recent call last)
Input In [12], in <cell line: 12>()
      8     inputs = {"input_ids": random_input_ids}
      9     return inputs
---> 12 keep_fp32_encoder = get_keep_fp32_nodes(onnx_model_path=encoder_model_path, get_input=get_random_input_encoder)
     13 assert len(keep_fp32_encoder) > 0
     14 enc_model_onnx = convert_fp16(onnx_model=encoder_model_path, nodes_to_exclude=keep_fp32_encoder)

File /usr/local/lib/python3.8/dist-packages/transformer_deploy/backends/ort_utils.py:372, in get_keep_fp32_nodes(onnx_model_path, get_input, early_stop, device)
    368 inputs = get_input()
    369 outputs: Dict[str, torch.Tensor] = inference_onnx_binding(
    370     model_onnx=ort_model_fp32_all_nodes, inputs=inputs, device=device, binding=ort_binding, clone_tensor=False
    371 )
--> 372 keep_node_io = find_node_fp32(graph=output_mapping, output_nodes=outputs)
    374 nodes_to_add = [n for n in keep_node_io if n not in keep_fp32_nodes]
    375 keep_fp32_nodes += nodes_to_add

File /usr/local/lib/python3.8/dist-packages/transformer_deploy/backends/ort_utils.py:304, in find_node_fp32(graph, output_nodes)
    299     # out of FP16 range
    300     print("Tensor is ", tensor)
    301     if (
    302         torch.any(tensor > max_float16)
    303         or torch.any(tensor < min_float16)
--> 304         or (torch.any(tensor < resolution & tensor > -resolution & tensor != 0))  # limited memory footprint
    305     ):
    306         keep_fp32.append(graph[k])
    307 return keep_fp32

RuntimeError: "bitwise_and_cuda" not implemented for 'Float'

convert_model -m sentence-transformers/msmarco-distilbert-cos-v5 --backend onnx --task embedding --seq-len 128 128 128

Traceback (most recent call last):
  File "/usr/local/bin/convert_model", line 8, in <module>
    sys.exit(entrypoint())
  File "/usr/local/lib/python3.8/dist-packages/transformer_deploy/convert.py", line 388, in entrypoint
    main(commands=args)
  File "/usr/local/lib/python3.8/dist-packages/transformer_deploy/convert.py", line 355, in main
    ort_output, time_buffer = launch_inference(
  File "/usr/local/lib/python3.8/dist-packages/transformer_deploy/convert.py", line 113, in launch_inference
    output = infer(batch_input)
  File "/usr/local/lib/python3.8/dist-packages/transformer_deploy/convert.py", line 351, in infer_ort
    results = inference_onnx_binding(model_onnx=ort_model, inputs=inputs, device=commands.device)
  File "/usr/local/lib/python3.8/dist-packages/transformer_deploy/backends/ort_utils.py", line 255, in inference_onnx_binding
    binding.synchronize_inputs()
AttributeError: 'IOBinding' object has no attribute 'synchronize_inputs'

git clone [email protected]:ELS-RD/transformer-deploy.git
cd transformer-deploy
# docker image may take a few minutes
docker pull ghcr.io/els-rd/transformer-deploy:0.4.0 

# run the docker container
docker run -it --rm --gpus all  -v $PWD:/project ghcr.io/els-rd/transformer-deploy:0.4.0 bash

# run the following commands from inside the container
# run test command. This should run without any issues
convert_model -m sentence-transformers/msmarco-distilbert-cos-v5 --backend onnx --task embedding --seq-len 128 128 128

# install transformer-deploy from the latest commit
pip install .

# run the test command again and it will error out
convert_model -m sentence-transformers/msmarco-distilbert-cos-v5 --backend onnx --task embedding --seq-len 128 128 128

+--------------------------------+---------+----------------------------------------------------------------------------------------------------------------------------------+
| Model                          | Version | Status                                                                                                                           |
+--------------------------------+---------+----------------------------------------------------------------------------------------------------------------------------------+
| transformer_onnx_model         | 1       | UNAVAILABLE: Internal: onnx runtime error 1: Load model from /models/transformer_onnx_model/1/model.bin failed:/workspace/onnxru |
|                                |         | ntime/onnxruntime/core/graph/model_load_utils.h:47 void onnxruntime::model_load_utils::ValidateOpsetForDomain(const std::unorder |
|                                |         | ed_map<std::__cxx11::basic_string<char>, int>&, const onnxruntime::logging::Logger&, bool, const string&, int) ONNX Runtime only |
|                                |         |  *guarantees* support for models stamped with official released onnx opset versions. Opset 3 is under development and support fo |
|                                |         | r this is limited. The operator schemas and or other functionality may change before next ONNX release and in this case ONNX Run |
|                                |         | time will not guarantee backward compatibility. Current official support for domain ai.onnx.ml is till opset 2.                  |
| transformer_onnx_tokenize      | 1       | READY                                                                                                                            |
| transformer_tensorrt_inference | 1       | READY                                                                                                                            |
| transformer_tensorrt_model     | 1       | READY                                                                                                                            |
| transformer_tensorrt_tokenize  | 1       | READY                                                                                                                            |
+--------------------------------+---------+----------------------------------------------------------------------------------------------------------------------------------+