ramsrigouthamg / questgen.ai Goto Github PK
View Code? Open in Web Editor NEWQuestion generation using state-of-the-art Natural Language Processing algorithms
Home Page: https://questgen.ai/
License: MIT License
Question generation using state-of-the-art Natural Language Processing algorithms
Home Page: https://questgen.ai/
License: MIT License
Using cached sentencepiece-0.1.97-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)
Requirement already satisfied: filelock in /usr/local/lib/python3.9/dist-packages (from transformers==3.0.2->Questgen==1.0.0) (3.9.0)
Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.9/dist-packages (from transformers==3.0.2->Questgen==1.0.0) (2022.6.2)
Collecting tokenizers==0.8.1.rc1
Using cached tokenizers-0.8.1rc1.tar.gz (97 kB)
Installing build dependencies ... done
error: subprocess-exited-with-error
× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
Getting requirements to build wheel ... error
error: subprocess-exited-with-error
× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.
I am getting following error
"Token indices sequence length is longer than the specified maximum sequence length for this model (988 > 512). Running this sequence through the model will result in indexing errors"
My most of documents about 1000 length. Please have a look and guide me for the solution.
This is the error I'm getting
Defaulting to user installation because normal site-packages is not writeable Collecting git+https://github.com/ramsrigouthamg/Questgen.ai Cloning https://github.com/ramsrigouthamg/Questgen.ai to /tmp/pip-req-build-o9y6mywf Running command git clone --filter=blob:none --quiet https://github.com/ramsrigouthamg/Questgen.ai /tmp/pip-req-build-o9y6mywf Resolved https://github.com/ramsrigouthamg/Questgen.ai to commit 66c43d37f972c900414461920498644ef47e856f Preparing metadata (setup.py) ... done ERROR: Could not find a version that satisfies the requirement torch==1.10.0 (from questgen) (from versions: 1.11.0) ERROR: No matching distribution found for torch==1.10.0
when I do pip install torch==1.10.0
it says cant install from versions: 1.11.0
I dont know if its a problem with pip, python, pytorch or what
pls help :)
Are there any contributing guidelines for this project?
Hi,
I am getting ValueError: Can't read file: s2v_old/cfg
at line main.QGen()
I tried with other call qe= main.BoolQGen()
which is running fine. I am using Python 3.7
I find some projects about table question answering task.
Such as https://github.com/abhijithneilabraham/tableQA use embedding and rule based method.
and tapas : https://huggingface.co/lysandre/tapas-temporary-repo
I think you are interested in question generation.
Do you have some ideas about generate question and answer on tableau data ?
Or can you share me some projects or papers as materials ?
Thanks for your wonderful work
I am using ubuntu 21.04 and python 3.9.5 when I do pip install https://github.com/ramsrigouthamg/Questgen.ai.git
i get the following error, previously it was due to a rust compiler and I installed the rust compiler. Now I am facing this issue. I do realize this is not a direct issue in your code but a dependency that is defined in your requirements.txt transformers
i will fork your repo and try with a slightly older version of transformers and keep you posted.
Building wheels for collected packages: Questgen, tokenizers
Building wheel for Questgen (setup.py) ... done
Created wheel for Questgen: filename=Questgen-1.0.0-py3-none-any.whl size=8641 sha256=f0826c98ace2675e3e76b46010252e5b6c1c7eaaf63f1343210e08c849852ca5
Stored in directory: /tmp/pip-ephem-wheel-cache-zycnpbr8/wheels/3a/1b/cc/37d8e05f24fc92467af00e01bbe847700db1bc05f29b850cb4
Building wheel for tokenizers (PEP 517) ... error
ERROR: Command errored out with exit status 1:
command: /usr/bin/python3 /tmp/tmplfvq6iuo_in_process.py build_wheel /tmp/tmpdszda4_4
cwd: /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b
Complete output (227 lines):
/tmp/pip-build-env-uvf7i3x8/overlay/lib/python3.9/site-packages/setuptools/dist.py:487: UserWarning: Normalizing '0.8.1.rc1' to '0.8.1rc1'
warnings.warn(tmpl.format(**locals()))
running bdist_wheel
running build
running build_py
creating build
creating build/lib
creating build/lib/tokenizers
copying tokenizers/__init__.py -> build/lib/tokenizers
creating build/lib/tokenizers/models
copying tokenizers/models/__init__.py -> build/lib/tokenizers/models
creating build/lib/tokenizers/decoders
copying tokenizers/decoders/__init__.py -> build/lib/tokenizers/decoders
creating build/lib/tokenizers/normalizers
copying tokenizers/normalizers/__init__.py -> build/lib/tokenizers/normalizers
creating build/lib/tokenizers/pre_tokenizers
copying tokenizers/pre_tokenizers/__init__.py -> build/lib/tokenizers/pre_tokenizers
creating build/lib/tokenizers/processors
copying tokenizers/processors/__init__.py -> build/lib/tokenizers/processors
creating build/lib/tokenizers/trainers
copying tokenizers/trainers/__init__.py -> build/lib/tokenizers/trainers
creating build/lib/tokenizers/implementations
copying tokenizers/implementations/base_tokenizer.py -> build/lib/tokenizers/implementations
copying tokenizers/implementations/sentencepiece_bpe.py -> build/lib/tokenizers/implementations
copying tokenizers/implementations/bert_wordpiece.py -> build/lib/tokenizers/implementations
copying tokenizers/implementations/__init__.py -> build/lib/tokenizers/implementations
copying tokenizers/implementations/byte_level_bpe.py -> build/lib/tokenizers/implementations
copying tokenizers/implementations/char_level_bpe.py -> build/lib/tokenizers/implementations
copying tokenizers/__init__.pyi -> build/lib/tokenizers
copying tokenizers/models/__init__.pyi -> build/lib/tokenizers/models
copying tokenizers/decoders/__init__.pyi -> build/lib/tokenizers/decoders
copying tokenizers/normalizers/__init__.pyi -> build/lib/tokenizers/normalizers
copying tokenizers/pre_tokenizers/__init__.pyi -> build/lib/tokenizers/pre_tokenizers
copying tokenizers/processors/__init__.pyi -> build/lib/tokenizers/processors
copying tokenizers/trainers/__init__.pyi -> build/lib/tokenizers/trainers
running build_ext
running build_rust
Updating crates.io index
Updating git repository `https://github.com/n1t0/rayon-cond`
cargo rustc --lib --manifest-path Cargo.toml --features pyo3/extension-module --release --verbose -- --crate-type cdylib
warning: unused manifest key: target.x86_64-apple-darwin.rustflags
Compiling proc-macro2 v1.0.29
Compiling unicode-xid v0.2.2
Compiling syn v1.0.80
Compiling libc v0.2.103
Compiling autocfg v1.0.1
Compiling cfg-if v1.0.0
Compiling lazy_static v1.4.0
Compiling memchr v2.4.1
Running `rustc --crate-name build_script_build --edition=2018 /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/proc-macro2-1.0.29/build.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type bin --emit=dep-info,link -C opt-level=3 -Cembed-bitcode=no --cfg 'feature="default"' --cfg 'feature="proc-macro"' -C metadata=138e404a51655eac -C extra-filename=-138e404a51655eac --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/build/proc-macro2-138e404a51655eac -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --cap-lints allow`
Running `rustc --crate-name unicode_xid /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/unicode-xid-0.2.2/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -Cembed-bitcode=no --cfg 'feature="default"' -C metadata=5169222e9a9224f7 -C extra-filename=-5169222e9a9224f7 --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --cap-lints allow`
Running `rustc --crate-name build_script_build --edition=2018 /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/syn-1.0.80/build.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type bin --emit=dep-info,link -C opt-level=3 -Cembed-bitcode=no --cfg 'feature="clone-impls"' --cfg 'feature="default"' --cfg 'feature="derive"' --cfg 'feature="extra-traits"' --cfg 'feature="full"' --cfg 'feature="parsing"' --cfg 'feature="printing"' --cfg 'feature="proc-macro"' --cfg 'feature="quote"' -C metadata=0f030b9755d18eeb -C extra-filename=-0f030b9755d18eeb --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/build/syn-0f030b9755d18eeb -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --cap-lints allow`
Running `rustc --crate-name build_script_build /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/libc-0.2.103/build.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type bin --emit=dep-info,link -C opt-level=3 -Cembed-bitcode=no --cfg 'feature="default"' --cfg 'feature="std"' -C metadata=7a7aa92cf75f8e53 -C extra-filename=-7a7aa92cf75f8e53 --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/build/libc-7a7aa92cf75f8e53 -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --cap-lints allow`
Running `rustc --crate-name autocfg /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/autocfg-1.0.1/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -Cembed-bitcode=no -C metadata=4b64b452f871448c -C extra-filename=-4b64b452f871448c --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --cap-lints allow`
Running `rustc --crate-name cfg_if --edition=2018 /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/cfg-if-1.0.0/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -Cembed-bitcode=no -C metadata=2deae1b316fdaf6f -C extra-filename=-2deae1b316fdaf6f --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --cap-lints allow`
Running `rustc --crate-name lazy_static /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/lazy_static-1.4.0/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -Cembed-bitcode=no -C metadata=79cb8478fcd4b466 -C extra-filename=-79cb8478fcd4b466 --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --cap-lints allow`
Running `rustc --crate-name build_script_build --edition=2018 /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/memchr-2.4.1/build.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type bin --emit=dep-info,link -C opt-level=3 -Cembed-bitcode=no --cfg 'feature="default"' --cfg 'feature="std"' -C metadata=c28282e528ea8998 -C extra-filename=-c28282e528ea8998 --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/build/memchr-c28282e528ea8998 -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --cap-lints allow`
Compiling serde_derive v1.0.130
Running `rustc --crate-name build_script_build /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/serde_derive-1.0.130/build.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type bin --emit=dep-info,link -C opt-level=3 -Cembed-bitcode=no --cfg 'feature="default"' -C metadata=3b9d8f340a2663a5 -C extra-filename=-3b9d8f340a2663a5 --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/build/serde_derive-3b9d8f340a2663a5 -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --cap-lints allow`
Compiling scopeguard v1.1.0
Running `rustc --crate-name scopeguard /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/scopeguard-1.1.0/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -Cembed-bitcode=no -C metadata=3c7ae9dd20140962 -C extra-filename=-3c7ae9dd20140962 --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --cap-lints allow`
Compiling crossbeam-utils v0.8.5
Running `rustc --crate-name build_script_build --edition=2018 /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/crossbeam-utils-0.8.5/build.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type bin --emit=dep-info,link -C opt-level=3 -Cembed-bitcode=no --cfg 'feature="default"' --cfg 'feature="lazy_static"' --cfg 'feature="std"' -C metadata=7ca04d908bb2896d -C extra-filename=-7ca04d908bb2896d --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/build/crossbeam-utils-7ca04d908bb2896d -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --cap-lints allow`
Compiling serde v1.0.130
Running `rustc --crate-name build_script_build /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-1.0.130/build.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type bin --emit=dep-info,link -C opt-level=3 -Cembed-bitcode=no --cfg 'feature="default"' --cfg 'feature="derive"' --cfg 'feature="serde_derive"' --cfg 'feature="std"' -C metadata=33c06ec10529aebe -C extra-filename=-33c06ec10529aebe --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/build/serde-33c06ec10529aebe -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --cap-lints allow`
Compiling regex-syntax v0.6.25
Running `rustc --crate-name regex_syntax --edition=2018 /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/regex-syntax-0.6.25/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -Cembed-bitcode=no --cfg 'feature="default"' --cfg 'feature="unicode"' --cfg 'feature="unicode-age"' --cfg 'feature="unicode-bool"' --cfg 'feature="unicode-case"' --cfg 'feature="unicode-gencat"' --cfg 'feature="unicode-perl"' --cfg 'feature="unicode-script"' --cfg 'feature="unicode-segment"' -C metadata=79e1cae865a51a95 -C extra-filename=-79e1cae865a51a95 --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --cap-lints allow`
Compiling ryu v1.0.5
Running `rustc --crate-name build_script_build --edition=2018 /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/ryu-1.0.5/build.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type bin --emit=dep-info,link -C opt-level=3 -Cembed-bitcode=no -C metadata=e99f6f87a9604525 -C extra-filename=-e99f6f87a9604525 --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/build/ryu-e99f6f87a9604525 -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --cap-lints allow`
Compiling proc-macro-hack v0.5.19
Running `rustc --crate-name build_script_build --edition=2018 /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/proc-macro-hack-0.5.19/build.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type bin --emit=dep-info,link -C opt-level=3 -Cembed-bitcode=no -C metadata=235c58df4bfc2d26 -C extra-filename=-235c58df4bfc2d26 --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/build/proc-macro-hack-235c58df4bfc2d26 -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --cap-lints allow`
Compiling crossbeam-epoch v0.9.5
Running `rustc --crate-name build_script_build --edition=2018 /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/crossbeam-epoch-0.9.5/build.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type bin --emit=dep-info,link -C opt-level=3 -Cembed-bitcode=no --cfg 'feature="alloc"' --cfg 'feature="lazy_static"' --cfg 'feature="std"' -C metadata=e61162bae36a78f2 -C extra-filename=-e61162bae36a78f2 --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/build/crossbeam-epoch-e61162bae36a78f2 -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --cap-lints allow`
Compiling getrandom v0.1.16
Running `rustc --crate-name build_script_build --edition=2018 /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/getrandom-0.1.16/build.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type bin --emit=dep-info,link -C opt-level=3 -Cembed-bitcode=no --cfg 'feature="std"' -C metadata=f7f1f89d3879f12a -C extra-filename=-f7f1f89d3879f12a --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/build/getrandom-f7f1f89d3879f12a -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --cap-lints allow`
Compiling serde_json v1.0.68
Running `rustc --crate-name build_script_build --edition=2018 /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/serde_json-1.0.68/build.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type bin --emit=dep-info,link -C opt-level=3 -Cembed-bitcode=no --cfg 'feature="default"' --cfg 'feature="std"' -C metadata=b0558d82f89a6139 -C extra-filename=-b0558d82f89a6139 --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/build/serde_json-b0558d82f89a6139 -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --cap-lints allow`
Compiling rayon-core v1.9.1
Running `rustc --crate-name build_script_build --edition=2018 /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-core-1.9.1/build.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type bin --emit=dep-info,link -C opt-level=3 -Cembed-bitcode=no -C metadata=d2283c246bafe80c -C extra-filename=-d2283c246bafe80c --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/build/rayon-core-d2283c246bafe80c -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --cap-lints allow`
Compiling smallvec v1.7.0
Running `rustc --crate-name smallvec --edition=2018 /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/smallvec-1.7.0/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -Cembed-bitcode=no -C metadata=80382cd50be7e711 -C extra-filename=-80382cd50be7e711 --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --cap-lints allow`
Compiling itoa v0.4.8
Running `rustc --crate-name itoa /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/itoa-0.4.8/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -Cembed-bitcode=no -C metadata=b1d5f3636cb98e06 -C extra-filename=-b1d5f3636cb98e06 --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --cap-lints allow`
Compiling cc v1.0.71
Running `rustc --crate-name cc --edition=2018 /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/cc-1.0.71/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -Cembed-bitcode=no -C metadata=2553afeeea44d791 -C extra-filename=-2553afeeea44d791 --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --cap-lints allow`
Compiling inventory v0.1.10
Running `rustc --crate-name build_script_build --edition=2018 /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/inventory-0.1.10/build.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type bin --emit=dep-info,link -C opt-level=3 -Cembed-bitcode=no -C metadata=04864ca25a92bb07 -C extra-filename=-04864ca25a92bb07 --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/build/inventory-04864ca25a92bb07 -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --cap-lints allow`
Compiling pkg-config v0.3.20
Running `rustc --crate-name pkg_config /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/pkg-config-0.3.20/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -Cembed-bitcode=no -C metadata=8f47cf49a7d07ce1 -C extra-filename=-8f47cf49a7d07ce1 --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --cap-lints allow`
Compiling unicode-width v0.1.9
Running `rustc --crate-name unicode_width /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/unicode-width-0.1.9/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -Cembed-bitcode=no --cfg 'feature="default"' -C metadata=bfedf134330d4bf0 -C extra-filename=-bfedf134330d4bf0 --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --cap-lints allow`
Compiling either v1.6.1
Running `rustc --crate-name either /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/either-1.6.1/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -Cembed-bitcode=no --cfg 'feature="default"' --cfg 'feature="use_std"' -C metadata=7a8a5a271dc7cb51 -C extra-filename=-7a8a5a271dc7cb51 --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --cap-lints allow`
Compiling erased-serde v0.3.16
Running `rustc --crate-name build_script_build --edition=2018 /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/erased-serde-0.3.16/build.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type bin --emit=dep-info,link -C opt-level=3 -Cembed-bitcode=no --cfg 'feature="default"' --cfg 'feature="std"' -C metadata=6a691a9f8dddbbe1 -C extra-filename=-6a691a9f8dddbbe1 --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/build/erased-serde-6a691a9f8dddbbe1 -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --cap-lints allow`
Compiling cfg-if v0.1.10
Running `rustc --crate-name cfg_if --edition=2018 /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/cfg-if-0.1.10/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -Cembed-bitcode=no -C metadata=33adbc7c23a934c7 -C extra-filename=-33adbc7c23a934c7 --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --cap-lints allow`
Compiling ppv-lite86 v0.2.10
Running `rustc --crate-name ppv_lite86 --edition=2018 /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/ppv-lite86-0.2.10/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -Cembed-bitcode=no --cfg 'feature="simd"' --cfg 'feature="std"' -C metadata=06ceeaec32b944ff -C extra-filename=-06ceeaec32b944ff --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --cap-lints allow`
Compiling bitflags v1.3.2
Running `rustc --crate-name bitflags --edition=2018 /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/bitflags-1.3.2/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -Cembed-bitcode=no --cfg 'feature="default"' -C metadata=07e4b438a47db117 -C extra-filename=-07e4b438a47db117 --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --cap-lints allow`
Compiling unindent v0.1.7
Running `rustc --crate-name unindent --edition=2018 /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/unindent-0.1.7/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -Cembed-bitcode=no -C metadata=ffdf58ea3b9e347d -C extra-filename=-ffdf58ea3b9e347d --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --cap-lints allow`
Compiling version_check v0.9.3
Running `rustc --crate-name version_check /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/version_check-0.9.3/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -Cembed-bitcode=no -C metadata=07b01e06a1065a79 -C extra-filename=-07b01e06a1065a79 --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --cap-lints allow`
Compiling ansi_term v0.11.0
Running `rustc --crate-name ansi_term /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/ansi_term-0.11.0/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -Cembed-bitcode=no -C metadata=7f603fb2692868e9 -C extra-filename=-7f603fb2692868e9 --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --cap-lints allow`
Compiling vec_map v0.8.2
Running `rustc --crate-name vec_map /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/vec_map-0.8.2/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -Cembed-bitcode=no -C metadata=99222fb3abba5a69 -C extra-filename=-99222fb3abba5a69 --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --cap-lints allow`
Compiling strsim v0.8.0
Running `rustc --crate-name strsim /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/strsim-0.8.0/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -Cembed-bitcode=no -C metadata=0b9af79a9eabd5cc -C extra-filename=-0b9af79a9eabd5cc --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --cap-lints allow`
Compiling number_prefix v0.3.0
Running `rustc --crate-name number_prefix /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/number_prefix-0.3.0/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -Cembed-bitcode=no --cfg 'feature="default"' --cfg 'feature="std"' -C metadata=fa210d0967aff1ff -C extra-filename=-fa210d0967aff1ff --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --cap-lints allow`
Compiling unicode_categories v0.1.1
Running `rustc --crate-name unicode_categories /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/unicode_categories-0.1.1/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -Cembed-bitcode=no -C metadata=6796d54f4a15ffb4 -C extra-filename=-6796d54f4a15ffb4 --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --cap-lints allow`
Compiling lock_api v0.3.4
Running `rustc --crate-name lock_api --edition=2018 /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/lock_api-0.3.4/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -Cembed-bitcode=no --cfg 'feature="nightly"' -C metadata=0e8ba3e8e8af44f0 -C extra-filename=-0e8ba3e8e8af44f0 --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --extern scopeguard=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps/libscopeguard-3c7ae9dd20140962.rmeta --cap-lints allow`
Running `/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/build/crossbeam-utils-7ca04d908bb2896d/build-script-build`
Running `/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/build/memchr-c28282e528ea8998/build-script-build`
Running `/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/build/serde_derive-3b9d8f340a2663a5/build-script-build`
Running `/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/build/syn-0f030b9755d18eeb/build-script-build`
Running `/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/build/libc-7a7aa92cf75f8e53/build-script-build`
Running `/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/build/serde-33c06ec10529aebe/build-script-build`
Running `/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/build/proc-macro2-138e404a51655eac/build-script-build`
Running `/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/build/ryu-e99f6f87a9604525/build-script-build`
Running `/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/build/crossbeam-epoch-e61162bae36a78f2/build-script-build`
Running `/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/build/rayon-core-d2283c246bafe80c/build-script-build`
Running `/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/build/getrandom-f7f1f89d3879f12a/build-script-build`
Running `/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/build/proc-macro-hack-235c58df4bfc2d26/build-script-build`
Compiling unicode-normalization-alignments v0.1.12
Running `rustc --crate-name unicode_normalization_alignments /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/unicode-normalization-alignments-0.1.12/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -Cembed-bitcode=no -C metadata=b83e8ec0575ef893 -C extra-filename=-b83e8ec0575ef893 --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --extern smallvec=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps/libsmallvec-80382cd50be7e711.rmeta --cap-lints allow`
Compiling textwrap v0.11.0
Running `rustc --crate-name textwrap /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/textwrap-0.11.0/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -Cembed-bitcode=no -C metadata=1e5358ff7f49d80c -C extra-filename=-1e5358ff7f49d80c --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --extern unicode_width=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps/libunicode_width-bfedf134330d4bf0.rmeta --cap-lints allow`
Running `/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/build/serde_json-b0558d82f89a6139/build-script-build`
Compiling itertools v0.8.2
Running `rustc --crate-name itertools /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/itertools-0.8.2/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -Cembed-bitcode=no --cfg 'feature="default"' --cfg 'feature="use_std"' -C metadata=ea6969d200b46984 -C extra-filename=-ea6969d200b46984 --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --extern either=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps/libeither-7a8a5a271dc7cb51.rmeta --cap-lints allow`
Compiling memoffset v0.6.4
Running `rustc --crate-name build_script_build /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/memoffset-0.6.4/build.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type bin --emit=dep-info,link -C opt-level=3 -Cembed-bitcode=no --cfg 'feature="default"' -C metadata=83977b95891b55dc -C extra-filename=-83977b95891b55dc --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/build/memoffset-83977b95891b55dc -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --extern autocfg=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps/libautocfg-4b64b452f871448c.rlib --cap-lints allow`
Compiling rayon v1.5.1
Running `rustc --crate-name build_script_build --edition=2018 /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-1.5.1/build.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type bin --emit=dep-info,link -C opt-level=3 -Cembed-bitcode=no -C metadata=6090863c6b754e05 -C extra-filename=-6090863c6b754e05 --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/build/rayon-6090863c6b754e05 -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --extern autocfg=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps/libautocfg-4b64b452f871448c.rlib --cap-lints allow`
Compiling num-traits v0.2.14
Running `rustc --crate-name build_script_build /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/num-traits-0.2.14/build.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type bin --emit=dep-info,link -C opt-level=3 -Cembed-bitcode=no --cfg 'feature="default"' --cfg 'feature="std"' -C metadata=8629b96a01b4610e -C extra-filename=-8629b96a01b4610e --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/build/num-traits-8629b96a01b4610e -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --extern autocfg=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps/libautocfg-4b64b452f871448c.rlib --cap-lints allow`
Running `/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/build/inventory-04864ca25a92bb07/build-script-build`
Running `/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/build/erased-serde-6a691a9f8dddbbe1/build-script-build`
Running `rustc --crate-name crossbeam_utils --edition=2018 /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/crossbeam-utils-0.8.5/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -Cembed-bitcode=no --cfg 'feature="default"' --cfg 'feature="lazy_static"' --cfg 'feature="std"' -C metadata=83a8773ded6f4747 -C extra-filename=-83a8773ded6f4747 --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --extern cfg_if=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps/libcfg_if-2deae1b316fdaf6f.rmeta --extern lazy_static=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps/liblazy_static-79cb8478fcd4b466.rmeta --cap-lints allow`
Running `rustc --crate-name memchr --edition=2018 /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/memchr-2.4.1/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -Cembed-bitcode=no --cfg 'feature="default"' --cfg 'feature="std"' -C metadata=671cb2e32aa232f7 -C extra-filename=-671cb2e32aa232f7 --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --cap-lints allow --cfg memchr_runtime_simd --cfg memchr_runtime_sse2 --cfg memchr_runtime_sse42 --cfg memchr_runtime_avx`
Running `rustc --crate-name libc /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/libc-0.2.103/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -Cembed-bitcode=no --cfg 'feature="default"' --cfg 'feature="std"' -C metadata=f2cbad5717f42b19 -C extra-filename=-f2cbad5717f42b19 --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --cap-lints allow --cfg freebsd11 --cfg libc_priv_mod_use --cfg libc_union --cfg libc_const_size_of --cfg libc_align --cfg libc_core_cvoid --cfg libc_packedN --cfg libc_cfg_target_vendor`
Running `rustc --crate-name proc_macro2 --edition=2018 /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/proc-macro2-1.0.29/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -Cembed-bitcode=no --cfg 'feature="default"' --cfg 'feature="proc-macro"' -C metadata=0ed316da2ce5ab27 -C extra-filename=-0ed316da2ce5ab27 --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --extern unicode_xid=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps/libunicode_xid-5169222e9a9224f7.rmeta --cap-lints allow --cfg lexerror_display --cfg hygiene --cfg use_proc_macro --cfg wrap_proc_macro --cfg proc_macro_span`
Running `rustc --crate-name ryu --edition=2018 /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/ryu-1.0.5/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -Cembed-bitcode=no -C metadata=0ddefef4682d0eba -C extra-filename=-0ddefef4682d0eba --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --cap-lints allow --cfg integer128 --cfg maybe_uninit`
Running `rustc --crate-name proc_macro_hack --edition=2018 /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/proc-macro-hack-0.5.19/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type proc-macro --emit=dep-info,link -C prefer-dynamic -C opt-level=3 -Cembed-bitcode=no -C metadata=940fbea956d42b7d -C extra-filename=-940fbea956d42b7d --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --extern proc_macro --cap-lints allow`
Running `/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/build/memoffset-83977b95891b55dc/build-script-build`
Running `/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/build/num-traits-8629b96a01b4610e/build-script-build`
Running `/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/build/rayon-6090863c6b754e05/build-script-build`
Compiling crossbeam-channel v0.5.1
Running `rustc --crate-name crossbeam_channel --edition=2018 /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/crossbeam-channel-0.5.1/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -Cembed-bitcode=no --cfg 'feature="crossbeam-utils"' --cfg 'feature="default"' --cfg 'feature="std"' -C metadata=d31f5ff11a3117cc -C extra-filename=-d31f5ff11a3117cc --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --extern cfg_if=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps/libcfg_if-2deae1b316fdaf6f.rmeta --extern crossbeam_utils=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps/libcrossbeam_utils-83a8773ded6f4747.rmeta --cap-lints allow`
Compiling quote v1.0.10
Running `rustc --crate-name quote --edition=2018 /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/quote-1.0.10/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -Cembed-bitcode=no --cfg 'feature="default"' --cfg 'feature="proc-macro"' -C metadata=e8f54a27eef7fb1b -C extra-filename=-e8f54a27eef7fb1b --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --extern proc_macro2=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps/libproc_macro2-0ed316da2ce5ab27.rmeta --cap-lints allow`
Compiling aho-corasick v0.7.18
Running `rustc --crate-name aho_corasick --edition=2018 /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/aho-corasick-0.7.18/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -Cembed-bitcode=no --cfg 'feature="default"' --cfg 'feature="std"' -C metadata=157f3d95428c1664 -C extra-filename=-157f3d95428c1664 --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --extern memchr=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps/libmemchr-671cb2e32aa232f7.rmeta --cap-lints allow`
Compiling num_cpus v1.13.0
Running `rustc --crate-name num_cpus /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/num_cpus-1.13.0/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -Cembed-bitcode=no -C metadata=3bb11ba5db4aee05 -C extra-filename=-3bb11ba5db4aee05 --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --extern libc=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps/liblibc-f2cbad5717f42b19.rmeta --cap-lints allow`
Running `rustc --crate-name getrandom --edition=2018 /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/getrandom-0.1.16/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -Cembed-bitcode=no --cfg 'feature="std"' -C metadata=019c834045ae5d3d -C extra-filename=-019c834045ae5d3d --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --extern cfg_if=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps/libcfg_if-2deae1b316fdaf6f.rmeta --extern libc=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps/liblibc-f2cbad5717f42b19.rmeta --cap-lints allow`
Compiling terminal_size v0.1.17
Running `rustc --crate-name terminal_size --edition=2018 /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/terminal_size-0.1.17/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -Cembed-bitcode=no -C metadata=8a288c71da52ce3a -C extra-filename=-8a288c71da52ce3a --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --extern libc=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps/liblibc-f2cbad5717f42b19.rmeta --cap-lints allow`
Compiling atty v0.2.14
Running `rustc --crate-name atty /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/atty-0.2.14/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -Cembed-bitcode=no -C metadata=effa741dd08e4083 -C extra-filename=-effa741dd08e4083 --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --extern libc=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps/liblibc-f2cbad5717f42b19.rmeta --cap-lints allow`
Compiling parking_lot_core v0.7.2
Running `rustc --crate-name parking_lot_core --edition=2018 /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/parking_lot_core-0.7.2/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -Cembed-bitcode=no --cfg 'feature="nightly"' -C metadata=c34fe336db83d3aa -C extra-filename=-c34fe336db83d3aa --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --extern cfg_if=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps/libcfg_if-33adbc7c23a934c7.rmeta --extern libc=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps/liblibc-f2cbad5717f42b19.rmeta --extern smallvec=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps/libsmallvec-80382cd50be7e711.rmeta --cap-lints allow`
Running `rustc --crate-name memoffset /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/memoffset-0.6.4/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -Cembed-bitcode=no --cfg 'feature="default"' -C metadata=baf75535ad0aa706 -C extra-filename=-baf75535ad0aa706 --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --cap-lints allow --cfg tuple_ty --cfg allow_clippy --cfg maybe_uninit --cfg doctests`
Running `rustc --crate-name num_traits /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/num-traits-0.2.14/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -Cembed-bitcode=no --cfg 'feature="default"' --cfg 'feature="std"' -C metadata=c8c722b05cb19780 -C extra-filename=-c8c722b05cb19780 --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --cap-lints allow --cfg has_i128 --cfg has_to_int_unchecked`
Running `rustc --crate-name syn --edition=2018 /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/syn-1.0.80/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -Cembed-bitcode=no --cfg 'feature="clone-impls"' --cfg 'feature="default"' --cfg 'feature="derive"' --cfg 'feature="extra-traits"' --cfg 'feature="full"' --cfg 'feature="parsing"' --cfg 'feature="printing"' --cfg 'feature="proc-macro"' --cfg 'feature="quote"' -C metadata=560b96560a31ee7a -C extra-filename=-560b96560a31ee7a --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --extern proc_macro2=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps/libproc_macro2-0ed316da2ce5ab27.rmeta --extern quote=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps/libquote-e8f54a27eef7fb1b.rmeta --extern unicode_xid=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps/libunicode_xid-5169222e9a9224f7.rmeta --cap-lints allow`
Compiling regex v1.5.4
Running `rustc --crate-name regex --edition=2018 /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/regex-1.5.4/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -Cembed-bitcode=no --cfg 'feature="aho-corasick"' --cfg 'feature="default"' --cfg 'feature="memchr"' --cfg 'feature="perf"' --cfg 'feature="perf-cache"' --cfg 'feature="perf-dfa"' --cfg 'feature="perf-inline"' --cfg 'feature="perf-literal"' --cfg 'feature="std"' --cfg 'feature="unicode"' --cfg 'feature="unicode-age"' --cfg 'feature="unicode-bool"' --cfg 'feature="unicode-case"' --cfg 'feature="unicode-gencat"' --cfg 'feature="unicode-perl"' --cfg 'feature="unicode-script"' --cfg 'feature="unicode-segment"' -C metadata=1e333a3fef530f25 -C extra-filename=-1e333a3fef530f25 --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --extern aho_corasick=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps/libaho_corasick-157f3d95428c1664.rmeta --extern memchr=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps/libmemchr-671cb2e32aa232f7.rmeta --extern regex_syntax=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps/libregex_syntax-79e1cae865a51a95.rmeta --cap-lints allow`
Compiling rand_core v0.5.1
Running `rustc --crate-name rand_core --edition=2018 /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/rand_core-0.5.1/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -Cembed-bitcode=no --cfg 'feature="alloc"' --cfg 'feature="getrandom"' --cfg 'feature="std"' -C metadata=b1a17f18772127c2 -C extra-filename=-b1a17f18772127c2 --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --extern getrandom=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps/libgetrandom-019c834045ae5d3d.rmeta --cap-lints allow`
Compiling paste-impl v0.1.18
Running `rustc --crate-name paste_impl --edition=2018 /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/paste-impl-0.1.18/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type proc-macro --emit=dep-info,link -C prefer-dynamic -C opt-level=3 -Cembed-bitcode=no -C metadata=7047fcfa649dd5ba -C extra-filename=-7047fcfa649dd5ba --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --extern proc_macro_hack=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps/libproc_macro_hack-940fbea956d42b7d.so --extern proc_macro --cap-lints allow`
Compiling clap v2.33.3
Running `rustc --crate-name clap /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/clap-2.33.3/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -Cembed-bitcode=no --cfg 'feature="ansi_term"' --cfg 'feature="atty"' --cfg 'feature="color"' --cfg 'feature="default"' --cfg 'feature="strsim"' --cfg 'feature="suggestions"' --cfg 'feature="vec_map"' -C metadata=41fb356e8e94df38 -C extra-filename=-41fb356e8e94df38 --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --extern ansi_term=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps/libansi_term-7f603fb2692868e9.rmeta --extern atty=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps/libatty-effa741dd08e4083.rmeta --extern bitflags=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps/libbitflags-07e4b438a47db117.rmeta --extern strsim=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps/libstrsim-0b9af79a9eabd5cc.rmeta --extern textwrap=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps/libtextwrap-1e5358ff7f49d80c.rmeta --extern unicode_width=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps/libunicode_width-bfedf134330d4bf0.rmeta --extern vec_map=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps/libvec_map-99222fb3abba5a69.rmeta --cap-lints allow`
Running `rustc --crate-name crossbeam_epoch --edition=2018 /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/crossbeam-epoch-0.9.5/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -Cembed-bitcode=no --cfg 'feature="alloc"' --cfg 'feature="lazy_static"' --cfg 'feature="std"' -C metadata=912afbe85c56463d -C extra-filename=-912afbe85c56463d --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --extern cfg_if=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps/libcfg_if-2deae1b316fdaf6f.rmeta --extern crossbeam_utils=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps/libcrossbeam_utils-83a8773ded6f4747.rmeta --extern lazy_static=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps/liblazy_static-79cb8478fcd4b466.rmeta --extern memoffset=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps/libmemoffset-baf75535ad0aa706.rmeta --extern scopeguard=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps/libscopeguard-3c7ae9dd20140962.rmeta --cap-lints allow`
error[E0658]: `if` is not allowed in a `const fn`
--> /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/clap-2.33.3/src/app/settings.rs:7:1
|
7 | / bitflags! {
8 | | struct Flags: u64 {
9 | | const SC_NEGATE_REQS = 1;
10 | | const SC_REQUIRED = 1 << 1;
... |
51 | | }
52 | | }
| |_^
|
= note: see issue #49146 <https://github.com/rust-lang/rust/issues/49146> for more information
= help: add `#![feature(const_if_match)]` to the crate attributes to enable
= note: this error originates in a macro (in Nightly builds, run with -Z macro-backtrace for more info)
error[E0658]: `if` is not allowed in a `const fn`
--> /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/clap-2.33.3/src/args/settings.rs:6:1
|
6 | / bitflags! {
7 | | struct Flags: u32 {
8 | | const REQUIRED = 1;
9 | | const MULTIPLE = 1 << 1;
... |
28 | | }
29 | | }
| |_^
|
= note: see issue #49146 <https://github.com/rust-lang/rust/issues/49146> for more information
= help: add `#![feature(const_if_match)]` to the crate attributes to enable
= note: this error originates in a macro (in Nightly builds, run with -Z macro-backtrace for more info)
Compiling parking_lot v0.10.2
Running `rustc --crate-name parking_lot --edition=2018 /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/parking_lot-0.10.2/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -Cembed-bitcode=no --cfg 'feature="default"' --cfg 'feature="nightly"' -C metadata=08d9d8be478c0597 -C extra-filename=-08d9d8be478c0597 --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --extern lock_api=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps/liblock_api-0e8ba3e8e8af44f0.rmeta --extern parking_lot_core=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps/libparking_lot_core-c34fe336db83d3aa.rmeta --cap-lints allow`
Compiling rand_chacha v0.2.2
Running `rustc --crate-name rand_chacha --edition=2018 /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/rand_chacha-0.2.2/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -Cembed-bitcode=no --cfg 'feature="std"' -C metadata=c3f928b028b16586 -C extra-filename=-c3f928b028b16586 --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --extern ppv_lite86=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps/libppv_lite86-06ceeaec32b944ff.rmeta --extern rand_core=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps/librand_core-b1a17f18772127c2.rmeta --cap-lints allow`
error: aborting due to 2 previous errors
For more information about this error, try `rustc --explain E0658`.
error: could not compile `clap`.
Caused by:
process didn't exit successfully: `rustc --crate-name clap /home/nayana/.cargo/registry/src/github.com-1ecc6299db9ec823/clap-2.33.3/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -Cembed-bitcode=no --cfg 'feature="ansi_term"' --cfg 'feature="atty"' --cfg 'feature="color"' --cfg 'feature="default"' --cfg 'feature="strsim"' --cfg 'feature="suggestions"' --cfg 'feature="vec_map"' -C metadata=41fb356e8e94df38 -C extra-filename=-41fb356e8e94df38 --out-dir /tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps -L dependency=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps --extern ansi_term=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps/libansi_term-7f603fb2692868e9.rmeta --extern atty=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps/libatty-effa741dd08e4083.rmeta --extern bitflags=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps/libbitflags-07e4b438a47db117.rmeta --extern strsim=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps/libstrsim-0b9af79a9eabd5cc.rmeta --extern textwrap=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps/libtextwrap-1e5358ff7f49d80c.rmeta --extern unicode_width=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps/libunicode_width-bfedf134330d4bf0.rmeta --extern vec_map=/tmp/pip-install-r4ey_3t6/tokenizers_52b8a3ad36a540c7a66f546f81a2587b/target/release/deps/libvec_map-99222fb3abba5a69.rmeta --cap-lints allow` (exit code: 1)
warning: build failed, waiting for other jobs to finish...
error: build failed
error: cargo failed with code: 101
----------------------------------------
ERROR: Failed building wheel for tokenizers
Successfully built Questgen
Failed to build tokenizers
ERROR: Could not build wheels for tokenizers which use PEP 517 and cannot be installed directly
Hi, I noticed that you said you use OpenAI APIs to build the questgen,( https://community.openai.com/t/questgen-ai-text-to-classroom-quizzes/165612)
So, i wonder if you build questgen.ai use this repo or just use OpenAI APIs .
Hi Team,
Questgen is very useful library for question generation.
We are able to use it well on CPU, but it consumes high runtime on large datasets.
Does Questgen also gives GPU support?
Hello,
This project is really interesting!
Could you share which datasets are used for training? I didn't find it in the code.
Thanks in advance!
from Questgen import main
. I think it might be an issue with nltk trying to download all the files at once.LookupError Traceback (most recent call last)
/usr/local/lib/python3.7/dist-packages/nltk/corpus/util.py in __load(self)
79 except LookupError as e:
---> 80 try: root = nltk.data.find('{}/{}'.format(self.subdir, zip_name))
81 except LookupError: raise e
9 frames
LookupError:
Resource stopwords not found.
Please use the NLTK Downloader to obtain the resource:
import nltk
nltk.download('stopwords')
Searched in:
- '/root/nltk_data'
- '/usr/share/nltk_data'
- '/usr/local/share/nltk_data'
- '/usr/lib/nltk_data'
- '/usr/local/lib/nltk_data'
- '/usr/nltk_data'
- '/usr/lib/nltk_data'
During handling of the above exception, another exception occurred:
LookupError Traceback (most recent call last)
/usr/local/lib/python3.7/dist-packages/nltk/data.py in find(resource_name, paths)
671 sep = '*' * 70
672 resource_not_found = '\n%s\n%s\n%s\n' % (sep, msg, sep)
--> 673 raise LookupError(resource_not_found)
674
675
LookupError:
Resource stopwords not found.
Please use the NLTK Downloader to obtain the resource:
import nltk
nltk.download('stopwords')
Searched in:
- '/root/nltk_data'
- '/usr/share/nltk_data'
- '/usr/local/share/nltk_data'
- '/usr/lib/nltk_data'
- '/usr/local/lib/nltk_data'
- '/usr/nltk_data'
- '/usr/lib/nltk_data'
TypeError Traceback (most recent call last)
Cell In[16], line 1
----> 1 qg = main.QGen()
2 output = qg.predict_mcq(payload)
3 pprint (output)
File ~\anaconda3\envs\PyTorch\lib\site-packages\Questgen\main.py:46, in QGen.init(self)
44 self.device = device
45 self.model = model
---> 46 self.nlp = spacy.load('en_core_web_sm', quiet=True)
48 self.s2v = Sense2Vec().from_disk('s2v_old')
50 self.fdist = FreqDist(brown.words())
TypeError: load() got an unexpected keyword argument 'quiet'
I installed it following the requirements.txt provided in this repo, and also followed the guidelines written in the README.md file.
However, the above error occurred. Please let me know how to resolve this error.
This is an issue of spaCy 2.2.4 with Python 3.9.1: when initializing QGen()
the following error occurs:
AttribureError: 'getset_descriptor' object has no attribute 'setdefault'
Should consider upgrading to spaCy 3.0.0
Awesome library! Beautifully written and easy-to-read code and well thought API/module design.
I have one request that if a mode debug:true
in payload or logging could be introduced, then it's much neater to use in a for-loop. tqdm
progress-bar breaks with print statements like Sense2vec_distractors successful for word
@ramsrigouthamg @parthplc @Vaibhav-nn @okhiroyuki
Results observed on https://questgen.ai/ are far better than what we get locally after setting up the code for the same input text.
Could you help me understand the gap here?
Example text: "Python is a programming language. Created by Guido van Rossum and first released in 1991"
MCQ results on the URL:
Q1) Who created Python?
A) Guido van Rossum
B) John G. Kemeny
C) Tim Peters
D) Bill Joy
E) None of the above
Q2) When was Python first released?
A) 1991
B) 1993
C) 1994
Q3) What programming language was created by Guido van Rossum?
A) Python
B) Perl
C) C++
D) Java
MCQ results from local code run:
Q1) What is Python?
A) C++
B) Scripting Language
C) Design Patterns
Q2) What is the programming language for Python?
A) C++
B) Java
C) Perl
As you can see, the quality of these questions is inferior, the second question doesn’t make sense!
Is there any package for this project?
Hi,
The MCQ always generates a set of 2 questions/answers.
I have tried to play with the playload in order to pass max_question in addition to input_text but without success.
payload = {
"input_text": "<input>",
"max_question": 5
}
Thanks
error in google colab due to
TypeError Traceback (most recent call last)
in <cell line: 1>()
----> 1 qg=QGen()
/usr/local/lib/python3.10/dist-packages/Questgen/main.py in init(self)
44 self.device = device
45 self.model = model
---> 46 self.nlp = spacy.load('en_core_web_sm', quiet=True)
47
48 self.s2v = Sense2Vec().from_disk('s2v_old')
TypeError: load() got an unexpected keyword argument 'quiet'
error in google colab due to
TypeError Traceback (most recent call last)
in <cell line: 1>()
----> 1 qg=QGen()
/usr/local/lib/python3.10/dist-packages/Questgen/main.py in init(self)
44 self.device = device
45 self.model = model
---> 46 self.nlp = spacy.load('en_core_web_sm', quiet=True)
47
48 self.s2v = Sense2Vec().from_disk('s2v_old')
TypeError: load() got an unexpected keyword argument 'quiet'
Is there a way to use other languages? It seems only English is supported by the library.
Hi Ramsri,I have found your wonderful library to generate questions and answers from documents, but when I am following the colab notebook you have shared on github repo and generating FAQ questions and answers it is only generating questions with one word answer.I need questions which can have answers of more than one words which is the case on your application website.Can you help me with what changes need to be done to get answers to more than one word from questions.Thanks in advance.
Can you provide deployment code ?
I did all these:
pip install git+https://github.com/ramsrigouthamg/Questgen.ai
pip install git+https://github.com/boudinfl/pke.git@69337af9f9e72a25af6d7991eaa9869f1322dd72
python -m nltk.downloader universal_tagset
python -m spacy download en
Then when I run:
from pprint import pprint
import nltk
nltk.download('stopwords')
from Questgen import main
qe= main.BoolQGen()
payload = {
"input_text": "Sachin Ramesh Tendulkar is a former international cricketer from India and a former captain of the Indian national team. He is widely regarded as one of the greatest batsmen in the history of cricket. He is the highest run scorer of all time in International cricket."
}
output = qe.predict_boolq(payload)
pprint (output)
I get the following error:
`---------------------------------------------------------------------------
ModuleNotFoundError Traceback (most recent call last)
in
2 import nltk
3 nltk.download('stopwords')
----> 4 from Questgen import main
5 qe= main.BoolQGen()
6 payload = {
ModuleNotFoundError: No module named 'Questgen'`
Please help
Hi, thank you for this project but I am facing some issues while running the code for Boolean Question Generation:
/home/codespace/.python/current/lib/python3.10/site-packages/transformers/models/t5/tokenization_t5.py:163: FutureWarning: This tokenizer was incorrectly instantiated with a model max length of 512 which will be corrected in Transformers v5.
For now, this behavior is kept to avoid breaking backwards compatibility when padding/encoding with truncation is True
.
model_max_length
or pass max_length
when encoding/padding.model_max_length
set to your preferred value.Canceled future for execute_request message before replies were done
The Kernel crashed while executing code in the the current cell or a previous cell. Please review the code in the cell(s) to identify a possible cause of the failure. Click here for more info. View Jupyter log for further details.
When i add max_questions to the payload and feed in a number like 10. It is only able to generate 4 questions at maximum.
Did the contributors intentionally restricted it to 4, or is there any scope of increasing it in any other way?
And what is the recommended number of words for the "input_text" in MCQ generation?
Thanks
Questgen is not working. Its constantly giving error related to thincneural ....even after downloading it .
PLEASE HELP
Hello, It seems pandas is required by "Questgen/mcq/mcq.py" please add if that's required or some bug.
(venv) gdev@GDevs-iMac gen_q % python test.py
Traceback (most recent call last):
File "test.py", line 2, in
from Questgen import main
File "/Users/gdev/PycharmProjects/gen_q_2/venv/lib/python3.7/site-packages/Questgen/init.py", line 4, in
from Questgen.mcq import mcq
File "/Users/gdev/PycharmProjects/gen_q_2/venv/lib/python3.7/site-packages/Questgen/mcq/mcq.py", line 2, in
import pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)
ModuleNotFoundError: No module named 'pandas'
Thanks
Hi @ramsrigouthamg any idea why I'm getting this error? am I missing anything?
OSError: Can't load weights for 'ramsrigouthamg/t5_boolean_questions'. Make sure that:
'ramsrigouthamg/t5_boolean_questions' is a correct model identifier listed on 'https://huggingface.co/models'
or 'ramsrigouthamg/t5_boolean_questions' is the correct path to a directory containing a file named one of pytorch_model.bin, tf_model.h5, model.ckpt.
Environment: Azure databricks
Also,
qg = main.QGen()
2
output = qg.predict_mcq(payload)
3
pprint (output)
Downloading: 100%|██████████| 1.21k/1.21k [00:00<00:00, 1.06MB/s]
OSError: Can't load weights for 'Parth/result'. Make sure that:
How can I increase the number of questions to generate (MCQ)? Should I set maximum_question to 10-15?
Because it just generates 3 mcq_ questions (sometimes ONLY 1) from about 700 words paragraph.
So, If it is the actual code you're using to power your website then Why does your site questgen.ai/ generates about 10-15 questions with same paragraph but only 3 questions when generated with this provided code (programs)? Are there any anti-patterns between the website's backend generation and this repo?
Hi
Trying to finetune your "ramsrigouthamg / t5-large-paraphraser-diverse-high-quality" model presented on the HuggingFace.
With:
model = AutoModelForSeq2SeqLM.from_pretrained ("ramsrigouthamg / t5-large-paraphraser-diverse-high-quality")
I get the following error: CUDA out of memory error. Tried to allocate 20.00 MiB (GPU 0; 10.76 GiB total capacity; 4.29 GiB already allocated; 10.12 MiB free; 4.46 GiB reserved in total by PyTorch.
Although loading "t5-base", like here, as well as other checkpoints from HuggingFace (for example, the model for translation "Helsinki-NLP/opus-mt-en-ro") doesn't cause such a problem and the model is finetuned.
How could I fix it? Is it really the problem that my GPU is not powerful enough to finetune the "t5-large-paraphraser-diverse-high-quality" model? Thanks in advance for answer
Hello,
I would like to buy and use the Questgen web application. Is there any kind of API coming with its membership plan?
I have downloaded every model required while running the FAQ generator I'm getting the below-mentioned error,
\env\lib\site-packages\srsly\util.py", line 24, in force_path
raise ValueError(f"Can't read file: {location}")
ValueError: Can't read file: s2v_old\strings.json
Have you thought at all about incorporating something like this into the project?
Maybe I just need to take a deeper look at the model being used but being able to see why certain questions/answers are being generated would help me verify that the output is sound.
Hi Team,
I need help. When i run the below code it give me
qg = main.QGen()
output = qg.predict_mcq(payload)
pprint (output)
KeyError Traceback (most recent call last)
in
----> 1 qg = main.QGen()
2 output = qg.predict_mcq(payload)
3 pprint (output)
~\AppData\Roaming\Python\Python38\site-packages\Questgen\main.py in init(self)
46 self.device = device
47 self.model = model
---> 48 self.nlp = spacy.load('en_core_web_sm')
49
50 self.s2v = Sense2Vec().from_disk('s2v_old')
~\AppData\Roaming\Python\Python38\site-packages\spacy_init_.py in load(name, **overrides)
28 if depr_path not in (True, False, None):
29 deprecation_warning(Warnings.W001.format(path=depr_path))
---> 30 return util.load_model(name, **overrides)
31
32
~\AppData\Roaming\Python\Python38\site-packages\spacy\util.py in load_model(name, **overrides)
162 return load_model_from_link(name, **overrides)
163 if is_package(name): # installed as package
--> 164 return load_model_from_package(name, **overrides)
165 if Path(name).exists(): # path to model data directory
166 return load_model_from_path(Path(name), **overrides)
~\AppData\Roaming\Python\Python38\site-packages\spacy\util.py in load_model_from_package(name, **overrides)
183 """Load a model from an installed package."""
184 cls = importlib.import_module(name)
--> 185 return cls.load(**overrides)
186
187
~\AppData\Roaming\Python\Python38\site-packages\en_core_web_sm_init_.py in load(**overrides)
8
9 def load(**overrides):
---> 10 return load_model_from_init_py(file, **overrides)
~\AppData\Roaming\Python\Python38\site-packages\spacy\util.py in load_model_from_init_py(init_file, **overrides)
226 if not model_path.exists():
227 raise IOError(Errors.E052.format(path=path2str(data_path)))
--> 228 return load_model_from_path(data_path, meta, **overrides)
229
230
~\AppData\Roaming\Python\Python38\site-packages\spacy\util.py in load_model_from_path(model_path, meta, **overrides)
207 config = meta.get("pipeline_args", {}).get(name, {})
208 factory = factories.get(name, name)
--> 209 component = nlp.create_pipe(factory, config=config)
210 nlp.add_pipe(component, name=name)
211 return nlp.from_disk(model_path, exclude=disable)
~\AppData\Roaming\Python\Python38\site-packages\spacy\language.py in create_pipe(self, name, config)
298 raise KeyError(Errors.E108.format(name=name))
299 else:
--> 300 raise KeyError(Errors.E002.format(name=name))
301 factory = self.factories[name]
302 return factory(self, **config)
KeyError: "[E002] Can't find factory for 'tok2vec'. This usually happens when spaCy calls nlp.create_pipe
with a component name that's not built in - for example, when constructing the pipeline from a model's meta.json. If you're using a custom component, you can write to Language.factories['tok2vec']
or remove it from the model meta and add it via nlp.add_pipe
instead."
In readme, a diversity penalty is used.
Can you please explain it and how the model is trained?
Is this released on PyPI? If not any chance of doing so?
While trying to run the code in colab I am getting following error
ERROR: Could not find a version that satisfies the requirement numpy==1.22.0 (from questgen) (from versions: 1.3.0, 1.4.1, 1.5.0, 1.5.1, 1.6.0, 1.6.1, 1.6.2, 1.7.0, 1.7.1, 1.7.2, 1.8.0, 1.8.1, 1.8.2, 1.9.0, 1.9.1, 1.9.2, 1.9.3, 1.10.0.post2, 1.10.1, 1.10.2, 1.10.4, 1.11.0, 1.11.1, 1.11.2, 1.11.3, 1.12.0, 1.12.1, 1.13.0rc1, 1.13.0rc2, 1.13.0, 1.13.1, 1.13.3, 1.14.0rc1, 1.14.0, 1.14.1, 1.14.2, 1.14.3, 1.14.4, 1.14.5, 1.14.6, 1.15.0rc1, 1.15.0rc2, 1.15.0, 1.15.1, 1.15.2, 1.15.3, 1.15.4, 1.16.0rc1, 1.16.0rc2, 1.16.0, 1.16.1, 1.16.2, 1.16.3, 1.16.4, 1.16.5, 1.16.6, 1.17.0rc1, 1.17.0rc2, 1.17.0, 1.17.1, 1.17.2, 1.17.3, 1.17.4, 1.17.5, 1.18.0rc1, 1.18.0, 1.18.1, 1.18.2, 1.18.3, 1.18.4, 1.18.5, 1.19.0rc1, 1.19.0rc2, 1.19.0, 1.19.1, 1.19.2, 1.19.3, 1.19.4, 1.19.5, 1.20.0rc1, 1.20.0rc2, 1.20.0, 1.20.1, 1.20.2, 1.20.3, 1.21.0rc1, 1.21.0rc2, 1.21.0, 1.21.1, 1.21.2, 1.21.3, 1.21.4, 1.21.5, 1.21.6)
ERROR: No matching distribution found for numpy==1.22.0
The example below is a problem for the current "main" module when inside the def main():
def main():
qe = main.BoolQGen()
NOTE: we must import using for example:
from Questgen import main as qdmain
The from similarity.normalized_levenshtein import NormalizedLevenshtein
is giving an error that there is no module. Changing it to from strsimpy.normalized_levenshtein import NormalizedLevenshtein
worked for me.
Thank you so much for this project. I'm getting same questions every time.
Is there any evaluation metric created for this model? How can we be sure that it answers the question properly from the provided context?
Please let us know the accuracy and the evaluation metrics for Questgen model
Questgen.ai/Questgen/mcq/mcq.py
Line 209 in 66c43d3
Hi Team,
Great work. One question while trying to replicate the code in github i got an error
ERROR: No matching distribution found for spacy==2.2.4
will this code work only in 2.2.4?
i have recently upgraded my spacy to the latest version. thanks
Hello,
When I try to download the Repository, it gives the error below.
ERROR: No matching distribution found for spacy==2.2.4
It happens after the "!pip install git+https://github.com/ramsrigouthamg/Questgen.ai
" command.
Any idea?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.