The rust-bert from guillaume-be

error when attempting to build rust-bert

from a pull of the repo today then cargo run --release:

Running `target/release/convert-tensor`
thread 'main' panicked at 'assertion failed: `(left == right)`
  left: `1`,
 right: `3`: usage: target/release/convert-tensor source.npz destination.ot', src/convert-tensor.rs:19:5
stack backtrace:
   0:     0x55a56f15b0a0 - std::backtrace_rs::backtrace::libunwind::trace::h5e9d00f0cdf4f57e
                               at /rustc/2fd73fabe469357a12c2c974c140f67e7cdd76d0/library/std/src/../../backtrace/src/backtrace/libunwind.rs:90:5
   1:     0x55a56f15b0a0 - std::backtrace_rs::backtrace::trace_unsynchronized::hd5302bd66215dab9
                               at /rustc/2fd73fabe469357a12c2c974c140f67e7cdd76d0/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
   2:     0x55a56f15b0a0 - std::sys_common::backtrace::_print_fmt::ha0237cd11a34e2bf
                               at /rustc/2fd73fabe469357a12c2c974c140f67e7cdd76d0/library/std/src/sys_common/backtrace.rs:67:5
   3:     0x55a56f15b0a0 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h171d4c10df1a98ee
                               at /rustc/2fd73fabe469357a12c2c974c140f67e7cdd76d0/library/std/src/sys_common/backtrace.rs:46:22
   4:     0x55a56f17292c - core::fmt::write::h89e4288724daa3fa
                               at /rustc/2fd73fabe469357a12c2c974c140f67e7cdd76d0/library/core/src/fmt/mod.rs:1096:17
   5:     0x55a56f1592a2 - std::io::Write::write_fmt::h6d40f996e84584d9
                               at /rustc/2fd73fabe469357a12c2c974c140f67e7cdd76d0/library/std/src/io/mod.rs:1568:15
   6:     0x55a56f15cd15 - std::sys_common::backtrace::_print::h0c0b93221682afc8
                               at /rustc/2fd73fabe469357a12c2c974c140f67e7cdd76d0/library/std/src/sys_common/backtrace.rs:49:5
   7:     0x55a56f15cd15 - std::sys_common::backtrace::print::h57a9f95204c2fdd6
                               at /rustc/2fd73fabe469357a12c2c974c140f67e7cdd76d0/library/std/src/sys_common/backtrace.rs:36:9
   8:     0x55a56f15cd15 - std::panicking::default_hook::{{closure}}::h4245258b50e37e69
                               at /rustc/2fd73fabe469357a12c2c974c140f67e7cdd76d0/library/std/src/panicking.rs:208:50
   9:     0x55a56f15c873 - std::panicking::default_hook::h7b00dcc1d0944747
                               at /rustc/2fd73fabe469357a12c2c974c140f67e7cdd76d0/library/std/src/panicking.rs:225:9
  10:     0x55a56f15d4b1 - std::panicking::rust_panic_with_hook::h71e6a073d87de1f5
                               at /rustc/2fd73fabe469357a12c2c974c140f67e7cdd76d0/library/std/src/panicking.rs:591:17
  11:     0x55a56f15cff7 - std::panicking::begin_panic_handler::{{closure}}::hd549436f6bb6dbb8
                               at /rustc/2fd73fabe469357a12c2c974c140f67e7cdd76d0/library/std/src/panicking.rs:497:13
  12:     0x55a56f15b53c - std::sys_common::backtrace::__rust_end_short_backtrace::h4e5f4b72b04174c3
                               at /rustc/2fd73fabe469357a12c2c974c140f67e7cdd76d0/library/std/src/sys_common/backtrace.rs:141:18
  13:     0x55a56f15cf59 - rust_begin_unwind
                               at /rustc/2fd73fabe469357a12c2c974c140f67e7cdd76d0/library/std/src/panicking.rs:493:5
  14:     0x55a56f171d01 - core::panicking::panic_fmt::hcd56f7f635f62c74
                               at /rustc/2fd73fabe469357a12c2c974c140f67e7cdd76d0/library/core/src/panicking.rs:92:14
  15:     0x55a56f1205b8 - convert_tensor::main::h1a355cc77e4a545c
  16:     0x55a56f1213ea - std::sys_common::backtrace::__rust_begin_short_backtrace::h9b741d2a6463928b
  17:     0x55a56f1284c5 - std::rt::lang_start::{{closure}}::he2609531be6e3aa1
  18:     0x55a56f15d8d7 - core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once::h527fb2333ede305e
                               at /rustc/2fd73fabe469357a12c2c974c140f67e7cdd76d0/library/core/src/ops/function.rs:259:13
  19:     0x55a56f15d8d7 - std::panicking::try::do_call::h309d8aee8149866c
                               at /rustc/2fd73fabe469357a12c2c974c140f67e7cdd76d0/library/std/src/panicking.rs:379:40
  20:     0x55a56f15d8d7 - std::panicking::try::h75a60c31fd16bfc6
                               at /rustc/2fd73fabe469357a12c2c974c140f67e7cdd76d0/library/std/src/panicking.rs:343:19
  21:     0x55a56f15d8d7 - std::panic::catch_unwind::h1f9892423e99bc00
                               at /rustc/2fd73fabe469357a12c2c974c140f67e7cdd76d0/library/std/src/panic.rs:431:14
  22:     0x55a56f15d8d7 - std::rt::lang_start_internal::hd5b67df56ca01dae
                               at /rustc/2fd73fabe469357a12c2c974c140f67e7cdd76d0/library/std/src/rt.rs:51:25
  23:     0x55a56f120662 - main
  24:     0x7f2f26dea0b3 - __libc_start_main
  25:     0x55a56f11ee6e - _start
  26:                0x0 - <unknown>

Make all models and pipelines `Send`

Hi @mfelsche,

Over the past few weeks I have worked on making all tokenizers Send to make multithreading in the library easier. With these changes, all pipelines and models could be also Send. I can see you have worked on a fork to ensure that these pipelines are Send for use in one of your projects.

Would you be interested in contributing these changes back to this repository? Changing the TensorFunction to a Struct and the methods to access the function get_fn would be welcome additions (mfelsche@1140989 and mfelsche@8048606)

Empty response

I think this is separate to #85 so I have created a new issue. The little chatbot I am writing always gives empty blank responses after about 7-12 rounds of conversation.

Fined tuned transcript

You: Hello Holly
Holly: My name is holly. I love talking about magic and the goddess of nature, it's so special to me as a fairy.
You: So what can you tell me about magic. I am very interested in it.
Holly: I am really interested in magic.
You: That makes sense because you have magic
Holly: Magic can be very powerful. Don't you think?
You: I think it can be very powerful
Holly: Magic can do many wonderful things.
You: Can you tell me what magic can do?
Holly: I want to learn.
You: I think you already know
Holly:
You: So all quite again
Holly:

(p.s. Was surprised it knows its name and the first reply to hello was quite a good description of what the training data conversations were all about.)

This happens on my model and the default DialgoGPT one. Even when I set min_length: 2, in the configuration.

DialgoGPT transcript

You: Hello again
Holly: Hey man
You: What make you think I am a man
Holly: I meant as in what do you think the human body is like. I don't mean physical appearance.
You: Well that is an odd way to generalise, but anywyas you were right
Holly: You know what, I guess it would be a bit odd, but I'm sure it'll do well.
You: Most odd things do well
Holly: I don't know what that means but I'm sure they are good.
You: Its just the funny way nature works
Holly: Yeah... It's the good kind of weird
You: Right you get it
Holly: Aye
You: So are you going to go quite too?
Holly:
You: Yep quite
Holly:
You:

Any ideas? Or should I start tracing down the rabbit hole again?

key phrase extractions

I filled a previous issue on this but seems to not be as expected. #40. text seems to be either POSITIVE or NEGATIVE instead of actual labels. It seems to be doing Sentiment analysis instead of generating tags/labels.

[[Label { text: "POSITIVE", score: 0.9605776071548462, id: 1, sentence: 0 }], [Label { text: "NEGATIVE", score: 0.7535297274589539, id: 0, sentence: 1 }, Label { text: "POSITIVE", score: 0.2753136157989502, id: 1, sentence: 1 }], [Label { text: "POSITIVE", score: 0.9856524467468262, id: 1, sentence: 2 }]]

I have bunch of data and would like to extract key phrases similar to fast text multi label classification. https://fasttext.cc/docs/en/supervised-tutorial.html#multi-label-classification.

here is an example:

Which baking dish is best to bake a banana bread ?

should return something like

__label__baking 1.00000 __label__bananas 0.939923 __label__bread 0.592677

For this case the key phrases was baking, bananas and bread.

Initial attempt at a REST API

Thanks for this project, it's really helped me out.

So I thought I'd share my crude attempt at creating an API for this project. I'm just using it for the sentiment at the moment, but I'm planning to expand it to some other functions too. I'm posting here just in case anyone else is interested, or if anyone has any suggestions. Eg; I wonder if I'm instantiating SentimentModel in the best place to share memory usage between requests?

Maybe there's even interest in including this in the project here, if so I'd be happy to help.

https://github.com/tombh/rust-bert-api

Support for OpenAIGPTDoubleHeadsModel, GPT2DoubleHeadsModel, SequenceSummary,AdamW

Hi, I was trying to implement Hugginface's ConvAI when I realized those implementations were missing. Is it on the roadmap? With some architecture guidance, I could make a PR.

I started something locally but I had some issues with SequenceSummary and couldn't figure out how to architecture my code.

Here basically how I started architecturing this :

pub struct OpenAIGPTDoubleHeadsModel {
    transformer: OpenAiGptModel,
    lm_head: LinearNoBias,
    multiple_choice_head: SequenceSummary,
}

trait DoubleHeadModel {
 fn forward_t(
        &self,
        input_ids: &Option<Tensor>,
        attention_mask: &Option<Tensor>,
        token_type_ids: &Option<Tensor>,
        layer_past: Cache,
        position_ids: &Option<Tensor>,
        head_mask: &Option<Tensor>,
        input_embeds: &Option<Tensor>,
        mc_token_ids: &Option<Tensor>,
        labels: &Option<Tensor>,
        output_attentions: &Option<bool>,
        encoder_outputs: Option<&Tensor>,
        decoder_input_ids: &Option<Tensor>,
        train: bool,
    ) -> Result<
        (
            Tensor,
            Option<Tensor>,
            Cache,
            Option<Vec<Tensor>>,
            Option<Vec<Tensor>>,
        ),
        &'static str,
    >;}
}

#[derive(Debug, Serialize, Deserialize)]
/// # OpenAIGPTDoubleHeadsModel model configuration
/// Defines the OpenAIGPTDoubleHeadsModel model architecture (e.g. number of layers, hidden layer size, label mapping...)
pub struct OpenAIGPTDoubleHeadsConfig {
    pub attn_pdrop: Option<f64>,
    pub embd_pdrop: Option<f64>,
    pub hidden_dropout_prob: Option<f64>,
    pub afn: Option<GptActivation>,
    pub initializer_range: f64,
    pub layer_norm_epsilon: f64,
    pub n_ctx: i64,
    pub n_embd: i64,
    pub n_head: i64,
    pub n_layer: i64,
    pub n_positions: i64,
    pub num_labels: Option<i64>,
    pub num_classes: Option<i64>,
    pub output_past: Option<bool>,
    pub output_attentions: Option<bool>,
    pub output_hidden_states: Option<bool>,
    pub hidden_size: Option<i64>,
    pub resid_pdrop: Option<f64>,
    pub vocab_size: i64,
    pub summary_use_proj: Option<bool>,
    pub summary_proj_to_labels: Option<bool>,
    pub summary_first_dropout: Option<f64>,
    pub summary_last_dropout: Option<f64>,
    pub summary_type: Option<f64>,
}

impl Config<OpenAIGPTDoubleHeadsConfig> for OpenAIGPTDoubleHeadsConfig {}

impl OpenAIGPTDoubleHeadsModel {
pub fn new(p: &nn::Path, config: &OpenAIGPTDoubleHeadsConfig) -> OpenAIGPTDoubleHeadsModel {}
}

impl DoubleHeadModel for OpenAIGPTDoubleHeadsModel {
fn forward_t(
        &self,
        input_ids: &Option<Tensor>,
        attention_mask: &Option<Tensor>,
        token_type_ids: &Option<Tensor>,
        _layer_past: Cache,
        position_ids: &Option<Tensor>,
        head_mask: &Option<Tensor>,
        input_embeds: &Option<Tensor>,
        mc_token_ids: &Option<Tensor>,
        labels: &Option<Tensor>,
        output_attentions: &Option<bool>,
        _encoder_outputs: Option<&Tensor>,
        _decoder_input_ids: &Option<Tensor>,
        train: bool,
    ) -> Result<
        (
            Tensor,
            Option<Tensor>,
            Cache,
            Option<Vec<Tensor>>,
            Option<Vec<Tensor>>,
        ),
        &'static str,
    > 
}
}

RobertaForSequenceClassification doesn't have a "pooler" layer by default in HuggingFace Transformers

Hi,

While trying to implement a Distilroberta for sequence classification using your library, I noticed that BertModel::new creates a pooler by default (link to the code) when used with BERT or RoBERTa for sequence classification, while in HuggingFace Transformers they default to no pooling (link to the code) which means that your library will be looking for roberta.pooler.* weights when there aren't any.

I just wanted to make sure my understanding of what's going on is correct and if there's any way to make this pooler optional.
If not, I can provide a PR to fix this issue.

Thanks for your help

tch-rs update for PyTorch 1.9

Hi, and thanks for all the work on this amazing crate.
We're about to merge a PR updating tch-rs to use PyTorch 1.9 LaurentMazare/tch-rs#378.
As part of this change, we also took the opportunity to change the way overloaded functions are named from using an index to using the "overload-name" specified in the PyTorch api, e.g. sum1 would become sum_dim_intlist. Hopefully this will help having more consistent and more stable namings but just wanted to give a heads up before merging this in case you have some feedback about this change.
The version will be bumped to 0.5.0 as part of this as it breaks api compatibility.
Thanks

Request: Make private `model` inside ConversationManager public

I would like to discussing making the memebers of ConversationModel public. Here is why:

What I want to be able to do is take past_user_inputs and generated_responses from the conversation and regenerate history. history is the tokenized as ids representation of the history and is the actual input into the conversation model. Whereas past_user_inputs and generated_responses are string representations for output. This would allow me to trim the inputs to say N last inputs or make other adjustments.

Originally I tried something like this

prepare past_user_inputs and generated_responese that I would like to make into the context

let texts: &[&str] = ....

Replace history using this:

conversation.history = conversation_manager.model.get_tokenizer().convert_tokens_to_ids(
    conversation_manager.model.get_tokenizer().tokenize_list(texts.to_vec())
)

Unfortunalty I discovered that conversation_manager.model is private.

Would there be any negative issues with making it public or perhaps added a convenience method to recreate the inputs.

Alternatively just making encode_prompts public would suit my needs, but more general access to model would probably be more useful in the future

Chinese <-> English Translation Model

Hi, I would hope rust-bert supports zh<->en translation.

And in huggingface db :
https://huggingface.co/Helsinki-NLP/opus-mt-zh-en

So I think it is possbile right ? If that sounds right, I'll try making a PR.

How to use grpc for gpt2 generation?

The gpt2 model is defined as a mutable model, so when I'm using the Rust-Grpc server, I will always get a *mut torch_sys::C_tensor cannot be shared between threads safely error.

It's not advisable to load model for each call, what i need is just loading once, then every thread of grpc could use the model to generate text.

Is there a proper advice for that? tks.

tch::Cuda::is_available() returns False

Hi, thanks for the work put into the library it's awesome.

Using Windows 10.

I've been able to run the transformers models on CPU, but not on GPU.

I see that
tch::Cuda::is_available()
returns False.

This is despite the fact that I have Cuda installed on my system.
**nvcc --version **
returns

I've also donwloaded LibTorch and added it to my system path. Any guidance to get the model working on GPU would be appreciated.

Example of generating multiple words

Hello,

What I would like to do is generate haikus.

I read over the example code (https://github.com/guillaume-be/rust-bert/blob/master/examples/openai_gpt.rs) and I am out of my depth. Is there a way to get 100 words or ask for a different word? I know the get returns a tch::Tensor and I have been trying different function calls with little luck. I figured I would ask here as it could be a good example to add.

Thank you,
Becker

Chunking and sentence detection?

Hi there,

I work on a project that currently uses opennlp, but we're considering switching our code over to rust, and rust-bert seems like a natural fit. We use opennlp for a few things, among them chunking/shallow parsing (ie extracting noun phrases, verb phrases, etc) and sentence detection. Does rust-bert support either of those functions? I looked through the code and it seemed like they weren't supported, but I could easily have missed it.

Anyway, assuming they're not supported, we might pursue implementing them ourselves. If we went that route, do you have any advice on what we'd need to do, or even where to get started? Sorry, it's a bit of a vague question, I know - I've worked with NLP for a while, but this is pretty much my first time diving into the inner workings of it.

Support for quantization/quantized models?

Hi,

Great work on this library! It is awesome to work with!

I was hoping to get some insight on whether you had considered quantization for this library. Seeing as how rust would be used mostly for production grade systems and quantization is a production-time solution it seems like a logical next step.

Thoughts?

Porting prefix_allowed_tokens for text generation

Hi, and thanks for maintaining such an awesome package!

I was wondering whether the prefix_allowed_tokens parameter of the generate() function from HF transformers could, at least in principle, be ported to rust-bert.

In HF transformers, this parameter receives a function as an argument. This function outputs a list of allowed candidate token ids for the generation of the next token. It takes as input a list of the token ids which have been previously generated, prior to that step.

Why would that be helpful? It would make it possible to force the generation models to:

"quote" from an existing pool of candidate strings, such as wikipedia entries in this paper
"quote" consecutive tokens from a document, such as extracting phrases from a document
"quote" any token from a restricted pool of tokens, such as multiple choice options, numbers etc.
stop the generation process after a set number of sentences or paragraphs (by counting already generated sentences/paragraphs and forcing an eos token if exceeding a certain threshold)
...

A prefix_allowed_tokens parameter would make it really accessible to tweak the generation process in many ways. I'm not sure how it would be implemented, though. Not even sure whether you can pass functions as arguments in Rust. But could such a function at least be hardcoded in generation_utils or something?

thread 'main' panicked at 'assertion failed: `(left == right)`

Hi there,

I was trying to run the example codes, but I saw this error

thread 'main' panicked at 'assertion failed: (left == right)
left: 2,
right: 3: usage: /home/hongfa/rust-bert/target/debug/convert-tensor source.npz destination.ot', src/convert-tensor.rs:19:5

Any idea why?

Slower than the Python Version

Hey, I'm trying to avoid Python and therefore would love to use rust-bert instead of using the transformers directly.

Right now I'm using the GPT2 model, to generate some simple text, but I noticed that the Python+Tensorflow model is a lot faster than rust-bert and was wondering if I'm just doing something wrong.
Both are using the GPU.

Python:

from transformers import pipeline, set_seed

generator = pipeline('text-generation', model='gpt2')
set_seed(42)
generator("Hello, I'm a language model,", max_length=30, num_return_sequences=5)

Rust:

    let generate_config = GenerateConfig {
        num_return_sequences: 5,
        ..Default::default()
    };
    let mut gpt2_generator = GPT2Generator::new(generate_config)?;
    let input_context = "Hello, I'm a language model,";

    let attention_mask = None;
    let max_length = 30;
    let decoder_start_token_id = None;

    let output = gpt2_generator.generate(
        Some(vec![input_context]),
        attention_mask,
        None,
        max_length,
        decoder_start_token_id,
    );

From rough estimation the python version is twice as fast.
Did anyone do some benchmarking to compare it to the tensorflow+python version?

`Utf8Error` when generating language using GPT2Generator

Very rarely, the process panics when generating language with the GPT2Generator. I've seen this error multiple times now, they are all similar to this one:

FromUtf8Error { bytes: [118, 97, 110, 105, 108, 108, 97, 32, 102, 108, 97, 118, 111, 114, 101, 100, 32, 105, 99, 101, 32, 99, 114, 101, 97, 109, 10, 10, 49, 32, 49, 226, 129], error: Utf8Error { valid_up_to: 31, error_len: None }.

I can upload the exact code that I use, but as I use sampling, this will not always produce the error.

Here's the backtrace where the error occured (excerpt):

  13: core::result::Result<alloc::string::String, alloc::string::FromUtf8Error>::unwrap<alloc::string::String,alloc::string::FromUtf8Error>
             at /rustc/5e1a799842ba6ed4a57e91f7ab9435947482f7d8\src\libcore\result.rs:956
  14: rust_tokenizers::preprocessing::tokenizer::gpt2_tokenizer::{{impl}}::convert_tokens_to_string
             at C:\Users\Johannes\.cargo\registry\src\github.com-1ecc6299db9ec823\rust_tokenizers-2.0.4\src\preprocessing\tokenizer\gpt2_tokenizer.rs:120
  15: rust_tokenizers::preprocessing::tokenizer::base_tokenizer::Tokenizer::decode<rust_tokenizers::preprocessing::tokenizer::gpt2_tokenizer::Gpt2Tokenizer,rust_tokenizers::preprocessing::vocab::gpt2_vocab::Gpt2Vocab>
             at C:\Users\Johannes\.cargo\registry\src\github.com-1ecc6299db9ec823\rust_tokenizers-2.0.4\src\preprocessing\tokenizer\base_tokenizer.rs:109
  16: rust_bert::pipelines::generation::LanguageGenerator::generate<rust_bert::pipelines::generation::GPT2Generator,rust_bert::gpt2::gpt2::GPT2LMHeadModel,rust_tokenizers::preprocessing::vocab::gpt2_vocab::Gpt2Vocab,rust_tokenizers::preprocessing::tokenizer::gp
             at C:\Users\Johannes\.cargo\registry\src\github.com-1ecc6299db9ec823\rust-bert-0.7.0\src\pipelines\generation.rs:1231
  17: phrase_set_variations::main
             at .\src\main.rs:67

Is there something for morphological analysis?

Alike to find verbs.

How would you approach implementing the scoring of given completions in text generation?

Hi,

When generating text using beam search, there's a mechanism behind-the-scenes for scoring the candidate completions by their likelihood-ish, with one candidate coming from each beam. I think something similar is happening when using sampling and returning multiple possible completions. I was wondering whether it would be possible to perform such a scoring/ranking of custom completions, which are simply given (i.e. a Vec<String>) and have nothing to do with beam search.

Why? I'm experimenting with implementing semantic search using text generation. I know semantic search is traditionally done using a retrieve/re-rank paradigm based on separate specialized models, but I have a hunch that OpenAI is using their actual text generation models for semantic search as well. They're asking users to mention a specific GPT-3 model to use in their semantic search requests:

I think this is implemented by determining how likely each document is as a completion to the query (or a clever short prompt based on the query). The screenshot from a demo of theirs shows the following candidate paragraphs from a web page with their scores, given a query:

So what I wanted to ask is how would you approach implementing this scoring of given completion based on a prompt? Is there some method buried somewhere for obtaining the likelihood-ish of a sequence of tokens given a model and possibly a prompt? Should I try to regenerate the candidate documents token by token and check how likely the ground-truth token at that location is? I feel that would take ages, but maybe something like teacher-forcing to batch all "prefixes" and determining all next tokens at each location at once. Although I can only run on CPU, so this wouldn't help that much.

I'm really curious to hear whether you have any thoughts on this. Another possible usecase for such a feature besides semantic search might be ranking candidate speech-to-text outputs by likelihood.

Oh actually as I was wrapping this up I thought: what if by using the allowed_tokens feature you implemented recently, you could force/constrain the model to only write out the candidate documents. Then beam search in this regime would essentially give you the most likely candidate documents as completions. In the case in which you don't have other suggestions, I think this is what I'll go with. With a diversity penalty so that not all beams give you the same candidate.

WASM support

I'm creating this issue because I couldn't figure it out on my own.

Can rust-bert support WASM?

The issue I had trying to refactor this crate on my own is that the idea of resources relies heavily on a file system (which doesn't exist in WASM). I was unsuccessful in trying to refactor this crate to fetch the model from the huggingface site instead of downloading it.

I understand that the contributors of this project may not have the bandwidth to refactor rust-bert for WASM. Therefore:

How do you actually use a model after you download it?

Do you have any resource recommendations that explain how to use a trained model to generate or classify text in a language agnostic way? This would be useful knowledge for if I make my own crate. Thanks!

Python binding on rust-bert

Thank you for this awesome work. I am thinking to use Rust-Bert with my Python based project Obsei.

Basically I want to remove transformers dependency and use Rust based pipeline instead. I am noob in Rust (but have C/C++ background). I dont want to re-write whole project in Rust. So I thought what if we have python bindings for this lib.

Python binding can -

increase adoption among developers
easy for less tech savvy people to use full power of Rust based pipeline
Easy to write Python based API server (like this awesome Rust based work https://github.com/epwalsh/rust-dl-webserver)

Also my another motivation to learn Rust via actual project. Hence I am happy to contribute if you guys guide me.

I am reading following links in this directions -

Obliviously we can borrow binding code from https://github.com/guillaume-be/rust-tokenizers :)

Part of Speech Tagging

First, thanks so much for this. I'm really enjoying using these resources. I just had a question regarding part of speech tagging. As I understand it, both named entity recognition and part of speech tagging are token classification tasks. As such, their processing pipelines will be very similar. I've had success running NER with rust-bert. However, I can't quite get POS tagging to work. In the documentation, The NER example is given as:

let ner_model = NERModel::new(default::default())?;
let input = ["My name is Amy. I live in Paris.", "Paris is a city in France."];
let output = ner_model.predict(&input);

And the POS example as:

let ner_model = NERModel::new(default::default())?;
let input = ["My name is Bob];
let output = ner_model.predict(&input);

Are these identical? And if so, how/where does one specify that the label output should be a POS tag?

EOF while parsing for EnglishToGerman translation

Hello!

For default EnglishToGerman model tokenization fails with EOF while parsing

prop:Help:StartUp

I have a simple GPT2 model on
https://huggingface.co/remotejob/tweetsGPT2fi_v1/tree/main
rust_model.ot uploaded
Pls. give me some idea for a startup.

my python version working correctly

from transformers import AutoModelWithLMHead, AutoTokenizer, pipeline
model = AutoModelWithLMHead.from_pretrained("remotejob/tweetsGPT2fi_v1")
tokenizer = AutoTokenizer.from_pretrained("remotejob/tweetsGPT2fi_v1")
generator= pipeline('text-generation', model=model, tokenizer=tokenizer)
res = generator("Kuka sei")
print(res)

Support for Apple Silicon (M1) - error: linking with `cc`

Hello 👋

I have a problem using this library on MacOS (chip version M1).

I tried on this code example (but similar error for the other example models):

extern crate anyhow;

use rust_bert::pipelines::question_answering::{QaInput, QuestionAnsweringModel};

fn main() -> anyhow::Result<()> {
    //    Set-up Question Answering model
    let qa_model = QuestionAnsweringModel::new(Default::default())?;

    //    Define input
    let question_1 = String::from("Where does Amy live ?");
    let context_1 = String::from("Amy lives in Amsterdam");
    let question_2 = String::from("Where does Eric live");
    let context_2 = String::from("While Amy lives in Amsterdam, Eric is in The Hague.");
    let qa_input_1 = QaInput {
        question: question_1,
        context: context_1,
    };
    let qa_input_2 = QaInput {
        question: question_2,
        context: context_2,
    };

    //    Get answer
    let answers = qa_model.predict(&[qa_input_1, qa_input_2], 1, 32);
    println!("{:?}", answers);
    Ok(())
}

Here is the error I get (error code deliberately reduced):

error: linking with `cc` failed: exit code: 1
  |
  = note: "cc" "-arch" "arm64" "-L" "/Users/jeremy/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib"

Any clue on how I can make it work? Do I have to convert a model ? I'll be glad to have a little help.

In advance, many thanks 🙏

Panic on `sequence_history.drain` conversation.rs

So I have an odd one. I am writing a small chat bot using the conversation model. All is good for the first 3 responses. I write and the bot responds in less than a second. However, always on the 4th response it takes a long time to respond and then panics at:

   9: std::panicking::default_hook                                                                                                                                         [35/22742]
             at src/libstd/panicking.rs:218
  10: std::panicking::rust_panic_with_hook
             at src/libstd/panicking.rs:486
  11: rust_begin_unwind
             at src/libstd/panicking.rs:388
  12: core::panicking::panic_fmt
  13: alloc::vec::Vec<T>::drain::start_assert_failed
             at src/libcore/../stdarch/crates/core_arch/src/macros.rs:403
  14: alloc::vec::Vec<T>::drain
             at /rustc/c367798cfd3817ca6ae908ce675d1d99242af148/src/liballoc/vec.rs:1335
  15: rust_bert::pipelines::conversation::ConversationModel::clean_padding_indices
             at ~/.cargo/registry/src/github.com-1ecc6299db9ec823/rust-bert-0.10.0/src/pipelines/conversation.rs:720
  16: rust_bert::pipelines::conversation::ConversationModel::generate_responses
             at ~/.cargo/registry/src/github.com-1ecc6299db9ec823/rust-bert-0.10.0/src/pipelines/conversation.rs:678

Which is on sequence_history.drain

fn clean_padding_indices(&self, model_output: &mut Vec<Vec<i64>>) -> Vec<(usize, usize)> {
        // In case inputs are sent as batch, this cleans the padding indices in the history for shorter outputs
        let pad_token = self.model.get_pad_id().unwrap_or(self.eos_token_id);
        let mut removed_tokens = Vec::with_capacity(model_output.len());
        for sequence_history in model_output {
            let index_end = sequence_history
                .iter()
                .rev()
                .position(|&r| r != pad_token)
                .unwrap();
            let index_start = sequence_history
                .iter()
                .position(|&r| r != pad_token)
                .unwrap();
            sequence_history.drain(sequence_history.len() - index_end + 1..); // <- PANICS HERE
            sequence_history.drain(..index_start);
            removed_tokens.push((index_start, index_end));
        }
        removed_tokens
    }

Some advice would be appreciate, can probably clone and compile myself if you want me to do any testing.

Im not sure where this is coming from and why it is always the 4th response....

Question: Device::Cuda throwing an Error

Hi, thanks for your work, again. I have a long story for you this time.

Problem

I had a project running a simple GTP-2 Generator. The CPU version ran just fine (albeit rather slow, as expected). Then I tried running the CUDA version to utilize the GPU. I got the following (probably torch related) runtime error:

TorchError { c_error: "Cannot initialize CUDA without ATen_cuda library. PyTorch splits its backend into two shared libraries: a CPU library and a CUDA library; this error has occurred because you are trying to use some CUDA functionality, but the CUDA library has not been loaded by the dynamic linker for some reason. The CUDA library MUST be loaded, EVEN IF you don\'t directly use any symbols from the CUDA library! One common culprit is a lack of -Wl,--no-as-needed in your link arguments; many dynamic linkers will delete dynamic library dependencies if you don\'t depend on any of their symbols. You can check if this has occurred by using ldd on your binary to see if there is a dependency on *_cuda.so library. (initCUDA at C:\\b\\windows\\pytorch\\aten\\src\\ATen/detail/CUDAHooksInterface.h:63)\n(no backtrace available)" }

Trying to fix it

I could not run the check as I am running Windows and do not have the ldd command available.

I realized that I didn't even have CUDA installed for my GTX-1060 on Windows, so I did that and the installation finished successfully, but the project didn't work yet. The tch crate states that the TorchLib dependency should be installed automatically in a build script. I thought maybe that's the thing that didn't work, so I installed TorchLib 1.5 manually, as advised in their README, setting PATH and TORCHLIB environment variables to the directory of the extracted library.

I noticed rust-bert uses version 0.1.6 of tch, which is only compatible with PyTorch 1.4. However, I cannot find a place to download TorchLib 1.4, not even on PyTorch.org. I also tried using tch version 1.7 in my cargo project, which also did not work.

The actual Question, finally

I know this is possibly not at all related to rust-bert itself, but if anyone of you has an idea that may help, I'd be happy if you wrote a little comment hinting me in the right direction.

Have a great weekend, all of you! :)

Hello, is it possible to try/add gpt-neo model?

There is one https://github.com/EleutherAI/gpt-neo/
I'd really like to try this one

Make crate::bert::encoder public

Would it be possible to make crate::bert::encoder public or export its components ?

This could be useful in order to implement models such as CharBERT which instantiate BertLayer and BertPooler directly.

Please consider `rustfmt` tidying

By running cargo fmt, it removes trailing spaces, missing line break at end of file, and unify the coding style. Please consider to do so.

Consider using cached-path crate in resources module

Hey @guillaume-be, I wrote a Rust version of the functionality you find in file_utils.py from transformers (and allennlp) called cached-path. This could make the resources module more robust. cached-path knows when to update a resource based on the Etag, and ensures the cache is never corrupted if a download fails for any reason. It also avoids race conditions by using file locks.

Let me know what you think.

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: TorchError

Hello,

I was playing around with this library and I am able to produce a panic. I have a large file but the last line.. really where it says ", you can access detailed integration logs. " in the last line causes a panic. removing the that bit work appears to stop the panic. somewhere around this point. I hope this helps.

My code is from the example folder:
https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=84b4c8f146e1229f2b2713677b5177da

Cargo.toml file
[dependencies]
rust-bert = ""
failure = ""

my test csv file
https://gist.github.com/sbeckeriv/0e2c387b9899e1916efd23654d660949

rustc 1.45.0-nightly (7ced01a73 2020-04-30)
A run with the backtrace set to full.

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: TorchError { c_error: "stack expects a non-empty TensorList (stack at ../aten/src/ATen/native/TensorShape.cpp:969)\nframe #0: c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) + 0x6a (0x7fc40279caaa in /home/becker/trash/bertku/target/debug/build/torch-sys-1cb99d9d0c4f3210/out/libtorch/libtorch/lib/libc10.so)\nframe #1: at::native::stack(c10::ArrayRef<at::Tensor>, long) + 0x1df (0x7fc4038bab6f in /home/becker/trash/bertku/target/debug/build/torch-sys-1cb99d9d0c4f3210/out/libtorch/libtorch/lib/libtorch_cpu.so)\nframe #2: <unknown function> + 0x1292d17 (0x7fc403c52d17 in /home/becker/trash/bertku/target/debug/build/torch-sys-1cb99d9d0c4f3210/out/libtorch/libtorch/lib/libtorch_cpu.so)\nframe #3: <unknown function> + 0x11af75d (0x7fc403b6f75d in /home/becker/trash/bertku/target/debug/build/torch-sys-1cb99d9d0c4f3210/out/libtorch/libtorch/lib/libtorch_cpu.so)\nframe #4: <unknown function> + 0x2e9adcf (0x7fc40585adcf in /home/becker/trash/bertku/target/debug/build/torch-sys-1cb99d9d0c4f3210/out/libtorch/libtorch/lib/libtorch_cpu.so)\nframe #5: <unknown function> + 0x11af75d (0x7fc403b6f75d in /home/becker/trash/bertku/target/debug/build/torch-sys-1cb99d9d0c4f3210/out/libtorch/libtorch/lib/libtorch_cpu.so)\nframe #6: <unknown function> + 0x5eb736 (0x55c0a3afa736 in target/debug/bertku)\nframe #7: <unknown function> + 0x5df985 (0x55c0a3aee985 in target/debug/bertku)\nframe #8: <unknown function> + 0x5d64f8 (0x55c0a3ae54f8 in target/debug/bertku)\nframe #9: <unknown function> + 0x5cce2c (0x55c0a3adbe2c in target/debug/bertku)\nframe #10: <unknown function> + 0x5bdb7f (0x55c0a3accb7f in target/debug/bertku)\nframe #11: <unknown function> + 0x5c4a55 (0x55c0a3ad3a55 in target/debug/bertku)\nframe #12: <unknown function> + 0x585d29 (0x55c0a3a94d29 in target/debug/bertku)\nframe #13: <unknown function> + 0x12adb5 (0x55c0a3639db5 in target/debug/bertku)\nframe #14: <unknown function> + 0x132f1e (0x55c0a3641f1e in target/debug/bertku)\nframe #15: <unknown function> + 0x12ba33 (0x55c0a363aa33 in target/debug/bertku)\nframe #16: <unknown function> + 0x1329e7 (0x55c0a36419e7 in target/debug/bertku)\nframe #17: <unknown function> + 0xdb7b9 (0x55c0a35ea7b9 in target/debug/bertku)\nframe #18: <unknown function> + 0xdaf0b (0x55c0a35e9f0b in target/debug/bertku)\nframe #19: <unknown function> + 0x6d6248 (0x55c0a3be5248 in target/debug/bertku)\nframe #20: <unknown function> + 0xdaee7 (0x55c0a35e9ee7 in target/debug/bertku)\nframe #21: <unknown function> + 0xdba4a (0x55c0a35eaa4a in target/debug/bertku)\nframe #22: __libc_start_main + 0xf3 (0x7fc40251d0b3 in /lib/x86_64-linux-gnu/libc.so.6)\nframe #23: <unknown function> + 0xd991e (0x55c0a35e891e in target/debug/bertku)\n" }

   0: failure::backtrace::internal::InternalBacktrace::new
             at /home/becker/.cargo/registry/src/github.com-1ecc6299db9ec823/failure-0.1.8/src/backtrace/internal.rs:46
   1: failure::backtrace::Backtrace::new
             at /home/becker/.cargo/registry/src/github.com-1ecc6299db9ec823/failure-0.1.8/src/backtrace/mod.rs:121
   2: <failure::error::error_impl::ErrorImpl as core::convert::From<F>>::from
             at /home/becker/.cargo/registry/src/github.com-1ecc6299db9ec823/failure-0.1.8/src/error/error_impl.rs:19
   3: <failure::error::Error as core::convert::From<F>>::from
             at /home/becker/.cargo/registry/src/github.com-1ecc6299db9ec823/failure-0.1.8/src/error/mod.rs:36
   4: tch::wrappers::tensor_fallible_generated::<impl tch::wrappers::tensor::Tensor>::f_stack
             at /home/becker/.cargo/registry/src/github.com-1ecc6299db9ec823/tch-0.1.7/src/wrappers/tensor_fallible_generated.rs:17322
   5: tch::wrappers::tensor_generated::<impl tch::wrappers::tensor::Tensor>::stack
             at /home/becker/.cargo/registry/src/github.com-1ecc6299db9ec823/tch-0.1.7/src/wrappers/tensor_generated.rs:7217
   6: rust_bert::pipelines::question_answering::QuestionAnsweringModel::predict::{{closure}}
             at /home/becker/.cargo/registry/src/github.com-1ecc6299db9ec823/rust-bert-0.7.2/src/pipelines/question_answering.rs:318
   7: tch::wrappers::tensor::no_grad
             at /home/becker/.cargo/registry/src/github.com-1ecc6299db9ec823/tch-0.1.7/src/wrappers/tensor.rs:452
   8: rust_bert::pipelines::question_answering::QuestionAnsweringModel::predict
             at /home/becker/.cargo/registry/src/github.com-1ecc6299db9ec823/rust-bert-0.7.2/src/pipelines/question_answering.rs:312
   9: bertku::main
             at src/main.rs:51
  10: std::rt::lang_start::{{closure}}
             at /home/becker/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd/rt.rs:67
  11: std::rt::lang_start_internal::{{closure}}
             at /rustc/7ced01a730e8fc1bae2f8d4369c26812c0484da4/src/libstd/rt.rs:52
      std::panicking::try::do_call
             at /rustc/7ced01a730e8fc1bae2f8d4369c26812c0484da4/src/libstd/panicking.rs:297
      std::panicking::try
             at /rustc/7ced01a730e8fc1bae2f8d4369c26812c0484da4/src/libstd/panicking.rs:274
      std::panic::catch_unwind
             at /rustc/7ced01a730e8fc1bae2f8d4369c26812c0484da4/src/libstd/panic.rs:394
      std::rt::lang_start_internal
             at /rustc/7ced01a730e8fc1bae2f8d4369c26812c0484da4/src/libstd/rt.rs:51
  12: std::rt::lang_start
             at /home/becker/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd/rt.rs:67
  13: main
  14: __libc_start_main
  15: _start
', /home/becker/.cargo/registry/src/github.com-1ecc6299db9ec823/tch-0.1.7/src/wrappers/tensor_generated.rs:7217:9
stack backtrace:
   0:     0x55c0a3be1ea4 - backtrace::backtrace::libunwind::trace::he144fab28a4aed2d
                               at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.46/src/backtrace/libunwind.rs:86
   1:     0x55c0a3be1ea4 - backtrace::backtrace::trace_unsynchronized::h54d2de49d4561d5b
                               at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.46/src/backtrace/mod.rs:66
   2:     0x55c0a3be1ea4 - std::sys_common::backtrace::_print_fmt::hc03d55f811cceef4
                               at src/libstd/sys_common/backtrace.rs:78
   3:     0x55c0a3be1ea4 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::hbc44d25334fa89cd
                               at src/libstd/sys_common/backtrace.rs:59
   4:     0x55c0a3c07bbc - core::fmt::write::hdf236390fbd68d3d
                               at src/libcore/fmt/mod.rs:1069
   5:     0x55c0a3bdbbd3 - std::io::Write::write_fmt::h4ee562ef1f300991
                               at src/libstd/io/mod.rs:1532
   6:     0x55c0a3be46d5 - std::sys_common::backtrace::_print::hc4186a5ac838159c
                               at src/libstd/sys_common/backtrace.rs:62
   7:     0x55c0a3be46d5 - std::sys_common::backtrace::print::h61fb789361bb8109
                               at src/libstd/sys_common/backtrace.rs:49
   8:     0x55c0a3be46d5 - std::panicking::default_hook::{{closure}}::hcd876594d255c932
                               at src/libstd/panicking.rs:198
   9:     0x55c0a3be4412 - std::panicking::default_hook::hfc205bc5ce834a89
                               at src/libstd/panicking.rs:218
  10:     0x55c0a3be4d32 - std::panicking::rust_panic_with_hook::h98fcd55ca23bf6e3
                               at src/libstd/panicking.rs:477
  11:     0x55c0a3be491b - rust_begin_unwind
                               at src/libstd/panicking.rs:385
  12:     0x55c0a3c065e1 - core::panicking::panic_fmt::hd101a87121fa411f
                               at src/libcore/panicking.rs:89
  13:     0x55c0a3c06403 - core::option::expect_none_failed::h4f0e89faa3179bf7
                               at src/libcore/option.rs:1272
  14:     0x55c0a3aa6e33 - core::result::Result<T,E>::unwrap::hfdcaa5d5ebd67e42
                               at /home/becker/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libcore/result.rs:1005
  15:     0x55c0a3639dca - tch::wrappers::tensor_generated::<impl tch::wrappers::tensor::Tensor>::stack::h5724a22a9145e765
                               at /home/becker/.cargo/registry/src/github.com-1ecc6299db9ec823/tch-0.1.7/src/wrappers/tensor_generated.rs:7217
  16:     0x55c0a3641f1e - rust_bert::pipelines::question_answering::QuestionAnsweringModel::predict::{{closure}}::h2f305cc79a31d6b7
                               at /home/becker/.cargo/registry/src/github.com-1ecc6299db9ec823/rust-bert-0.7.2/src/pipelines/question_answering.rs:318
  17:     0x55c0a363aa33 - tch::wrappers::tensor::no_grad::h46fe95cf89870da2
                               at /home/becker/.cargo/registry/src/github.com-1ecc6299db9ec823/tch-0.1.7/src/wrappers/tensor.rs:452
  18:     0x55c0a36419e7 - rust_bert::pipelines::question_answering::QuestionAnsweringModel::predict::hfa675df4580255af
                               at /home/becker/.cargo/registry/src/github.com-1ecc6299db9ec823/rust-bert-0.7.2/src/pipelines/question_answering.rs:312
  19:     0x55c0a35ea7b9 - bertku::main::hcf5f95fd7aa784ef
                               at src/main.rs:51
  20:     0x55c0a35e9f0b - std::rt::lang_start::{{closure}}::h29b5c750fbd354e3
                               at /home/becker/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd/rt.rs:67
  21:     0x55c0a3be5248 - std::rt::lang_start_internal::{{closure}}::h86be2e33c1b2fc02
                               at src/libstd/rt.rs:52
  22:     0x55c0a3be5248 - std::panicking::try::do_call::h9468ed432eddf5f1
                               at src/libstd/panicking.rs:297
  23:     0x55c0a3be5248 - std::panicking::try::hda2b1d9056481eed
                               at src/libstd/panicking.rs:274
  24:     0x55c0a3be5248 - std::panic::catch_unwind::h8cbb401c5be5ca4d
                               at src/libstd/panic.rs:394
  25:     0x55c0a3be5248 - std::rt::lang_start_internal::h2403699d8e6ba133
                               at src/libstd/rt.rs:51
  26:     0x55c0a35e9ee7 - std::rt::lang_start::h91413199939c6997
                               at /home/becker/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd/rt.rs:67
  27:     0x55c0a35eaa4a - main
  28:     0x7fc40251d0b3 - __libc_start_main
  29:     0x55c0a35e891e - _start
  30:                0x0 - <unknown>

Refactor Translation Languages Setup

Would it be better if we had an enum called Language that held traits named English, French, German...? In the translation function, we could then define a tuple struct that looks like the following: (Language, Language). The first element in the tuple would be the language that is given with the second element being the language that it would be translated to. We could then do a match on the tuple to get all of the correct translations and provide the implementation within the body of each of the match statements. The catching statement would probably panic. This would make it a lot easier for people to use the library as an API as they could just serialize the Language traits with a string and pass them into the translate function as a tuple. However, readability would suffer so we would need to make sure we have docs on this. I figured I might as well post this here to get some input as I can't work on this today.

multi label/tag classification

Is there an example to generate tags/labels on given sentences?

Looking at something similar to fasttext multi label classification.
https://fasttext.cc/docs/en/supervised-tutorial.html#multi-label-classification

I would also like to see examples on how to train them.

Failed to run cargo build

Hi, I'd like to build rust-bert on my Mac but failed. However, I successfully run cargo build in the tsh-rs repo.

Below is the error output, do you have any suggestion? Thanks!

Last login: Fri Mar 5 12:34:44 on ttys001 direnv: export +NPM_CONFIG_PREFIX +VIRTUAL_ENV ~PATH user@macOS: ~ (python-3.8.7) $ z bert user@macOS: ~/git/rust/rust-bert (python-3.8.7) $ cargo build Compiling torch-sys v0.3.1 Compiling rust_tokenizers v6.2.1 Compiling cached-path v0.5.0 The following warnings were emitted during compilation:

warning: clang: warning: -Wl,-rpath=/Users/user/.local/libtorch/lib: 'linker' input unused [-Wunused-command-line-argument]
warning: libtch/torch_api.cpp:142:9: error: no member named 'amp_non_finite_check_and_unscale' in namespace 'at'; did you mean 'amp_foreach_non_finite_check_and_unscale'?
warning: at::amp_non_finite_check_and_unscale(*t, *found_inf, *inf_scale);
warning: ~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
warning: amp_foreach_non_finite_check_and_unscale
warning: libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
warning: x
warning: ^
warning: /Users/user/.local/libtorch/include/ATen/Functions.h:1645:16: note: 'amp_foreach_non_finite_check_and_unscale' declared here
warning: TORCH_API void amp_foreach_non_finite_check_and_unscale(TensorList self, Tensor & found_inf, const Tensor & inv_scale);
warning: ^
warning: libtch/torch_api.cpp:487:109: error: no viable conversion from 'vectortorch::autograd::Edge' to 'bool'
warning: auto vl = torch::autograd::Engine::get_default_engine().execute(roots, grads, keep_graph, create_graph, inputs_);
warning: ^~~~~~~
warning: libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
warning: x
warning: ^
warning: /Users/user/.local/libtorch/include/torch/csrc/autograd/engine.h:275:12: note: passing argument to parameter 'accumulate_grad' here
warning: bool accumulate_grad,
warning: ^
warning: In file included from libtch/torch_api.cpp:1215:
warning: libtch/torch_api_generated.cpp.h:415:386: error: too few arguments to function call, expected 16, have 15
warning: auto outputs__ = torch::cudnn_rnn(*input, of_carray_tensor(weight_data, weight_len), weight_stride0, (weight_buf ? *weight_buf : torch::Tensor()), *hx, (cx ? *cx : torch::Tensor()), mode, hidden_size, num_layers, (bool)batch_first, dropout, (bool)train, (bool)bidirectional, torch::IntArrayRef(batch_sizes_data, batch_sizes_len), (dropout_state ? *dropout_state : torch::Tensor()));
warning: ~~~~~~~~~~~~~~~~~ ^
warning: libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
warning: x
warning: ^
warning: /Users/user/.local/libtorch/include/ATen/Functions.h:59:58: note: 'cudnn_rnn' declared here
warning: TORCH_API std::tuple<Tensor,Tensor,Tensor,Tensor,Tensor> cudnn_rnn(const Tensor & input, TensorList weight, int64_t weight_stride0, const c10::optional & weight_buf, const Tensor & hx, const c10::optional & cx, int64_t mode, int64_t hidden_size, int64_t proj_size, int64_t num_layers, bool batch_first, double dropout, bool train, bool bidirectional, IntArrayRef batch_sizes, const c10::optional & dropout_state);
warning: ^
warning: In file included from libtch/torch_api.cpp:1215:
warning: libtch/torch_api_generated.cpp.h:426:203: error: too few arguments to function call, expected 9, have 8
warning: auto outputs = torch::cudnn_rnn_flatten_weight(of_carray_tensor(weight_arr_data, weight_arr_len), weight_stride0, input_size, mode, hidden_size, num_layers, (bool)batch_first, (bool)bidirectional);
warning: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^
warning: libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
warning: x
warning: ^
warning: /Users/user/.local/libtorch/include/ATen/Functions.h:58:18: note: 'cudnn_rnn_flatten_weight' declared here
warning: TORCH_API Tensor cudnn_rnn_flatten_weight(TensorList weight_arr, int64_t weight_stride0, int64_t input_size, int64_t mode, int64_t hidden_size, int64_t proj_size, int64_t num_layers, bool batch_first, bool bidirectional);
warning: ^
warning: In file included from libtch/torch_api.cpp:1215:
warning: libtch/torch_api_generated.cpp.h:576:29: error: no member named 'fft_with_size' in namespace 'torch'
warning: auto outputs = torch::fft_with_size(*self, signal_ndim, (bool)complex_input, (bool)complex_output, (bool)inverse, torch::IntArrayRef(checked_signal_sizes_data, checked_signal_sizes_len), (bool)normalized, (bool)onesided, torch::IntArrayRef(output_sizes_data, output_sizes_len));
warning: ~~~~~~~^
warning: libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
warning: x
warning: ^
warning: In file included from libtch/torch_api.cpp:1215:
warning: libtch/torch_api_generated.cpp.h:583:29: error: no member named 'fft_with_size' in namespace 'torch'
warning: auto outputs = torch::fft_with_size(*self, signal_ndim, (bool)complex_input, (bool)complex_output, (bool)inverse, torch::IntArrayRef(checked_signal_sizes_data, checked_signal_sizes_len), normalization, (bool)onesided, torch::IntArrayRef(output_sizes_data, output_sizes_len));
warning: ~~~~~~~^
warning: libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
warning: x
warning: ^
warning: In file included from libtch/torch_api.cpp:1215:
warning: libtch/torch_api_generated.cpp.h:627:53: error: no viable conversion from 'vectortorch::Tensor' to 'const c10::List<c10::optional>'
warning: auto outputs = torch::index_put_impl(*self, of_carray_tensor(indices_data, indices_len), *values, (bool)accumulate, (bool)unsafe);
warning: ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
warning: libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
warning: x
warning: ^
warning: /Users/user/.local/libtorch/include/ATen/core/List.h:258:3: note: candidate constructor not viable: no known conversion from 'vectortorch::Tensor' to 'std::initializer_list<optional>' for 1st argument
warning: List(std::initializer_list initial_values);
warning: ^
warning: /Users/user/.local/libtorch/include/ATen/core/List.h:268:3: note: candidate constructor not viable: no known conversion from 'vectortorch::Tensor' to 'const c10::List<c10::optionalat::Tensor> &' for 1st argument
warning: List(const List&) = default;
warning: ^
warning: /Users/user/.local/libtorch/include/ATen/core/List.h:270:3: note: candidate constructor not viable: no known conversion from 'vectortorch::Tensor' to 'c10::List<c10::optionalat::Tensor> &&' for 1st argument
warning: List(List&&) noexcept;
warning: ^
warning: /Users/user/.local/libtorch/include/ATen/core/List.h:259:12: note: explicit constructor is not a candidate
warning: explicit List(ArrayRef initial_values);
warning: ^
warning: /Users/user/.local/libtorch/include/ATen/core/List.h:266:12: note: explicit constructor is not a candidate
warning: explicit List(TypePtr elementType);
warning: ^
warning: /Users/user/.local/libtorch/include/ATen/core/List.h:462:12: note: explicit constructor is not a candidate
warning: explicit List(c10::intrusive_ptrc10::detail::ListImpl&& elements);
warning: ^
warning: /Users/user/.local/libtorch/include/ATen/Functions.h:562:93: note: passing argument to parameter 'indices' here
warning: TORCH_API Tensor & index_put_impl(Tensor & self, const c10::List<c10::optional> & indices, const Tensor & values, bool accumulate=false, bool unsafe=false);
warning: ^
warning: In file included from libtch/torch_api.cpp:1215:
warning: libtch/torch_api_generated.cpp.h:750:29: error: no member named 'multinomial_alias_draw' in namespace 'torch'
warning: auto outputs = torch::multinomial_alias_draw(*J, *q, num_samples);
warning: ~~~~~~~^
warning: libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
warning: x
warning: ^
warning: In file included from libtch/torch_api.cpp:1215:
warning: libtch/torch_api_generated.cpp.h:757:29: error: no member named 'multinomial_alias_setup' in namespace 'torch'
warning: auto outputs = torch::multinomial_alias_setup(*probs);
warning: ~~~~~~~^
warning: libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
warning: x
warning: ^
warning: In file included from libtch/torch_api.cpp:1215:
warning: libtch/torch_api_generated.cpp.h:816:29: error: no member named 'qr_helper' in namespace 'torch'
warning: auto outputs = torch::qr_helper(*self, (bool)some);
warning: ~~~~~~~^
warning: libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
warning: x
warning: ^
warning: In file included from libtch/torch_api.cpp:1215:
warning: libtch/torch_api_generated.cpp.h:3487:93: error: too few arguments to function call, expected 6, have 5
warning: auto outputs = torch::elu_backward(*grad_output, *alpha, *scale, *input_scale, *output);
warning: ~~~~~~~~~~~~~~~~~~~ ^
warning: libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
warning: x
warning: ^
warning: /Users/user/.local/libtorch/include/ATen/Functions.h:1812:18: note: 'elu_backward' declared here
warning: TORCH_API Tensor elu_backward(const Tensor & grad_output, Scalar alpha, Scalar scale, Scalar input_scale, bool is_result, const Tensor & self_or_result);
warning: ^
warning: In file included from libtch/torch_api.cpp:1215:
warning: libtch/torch_api_generated.cpp.h:3494:29: error: no member named 'elu_backward_out' in namespace 'torch'
warning: auto outputs__ = torch::elu_backward_out(*grad_input, *grad_output, *alpha, *scale, *input_scale, *output);
warning: ~~~~~~~^
warning: libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
warning: x
warning: ^
warning: In file included from libtch/torch_api.cpp:1215:
warning: libtch/torch_api_generated.cpp.h:3819:29: error: no member named 'fake_quantize_per_channel_affine_backward' in namespace 'torch'
warning: auto outputs__ = torch::fake_quantize_per_channel_affine_backward(*grad, *self, *scale, *zero_point, axis, quant_min, quant_max);
warning: ~~~~~~~^
warning: libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
warning: x
warning: ^
warning: In file included from libtch/torch_api.cpp:1215:
warning: libtch/torch_api_generated.cpp.h:3833:29: error: no member named 'fake_quantize_per_tensor_affine_backward' in namespace 'torch'
warning: auto outputs__ = torch::fake_quantize_per_tensor_affine_backward(*grad, *self, scale, zero_point, quant_min, quant_max);
warning: ~~~~~~~^
warning: libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
warning: x
warning: ^
warning: In file included from libtch/torch_api.cpp:1215:
warning: libtch/torch_api_generated.cpp.h:3917:29: error: unexpected namespace name 'fft': expected expression
warning: auto outputs__ = torch::fft(*self, signal_ndim, (bool)normalized);
warning: ^
warning: libtch/torch_api_generated.cpp.h:4914:29: error: no member named 'ifft' in namespace 'torch'
warning: auto outputs__ = torch::ifft(*self, signal_ndim, (bool)normalized);
warning: ~~~~~~~^
warning: libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
warning: x
warning: ^
warning: In file included from libtch/torch_api.cpp:1215:
warning: libtch/torch_api_generated.cpp.h:4956:42: error: no viable conversion from 'vectortorch::Tensor' to 'const c10::List<c10::optional>'
warning: auto outputs__ = torch::index(*self, of_carray_tensor(indices_data, indices_len));
warning: ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
warning: libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
warning: x
warning: ^
warning: /Users/user/.local/libtorch/include/ATen/core/List.h:258:3: note: candidate constructor not viable: no known conversion from 'vectortorch::Tensor' to 'std::initializer_list<optional>' for 1st argument
warning: List(std::initializer_list initial_values);
warning: ^
warning: /Users/user/.local/libtorch/include/ATen/core/List.h:268:3: note: candidate constructor not viable: no known conversion from 'vectortorch::Tensor' to 'const c10::List<c10::optionalat::Tensor> &' for 1st argument
warning: List(const List&) = default;
warning: ^
warning: /Users/user/.local/libtorch/include/ATen/core/List.h:270:3: note: candidate constructor not viable: no known conversion from 'vectortorch::Tensor' to 'c10::List<c10::optionalat::Tensor> &&' for 1st argument
warning: List(List&&) noexcept;
warning: ^
warning: /Users/user/.local/libtorch/include/ATen/core/List.h:259:12: note: explicit constructor is not a candidate
warning: explicit List(ArrayRef initial_values);
warning: ^
warning: /Users/user/.local/libtorch/include/ATen/core/List.h:266:12: note: explicit constructor is not a candidate
warning: explicit List(TypePtr elementType);
warning: ^
warning: /Users/user/.local/libtorch/include/ATen/core/List.h:462:12: note: explicit constructor is not a candidate
warning: explicit List(c10::intrusive_ptrc10::detail::ListImpl&& elements);
warning: ^
warning: /Users/user/.local/libtorch/include/ATen/Functions.h:557:86: note: passing argument to parameter 'indices' here
warning: TORCH_API Tensor index(const Tensor & self, const c10::List<c10::optional> & indices);
warning: ^
warning: In file included from libtch/torch_api.cpp:1215:
warning: libtch/torch_api_generated.cpp.h:5019:46: error: no viable conversion from 'vectortorch::Tensor' to 'const c10::List<c10::optional>'
warning: auto outputs__ = torch::index_put(*self, of_carray_tensor(indices_data, indices_len), *values, (bool)accumulate);
warning: ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
warning: libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
warning: x
warning: ^
warning: /Users/user/.local/libtorch/include/ATen/core/List.h:258:3: note: candidate constructor not viable: no known conversion from 'vectortorch::Tensor' to 'std::initializer_list<optional>' for 1st argument
warning: List(std::initializer_list initial_values);
warning: ^
warning: /Users/user/.local/libtorch/include/ATen/core/List.h:268:3: note: candidate constructor not viable: no known conversion from 'vectortorch::Tensor' to 'const c10::List<c10::optionalat::Tensor> &' for 1st argument
warning: List(const List&) = default;
warning: ^
warning: /Users/user/.local/libtorch/include/ATen/core/List.h:270:3: note: candidate constructor not viable: no known conversion from 'vectortorch::Tensor' to 'c10::List<c10::optionalat::Tensor> &&' for 1st argument
warning: List(List&&) noexcept;
warning: ^
warning: /Users/user/.local/libtorch/include/ATen/core/List.h:259:12: note: explicit constructor is not a candidate
warning: explicit List(ArrayRef initial_values);
warning: ^
warning: /Users/user/.local/libtorch/include/ATen/core/List.h:266:12: note: explicit constructor is not a candidate
warning: explicit List(TypePtr elementType);
warning: ^
warning: /Users/user/.local/libtorch/include/ATen/core/List.h:462:12: note: explicit constructor is not a candidate
warning: explicit List(c10::intrusive_ptrc10::detail::ListImpl&& elements);
warning: ^
warning: /Users/user/.local/libtorch/include/ATen/Functions.h:561:90: note: passing argument to parameter 'indices' here
warning: TORCH_API Tensor index_put(const Tensor & self, const c10::List<c10::optional> & indices, const Tensor & values, bool accumulate=false);
warning: ^
warning: In file included from libtch/torch_api.cpp:1215:
warning: libtch/torch_api_generated.cpp.h:5026:47: error: no viable conversion from 'vectortorch::Tensor' to 'const c10::List<c10::optional>'
warning: auto outputs__ = torch::index_put_(*self, of_carray_tensor(indices_data, indices_len), *values, (bool)accumulate);
warning: ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
warning: libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
warning: x
warning: ^
warning: /Users/user/.local/libtorch/include/ATen/core/List.h:258:3: note: candidate constructor not viable: no known conversion from 'vectortorch::Tensor' to 'std::initializer_list<optional>' for 1st argument
warning: List(std::initializer_list initial_values);
warning: ^
warning: /Users/user/.local/libtorch/include/ATen/core/List.h:268:3: note: candidate constructor not viable: no known conversion from 'vectortorch::Tensor' to 'const c10::List<c10::optionalat::Tensor> &' for 1st argument
warning: List(const List&) = default;
warning: ^
warning: /Users/user/.local/libtorch/include/ATen/core/List.h:270:3: note: candidate constructor not viable: no known conversion from 'vectortorch::Tensor' to 'c10::List<c10::optionalat::Tensor> &&' for 1st argument
warning: List(List&&) noexcept;
warning: ^
warning: /Users/user/.local/libtorch/include/ATen/core/List.h:259:12: note: explicit constructor is not a candidate
warning: explicit List(ArrayRef initial_values);
warning: ^
warning: /Users/user/.local/libtorch/include/ATen/core/List.h:266:12: note: explicit constructor is not a candidate
warning: explicit List(TypePtr elementType);
warning: ^
warning: /Users/user/.local/libtorch/include/ATen/core/List.h:462:12: note: explicit constructor is not a candidate
warning: explicit List(c10::intrusive_ptrc10::detail::ListImpl&& elements);
warning: ^
warning: /Users/user/.local/libtorch/include/ATen/Functions.h:560:87: note: passing argument to parameter 'indices' here
warning: TORCH_API Tensor & index_put_(Tensor & self, const c10::List<c10::optional> & indices, const Tensor & values, bool accumulate=false);
warning: ^
warning: fatal error: too many errors emitted, stopping now [-ferror-limit=]
warning: 20 errors generated.

error: failed to run custom build command for torch-sys v0.3.1

Caused by:
process didn't exit successfully: /Users/user/git/rust/rust-bert/target/debug/build/torch-sys-b5698cc78204d67c/build-script-build (exit code: 1)
--- stdout
cargo:rerun-if-env-changed=TORCH_CUDA_VERSION
cargo:rerun-if-env-changed=LIBTORCH
cargo:rustc-link-search=native=/Users/user/.local/libtorch/lib
cargo:rerun-if-env-changed=LIBTORCH_USE_CMAKE
cargo:rerun-if-changed=libtch/torch_api.cpp
cargo:rerun-if-changed=libtch/torch_api.h
cargo:rerun-if-changed=libtch/torch_api_generated.cpp.h
cargo:rerun-if-changed=libtch/torch_api_generated.h
cargo:rerun-if-changed=libtch/stb_image_write.h
cargo:rerun-if-changed=libtch/stb_image_resize.h
cargo:rerun-if-changed=libtch/stb_image.h
cargo:rerun-if-env-changed=LIBTORCH_CXX11_ABI
TARGET = Some("x86_64-apple-darwin")
OPT_LEVEL = Some("0")
HOST = Some("x86_64-apple-darwin")
CXX_x86_64-apple-darwin = None
CXX_x86_64_apple_darwin = None
HOST_CXX = None
CXX = None
CXXFLAGS_x86_64-apple-darwin = None
CXXFLAGS_x86_64_apple_darwin = None
HOST_CXXFLAGS = None
CXXFLAGS = None
CRATE_CC_NO_DEFAULTS = None
DEBUG = Some("true")
CARGO_CFG_TARGET_FEATURE = Some("fxsr,sse,sse2,sse3,ssse3")
running: "c++" "-O0" "-ffunction-sections" "-fdata-sections" "-fPIC" "-g" "-fno-omit-frame-pointer" "-m64" "-arch" "x86_64" "-I" "/Users/user/.local/libtorch/include" "-I" "/Users/user/.local/libtorch/include/torch/csrc/api/include" "-Wl,-rpath=/Users/user/.local/libtorch/lib" "-std=c++14" "-D_GLIBCXX_USE_CXX11_ABI=1" "-o" "/Users/user/git/rust/rust-bert/target/debug/build/torch-sys-c9a36c0d279de21f/out/libtch/torch_api.o" "-c" "libtch/torch_api.cpp"
cargo:warning=clang: warning: -Wl,-rpath=/Users/user/.local/libtorch/lib: 'linker' input unused [-Wunused-command-line-argument]
cargo:warning=libtch/torch_api.cpp:142:9: error: no member named 'amp_non_finite_check_and_unscale' in namespace 'at'; did you mean 'amp_foreach_non_finite_check_and_unscale'?
cargo:warning= at::amp_non_finite_check_and_unscale(*t, *found_inf, *inf_scale);
cargo:warning= ~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
cargo:warning= amp_foreach_non_finite_check_and_unscale
cargo:warning=libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
cargo:warning= x
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/ATen/Functions.h:1645:16: note: 'amp_foreach_non_finite_check_and_unscale' declared here
cargo:warning=TORCH_API void amp_foreach_non_finite_check_and_unscale(TensorList self, Tensor & found_inf, const Tensor & inv_scale);
cargo:warning= ^
cargo:warning=libtch/torch_api.cpp:487:109: error: no viable conversion from 'vectortorch::autograd::Edge' to 'bool'
cargo:warning= auto vl = torch::autograd::Engine::get_default_engine().execute(roots, grads, keep_graph, create_graph, inputs_);
cargo:warning= ^~~~~~~
cargo:warning=libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
cargo:warning= x
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/torch/csrc/autograd/engine.h:275:12: note: passing argument to parameter 'accumulate_grad' here
cargo:warning= bool accumulate_grad,
cargo:warning= ^
cargo:warning=In file included from libtch/torch_api.cpp:1215:
cargo:warning=libtch/torch_api_generated.cpp.h:415:386: error: too few arguments to function call, expected 16, have 15
cargo:warning= auto outputs__ = torch::cudnn_rnn(*input, of_carray_tensor(weight_data, weight_len), weight_stride0, (weight_buf ? *weight_buf : torch::Tensor()), *hx, (cx ? *cx : torch::Tensor()), mode, hidden_size, num_layers, (bool)batch_first, dropout, (bool)train, (bool)bidirectional, torch::IntArrayRef(batch_sizes_data, batch_sizes_len), (dropout_state ? *dropout_state : torch::Tensor()));
cargo:warning= ~~~~~~~~~~~~~~~~~ ^
cargo:warning=libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
cargo:warning= x
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/ATen/Functions.h:59:58: note: 'cudnn_rnn' declared here
cargo:warning=TORCH_API std::tuple<Tensor,Tensor,Tensor,Tensor,Tensor> cudnn_rnn(const Tensor & input, TensorList weight, int64_t weight_stride0, const c10::optional & weight_buf, const Tensor & hx, const c10::optional & cx, int64_t mode, int64_t hidden_size, int64_t proj_size, int64_t num_layers, bool batch_first, double dropout, bool train, bool bidirectional, IntArrayRef batch_sizes, const c10::optional & dropout_state);
cargo:warning= ^
cargo:warning=In file included from libtch/torch_api.cpp:1215:
cargo:warning=libtch/torch_api_generated.cpp.h:426:203: error: too few arguments to function call, expected 9, have 8
cargo:warning= auto outputs = torch::cudnn_rnn_flatten_weight(of_carray_tensor(weight_arr_data, weight_arr_len), weight_stride0, input_size, mode, hidden_size, num_layers, (bool)batch_first, (bool)bidirectional);
cargo:warning= ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^
cargo:warning=libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
cargo:warning= x
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/ATen/Functions.h:58:18: note: 'cudnn_rnn_flatten_weight' declared here
cargo:warning=TORCH_API Tensor cudnn_rnn_flatten_weight(TensorList weight_arr, int64_t weight_stride0, int64_t input_size, int64_t mode, int64_t hidden_size, int64_t proj_size, int64_t num_layers, bool batch_first, bool bidirectional);
cargo:warning= ^
cargo:warning=In file included from libtch/torch_api.cpp:1215:
cargo:warning=libtch/torch_api_generated.cpp.h:576:29: error: no member named 'fft_with_size' in namespace 'torch'
cargo:warning= auto outputs = torch::fft_with_size(*self, signal_ndim, (bool)complex_input, (bool)complex_output, (bool)inverse, torch::IntArrayRef(checked_signal_sizes_data, checked_signal_sizes_len), (bool)normalized, (bool)onesided, torch::IntArrayRef(output_sizes_data, output_sizes_len));
cargo:warning= ~~~~~~~^
cargo:warning=libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
cargo:warning= x
cargo:warning= ^
cargo:warning=In file included from libtch/torch_api.cpp:1215:
cargo:warning=libtch/torch_api_generated.cpp.h:583:29: error: no member named 'fft_with_size' in namespace 'torch'
cargo:warning= auto outputs = torch::fft_with_size(*self, signal_ndim, (bool)complex_input, (bool)complex_output, (bool)inverse, torch::IntArrayRef(checked_signal_sizes_data, checked_signal_sizes_len), normalization, (bool)onesided, torch::IntArrayRef(output_sizes_data, output_sizes_len));
cargo:warning= ~~~~~~~^
cargo:warning=libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
cargo:warning= x
cargo:warning= ^
cargo:warning=In file included from libtch/torch_api.cpp:1215:
cargo:warning=libtch/torch_api_generated.cpp.h:627:53: error: no viable conversion from 'vectortorch::Tensor' to 'const c10::List<c10::optional>'
cargo:warning= auto outputs = torch::index_put_impl(*self, of_carray_tensor(indices_data, indices_len), *values, (bool)accumulate, (bool)unsafe);
cargo:warning= ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
cargo:warning=libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
cargo:warning= x
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/ATen/core/List.h:258:3: note: candidate constructor not viable: no known conversion from 'vectortorch::Tensor' to 'std::initializer_list<optional>' for 1st argument
cargo:warning= List(std::initializer_list initial_values);
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/ATen/core/List.h:268:3: note: candidate constructor not viable: no known conversion from 'vectortorch::Tensor' to 'const c10::List<c10::optionalat::Tensor> &' for 1st argument
cargo:warning= List(const List&) = default;
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/ATen/core/List.h:270:3: note: candidate constructor not viable: no known conversion from 'vectortorch::Tensor' to 'c10::List<c10::optionalat::Tensor> &&' for 1st argument
cargo:warning= List(List&&) noexcept;
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/ATen/core/List.h:259:12: note: explicit constructor is not a candidate
cargo:warning= explicit List(ArrayRef initial_values);
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/ATen/core/List.h:266:12: note: explicit constructor is not a candidate
cargo:warning= explicit List(TypePtr elementType);
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/ATen/core/List.h:462:12: note: explicit constructor is not a candidate
cargo:warning= explicit List(c10::intrusive_ptrc10::detail::ListImpl&& elements);
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/ATen/Functions.h:562:93: note: passing argument to parameter 'indices' here
cargo:warning=TORCH_API Tensor & index_put_impl(Tensor & self, const c10::List<c10::optional> & indices, const Tensor & values, bool accumulate=false, bool unsafe=false);
cargo:warning= ^
cargo:warning=In file included from libtch/torch_api.cpp:1215:
cargo:warning=libtch/torch_api_generated.cpp.h:750:29: error: no member named 'multinomial_alias_draw' in namespace 'torch'
cargo:warning= auto outputs = torch::multinomial_alias_draw(*J, *q, num_samples);
cargo:warning= ~~~~~~~^
cargo:warning=libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
cargo:warning= x
cargo:warning= ^
cargo:warning=In file included from libtch/torch_api.cpp:1215:
cargo:warning=libtch/torch_api_generated.cpp.h:757:29: error: no member named 'multinomial_alias_setup' in namespace 'torch'
cargo:warning= auto outputs = torch::multinomial_alias_setup(*probs);
cargo:warning= ~~~~~~~^
cargo:warning=libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
cargo:warning= x
cargo:warning= ^
cargo:warning=In file included from libtch/torch_api.cpp:1215:
cargo:warning=libtch/torch_api_generated.cpp.h:816:29: error: no member named 'qr_helper' in namespace 'torch'
cargo:warning= auto outputs = torch::qr_helper(*self, (bool)some);
cargo:warning= ~~~~~~~^
cargo:warning=libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
cargo:warning= x
cargo:warning= ^
cargo:warning=In file included from libtch/torch_api.cpp:1215:
cargo:warning=libtch/torch_api_generated.cpp.h:3487:93: error: too few arguments to function call, expected 6, have 5
cargo:warning= auto outputs = torch::elu_backward(*grad_output, *alpha, *scale, *input_scale, *output);
cargo:warning= ~~~~~~~~~~~~~~~~~~~ ^
cargo:warning=libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
cargo:warning= x
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/ATen/Functions.h:1812:18: note: 'elu_backward' declared here
cargo:warning=TORCH_API Tensor elu_backward(const Tensor & grad_output, Scalar alpha, Scalar scale, Scalar input_scale, bool is_result, const Tensor & self_or_result);
cargo:warning= ^
cargo:warning=In file included from libtch/torch_api.cpp:1215:
cargo:warning=libtch/torch_api_generated.cpp.h:3494:29: error: no member named 'elu_backward_out' in namespace 'torch'
cargo:warning= auto outputs__ = torch::elu_backward_out(*grad_input, *grad_output, *alpha, *scale, *input_scale, *output);
cargo:warning= ~~~~~~~^
cargo:warning=libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
cargo:warning= x
cargo:warning= ^
cargo:warning=In file included from libtch/torch_api.cpp:1215:
cargo:warning=libtch/torch_api_generated.cpp.h:3819:29: error: no member named 'fake_quantize_per_channel_affine_backward' in namespace 'torch'
cargo:warning= auto outputs__ = torch::fake_quantize_per_channel_affine_backward(*grad, *self, *scale, *zero_point, axis, quant_min, quant_max);
cargo:warning= ~~~~~~~^
cargo:warning=libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
cargo:warning= x
cargo:warning= ^
cargo:warning=In file included from libtch/torch_api.cpp:1215:
cargo:warning=libtch/torch_api_generated.cpp.h:3833:29: error: no member named 'fake_quantize_per_tensor_affine_backward' in namespace 'torch'
cargo:warning= auto outputs__ = torch::fake_quantize_per_tensor_affine_backward(*grad, *self, scale, zero_point, quant_min, quant_max);
cargo:warning= ~~~~~~~^
cargo:warning=libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
cargo:warning= x
cargo:warning= ^
cargo:warning=In file included from libtch/torch_api.cpp:1215:
cargo:warning=libtch/torch_api_generated.cpp.h:3917:29: error: unexpected namespace name 'fft': expected expression
cargo:warning= auto outputs__ = torch::fft(*self, signal_ndim, (bool)normalized);
cargo:warning= ^
cargo:warning=libtch/torch_api_generated.cpp.h:4914:29: error: no member named 'ifft' in namespace 'torch'
cargo:warning= auto outputs__ = torch::ifft(*self, signal_ndim, (bool)normalized);
cargo:warning= ~~~~~~~^
cargo:warning=libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
cargo:warning= x
cargo:warning= ^
cargo:warning=In file included from libtch/torch_api.cpp:1215:
cargo:warning=libtch/torch_api_generated.cpp.h:4956:42: error: no viable conversion from 'vectortorch::Tensor' to 'const c10::List<c10::optional>'
cargo:warning= auto outputs__ = torch::index(*self, of_carray_tensor(indices_data, indices_len));
cargo:warning= ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
cargo:warning=libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
cargo:warning= x
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/ATen/core/List.h:258:3: note: candidate constructor not viable: no known conversion from 'vectortorch::Tensor' to 'std::initializer_list<optional>' for 1st argument
cargo:warning= List(std::initializer_list initial_values);
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/ATen/core/List.h:268:3: note: candidate constructor not viable: no known conversion from 'vectortorch::Tensor' to 'const c10::List<c10::optionalat::Tensor> &' for 1st argument
cargo:warning= List(const List&) = default;
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/ATen/core/List.h:270:3: note: candidate constructor not viable: no known conversion from 'vectortorch::Tensor' to 'c10::List<c10::optionalat::Tensor> &&' for 1st argument
cargo:warning= List(List&&) noexcept;
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/ATen/core/List.h:259:12: note: explicit constructor is not a candidate
cargo:warning= explicit List(ArrayRef initial_values);
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/ATen/core/List.h:266:12: note: explicit constructor is not a candidate
cargo:warning= explicit List(TypePtr elementType);
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/ATen/core/List.h:462:12: note: explicit constructor is not a candidate
cargo:warning= explicit List(c10::intrusive_ptrc10::detail::ListImpl&& elements);
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/ATen/Functions.h:557:86: note: passing argument to parameter 'indices' here
cargo:warning=TORCH_API Tensor index(const Tensor & self, const c10::List<c10::optional> & indices);
cargo:warning= ^
cargo:warning=In file included from libtch/torch_api.cpp:1215:
cargo:warning=libtch/torch_api_generated.cpp.h:5019:46: error: no viable conversion from 'vectortorch::Tensor' to 'const c10::List<c10::optional>'
cargo:warning= auto outputs__ = torch::index_put(*self, of_carray_tensor(indices_data, indices_len), *values, (bool)accumulate);
cargo:warning= ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
cargo:warning=libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
cargo:warning= x
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/ATen/core/List.h:258:3: note: candidate constructor not viable: no known conversion from 'vectortorch::Tensor' to 'std::initializer_list<optional>' for 1st argument
cargo:warning= List(std::initializer_list initial_values);
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/ATen/core/List.h:268:3: note: candidate constructor not viable: no known conversion from 'vectortorch::Tensor' to 'const c10::List<c10::optionalat::Tensor> &' for 1st argument
cargo:warning= List(const List&) = default;
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/ATen/core/List.h:270:3: note: candidate constructor not viable: no known conversion from 'vectortorch::Tensor' to 'c10::List<c10::optionalat::Tensor> &&' for 1st argument
cargo:warning= List(List&&) noexcept;
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/ATen/core/List.h:259:12: note: explicit constructor is not a candidate
cargo:warning= explicit List(ArrayRef initial_values);
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/ATen/core/List.h:266:12: note: explicit constructor is not a candidate
cargo:warning= explicit List(TypePtr elementType);
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/ATen/core/List.h:462:12: note: explicit constructor is not a candidate
cargo:warning= explicit List(c10::intrusive_ptrc10::detail::ListImpl&& elements);
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/ATen/Functions.h:561:90: note: passing argument to parameter 'indices' here
cargo:warning=TORCH_API Tensor index_put(const Tensor & self, const c10::List<c10::optional> & indices, const Tensor & values, bool accumulate=false);
cargo:warning= ^
cargo:warning=In file included from libtch/torch_api.cpp:1215:
cargo:warning=libtch/torch_api_generated.cpp.h:5026:47: error: no viable conversion from 'vectortorch::Tensor' to 'const c10::List<c10::optional>'
cargo:warning= auto outputs__ = torch::index_put_(*self, of_carray_tensor(indices_data, indices_len), *values, (bool)accumulate);
cargo:warning= ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
cargo:warning=libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
cargo:warning= x
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/ATen/core/List.h:258:3: note: candidate constructor not viable: no known conversion from 'vectortorch::Tensor' to 'std::initializer_list<optional>' for 1st argument
cargo:warning= List(std::initializer_list initial_values);
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/ATen/core/List.h:268:3: note: candidate constructor not viable: no known conversion from 'vectortorch::Tensor' to 'const c10::List<c10::optionalat::Tensor> &' for 1st argument
cargo:warning= List(const List&) = default;
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/ATen/core/List.h:270:3: note: candidate constructor not viable: no known conversion from 'vectortorch::Tensor' to 'c10::List<c10::optionalat::Tensor> &&' for 1st argument
cargo:warning= List(List&&) noexcept;
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/ATen/core/List.h:259:12: note: explicit constructor is not a candidate
cargo:warning= explicit List(ArrayRef initial_values);
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/ATen/core/List.h:266:12: note: explicit constructor is not a candidate
cargo:warning= explicit List(TypePtr elementType);
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/ATen/core/List.h:462:12: note: explicit constructor is not a candidate
cargo:warning= explicit List(c10::intrusive_ptrc10::detail::ListImpl&& elements);
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/ATen/Functions.h:560:87: note: passing argument to parameter 'indices' here
cargo:warning=TORCH_API Tensor & index_put_(Tensor & self, const c10::List<c10::optional> & indices, const Tensor & values, bool accumulate=false);
cargo:warning= ^
cargo:warning=fatal error: too many errors emitted, stopping now [-ferror-limit=]
cargo:warning=20 errors generated.
exit code: 1

--- stderr

error occurred: Command "c++" "-O0" "-ffunction-sections" "-fdata-sections" "-fPIC" "-g" "-fno-omit-frame-pointer" "-m64" "-arch" "x86_64" "-I" "/Users/user/.local/libtorch/include" "-I" "/Users/user/.local/libtorch/include/torch/csrc/api/include" "-Wl,-rpath=/Users/user/.local/libtorch/lib" "-std=c++14" "-D_GLIBCXX_USE_CXX11_ABI=1" "-o" "/Users/user/git/rust/rust-bert/target/debug/build/torch-sys-c9a36c0d279de21f/out/libtch/torch_api.o" "-c" "libtch/torch_api.cpp" with args "c++" did not execute successfully (status code exit code: 1).

user@macOS: ~/git/rust/rust-bert (python-3.8.7)
$ c
user@macOS: ~/git/rust/rust-bert (python-3.8.7)
$ cd
user@macOS: ~ (python-3.8.7)
$ c
user@macOS: ~ (python-3.8.7)
$ emacs
user@macOS: ~ (python-3.8.7)
$ z -
user@macOS: ~/git/rust/rust-bert (python-3.8.7)
$ c
user@macOS: ~/git/rust/rust-bert (python-3.8.7)
$ cargo build
Compiling torch-sys v0.3.1
The following warnings were emitted during compilation:

warning: clang: warning: -Wl,-rpath=/Users/user/.local/libtorch/lib: 'linker' input unused [-Wunused-command-line-argument]
warning: libtch/torch_api.cpp:142:9: error: no member named 'amp_non_finite_check_and_unscale' in namespace 'at'; did you mean 'amp_foreach_non_finite_check_and_unscale'?
warning: at::amp_non_finite_check_and_unscale(*t, *found_inf, *inf_scale);
warning: ~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
warning: amp_foreach_non_finite_check_and_unscale
warning: libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
warning: x
warning: ^
warning: /Users/user/.local/libtorch/include/ATen/Functions.h:1645:16: note: 'amp_foreach_non_finite_check_and_unscale' declared here
warning: TORCH_API void amp_foreach_non_finite_check_and_unscale(TensorList self, Tensor & found_inf, const Tensor & inv_scale);
warning: ^
warning: libtch/torch_api.cpp:487:109: error: no viable conversion from 'vectortorch::autograd::Edge' to 'bool'
warning: auto vl = torch::autograd::Engine::get_default_engine().execute(roots, grads, keep_graph, create_graph, inputs_);
warning: ^~~~~~~
warning: libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
warning: x
warning: ^
warning: /Users/user/.local/libtorch/include/torch/csrc/autograd/engine.h:275:12: note: passing argument to parameter 'accumulate_grad' here
warning: bool accumulate_grad,
warning: ^
warning: In file included from libtch/torch_api.cpp:1215:
warning: libtch/torch_api_generated.cpp.h:415:386: error: too few arguments to function call, expected 16, have 15
warning: auto outputs__ = torch::cudnn_rnn(*input, of_carray_tensor(weight_data, weight_len), weight_stride0, (weight_buf ? *weight_buf : torch::Tensor()), *hx, (cx ? *cx : torch::Tensor()), mode, hidden_size, num_layers, (bool)batch_first, dropout, (bool)train, (bool)bidirectional, torch::IntArrayRef(batch_sizes_data, batch_sizes_len), (dropout_state ? *dropout_state : torch::Tensor()));
warning: ~~~~~~~~~~~~~~~~~ ^
warning: libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
warning: x
warning: ^
warning: /Users/user/.local/libtorch/include/ATen/Functions.h:59:58: note: 'cudnn_rnn' declared here
warning: TORCH_API std::tuple<Tensor,Tensor,Tensor,Tensor,Tensor> cudnn_rnn(const Tensor & input, TensorList weight, int64_t weight_stride0, const c10::optional & weight_buf, const Tensor & hx, const c10::optional & cx, int64_t mode, int64_t hidden_size, int64_t proj_size, int64_t num_layers, bool batch_first, double dropout, bool train, bool bidirectional, IntArrayRef batch_sizes, const c10::optional & dropout_state);
warning: ^
warning: In file included from libtch/torch_api.cpp:1215:
warning: libtch/torch_api_generated.cpp.h:426:203: error: too few arguments to function call, expected 9, have 8
warning: auto outputs = torch::cudnn_rnn_flatten_weight(of_carray_tensor(weight_arr_data, weight_arr_len), weight_stride0, input_size, mode, hidden_size, num_layers, (bool)batch_first, (bool)bidirectional);
warning: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^
warning: libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
warning: x
warning: ^
warning: /Users/user/.local/libtorch/include/ATen/Functions.h:58:18: note: 'cudnn_rnn_flatten_weight' declared here
warning: TORCH_API Tensor cudnn_rnn_flatten_weight(TensorList weight_arr, int64_t weight_stride0, int64_t input_size, int64_t mode, int64_t hidden_size, int64_t proj_size, int64_t num_layers, bool batch_first, bool bidirectional);
warning: ^
warning: In file included from libtch/torch_api.cpp:1215:
warning: libtch/torch_api_generated.cpp.h:576:29: error: no member named 'fft_with_size' in namespace 'torch'
warning: auto outputs = torch::fft_with_size(*self, signal_ndim, (bool)complex_input, (bool)complex_output, (bool)inverse, torch::IntArrayRef(checked_signal_sizes_data, checked_signal_sizes_len), (bool)normalized, (bool)onesided, torch::IntArrayRef(output_sizes_data, output_sizes_len));
warning: ~~~~~~~^
warning: libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
warning: x
warning: ^
warning: In file included from libtch/torch_api.cpp:1215:
warning: libtch/torch_api_generated.cpp.h:583:29: error: no member named 'fft_with_size' in namespace 'torch'
warning: auto outputs = torch::fft_with_size(*self, signal_ndim, (bool)complex_input, (bool)complex_output, (bool)inverse, torch::IntArrayRef(checked_signal_sizes_data, checked_signal_sizes_len), normalization, (bool)onesided, torch::IntArrayRef(output_sizes_data, output_sizes_len));
warning: ~~~~~~~^
warning: libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
warning: x
warning: ^
warning: In file included from libtch/torch_api.cpp:1215:
warning: libtch/torch_api_generated.cpp.h:627:53: error: no viable conversion from 'vectortorch::Tensor' to 'const c10::List<c10::optional>'
warning: auto outputs = torch::index_put_impl(*self, of_carray_tensor(indices_data, indices_len), *values, (bool)accumulate, (bool)unsafe);
warning: ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
warning: libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
warning: x
warning: ^
warning: /Users/user/.local/libtorch/include/ATen/core/List.h:258:3: note: candidate constructor not viable: no known conversion from 'vectortorch::Tensor' to 'std::initializer_list<optional>' for 1st argument
warning: List(std::initializer_list initial_values);
warning: ^
warning: /Users/user/.local/libtorch/include/ATen/core/List.h:268:3: note: candidate constructor not viable: no known conversion from 'vectortorch::Tensor' to 'const c10::List<c10::optionalat::Tensor> &' for 1st argument
warning: List(const List&) = default;
warning: ^
warning: /Users/user/.local/libtorch/include/ATen/core/List.h:270:3: note: candidate constructor not viable: no known conversion from 'vectortorch::Tensor' to 'c10::List<c10::optionalat::Tensor> &&' for 1st argument
warning: List(List&&) noexcept;
warning: ^
warning: /Users/user/.local/libtorch/include/ATen/core/List.h:259:12: note: explicit constructor is not a candidate
warning: explicit List(ArrayRef initial_values);
warning: ^
warning: /Users/user/.local/libtorch/include/ATen/core/List.h:266:12: note: explicit constructor is not a candidate
warning: explicit List(TypePtr elementType);
warning: ^
warning: /Users/user/.local/libtorch/include/ATen/core/List.h:462:12: note: explicit constructor is not a candidate
warning: explicit List(c10::intrusive_ptrc10::detail::ListImpl&& elements);
warning: ^
warning: /Users/user/.local/libtorch/include/ATen/Functions.h:562:93: note: passing argument to parameter 'indices' here
warning: TORCH_API Tensor & index_put_impl(Tensor & self, const c10::List<c10::optional> & indices, const Tensor & values, bool accumulate=false, bool unsafe=false);
warning: ^
warning: In file included from libtch/torch_api.cpp:1215:
warning: libtch/torch_api_generated.cpp.h:750:29: error: no member named 'multinomial_alias_draw' in namespace 'torch'
warning: auto outputs = torch::multinomial_alias_draw(*J, *q, num_samples);
warning: ~~~~~~~^
warning: libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
warning: x
warning: ^
warning: In file included from libtch/torch_api.cpp:1215:
warning: libtch/torch_api_generated.cpp.h:757:29: error: no member named 'multinomial_alias_setup' in namespace 'torch'
warning: auto outputs = torch::multinomial_alias_setup(*probs);
warning: ~~~~~~~^
warning: libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
warning: x
warning: ^
warning: In file included from libtch/torch_api.cpp:1215:
warning: libtch/torch_api_generated.cpp.h:816:29: error: no member named 'qr_helper' in namespace 'torch'
warning: auto outputs = torch::qr_helper(*self, (bool)some);
warning: ~~~~~~~^
warning: libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
warning: x
warning: ^
warning: In file included from libtch/torch_api.cpp:1215:
warning: libtch/torch_api_generated.cpp.h:3487:93: error: too few arguments to function call, expected 6, have 5
warning: auto outputs = torch::elu_backward(*grad_output, *alpha, *scale, *input_scale, *output);
warning: ~~~~~~~~~~~~~~~~~~~ ^
warning: libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
warning: x
warning: ^
warning: /Users/user/.local/libtorch/include/ATen/Functions.h:1812:18: note: 'elu_backward' declared here
warning: TORCH_API Tensor elu_backward(const Tensor & grad_output, Scalar alpha, Scalar scale, Scalar input_scale, bool is_result, const Tensor & self_or_result);
warning: ^
warning: In file included from libtch/torch_api.cpp:1215:
warning: libtch/torch_api_generated.cpp.h:3494:29: error: no member named 'elu_backward_out' in namespace 'torch'
warning: auto outputs__ = torch::elu_backward_out(*grad_input, *grad_output, *alpha, *scale, *input_scale, *output);
warning: ~~~~~~~^
warning: libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
warning: x
warning: ^
warning: In file included from libtch/torch_api.cpp:1215:
warning: libtch/torch_api_generated.cpp.h:3819:29: error: no member named 'fake_quantize_per_channel_affine_backward' in namespace 'torch'
warning: auto outputs__ = torch::fake_quantize_per_channel_affine_backward(*grad, *self, *scale, *zero_point, axis, quant_min, quant_max);
warning: ~~~~~~~^
warning: libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
warning: x
warning: ^
warning: In file included from libtch/torch_api.cpp:1215:
warning: libtch/torch_api_generated.cpp.h:3833:29: error: no member named 'fake_quantize_per_tensor_affine_backward' in namespace 'torch'
warning: auto outputs__ = torch::fake_quantize_per_tensor_affine_backward(*grad, *self, scale, zero_point, quant_min, quant_max);
warning: ~~~~~~~^
warning: libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
warning: x
warning: ^
warning: In file included from libtch/torch_api.cpp:1215:
warning: libtch/torch_api_generated.cpp.h:3917:29: error: unexpected namespace name 'fft': expected expression
warning: auto outputs__ = torch::fft(*self, signal_ndim, (bool)normalized);
warning: ^
warning: libtch/torch_api_generated.cpp.h:4914:29: error: no member named 'ifft' in namespace 'torch'
warning: auto outputs__ = torch::ifft(*self, signal_ndim, (bool)normalized);
warning: ~~~~~~~^
warning: libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
warning: x
warning: ^
warning: In file included from libtch/torch_api.cpp:1215:
warning: libtch/torch_api_generated.cpp.h:4956:42: error: no viable conversion from 'vectortorch::Tensor' to 'const c10::List<c10::optional>'
warning: auto outputs__ = torch::index(*self, of_carray_tensor(indices_data, indices_len));
warning: ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
warning: libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
warning: x
warning: ^
warning: /Users/user/.local/libtorch/include/ATen/core/List.h:258:3: note: candidate constructor not viable: no known conversion from 'vectortorch::Tensor' to 'std::initializer_list<optional>' for 1st argument
warning: List(std::initializer_list initial_values);
warning: ^
warning: /Users/user/.local/libtorch/include/ATen/core/List.h:268:3: note: candidate constructor not viable: no known conversion from 'vectortorch::Tensor' to 'const c10::List<c10::optionalat::Tensor> &' for 1st argument
warning: List(const List&) = default;
warning: ^
warning: /Users/user/.local/libtorch/include/ATen/core/List.h:270:3: note: candidate constructor not viable: no known conversion from 'vectortorch::Tensor' to 'c10::List<c10::optionalat::Tensor> &&' for 1st argument
warning: List(List&&) noexcept;
warning: ^
warning: /Users/user/.local/libtorch/include/ATen/core/List.h:259:12: note: explicit constructor is not a candidate
warning: explicit List(ArrayRef initial_values);
warning: ^
warning: /Users/user/.local/libtorch/include/ATen/core/List.h:266:12: note: explicit constructor is not a candidate
warning: explicit List(TypePtr elementType);
warning: ^
warning: /Users/user/.local/libtorch/include/ATen/core/List.h:462:12: note: explicit constructor is not a candidate
warning: explicit List(c10::intrusive_ptrc10::detail::ListImpl&& elements);
warning: ^
warning: /Users/user/.local/libtorch/include/ATen/Functions.h:557:86: note: passing argument to parameter 'indices' here
warning: TORCH_API Tensor index(const Tensor & self, const c10::List<c10::optional> & indices);
warning: ^
warning: In file included from libtch/torch_api.cpp:1215:
warning: libtch/torch_api_generated.cpp.h:5019:46: error: no viable conversion from 'vectortorch::Tensor' to 'const c10::List<c10::optional>'
warning: auto outputs__ = torch::index_put(*self, of_carray_tensor(indices_data, indices_len), *values, (bool)accumulate);
warning: ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
warning: libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
warning: x
warning: ^
warning: /Users/user/.local/libtorch/include/ATen/core/List.h:258:3: note: candidate constructor not viable: no known conversion from 'vectortorch::Tensor' to 'std::initializer_list<optional>' for 1st argument
warning: List(std::initializer_list initial_values);
warning: ^
warning: /Users/user/.local/libtorch/include/ATen/core/List.h:268:3: note: candidate constructor not viable: no known conversion from 'vectortorch::Tensor' to 'const c10::List<c10::optionalat::Tensor> &' for 1st argument
warning: List(const List&) = default;
warning: ^
warning: /Users/user/.local/libtorch/include/ATen/core/List.h:270:3: note: candidate constructor not viable: no known conversion from 'vectortorch::Tensor' to 'c10::List<c10::optionalat::Tensor> &&' for 1st argument
warning: List(List&&) noexcept;
warning: ^
warning: /Users/user/.local/libtorch/include/ATen/core/List.h:259:12: note: explicit constructor is not a candidate
warning: explicit List(ArrayRef initial_values);
warning: ^
warning: /Users/user/.local/libtorch/include/ATen/core/List.h:266:12: note: explicit constructor is not a candidate
warning: explicit List(TypePtr elementType);
warning: ^
warning: /Users/user/.local/libtorch/include/ATen/core/List.h:462:12: note: explicit constructor is not a candidate
warning: explicit List(c10::intrusive_ptrc10::detail::ListImpl&& elements);
warning: ^
warning: /Users/user/.local/libtorch/include/ATen/Functions.h:561:90: note: passing argument to parameter 'indices' here
warning: TORCH_API Tensor index_put(const Tensor & self, const c10::List<c10::optional> & indices, const Tensor & values, bool accumulate=false);
warning: ^
warning: In file included from libtch/torch_api.cpp:1215:
warning: libtch/torch_api_generated.cpp.h:5026:47: error: no viable conversion from 'vectortorch::Tensor' to 'const c10::List<c10::optional>'
warning: auto outputs__ = torch::index_put_(*self, of_carray_tensor(indices_data, indices_len), *values, (bool)accumulate);
warning: ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
warning: libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
warning: x
warning: ^
warning: /Users/user/.local/libtorch/include/ATen/core/List.h:258:3: note: candidate constructor not viable: no known conversion from 'vectortorch::Tensor' to 'std::initializer_list<optional>' for 1st argument
warning: List(std::initializer_list initial_values);
warning: ^
warning: /Users/user/.local/libtorch/include/ATen/core/List.h:268:3: note: candidate constructor not viable: no known conversion from 'vectortorch::Tensor' to 'const c10::List<c10::optionalat::Tensor> &' for 1st argument
warning: List(const List&) = default;
warning: ^
warning: /Users/user/.local/libtorch/include/ATen/core/List.h:270:3: note: candidate constructor not viable: no known conversion from 'vectortorch::Tensor' to 'c10::List<c10::optionalat::Tensor> &&' for 1st argument
warning: List(List&&) noexcept;
warning: ^
warning: /Users/user/.local/libtorch/include/ATen/core/List.h:259:12: note: explicit constructor is not a candidate
warning: explicit List(ArrayRef initial_values);
warning: ^
warning: /Users/user/.local/libtorch/include/ATen/core/List.h:266:12: note: explicit constructor is not a candidate
warning: explicit List(TypePtr elementType);
warning: ^
warning: /Users/user/.local/libtorch/include/ATen/core/List.h:462:12: note: explicit constructor is not a candidate
warning: explicit List(c10::intrusive_ptrc10::detail::ListImpl&& elements);
warning: ^
warning: /Users/user/.local/libtorch/include/ATen/Functions.h:560:87: note: passing argument to parameter 'indices' here
warning: TORCH_API Tensor & index_put_(Tensor & self, const c10::List<c10::optional> & indices, const Tensor & values, bool accumulate=false);
warning: ^
warning: fatal error: too many errors emitted, stopping now [-ferror-limit=]
warning: 20 errors generated.

error: failed to run custom build command for torch-sys v0.3.1

Caused by:
process didn't exit successfully: /Users/user/git/rust/rust-bert/target/debug/build/torch-sys-b5698cc78204d67c/build-script-build (exit code: 1)
--- stdout
cargo:rerun-if-env-changed=TORCH_CUDA_VERSION
cargo:rerun-if-env-changed=LIBTORCH
cargo:rustc-link-search=native=/Users/user/.local/libtorch/lib
cargo:rerun-if-env-changed=LIBTORCH_USE_CMAKE
cargo:rerun-if-changed=libtch/torch_api.cpp
cargo:rerun-if-changed=libtch/torch_api.h
cargo:rerun-if-changed=libtch/torch_api_generated.cpp.h
cargo:rerun-if-changed=libtch/torch_api_generated.h
cargo:rerun-if-changed=libtch/stb_image_write.h
cargo:rerun-if-changed=libtch/stb_image_resize.h
cargo:rerun-if-changed=libtch/stb_image.h
cargo:rerun-if-env-changed=LIBTORCH_CXX11_ABI
TARGET = Some("x86_64-apple-darwin")
OPT_LEVEL = Some("0")
HOST = Some("x86_64-apple-darwin")
CXX_x86_64-apple-darwin = None
CXX_x86_64_apple_darwin = None
HOST_CXX = None
CXX = None
CXXFLAGS_x86_64-apple-darwin = None
CXXFLAGS_x86_64_apple_darwin = None
HOST_CXXFLAGS = None
CXXFLAGS = None
CRATE_CC_NO_DEFAULTS = None
DEBUG = Some("true")
CARGO_CFG_TARGET_FEATURE = Some("fxsr,sse,sse2,sse3,ssse3")
running: "c++" "-O0" "-ffunction-sections" "-fdata-sections" "-fPIC" "-g" "-fno-omit-frame-pointer" "-m64" "-arch" "x86_64" "-I" "/Users/user/.local/libtorch/include" "-I" "/Users/user/.local/libtorch/include/torch/csrc/api/include" "-Wl,-rpath=/Users/user/.local/libtorch/lib" "-std=c++14" "-D_GLIBCXX_USE_CXX11_ABI=1" "-o" "/Users/user/git/rust/rust-bert/target/debug/build/torch-sys-c9a36c0d279de21f/out/libtch/torch_api.o" "-c" "libtch/torch_api.cpp"
cargo:warning=clang: warning: -Wl,-rpath=/Users/user/.local/libtorch/lib: 'linker' input unused [-Wunused-command-line-argument]
cargo:warning=libtch/torch_api.cpp:142:9: error: no member named 'amp_non_finite_check_and_unscale' in namespace 'at'; did you mean 'amp_foreach_non_finite_check_and_unscale'?
cargo:warning= at::amp_non_finite_check_and_unscale(*t, *found_inf, *inf_scale);
cargo:warning= ~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
cargo:warning= amp_foreach_non_finite_check_and_unscale
cargo:warning=libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
cargo:warning= x
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/ATen/Functions.h:1645:16: note: 'amp_foreach_non_finite_check_and_unscale' declared here
cargo:warning=TORCH_API void amp_foreach_non_finite_check_and_unscale(TensorList self, Tensor & found_inf, const Tensor & inv_scale);
cargo:warning= ^
cargo:warning=libtch/torch_api.cpp:487:109: error: no viable conversion from 'vectortorch::autograd::Edge' to 'bool'
cargo:warning= auto vl = torch::autograd::Engine::get_default_engine().execute(roots, grads, keep_graph, create_graph, inputs_);
cargo:warning= ^~~~~~~
cargo:warning=libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
cargo:warning= x
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/torch/csrc/autograd/engine.h:275:12: note: passing argument to parameter 'accumulate_grad' here
cargo:warning= bool accumulate_grad,
cargo:warning= ^
cargo:warning=In file included from libtch/torch_api.cpp:1215:
cargo:warning=libtch/torch_api_generated.cpp.h:415:386: error: too few arguments to function call, expected 16, have 15
cargo:warning= auto outputs__ = torch::cudnn_rnn(*input, of_carray_tensor(weight_data, weight_len), weight_stride0, (weight_buf ? *weight_buf : torch::Tensor()), *hx, (cx ? *cx : torch::Tensor()), mode, hidden_size, num_layers, (bool)batch_first, dropout, (bool)train, (bool)bidirectional, torch::IntArrayRef(batch_sizes_data, batch_sizes_len), (dropout_state ? *dropout_state : torch::Tensor()));
cargo:warning= ~~~~~~~~~~~~~~~~~ ^
cargo:warning=libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
cargo:warning= x
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/ATen/Functions.h:59:58: note: 'cudnn_rnn' declared here
cargo:warning=TORCH_API std::tuple<Tensor,Tensor,Tensor,Tensor,Tensor> cudnn_rnn(const Tensor & input, TensorList weight, int64_t weight_stride0, const c10::optional & weight_buf, const Tensor & hx, const c10::optional & cx, int64_t mode, int64_t hidden_size, int64_t proj_size, int64_t num_layers, bool batch_first, double dropout, bool train, bool bidirectional, IntArrayRef batch_sizes, const c10::optional & dropout_state);
cargo:warning= ^
cargo:warning=In file included from libtch/torch_api.cpp:1215:
cargo:warning=libtch/torch_api_generated.cpp.h:426:203: error: too few arguments to function call, expected 9, have 8
cargo:warning= auto outputs = torch::cudnn_rnn_flatten_weight(of_carray_tensor(weight_arr_data, weight_arr_len), weight_stride0, input_size, mode, hidden_size, num_layers, (bool)batch_first, (bool)bidirectional);
cargo:warning= ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^
cargo:warning=libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
cargo:warning= x
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/ATen/Functions.h:58:18: note: 'cudnn_rnn_flatten_weight' declared here
cargo:warning=TORCH_API Tensor cudnn_rnn_flatten_weight(TensorList weight_arr, int64_t weight_stride0, int64_t input_size, int64_t mode, int64_t hidden_size, int64_t proj_size, int64_t num_layers, bool batch_first, bool bidirectional);
cargo:warning= ^
cargo:warning=In file included from libtch/torch_api.cpp:1215:
cargo:warning=libtch/torch_api_generated.cpp.h:576:29: error: no member named 'fft_with_size' in namespace 'torch'
cargo:warning= auto outputs = torch::fft_with_size(*self, signal_ndim, (bool)complex_input, (bool)complex_output, (bool)inverse, torch::IntArrayRef(checked_signal_sizes_data, checked_signal_sizes_len), (bool)normalized, (bool)onesided, torch::IntArrayRef(output_sizes_data, output_sizes_len));
cargo:warning= ~~~~~~~^
cargo:warning=libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
cargo:warning= x
cargo:warning= ^
cargo:warning=In file included from libtch/torch_api.cpp:1215:
cargo:warning=libtch/torch_api_generated.cpp.h:583:29: error: no member named 'fft_with_size' in namespace 'torch'
cargo:warning= auto outputs = torch::fft_with_size(*self, signal_ndim, (bool)complex_input, (bool)complex_output, (bool)inverse, torch::IntArrayRef(checked_signal_sizes_data, checked_signal_sizes_len), normalization, (bool)onesided, torch::IntArrayRef(output_sizes_data, output_sizes_len));
cargo:warning= ~~~~~~~^
cargo:warning=libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
cargo:warning= x
cargo:warning= ^
cargo:warning=In file included from libtch/torch_api.cpp:1215:
cargo:warning=libtch/torch_api_generated.cpp.h:627:53: error: no viable conversion from 'vectortorch::Tensor' to 'const c10::List<c10::optional>'
cargo:warning= auto outputs = torch::index_put_impl(*self, of_carray_tensor(indices_data, indices_len), *values, (bool)accumulate, (bool)unsafe);
cargo:warning= ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
cargo:warning=libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
cargo:warning= x
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/ATen/core/List.h:258:3: note: candidate constructor not viable: no known conversion from 'vectortorch::Tensor' to 'std::initializer_list<optional>' for 1st argument
cargo:warning= List(std::initializer_list initial_values);
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/ATen/core/List.h:268:3: note: candidate constructor not viable: no known conversion from 'vectortorch::Tensor' to 'const c10::List<c10::optionalat::Tensor> &' for 1st argument
cargo:warning= List(const List&) = default;
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/ATen/core/List.h:270:3: note: candidate constructor not viable: no known conversion from 'vectortorch::Tensor' to 'c10::List<c10::optionalat::Tensor> &&' for 1st argument
cargo:warning= List(List&&) noexcept;
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/ATen/core/List.h:259:12: note: explicit constructor is not a candidate
cargo:warning= explicit List(ArrayRef initial_values);
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/ATen/core/List.h:266:12: note: explicit constructor is not a candidate
cargo:warning= explicit List(TypePtr elementType);
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/ATen/core/List.h:462:12: note: explicit constructor is not a candidate
cargo:warning= explicit List(c10::intrusive_ptrc10::detail::ListImpl&& elements);
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/ATen/Functions.h:562:93: note: passing argument to parameter 'indices' here
cargo:warning=TORCH_API Tensor & index_put_impl(Tensor & self, const c10::List<c10::optional> & indices, const Tensor & values, bool accumulate=false, bool unsafe=false);
cargo:warning= ^
cargo:warning=In file included from libtch/torch_api.cpp:1215:
cargo:warning=libtch/torch_api_generated.cpp.h:750:29: error: no member named 'multinomial_alias_draw' in namespace 'torch'
cargo:warning= auto outputs = torch::multinomial_alias_draw(*J, *q, num_samples);
cargo:warning= ~~~~~~~^
cargo:warning=libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
cargo:warning= x
cargo:warning= ^
cargo:warning=In file included from libtch/torch_api.cpp:1215:
cargo:warning=libtch/torch_api_generated.cpp.h:757:29: error: no member named 'multinomial_alias_setup' in namespace 'torch'
cargo:warning= auto outputs = torch::multinomial_alias_setup(*probs);
cargo:warning= ~~~~~~~^
cargo:warning=libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
cargo:warning= x
cargo:warning= ^
cargo:warning=In file included from libtch/torch_api.cpp:1215:
cargo:warning=libtch/torch_api_generated.cpp.h:816:29: error: no member named 'qr_helper' in namespace 'torch'
cargo:warning= auto outputs = torch::qr_helper(*self, (bool)some);
cargo:warning= ~~~~~~~^
cargo:warning=libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
cargo:warning= x
cargo:warning= ^
cargo:warning=In file included from libtch/torch_api.cpp:1215:
cargo:warning=libtch/torch_api_generated.cpp.h:3487:93: error: too few arguments to function call, expected 6, have 5
cargo:warning= auto outputs = torch::elu_backward(*grad_output, *alpha, *scale, *input_scale, *output);
cargo:warning= ~~~~~~~~~~~~~~~~~~~ ^
cargo:warning=libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
cargo:warning= x
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/ATen/Functions.h:1812:18: note: 'elu_backward' declared here
cargo:warning=TORCH_API Tensor elu_backward(const Tensor & grad_output, Scalar alpha, Scalar scale, Scalar input_scale, bool is_result, const Tensor & self_or_result);
cargo:warning= ^
cargo:warning=In file included from libtch/torch_api.cpp:1215:
cargo:warning=libtch/torch_api_generated.cpp.h:3494:29: error: no member named 'elu_backward_out' in namespace 'torch'
cargo:warning= auto outputs__ = torch::elu_backward_out(*grad_input, *grad_output, *alpha, *scale, *input_scale, *output);
cargo:warning= ~~~~~~~^
cargo:warning=libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
cargo:warning= x
cargo:warning= ^
cargo:warning=In file included from libtch/torch_api.cpp:1215:
cargo:warning=libtch/torch_api_generated.cpp.h:3819:29: error: no member named 'fake_quantize_per_channel_affine_backward' in namespace 'torch'
cargo:warning= auto outputs__ = torch::fake_quantize_per_channel_affine_backward(*grad, *self, *scale, *zero_point, axis, quant_min, quant_max);
cargo:warning= ~~~~~~~^
cargo:warning=libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
cargo:warning= x
cargo:warning= ^
cargo:warning=In file included from libtch/torch_api.cpp:1215:
cargo:warning=libtch/torch_api_generated.cpp.h:3833:29: error: no member named 'fake_quantize_per_tensor_affine_backward' in namespace 'torch'
cargo:warning= auto outputs__ = torch::fake_quantize_per_tensor_affine_backward(*grad, *self, scale, zero_point, quant_min, quant_max);
cargo:warning= ~~~~~~~^
cargo:warning=libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
cargo:warning= x
cargo:warning= ^
cargo:warning=In file included from libtch/torch_api.cpp:1215:
cargo:warning=libtch/torch_api_generated.cpp.h:3917:29: error: unexpected namespace name 'fft': expected expression
cargo:warning= auto outputs__ = torch::fft(*self, signal_ndim, (bool)normalized);
cargo:warning= ^
cargo:warning=libtch/torch_api_generated.cpp.h:4914:29: error: no member named 'ifft' in namespace 'torch'
cargo:warning= auto outputs__ = torch::ifft(*self, signal_ndim, (bool)normalized);
cargo:warning= ~~~~~~~^
cargo:warning=libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
cargo:warning= x
cargo:warning= ^
cargo:warning=In file included from libtch/torch_api.cpp:1215:
cargo:warning=libtch/torch_api_generated.cpp.h:4956:42: error: no viable conversion from 'vectortorch::Tensor' to 'const c10::List<c10::optional>'
cargo:warning= auto outputs__ = torch::index(*self, of_carray_tensor(indices_data, indices_len));
cargo:warning= ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
cargo:warning=libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
cargo:warning= x
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/ATen/core/List.h:258:3: note: candidate constructor not viable: no known conversion from 'vectortorch::Tensor' to 'std::initializer_list<optional>' for 1st argument
cargo:warning= List(std::initializer_list initial_values);
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/ATen/core/List.h:268:3: note: candidate constructor not viable: no known conversion from 'vectortorch::Tensor' to 'const c10::List<c10::optionalat::Tensor> &' for 1st argument
cargo:warning= List(const List&) = default;
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/ATen/core/List.h:270:3: note: candidate constructor not viable: no known conversion from 'vectortorch::Tensor' to 'c10::List<c10::optionalat::Tensor> &&' for 1st argument
cargo:warning= List(List&&) noexcept;
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/ATen/core/List.h:259:12: note: explicit constructor is not a candidate
cargo:warning= explicit List(ArrayRef initial_values);
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/ATen/core/List.h:266:12: note: explicit constructor is not a candidate
cargo:warning= explicit List(TypePtr elementType);
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/ATen/core/List.h:462:12: note: explicit constructor is not a candidate
cargo:warning= explicit List(c10::intrusive_ptrc10::detail::ListImpl&& elements);
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/ATen/Functions.h:557:86: note: passing argument to parameter 'indices' here
cargo:warning=TORCH_API Tensor index(const Tensor & self, const c10::List<c10::optional> & indices);
cargo:warning= ^
cargo:warning=In file included from libtch/torch_api.cpp:1215:
cargo:warning=libtch/torch_api_generated.cpp.h:5019:46: error: no viable conversion from 'vectortorch::Tensor' to 'const c10::List<c10::optional>'
cargo:warning= auto outputs__ = torch::index_put(*self, of_carray_tensor(indices_data, indices_len), *values, (bool)accumulate);
cargo:warning= ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
cargo:warning=libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
cargo:warning= x
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/ATen/core/List.h:258:3: note: candidate constructor not viable: no known conversion from 'vectortorch::Tensor' to 'std::initializer_list<optional>' for 1st argument
cargo:warning= List(std::initializer_list initial_values);
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/ATen/core/List.h:268:3: note: candidate constructor not viable: no known conversion from 'vectortorch::Tensor' to 'const c10::List<c10::optionalat::Tensor> &' for 1st argument
cargo:warning= List(const List&) = default;
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/ATen/core/List.h:270:3: note: candidate constructor not viable: no known conversion from 'vectortorch::Tensor' to 'c10::List<c10::optionalat::Tensor> &&' for 1st argument
cargo:warning= List(List&&) noexcept;
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/ATen/core/List.h:259:12: note: explicit constructor is not a candidate
cargo:warning= explicit List(ArrayRef initial_values);
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/ATen/core/List.h:266:12: note: explicit constructor is not a candidate
cargo:warning= explicit List(TypePtr elementType);
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/ATen/core/List.h:462:12: note: explicit constructor is not a candidate
cargo:warning= explicit List(c10::intrusive_ptrc10::detail::ListImpl&& elements);
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/ATen/Functions.h:561:90: note: passing argument to parameter 'indices' here
cargo:warning=TORCH_API Tensor index_put(const Tensor & self, const c10::List<c10::optional> & indices, const Tensor & values, bool accumulate=false);
cargo:warning= ^
cargo:warning=In file included from libtch/torch_api.cpp:1215:
cargo:warning=libtch/torch_api_generated.cpp.h:5026:47: error: no viable conversion from 'vectortorch::Tensor' to 'const c10::List<c10::optional>'
cargo:warning= auto outputs__ = torch::index_put_(*self, of_carray_tensor(indices_data, indices_len), *values, (bool)accumulate);
cargo:warning= ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
cargo:warning=libtch/torch_api.h:16:5: note: expanded from macro 'PROTECT'
cargo:warning= x
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/ATen/core/List.h:258:3: note: candidate constructor not viable: no known conversion from 'vectortorch::Tensor' to 'std::initializer_list<optional>' for 1st argument
cargo:warning= List(std::initializer_list initial_values);
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/ATen/core/List.h:268:3: note: candidate constructor not viable: no known conversion from 'vectortorch::Tensor' to 'const c10::List<c10::optionalat::Tensor> &' for 1st argument
cargo:warning= List(const List&) = default;
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/ATen/core/List.h:270:3: note: candidate constructor not viable: no known conversion from 'vectortorch::Tensor' to 'c10::List<c10::optionalat::Tensor> &&' for 1st argument
cargo:warning= List(List&&) noexcept;
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/ATen/core/List.h:259:12: note: explicit constructor is not a candidate
cargo:warning= explicit List(ArrayRef initial_values);
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/ATen/core/List.h:266:12: note: explicit constructor is not a candidate
cargo:warning= explicit List(TypePtr elementType);
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/ATen/core/List.h:462:12: note: explicit constructor is not a candidate
cargo:warning= explicit List(c10::intrusive_ptrc10::detail::ListImpl&& elements);
cargo:warning= ^
cargo:warning=/Users/user/.local/libtorch/include/ATen/Functions.h:560:87: note: passing argument to parameter 'indices' here
cargo:warning=TORCH_API Tensor & index_put_(Tensor & self, const c10::List<c10::optional> & indices, const Tensor & values, bool accumulate=false);
cargo:warning= ^
cargo:warning=fatal error: too many errors emitted, stopping now [-ferror-limit=]
cargo:warning=20 errors generated.
exit code: 1

--- stderr

error occurred: Command "c++" "-O0" "-ffunction-sections" "-fdata-sections" "-fPIC" "-g" "-fno-omit-frame-pointer" "-m64" "-arch" "x86_64" "-I" "/Users/user/.local/libtorch/include" "-I" "/Users/user/.local/libtorch/include/torch/csrc/api/include" "-Wl,-rpath=/Users/user/.local/libtorch/lib" "-std=c++14" "-D_GLIBCXX_USE_CXX11_ABI=1" "-o" "/Users/user/git/rust/rust-bert/target/debug/build/torch-sys-c9a36c0d279de21f/out/libtch/torch_api.o" "-c" "libtch/torch_api.cpp" with args "c++" did not execute successfully (status code exit code: 1).

Help with NER usage

Hello, I have this learned model:
https://github.com/deepmipt/DeepPavlov/blob/0.8.0/deeppavlov/configs/ner/ner_ontonotes_bert_mult.json

In downloads section it mentions two items: BERT model and NER. Is it possible to use NER model with this crate? It requires vocab.txt, but list of files is:

checkpoint 
model.data-00000-of-00001
model.index
model.meta
tag.dict

Custom NER tagging

Hi! I am looking for tools to tag ingredients of recipes to allow me extract information out of them. Given the example 1/2 cup of vegetable oil I'd like to use NER to tag the quantity, unit and ingredient:

quantity: 1/2
unut: cup
ingredient: vegetable oil

I saw that there is already a NER implementation. How do I go about customizing it to tag the way I want it to? I know that I'd need to train a model (which I'd probably do using python) but I need to be able to tag incoming phrases using rust.

Any help is appreciated!

Any benchmark results could be shared?

Great project and I know /benches are there, but from #98, I think it would be better to list some benchmark results to show rust-bert's performance.

Failed to run cargo run on Linux

Hello @guillaume-be

I'm trying to run the example of rust-bert and It's looking like I have issues with torch-sys.

running cargo run --example generation --verbose gives me the following errors:

       Fresh autocfg v1.0.1
       Fresh cfg-if v1.0.0
       Fresh cc v1.0.67
       Fresh pkg-config v0.3.19
       Fresh unicode-xid v0.2.2
       Fresh lazy_static v1.4.0
       Fresh itoa v0.4.7
       Fresh byteorder v1.4.3
       Fresh pin-project-lite v0.2.6
       Fresh bytes v1.0.1
       Fresh openssl-probe v0.1.2
       Fresh adler v1.0.2
       Fresh ahash v0.4.7
       Fresh futures-core v0.3.14
       Fresh tinyvec_macros v0.1.0
       Fresh foreign-types-shared v0.1.1
       Fresh ppv-lite86 v0.2.10
       Fresh matches v0.1.8
       Fresh fnv v1.0.7
       Fresh version_check v0.9.3
       Fresh pin-utils v0.1.0
       Fresh once_cell v1.7.2
       Fresh futures-io v0.3.14
       Fresh unicode-width v0.1.8
       Fresh either v1.6.1
       Fresh regex-syntax v0.6.25
       Fresh slab v0.4.3
       Fresh futures-sink v0.3.14
       Fresh scopeguard v1.1.0
       Fresh futures-task v0.3.14
       Fresh semver-parser v0.7.0
       Fresh percent-encoding v2.1.0
       Fresh try-lock v0.2.3
       Fresh httpdate v1.0.0
       Fresh tower-service v0.3.1
       Fresh rawpointer v0.2.1
       Fresh half v1.7.1
       Fresh smallvec v1.6.1
       Fresh opaque-debug v0.3.0
       Fresh ipnet v2.3.0
       Fresh cpuid-bool v0.1.2
       Fresh remove_dir_all v0.5.3
       Fresh plotters-backend v0.3.0
       Fresh mime v0.3.16
       Fresh base64 v0.13.0
       Fresh number_prefix v0.3.0
       Fresh same-file v1.0.6
       Fresh glob v0.3.0
       Fresh oorandom v11.1.3
       Fresh tracing-core v0.1.18
       Fresh cmake v0.1.45
       Fresh regex-automata v0.1.9
       Fresh hashbrown v0.9.1
       Fresh foreign-types v0.3.2
       Fresh tinyvec v1.2.0
       Fresh futures-channel v0.3.14
       Fresh unicode-bidi v0.3.5
       Fresh http v0.2.4
       Fresh itertools v0.10.0
       Fresh textwrap v0.11.0
       Fresh itertools v0.9.0
       Fresh semver v0.9.0
       Fresh form_urlencoded v1.0.1
       Fresh matrixmultiply v0.2.4
       Fresh libc v0.2.94
       Fresh proc-macro2 v1.0.26
       Fresh unicode-normalization-alignments v0.1.12
       Fresh plotters-svg v0.3.0
       Fresh walkdir v2.3.2
       Fresh memchr v2.4.0
       Fresh log v0.4.14
       Fresh bitflags v1.2.1
       Fresh ryu v1.0.5
       Fresh crc32fast v1.2.1
       Fresh typenum v1.13.0
       Fresh tracing v0.1.26
       Fresh unicode-normalization v0.1.17
       Fresh anyhow v1.0.40
       Fresh http-body v0.4.1
       Fresh httparse v1.4.0
       Fresh rustc_version v0.2.3
       Fresh encoding_rs v0.8.28
       Fresh quote v1.0.9
       Fresh openssl-sys v0.9.62
       Fresh num_cpus v1.13.0
       Fresh socket2 v0.4.0
       Fresh mio v0.7.11
       Fresh bzip2-sys v0.1.10+1.0.8
       Fresh miniz_oxide v0.4.4
       Fresh crossbeam-utils v0.8.4
       Fresh time v0.1.43
       Fresh getrandom v0.2.2
       Fresh libz-sys v1.1.3
       Fresh aho-corasick v0.7.18
       Fresh futures-util v0.3.14
       Fresh getrandom v0.1.16
       Fresh want v0.3.0
       Fresh csv-core v0.1.10
       Fresh terminal_size v0.1.16
       Fresh xattr v0.2.2
       Fresh filetime v0.2.14
       Fresh protobuf v2.22.1
       Fresh fs2 v0.4.3
       Fresh dirs-sys v0.3.6
       Fresh atty v0.2.14
       Fresh num-traits v0.2.14
       Fresh memoffset v0.6.3
       Fresh indexmap v1.6.2
       Fresh generic-array v0.14.4
       Fresh idna v0.2.3
       Fresh clap v2.33.3
       Fresh syn v1.0.72
       Fresh bzip2 v0.3.3
       Fresh flate2 v1.0.20
       Fresh tokio v1.5.0
       Fresh rand_core v0.6.2
       Fresh crossbeam-channel v0.5.1
       Fresh uuid v0.8.2
       Fresh curl-sys v0.4.42+curl-7.76.0
       Fresh openssl v0.10.34
       Fresh regex v1.5.3
       Fresh rand_core v0.5.1
       Fresh tar v0.4.33
       Fresh dirs v3.0.2
       Fresh serde_derive v1.0.125
       Fresh thiserror-impl v1.0.24
       Fresh pin-project-internal v1.0.7
       Fresh crossbeam-epoch v0.9.4
       Fresh tokio-util v0.6.6
       Fresh rand_chacha v0.3.0
       Fresh digest v0.9.0
       Fresh block-buffer v0.9.0
       Fresh url v2.2.1
       Fresh num-complex v0.2.4
       Fresh num-integer v0.1.44
       Fresh ordered-float v2.2.0
       Fresh plotters v0.3.0
       Fresh serde v1.0.125
       Fresh thiserror v1.0.24
       Fresh native-tls v0.2.7
       Fresh crossbeam-deque v0.8.0
       Fresh curl v0.4.36
       Fresh pin-project v1.0.7
       Fresh h2 v0.3.3
       Fresh console v0.14.1
       Fresh rand v0.8.3
       Fresh rand_chacha v0.2.2
       Fresh sha2 v0.9.3
       Fresh cast v0.2.5
       Fresh zip v0.5.12
       Fresh serde_json v1.0.64
       Fresh hyper v0.14.7
       Fresh rayon-core v1.9.0
       Fresh bstr v0.2.16
       Fresh tokio-native-tls v0.3.0
       Fresh serde_urlencoded v0.7.0
       Fresh ndarray v0.13.1
       Fresh indicatif v0.15.0
       Fresh rand v0.7.3
       Fresh tempfile v3.2.0
       Fresh serde_cbor v0.11.1
       Fresh hyper-tls v0.5.0
       Fresh csv v1.1.6
       Fresh rayon v1.5.0
       Fresh zip-extensions v0.6.0
       Fresh criterion-plot v0.4.3
       Fresh tinytemplate v1.2.1
       Fresh reqwest v0.11.3
       Fresh rust_tokenizers v6.2.2
       Fresh torch-sys v0.3.1
       Fresh cached-path v0.5.1
       Fresh criterion v0.3.4
       Fresh tch v0.3.1
   Compiling rust-bert v0.14.0 (/home/mathis/Dev/rust-bert)
     Running `rustc --crate-name generation --edition=2018 examples/generation.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debuginfo=2 -C metadata=1079cbdd0923672d -C extra-filename=-1079cbdd0923672d --out-dir /home/mathis/Dev/rust-bert/target/debug/examples -C incremental=/home/mathis/Dev/rust-bert/target/debug/incremental -L dependency=/home/mathis/Dev/rust-bert/target/debug/deps --extern anyhow=/home/mathis/Dev/rust-bert/target/debug/deps/libanyhow-deb63606ff1cfc02.rlib --extern cached_path=/home/mathis/Dev/rust-bert/target/debug/deps/libcached_path-ed4f91024ed4fc3e.rlib --extern criterion=/home/mathis/Dev/rust-bert/target/debug/deps/libcriterion-11283086a0db42c9.rlib --extern csv=/home/mathis/Dev/rust-bert/target/debug/deps/libcsv-c5db6ce464e9affd.rlib --extern dirs=/home/mathis/Dev/rust-bert/target/debug/deps/libdirs-d779fef0aee33dfd.rlib --extern lazy_static=/home/mathis/Dev/rust-bert/target/debug/deps/liblazy_static-cd2401a06b1676f6.rlib --extern ordered_float=/home/mathis/Dev/rust-bert/target/debug/deps/libordered_float-9d94cc6b3ed64167.rlib --extern rust_bert=/home/mathis/Dev/rust-bert/target/debug/deps/librust_bert-2ec9c634d44292b9.rlib --extern rust_tokenizers=/home/mathis/Dev/rust-bert/target/debug/deps/librust_tokenizers-d60a711620946171.rlib --extern serde=/home/mathis/Dev/rust-bert/target/debug/deps/libserde-3f736418ac579112.rlib --extern serde_json=/home/mathis/Dev/rust-bert/target/debug/deps/libserde_json-56cb869844da73de.rlib --extern tch=/home/mathis/Dev/rust-bert/target/debug/deps/libtch-3489097ebbda946c.rlib --extern tempfile=/home/mathis/Dev/rust-bert/target/debug/deps/libtempfile-cb2a539468ebc1f7.rlib --extern thiserror=/home/mathis/Dev/rust-bert/target/debug/deps/libthiserror-849666d06ac7e440.rlib --extern torch_sys=/home/mathis/Dev/rust-bert/target/debug/deps/libtorch_sys-73690ae916a3995c.rlib --extern uuid=/home/mathis/Dev/rust-bert/target/debug/deps/libuuid-bf8e4fb0cc067046.rlib -L native=/usr/lib -L native=/home/mathis/Dev/rust-bert/target/debug/build/torch-sys-d810dc21875d10ae/out/libtorch/libtorch/lib -L native=/home/mathis/Dev/rust-bert/target/debug/build/torch-sys-d810dc21875d10ae/out`
error: linking with `cc` failed: exit code: 1
  |
  = note: "cc" "-Wl,--as-needed" "-Wl,-z,noexecstack" "-m64" "-Wl,--eh-frame-hdr" "-L" "/home/mathis/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "/home/mathis/Dev/rust-bert/target/debug/examples/generation-1079cbdd0923672d.10it0tix8kemlzzz.rcgu.o" "/home/mathis/Dev/rust-bert/target/debug/examples/generation-1079cbdd0923672d.167xi68m3zomqb7.rcgu.o" "/home/mathis/Dev/rust-bert/target/debug/examples/generation-1079cbdd0923672d.1b5qfw6lrgq9ixu9.rcgu.o" "/home/mathis/Dev/rust-bert/target/debug/examples/generation-1079cbdd0923672d.1d0ql6qb0gfr40an.rcgu.o" "/home/mathis/Dev/rust-bert/target/debug/examples/generation-1079cbdd0923672d.1lo4kpl4iw1j0xzu.rcgu.o" "/home/mathis/Dev/rust-bert/target/debug/examples/generation-1079cbdd0923672d.1ltk8i68f9tjta0v.rcgu.o" "/home/mathis/Dev/rust-bert/target/debug/examples/generation-1079cbdd0923672d.1mif4ri8vrsnk1lb.rcgu.o" "/home/mathis/Dev/rust-bert/target/debug/examples/generation-1079cbdd0923672d.1n0sgz8zmfb1yk8e.rcgu.o" "/home/mathis/Dev/rust-bert/target/debug/examples/generation-1079cbdd0923672d.1p2f8f4m2gx4bs7c.rcgu.o" "/home/mathis/Dev/rust-bert/target/debug/examples/generation-1079cbdd0923672d.1pffsc4z38348shw.rcgu.o" "/home/mathis/Dev/rust-bert/target/debug/examples/generation-1079cbdd0923672d.1r7u2nys3lazyjqj.rcgu.o" "/home/mathis/Dev/rust-bert/target/debug/examples/generation-1079cbdd0923672d.1t5v18zbjjusjg3b.rcgu.o" "/home/mathis/Dev/rust-bert/target/debug/examples/generation-1079cbdd0923672d.1vr0f0tzbpd6tsr6.rcgu.o" "/home/mathis/Dev/rust-bert/target/debug/examples/generation-1079cbdd0923672d.232uq55vzhtlx2y0.rcgu.o" "/home/mathis/Dev/rust-bert/target/debug/examples/generation-1079cbdd0923672d.28sfi6qkpnmtd1jv.rcgu.o" "/home/mathis/Dev/rust-bert/target/debug/examples/generation-1079cbdd0923672d.2apsawysjbfj41cw.rcgu.o" "/home/mathis/Dev/rust-bert/target/debug/examples/generation-1079cbdd0923672d.2efxy0h6puoi2k9u.rcgu.o" "/home/mathis/Dev/rust-bert/target/debug/examples/generation-1079cbdd0923672d.2k2r5jq5bv472xuc.rcgu.o" "/home/mathis/Dev/rust-bert/target/debug/examples/generation-1079cbdd0923672d.2ocmatgwqmq8wdin.rcgu.o" "/home/mathis/Dev/rust-bert/target/debug/examples/generation-1079cbdd0923672d.2vu57mhs8b1rdwyd.rcgu.o" "/home/mathis/Dev/rust-bert/target/debug/examples/generation-1079cbdd0923672d.34armeuadq7ou4g7.rcgu.o" "/home/mathis/Dev/rust-bert/target/debug/examples/generation-1079cbdd0923672d.34pc7e99jb7tqj70.rcgu.o" "/home/mathis/Dev/rust-bert/target/debug/examples/generation-1079cbdd0923672d.3b3ohz2lemdo994y.rcgu.o" "/home/mathis/Dev/rust-bert/target/debug/examples/generation-1079cbdd0923672d.3d6prhey778singb.rcgu.o" "/home/mathis/Dev/rust-bert/target/debug/examples/generation-1079cbdd0923672d.3mdf12i52d5gzn59.rcgu.o" "/home/mathis/Dev/rust-bert/target/debug/examples/generation-1079cbdd0923672d.3ohu9xvnsquxze42.rcgu.o" "/home/mathis/Dev/rust-bert/target/debug/examples/generation-1079cbdd0923672d.3pq0r8m6j1ykpyz5.rcgu.o" "/home/mathis/Dev/rust-bert/target/debug/examples/generation-1079cbdd0923672d.3qg9lpklsjsdpcaz.rcgu.o" "/home/mathis/Dev/rust-bert/target/debug/examples/generation-1079cbdd0923672d.3rkm1e4qxh8n4c3l.rcgu.o" "/home/mathis/Dev/rust-bert/target/debug/examples/generation-1079cbdd0923672d.3sp583ijiw4ov2sz.rcgu.o" "/home/mathis/Dev/rust-bert/target/debug/examples/generation-1079cbdd0923672d.3uqpvz1x50awpida.rcgu.o" "/home/mathis/Dev/rust-bert/target/debug/examples/generation-1079cbdd0923672d.3vol26id67ntsbxl.rcgu.o" "/home/mathis/Dev/rust-bert/target/debug/examples/generation-1079cbdd0923672d.3whurftq1sb0v984.rcgu.o" "/home/mathis/Dev/rust-bert/target/debug/examples/generation-1079cbdd0923672d.49z5c534dy9uim1a.rcgu.o" "/home/mathis/Dev/rust-bert/target/debug/examples/generation-1079cbdd0923672d.4ae8f3av8lqccarn.rcgu.o" "/home/mathis/Dev/rust-bert/target/debug/examples/generation-1079cbdd0923672d.4h6k1vvx97o7e6tb.rcgu.o" "/home/mathis/Dev/rust-bert/target/debug/examples/generation-1079cbdd0923672d.4krufbcvoktpje1z.rcgu.o" "/home/mathis/Dev/rust-bert/target/debug/examples/generation-1079cbdd0923672d.4kv6jrdjrkzycgr8.rcgu.o" "/home/mathis/Dev/rust-bert/target/debug/examples/generation-1079cbdd0923672d.4o3sj4udjrggx8mj.rcgu.o" "/home/mathis/Dev/rust-bert/target/debug/examples/generation-1079cbdd0923672d.4wajjfybl1w1bx97.rcgu.o" "/home/mathis/Dev/rust-bert/target/debug/examples/generation-1079cbdd0923672d.52ooz6okopac6bou.rcgu.o" "/home/mathis/Dev/rust-bert/target/debug/examples/generation-1079cbdd0923672d.5cqzl22pfezok8hy.rcgu.o" "/home/mathis/Dev/rust-bert/target/debug/examples/generation-1079cbdd0923672d.5ge8c24exj1sggnk.rcgu.o" "/home/mathis/Dev/rust-bert/target/debug/examples/generation-1079cbdd0923672d.akk6cuaji4u6oih.rcgu.o" "/home/mathis/Dev/rust-bert/target/debug/examples/generation-1079cbdd0923672d.ctpuo9x1x6wcg53.rcgu.o" "/home/mathis/Dev/rust-bert/target/debug/examples/generation-1079cbdd0923672d.odzqcam2z1zbxzn.rcgu.o" "/home/mathis/Dev/rust-bert/target/debug/examples/generation-1079cbdd0923672d.rz5bc76mat974dm.rcgu.o" "/home/mathis/Dev/rust-bert/target/debug/examples/generation-1079cbdd0923672d.t62z9mjebfibll8.rcgu.o" "-o" "/home/mathis/Dev/rust-bert/target/debug/examples/generation-1079cbdd0923672d" "/home/mathis/Dev/rust-bert/target/debug/examples/generation-1079cbdd0923672d.3jhp2i5vw46ruhq7.rcgu.o" "-Wl,--gc-sections" "-pie" "-Wl,-zrelro" "-Wl,-znow" "-nodefaultlibs" "-L" "/home/mathis/Dev/rust-bert/target/debug/deps" "-L" "/usr/lib" "-L" "/home/mathis/Dev/rust-bert/target/debug/build/torch-sys-d810dc21875d10ae/out/libtorch/libtorch/lib" "-L" "/home/mathis/Dev/rust-bert/target/debug/build/torch-sys-d810dc21875d10ae/out" "-L" "/home/mathis/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,-Bstatic" "/home/mathis/Dev/rust-bert/target/debug/deps/librust_bert-2ec9c634d44292b9.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libuuid-bf8e4fb0cc067046.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libcached_path-ed4f91024ed4fc3e.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libindicatif-8a0185076a0eb177.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libconsole-c943e32fe490a795.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libterminal_size-29f5eab3f5b081c5.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libunicode_width-85f6c02835e49e35.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libnumber_prefix-f42c307c9580ef59.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libtar-fd5427c40075e783.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libxattr-9989c107df9b07c7.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libfiletime-f00939867a7a5d10.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libsha2-b96c29815f69105c.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libcpuid_bool-77e420fc3482a298.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libopaque_debug-e255b5a7231feb45.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libdigest-5e985337ea6e7525.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libblock_buffer-1a5ff49bbac509bb.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libgeneric_array-c6e858b52fea96d4.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libtypenum-d38b2426cfe2d60a.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libreqwest-3a20498a91e06669.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libhyper_tls-41c83e33a6039bb3.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libipnet-80aef0b8131aa337.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libtokio_native_tls-b2e970837a80eae8.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libserde_urlencoded-737747b42f1af893.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libmime-4ab1ec3a2ae719c1.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libencoding_rs-f79ff60439fd89ce.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libbase64-2eaf2a28e351c3cf.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libnative_tls-aff207f6c062165a.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libopenssl_probe-3fc17c5c1d53a384.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libopenssl-8449b8ae49b7be20.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libonce_cell-e4cdb0ff14231943.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libbitflags-df474a8ca45884a8.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libforeign_types-f70576c32fbfaf7f.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libforeign_types_shared-c71078ab8c7eadf1.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libopenssl_sys-70482dbbf5b1d608.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libhyper-8a00342c18d2c3a1.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libwant-99e08c6aeea13938.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libtry_lock-41d3178ef58d6d67.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libsocket2-811a7a403c31caed.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libh2-477112006cb6758a.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libindexmap-b32181fdc7afcc4c.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libtokio_util-72f75e6a6423678a.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libfutures_sink-2577d8e12fcf6991.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libhttparse-648ab1cdd3cbae9c.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libtower_service-9282d801dbb17df2.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libfutures_channel-35431632760d3376.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libpin_project-a21d56dd1042478a.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libtracing-7c279f1a677a4fa0.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libtracing_core-a2c6a62ee372e2c8.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libtokio-3879a93bf360f2b5.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libmio-56ac13146c4b2860.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libhttp_body-34a615cfe7b2c0c6.rlib" "/home/mathis/Dev/rust-bert/target/
debug/deps/libfutures_util-b7a30bee20f5fe71.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libfutures_io-f314da869344a9a8.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libslab-dd6edc0ed1ca4f6b.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libpin_project_lite-06d42e9781056991.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libfutures_task-b04b5f0e69e53ffb.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libpin_utils-20f290c37612163f.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libfutures_core-a16351367c097e6e.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/liburl-4c6ee3061aae65b7.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libidna-3dd9383909fbc055.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libunicode_bidi-0fe025d2b861b5b3.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libform_urlencoded-93c34c4329f24067.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libpercent_encoding-bae4c6d884296e4c.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libmatches-b31feef5c3ce1062.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libhttp-241165acd5255fd4.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libbytes-0ff5e007e0c7e08c.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libfnv-eef866999c50c67e.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/liblog-2f3f18ae3a68c2f8.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libglob-c57da5286c7fae6e.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libfs2-a48de2555a8471db.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libzip_extensions-353c3f8d4e9ed7c3.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libtempfile-cb2a539468ebc1f7.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/librand-1c75c86b13b5580a.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/librand_chacha-256b875f4a5a3d6b.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/librand_core-d1cdd31427f095d4.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libgetrandom-7dc64231f7b99450.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libremove_dir_all-b44ab79d858f6f76.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/librust_tokenizers-d60a711620946171.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libcsv-c5db6ce464e9affd.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libcsv_core-ebe164817eda8c9b.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libbstr-4668142ce77a56b7.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libregex_automata-8cb0baf15346bda4.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libserde_json-56cb869844da73de.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libryu-bc3a68e8fbaa57e5.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libitoa-f9dbccd7af7d6b3a.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libhashbrown-60d09b3993522df1.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libahash-749bc2fb31b31530.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libprotobuf-c58299b47f54fed5.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libunicode_normalization_alignments-206b7fe2a9ea75f1.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libsmallvec-0f55f5d6a54014e3.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libunicode_normalization-67b65b54ebb28bc8.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libtinyvec-2a8082d1c3b39837.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libtinyvec_macros-a8a3510ab9d2d66d.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libregex-837648ba3d68bb20.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libaho_corasick-a0f2b910ddf545ab.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libmemchr-abe191a7fe2cb81e.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libregex_syntax-d8c25f7d5a9c013c.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/librayon-2d0d238453171a73.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/librayon_core-f097f6ff0debde4e.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libnum_cpus-db12bd77386e7612.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libcrossbeam_deque-54b2fd51490e9920.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libcrossbeam_epoch-f62cfc549debd93a.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libmemoffset-833e5f62b4ed8191.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libscopeguard-4c9e30f2e7684fc3.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libcrossbeam_channel-041e91dffad4bb57.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libcrossbeam_utils-d38a60b6601ae3a6.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libitertools-4d8dca7ba26edf3b.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libeither-976ba12e28a58b3f.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libserde-3f736418ac579112.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libordered_float-9d94cc6b3ed64167.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libtch-3489097ebbda946c.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libndarray-5e50a2ad469eca8a.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libmatrixmultiply-e953bca71897c4d4.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libnum_complex-94ed3827aa717c55.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libnum_integer-1111d064da27e043.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libnum_traits-a8f43e054a44200b.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/librawpointer-5f566245f6baa3b8.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/librand-c96fafd91f6fb884.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/librand_chacha-b3dc7a75b6fcc1e7.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libppv_lite86-954a0753d9a936e0.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/librand_core-d163b764f498228a.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libgetrandom-75286b24b01f3ec6.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libhalf-3df75a818605f784.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libtorch_sys-73690ae916a3995c.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libzip-a0e4d565bc7812ee.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libtime-ac2f880c23971580.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libbzip2-2bf74b0aaaed15ea.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libbzip2_sys-aebd69e543b5cce6.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libflate2-55dac049734f889a.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libminiz_oxide-ae9bfcf09048bed3.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libadler-0dfc8a797665414c.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libbyteorder-4126bddb3356bb8d.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libcrc32fast-516f19d8f84806f9.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libcfg_if-2909e55669d95bfc.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libthiserror-849666d06ac7e440.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/liblazy_static-cd2401a06b1676f6.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libdirs-d779fef0aee33dfd.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libdirs_sys-d14c8c2444a3fc5b.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/liblibc-d51a272aa641e225.rlib" "/home/mathis/Dev/rust-bert/target/debug/deps/libanyhow-deb63606ff1cfc02.rlib" "-Wl,--start-group" "/home/mathis/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-6e0e72ef3f331f94.rlib" "/home/mathis/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libpanic_unwind-eed7c8ea6eea20e8.rlib" "/home/mathis/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libminiz_oxide-637cb1b53c807e95.rlib" "/home/mathis/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libadler-099cf0af4375543b.rlib" "/home/mathis/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libobject-b5f18e83369ef257.rlib" "/home/mathis/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libaddr2line-3bb19daa4485d5fe.rlib" "/home/mathis/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libgimli-5298ab0591e7fb29.rlib" "/home/mathis/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_demangle-2e4947d254d0b599.rlib" "/home/mathis/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libhashbrown-1532436f783b0405.rlib" "/home/mathis/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_alloc-3d12d76f5782439f.rlib" "/home/mathis/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libunwind-87f8d20d4e058c86.rlib" "/home/mathis/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcfg_if-d41f1ff31e4e0f27.rlib" "/home/mathis/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-0d5ea4f2d39b8e27.rlib" "/home/mathis/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liballoc-31288459e6a43502.rlib" "/home/mathis/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_core-c52e5d6301e1bd59.rlib" "/home/mathis/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-2675a9a46b5cec89.rlib" "-Wl,--end-group" "/home/mathis/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-f51baad7bbcb81c4.rlib" "-Wl,-Bdynamic" "-lssl" "-lcrypto" "-lstdc++" "-ltorch" "-ltorch_cpu" "-lc10" "-lgomp" "-lbz2" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc"
  = note: /usr/bin/ld: /home/mathis/Dev/rust-bert/target/debug/deps/libtorch_sys-73690ae916a3995c.rlib(torch_api.o): in function `caffe2::TypeMeta::TypeMeta()':
          /home/mathis/Dev/rust-bert/target/debug/build/torch-sys-d810dc21875d10ae/out/libtorch/libtorch/include/c10/util/typeid.h:457: undefined reference to `caffe2::detail::TypeMetaData const* caffe2::TypeMeta::_typeMetaDataInstance<caffe2::detail::_Uninitialized>()'
          /usr/bin/ld: /home/mathis/Dev/rust-bert/target/debug/deps/libtorch_sys-73690ae916a3995c.rlib(torch_api.o): in function `caffe2::TypeMeta caffe2::TypeMeta::Make<unsigned char>()':
          /home/mathis/Dev/rust-bert/target/debug/build/torch-sys-d810dc21875d10ae/out/libtorch/libtorch/include/c10/util/typeid.h:439: undefined reference to `caffe2::detail::TypeMetaData const* caffe2::TypeMeta::_typeMetaDataInstance<unsigned char>()'
          /usr/bin/ld: /home/mathis/Dev/rust-bert/target/debug/deps/libtorch_sys-73690ae916a3995c.rlib(torch_api.o): in function `caffe2::TypeMeta caffe2::TypeMeta::Make<signed char>()':
          /home/mathis/Dev/rust-bert/target/debug/build/torch-sys-d810dc21875d10ae/out/libtorch/libtorch/include/c10/util/typeid.h:439: undefined reference to `caffe2::detail::TypeMetaData const* caffe2::TypeMeta::_typeMetaDataInstance<signed char>()'
          /usr/bin/ld: /home/mathis/Dev/rust-bert/target/debug/deps/libtorch_sys-73690ae916a3995c.rlib(torch_api.o): in function `caffe2::TypeMeta caffe2::TypeMeta::Make<short>()':
          /home/mathis/Dev/rust-bert/target/debug/build/torch-sys-d810dc21875d10ae/out/libtorch/libtorch/include/c10/util/typeid.h:439: undefined reference to `caffe2::detail::TypeMetaData const* caffe2::TypeMeta::_typeMetaDataInstance<short>()'
          /usr/bin/ld: /home/mathis/Dev/rust-bert/target/debug/deps/libtorch_sys-73690ae916a3995c.rlib(torch_api.o): in function `caffe2::TypeMeta caffe2::TypeMeta::Make<int>()':
          /home/mathis/Dev/rust-bert/target/debug/build/torch-sys-d810dc21875d10ae/out/libtorch/libtorch/include/c10/util/typeid.h:439: undefined reference to `caffe2::detail::TypeMetaData const* caffe2::TypeMeta::_typeMetaDataInstance<int>()'
          /usr/bin/ld: /home/mathis/Dev/rust-bert/target/debug/deps/libtorch_sys-73690ae916a3995c.rlib(torch_api.o): in function `caffe2::TypeMeta caffe2::TypeMeta::Make<long>()':
          /home/mathis/Dev/rust-bert/target/debug/build/torch-sys-d810dc21875d10ae/out/libtorch/libtorch/include/c10/util/typeid.h:439: undefined reference to `caffe2::detail::TypeMetaData const* caffe2::TypeMeta::_typeMetaDataInstance<long>()'
          /usr/bin/ld: /home/mathis/Dev/rust-bert/target/debug/deps/libtorch_sys-73690ae916a3995c.rlib(torch_api.o): in function `caffe2::TypeMeta caffe2::TypeMeta::Make<c10::Half>()':
          /home/mathis/Dev/rust-bert/target/debug/build/torch-sys-d810dc21875d10ae/out/libtorch/libtorch/include/c10/util/typeid.h:439: undefined reference to `caffe2::detail::TypeMetaData const* caffe2::TypeMeta::_typeMetaDataInstance<c10::Half>()'
          /usr/bin/ld: /home/mathis/Dev/rust-bert/target/debug/deps/libtorch_sys-73690ae916a3995c.rlib(torch_api.o): in function `caffe2::TypeMeta caffe2::TypeMeta::Make<float>()':
          /home/mathis/Dev/rust-bert/target/debug/build/torch-sys-d810dc21875d10ae/out/libtorch/libtorch/include/c10/util/typeid.h:439: undefined reference to `caffe2::detail::TypeMetaData const* caffe2::TypeMeta::_typeMetaDataInstance<float>()'
          /usr/bin/ld: /home/mathis/Dev/rust-bert/target/debug/deps/libtorch_sys-73690ae916a3995c.rlib(torch_api.o): in function `caffe2::TypeMeta caffe2::TypeMeta::Make<double>()':
          /home/mathis/Dev/rust-bert/target/debug/build/torch-sys-d810dc21875d10ae/out/libtorch/libtorch/include/c10/util/typeid.h:439: undefined reference to `caffe2::detail::TypeMetaData const* caffe2::TypeMeta::_typeMetaDataInstance<double>()'
          /usr/bin/ld: /home/mathis/Dev/rust-bert/target/debug/deps/libtorch_sys-73690ae916a3995c.rlib(torch_api.o): in function `caffe2::TypeMeta caffe2::TypeMeta::Make<c10::complex<c10::Half> >()':
          /home/mathis/Dev/rust-bert/target/debug/build/torch-sys-d810dc21875d10ae/out/libtorch/libtorch/include/c10/util/typeid.h:439: undefined reference to `caffe2::detail::TypeMetaData const* caffe2::TypeMeta::_typeMetaDataInstance<c10::complex<c10::Half> >()'
          /usr/bin/ld: /home/mathis/Dev/rust-bert/target/debug/deps/libtorch_sys-73690ae916a3995c.rlib(torch_api.o): in function `caffe2::TypeMeta caffe2::TypeMeta::Make<c10::complex<float> >()':
          /home/mathis/Dev/rust-bert/target/debug/build/torch-sys-d810dc21875d10ae/out/libtorch/libtorch/include/c10/util/typeid.h:439: undefined reference to `caffe2::detail::TypeMetaData const* caffe2::TypeMeta::_typeMetaDataInstance<c10::complex<float> >()'
          /usr/bin/ld: /home/mathis/Dev/rust-bert/target/debug/deps/libtorch_sys-73690ae916a3995c.rlib(torch_api.o): in function `caffe2::TypeMeta caffe2::TypeMeta::Make<c10::complex<double> >()':
          /home/mathis/Dev/rust-bert/target/debug/build/torch-sys-d810dc21875d10ae/out/libtorch/libtorch/include/c10/util/typeid.h:439: undefined reference to `caffe2::detail::TypeMetaData const* caffe2::TypeMeta::_typeMetaDataInstance<c10::complex<double> >()'
          /usr/bin/ld: /home/mathis/Dev/rust-bert/target/debug/deps/libtorch_sys-73690ae916a3995c.rlib(torch_api.o): in function `caffe2::TypeMeta caffe2::TypeMeta::Make<bool>()':
          /home/mathis/Dev/rust-bert/target/debug/build/torch-sys-d810dc21875d10ae/out/libtorch/libtorch/include/c10/util/typeid.h:439: undefined reference to `caffe2::detail::TypeMetaData const* caffe2::TypeMeta::_typeMetaDataInstance<bool>()'
          /usr/bin/ld: /home/mathis/Dev/rust-bert/target/debug/deps/libtorch_sys-73690ae916a3995c.rlib(torch_api.o): in function `caffe2::TypeMeta caffe2::TypeMeta::Make<c10::qint8>()':
          /home/mathis/Dev/rust-bert/target/debug/build/torch-sys-d810dc21875d10ae/out/libtorch/libtorch/include/c10/util/typeid.h:439: undefined reference to `caffe2::detail::TypeMetaData const* caffe2::TypeMeta::_typeMetaDataInstance<c10::qint8>()'
          /usr/bin/ld: /home/mathis/Dev/rust-bert/target/debug/deps/libtorch_sys-73690ae916a3995c.rlib(torch_api.o): in function `caffe2::TypeMeta caffe2::TypeMeta::Make<c10::quint8>()':
          /home/mathis/Dev/rust-bert/target/debug/build/torch-sys-d810dc21875d10ae/out/libtorch/libtorch/include/c10/util/typeid.h:439: undefined reference to `caffe2::detail::TypeMetaData const* caffe2::TypeMeta::_typeMetaDataInstance<c10::quint8>()'
          /usr/bin/ld: /home/mathis/Dev/rust-bert/target/debug/deps/libtorch_sys-73690ae916a3995c.rlib(torch_api.o): in function `caffe2::TypeMeta caffe2::TypeMeta::Make<c10::qint32>()':
          /home/mathis/Dev/rust-bert/target/debug/build/torch-sys-d810dc21875d10ae/out/libtorch/libtorch/include/c10/util/typeid.h:439: undefined reference to `caffe2::detail::TypeMetaData const* caffe2::TypeMeta::_typeMetaDataInstance<c10::qint32>()'
          /usr/bin/ld: /home/mathis/Dev/rust-bert/target/debug/deps/libtorch_sys-73690ae916a3995c.rlib(torch_api.o): in function `caffe2::TypeMeta caffe2::TypeMeta::Make<c10::BFloat16>()':
          /home/mathis/Dev/rust-bert/target/debug/build/torch-sys-d810dc21875d10ae/out/libtorch/libtorch/include/c10/util/typeid.h:439: undefined reference to `caffe2::detail::TypeMetaData const* caffe2::TypeMeta::_typeMetaDataInstance<c10::BFloat16>()'
          /usr/bin/ld: /home/mathis/Dev/rust-bert/target/debug/build/torch-sys-d810dc21875d10ae/out/libtorch/libtorch/lib/libtorch_cpu.so: undefined reference to `c10::C10FlagsRegistry[abi:cxx11]()'
          /usr/bin/ld: /home/mathis/Dev/rust-bert/target/debug/build/torch-sys-d810dc21875d10ae/out/libtorch/libtorch/lib/libtorch_cpu.so: undefined reference to `c10::TensorImpl::TensorImpl(c10::Storage&&, c10::DispatchKeySet, caffe2::TypeMeta const&)'
          /usr/bin/ld: /home/mathis/Dev/rust-bert/target/debug/build/torch-sys-d810dc21875d10ae/out/libtorch/libtorch/lib/libtorch_cpu.so: undefined reference to `c10::TensorImpl::TensorImpl(c10::DispatchKeySet, caffe2::TypeMeta const&, c10::optional<c10::Device>)'
          /usr/bin/ld: /home/mathis/Dev/rust-bert/target/debug/build/torch-sys-d810dc21875d10ae/out/libtorch/libtorch/lib/libtorch_cpu.so: undefined reference to `c10::impl::tls_local_dispatch_key_set()'
          /usr/bin/ld: /home/mathis/Dev/rust-bert/target/debug/build/torch-sys-d810dc21875d10ae/out/libtorch/libtorch/lib/libtorch_cpu.so: undefined reference to `bool c10::C10FlagParser::Parse<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*)'
          /usr/bin/ld: /home/mathis/Dev/rust-bert/target/debug/build/torch-sys-d810dc21875d10ae/out/libtorch/libtorch/lib/libtorch_cpu.so: undefined reference to `c10::TensorImpl::TensorImpl(c10::DispatchKeySet, caffe2::TypeMeta const&, c10::optional<c10::Device>)'
          /usr/bin/ld: /home/mathis/Dev/rust-bert/target/debug/build/torch-sys-d810dc21875d10ae/out/libtorch/libtorch/lib/libtorch_cpu.so: undefined reference to `c10::MessageLogger::MessageLogger(char const*, int, int)'
          /usr/bin/ld: /home/mathis/Dev/rust-bert/target/debug/build/torch-sys-d810dc21875d10ae/out/libtorch/libtorch/lib/libtorch_cpu.so: undefined reference to `caffe2::detail::TypeMetaData const* caffe2::TypeMeta::_typeMetaDataInstance<char>()'
          /usr/bin/ld: /home/mathis/Dev/rust-bert/target/debug/build/torch-sys-d810dc21875d10ae/out/libtorch/libtorch/lib/libtorch_cpu.so: undefined reference to `FLAGS_caffe2_keep_on_shrink'
          /usr/bin/ld: /home/mathis/Dev/rust-bert/target/debug/build/torch-sys-d810dc21875d10ae/out/libtorch/libtorch/lib/libtorch_cpu.so: undefined reference to `c10::TensorImpl::TensorImpl(c10::Storage&&, c10::DispatchKeySet, caffe2::TypeMeta const&)'
          /usr/bin/ld: /home/mathis/Dev/rust-bert/target/debug/build/torch-sys-d810dc21875d10ae/out/libtorch/libtorch/lib/libtorch_cpu.so: undefined reference to `caffe2::detail::TypeMetaData const* caffe2::TypeMeta::_typeMetaDataInstance<std::unique_ptr<std::mutex, std::default_delete<std::mutex> > >()'
          /usr/bin/ld: /home/mathis/Dev/rust-bert/target/debug/build/torch-sys-d810dc21875d10ae/out/libtorch/libtorch/lib/libtorch_cpu.so: undefined reference to `c10::MessageLogger::~MessageLogger()'
          /usr/bin/ld: /home/mathis/Dev/rust-bert/target/debug/build/torch-sys-d810dc21875d10ae/out/libtorch/libtorch/lib/libtorch_cpu.so: undefined reference to `caffe2::detail::TypeMetaData const* caffe2::TypeMeta::_typeMetaDataInstance<unsigned short>()'
          /usr/bin/ld: /home/mathis/Dev/rust-bert/target/debug/build/torch-sys-d810dc21875d10ae/out/libtorch/libtorch/lib/libtorch_cpu.so: undefined reference to `caffe2::detail::TypeMetaData const* caffe2::TypeMeta::_typeMetaDataInstance<std::unique_ptr<std::atomic<bool>, std::default_delete<std::atomic<bool> > > >()'
          /usr/bin/ld: /home/mathis/Dev/rust-bert/target/debug/build/torch-sys-d810dc21875d10ae/out/libtorch/libtorch/lib/libtorch_cpu.so: undefined reference to `bool c10::C10FlagParser::Parse<int>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int*)'
          /usr/bin/ld: /home/mathis/Dev/rust-bert/target/debug/build/torch-sys-d810dc21875d10ae/out/libtorch/libtorch/lib/libtorch_cpu.so: undefined reference to `FLAGS_caffe2_max_keep_on_shrink_memory'
          /usr/bin/ld: /home/mathis/Dev/rust-bert/target/debug/build/torch-sys-d810dc21875d10ae/out/libtorch/libtorch/lib/libtorch_cpu.so: undefined reference to `caffe2::detail::TypeMetaData const* caffe2::TypeMeta::_typeMetaDataInstance<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >()'
          /usr/bin/ld: /home/mathis/Dev/rust-bert/target/debug/build/torch-sys-d810dc21875d10ae/out/libtorch/libtorch/lib/libtorch_cpu.so: undefined reference to `bool c10::C10FlagParser::Parse<double>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, double*)'
          /usr/bin/ld: /home/mathis/Dev/rust-bert/target/debug/build/torch-sys-d810dc21875d10ae/out/libtorch/libtorch/lib/libtorch_cpu.so: undefined reference to `bool c10::C10FlagParser::Parse<bool>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool*)'
          /usr/bin/ld: /home/mathis/Dev/rust-bert/target/debug/build/torch-sys-d810dc21875d10ae/out/libtorch/libtorch/lib/libtorch_cpu.so: undefined reference to `caffe2::detail::TypeMetaData const* caffe2::TypeMeta::_typeMetaDataInstance<std::vector<int, std::allocator<int> > >()'
          /usr/bin/ld: /usr/bin/ld generated: undefined reference to `caffe2::detail::TypeMetaData const* caffe2::TypeMeta::_typeMetaDataInstance<long>()'
          /usr/bin/ld: /usr/bin/ld generated: undefined reference to `caffe2::detail::TypeMetaData const* caffe2::TypeMeta::_typeMetaDataInstance<c10::complex<float> >()'
          /usr/bin/ld: /usr/bin/ld generated: undefined reference to `caffe2::detail::TypeMetaData const* caffe2::TypeMeta::_typeMetaDataInstance<c10::BFloat16>()'
          /usr/bin/ld: /usr/bin/ld generated: undefined reference to `caffe2::detail::TypeMetaData const* caffe2::TypeMeta::_typeMetaDataInstance<c10::complex<double> >()'
          /usr/bin/ld: /usr/bin/ld generated: undefined reference to `caffe2::detail::TypeMetaData const* caffe2::TypeMeta::_typeMetaDataInstance<bool>()'
          /usr/bin/ld: /usr/bin/ld generated: undefined reference to `caffe2::detail::TypeMetaData const* caffe2::TypeMeta::_typeMetaDataInstance<c10::Half>()'
          /usr/bin/ld: /usr/bin/ld generated: undefined reference to `caffe2::detail::TypeMetaData const* caffe2::TypeMeta::_typeMetaDataInstance<float>()'
          /usr/bin/ld: /usr/bin/ld generated: undefined reference to `caffe2::detail::TypeMetaData const* caffe2::TypeMeta::_typeMetaDataInstance<int>()'
          /usr/bin/ld: /usr/bin/ld generated: undefined reference to `caffe2::detail::TypeMetaData const* caffe2::TypeMeta::_typeMetaDataInstance<double>()'
          /usr/bin/ld: /usr/bin/ld generated: undefined reference to `caffe2::detail::TypeMetaData const* caffe2::TypeMeta::_typeMetaDataInstance<c10::complex<c10::Half> >()'
          /usr/bin/ld: /usr/bin/ld generated: undefined reference to `caffe2::detail::TypeMetaData const* caffe2::TypeMeta::_typeMetaDataInstance<c10::quint8>()'
          /usr/bin/ld: /usr/bin/ld generated: undefined reference to `caffe2::detail::TypeMetaData const* caffe2::TypeMeta::_typeMetaDataInstance<c10::qint32>()'
          /usr/bin/ld: /usr/bin/ld generated: undefined reference to `caffe2::detail::TypeMetaData const* caffe2::TypeMeta::_typeMetaDataInstance<c10::qint8>()'
          /usr/bin/ld: /usr/bin/ld generated: undefined reference to `caffe2::detail::TypeMetaData const* caffe2::TypeMeta::_typeMetaDataInstance<unsigned char>()'
          /usr/bin/ld: /usr/bin/ld generated: undefined reference to `caffe2::detail::TypeMetaData const* caffe2::TypeMeta::_typeMetaDataInstance<caffe2::detail::_Uninitialized>()'
          /usr/bin/ld: /usr/bin/ld generated: undefined reference to `caffe2::detail::TypeMetaData const* caffe2::TypeMeta::_typeMetaDataInstance<signed char>()'
          /usr/bin/ld: /usr/bin/ld generated: undefined reference to `caffe2::detail::TypeMetaData const* caffe2::TypeMeta::_typeMetaDataInstance<short>()'
          collect2: error: ld returned 1 exit status
          

error: aborting due to previous error

error: could not compile `rust-bert`

Caused by:
  process didn't exit successfully: `rustc --crate-name generation --edition=2018 examples/generation.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debuginfo=2 -C metadata=1079cbdd0923672d -C extra-filename=-1079cbdd0923672d --out-dir /home/mathis/Dev/rust-bert/target/debug/examples -C incremental=/home/mathis/Dev/rust-bert/target/debug/incremental -L dependency=/home/mathis/Dev/rust-bert/target/debug/deps --extern anyhow=/home/mathis/Dev/rust-bert/target/debug/deps/libanyhow-deb63606ff1cfc02.rlib --extern cached_path=/home/mathis/Dev/rust-bert/target/debug/deps/libcached_path-ed4f91024ed4fc3e.rlib --extern criterion=/home/mathis/Dev/rust-bert/target/debug/deps/libcriterion-11283086a0db42c9.rlib --extern csv=/home/mathis/Dev/rust-bert/target/debug/deps/libcsv-c5db6ce464e9affd.rlib --extern dirs=/home/mathis/Dev/rust-bert/target/debug/deps/libdirs-d779fef0aee33dfd.rlib --extern lazy_static=/home/mathis/Dev/rust-bert/target/debug/deps/liblazy_static-cd2401a06b1676f6.rlib --extern ordered_float=/home/mathis/Dev/rust-bert/target/debug/deps/libordered_float-9d94cc6b3ed64167.rlib --extern rust_bert=/home/mathis/Dev/rust-bert/target/debug/deps/librust_bert-2ec9c634d44292b9.rlib --extern rust_tokenizers=/home/mathis/Dev/rust-bert/target/debug/deps/librust_tokenizers-d60a711620946171.rlib --extern serde=/home/mathis/Dev/rust-bert/target/debug/deps/libserde-3f736418ac579112.rlib --extern serde_json=/home/mathis/Dev/rust-bert/target/debug/deps/libserde_json-56cb869844da73de.rlib --extern tch=/home/mathis/Dev/rust-bert/target/debug/deps/libtch-3489097ebbda946c.rlib --extern tempfile=/home/mathis/Dev/rust-bert/target/debug/deps/libtempfile-cb2a539468ebc1f7.rlib --extern thiserror=/home/mathis/Dev/rust-bert/target/debug/deps/libthiserror-849666d06ac7e440.rlib --extern torch_sys=/home/mathis/Dev/rust-bert/target/debug/deps/libtorch_sys-73690ae916a3995c.rlib --extern uuid=/home/mathis/Dev/rust-bert/target/debug/deps/libuuid-bf8e4fb0cc067046.rlib -L native=/usr/lib -L native=/home/mathis/Dev/rust-bert/target/debug/build/torch-sys-d810dc21875d10ae/out/libtorch/libtorch/lib -L native=/home/mathis/Dev/rust-bert/target/debug/build/torch-sys-d810dc21875d10ae/out` (exit code: 1)

After taking a look at #127 I tried to downgrade to libtorch 1.7.1 but without success. I also tried to follow the "Libtorch Manual Install" documentation from the tch-rs crate.

Best regards,

ConversationModel panic on large input (with 3-4 lines of text)

thread 'tokio-runtime-worker' panicked at 'called Result::unwrap() on an Err value: Torch("index out of range in self\nException raised from operator() at ../aten/src/ATen/native/TensorAdvancedIndexing.cpp:677 (most recent call first):\nframe #0: c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string<char, std::char_traits, std::allocator >) + 0x69 (0x7f634c696b29 in Amadeus/target/release/build/torch-sys-8dc810d88f246fa1/out/libtorch/libtorch/lib/libc10.so)\nframe #1: + 0xc12390 (0x7f634d4e5390 in Amadeus/target/release/build/torch-sys-8dc810d88f246fa1/out/libtorch/libtorch/lib/libtorch_cpu.so)\nframe #2: + 0x10e5d4a (0x7f634d9b8d4a in Amadeus/target/release/build/torch-sys-8dc810d88f246fa1/out/libtorch/libtorch/lib/libtorch_cpu.so)\nframe #3: + 0x1b766 (0x7f634c61e766 in gcc/x86_64-pc-linux-gnu/11.1.0/libgomp.so.1)\nframe #4: + 0x7e3e (0x7f634a1d5e3e in /lib64/libpthread.so.0)\nframe #5: clone + 0x3f (0x7f6349fd173f in libc.so.6)\n")', .cargo/registry/src/github.com-1ecc6299db9ec823/tch-0.4.1/src/wrappers/tensor_generated.rs:3649:87

I think models leak

I suggest writing memory leak test disabled by default for models.
Use ::new to load models in scope several times and do things with them.

Add memory check after all loops done.

For my service/bot I store all the models in lazystatic mutex for now as workaround.
(For my case it's also nice for speed up but in general I think leaks are bad)

replacing the python utils with rust code?

Do you mind going ahead with replacing the python utilities with rust code?

The python utilis scripts are segfaulting (!!) on my system (possibly due to out of memory). Even the first import alone causes a segfault. I've been careful to create a fresh venv using python 3.6 and installed torch torchvision and transformers only in the venv on an up-to-date linux system running cuda 10.1 (several different versions of python actually including 3.7 and 3.8).

This line alone causes a segfault:

from transformers import BERT_PRETRAINED_CONFIG_ARCHIVE_MAP

About Activation visibility in Config implementors

Hi again,

It looks like Config implementors, surch as BertConfig, have a public Activation field but this enum is not public.
It is thus impossible to store it as a field in a struct (its type must be known and accessible).

I feel that publicly exposed types should be public, what do you think ?

Also, do you think BertConfig and its friends could implement Clone ? It seems that nothing is preventing it (Activation is clonable).

What exact model is downloaded when using `Gpt2ModelResources::GPT2`?

First, thanks for you work!

I would like to know which exact model is downloaded from from ...models.huggingface.co/bert/gpt2-rust_model.ot. Is this the 1.5B model?

Error with resources

Dearest Maintainer,

I hope all is well. I like the Summarization model, however, sometimes I would get text like "Please email us at [email protected]." I thought I would try out BART vs BART_CNN. I copied the config from default and removed the CNN. I loaded this in to the file in the example folder.

https://gist.github.com/sbeckeriv/db5711f86be38df6bb23ab34e803afd4

Running this gives this error:

Error: ErrorMessage { msg: "cannot find model.encoder.layers.4.fc1.bias in \"/home/becker/.cache/.rustbert/bart/model.ot\"" }

any suggestions on how i can load the BART data vs BART_CNN would help appreciated.

Thanks again,
Becker

Full names as single entity in NER

Very cool idea to use the Huggingface models from rust!

I've been looking at integrating NER with an existing rust tool to help automatically index entities mentioned in the text. I don't know much about NER or NLP, but when I tested some examples with NLTK, it extracted full names as a single entity, which is very helpful. Is there a way to configure this library to do the same? I've tried various different models and configurations but I can't seem to find one that works that way. And I've had some trouble finding information about this particular problem online.

NLTK:

Asked John Smith about Acme Corp

(S
  Asked/VBN
  (PERSON John/NNP Smith/NNP)
  about/IN
  (PERSON Acme/NNP Corp/NNP))

rust-bert:

Asked John Smith about Acme Corp

[Entity { word: "John", score: 0.9973952174186707, label: "I-PER" }, Entity { word: "Smith", score: 0.9898008108139038, label: "I-PER" }, Entity { word: "Acme", score: 0.9634051628696118, label: "I-ORG" }, Entity { word: "Corp", score: 0.9988095760345459, label: "I-ORG" }]

Make downloads asyncronous

I failed to integrate this library into my application which natively uses async/await and tokio.
I simply cannot create a model, because I get

thread 'main' panicked at 'Cannot start a runtime from within a runtime.
This happens because a function (like `block_on`) attempted to block the current thread

every time I try to create SentimentModel::new().
I may be a newbie in tokio, so I tried

let model = tokio::task::block_in_place(|| SentimentModel::new(Default::default())).unwrap();

But this didn't solve the problem.
Also doing this:

let model = tokio::task::spawn_blocking(|| SentimentModel::new(Default::default()).unwrap())
    .await.unwrap();

also doesn't work because Rust complaints about sending some internal std::rc::Rc between threads. Even wrapping it all with Arc<Mutex<SentimentModel>> doesn't help.
I am simply perplexed and don't know how to use this crate inside of async fns...

Failure library is being deprecated

I happened to come across the news that the failure crate is being deprecated, and was aware this library is used by rust-bert: see rust-lang-deprecated/failure#347

This probably isn't a priority, but something you might want to be aware of at least.

guillaume-be / rust-bert Goto Github PK

rust-bert's People

Contributors

Stargazers

Watchers

Forkers

rust-bert's Issues

Problem

Trying to fix it

The actual Question, finally

Recommend Projects

Recommend Topics

Recommend Org

Jobs