Comments (1)
Hello,
both y_pred and y_true are of type torch.Tensor
with the same shape: [batch_size, slate_length]
- y_true values are labels for each slate in the original order. The higher value the more relevant item is according in the context of a slate. Typically relevance takes integer (e. g. from 0 to 4 in MSLR-WEB30K) or binary values.
- y_pred values are real-valued scores from model which are used to produce new order of y_true according to these scores (descending sorting). They're also in the same order as y_true.
As for the question:
are all losses and metrics following this API?
In short: yes. For all the metrics and losses y_pred and y_true are mandatory arguments. Some of them have also additional arguments (most of them with default values specified) e. g. ats
for metrics which specifies top n items of a slate taken into account while calculating a metric.
For some of the losses there are other mandatory arguements - when using ordinal loss you need to pass the number of ordinal values and for pointwise rmse number of unique ground truth values is required.
Take a notice that some arguments of the loss functions can drastically change form of the function e. g. weighing_scheme
for [lambdaLoss].(https://github.com/allegro/allRank/blob/master/allrank/models/losses/lambdaLoss.py)
If in doubts I encourage to take a look at our real-life configs (which corresponds to experiments from our paper) from the reproducibility guide: https://github.com/allegro/allRank/tree/master/reproducibility/configs
Best,
Mikołaj
from allrank.
Related Issues (20)
- How to run "run_example.sh" HOT 3
- Which loss functions are suitable for sentence ordering? HOT 1
- Transformer implementation seemingly not corresponding to paper HOT 3
- Location of loss functions?? HOT 2
- Forward vs Backward Slash in Path definition HOT 3
- AssertionError: val ndcg_5 got 0.7586661577224731. It was expected to be at least 0.76 HOT 2
- Getting started without Docker HOT 1
- How to get predictions from the model? HOT 5
- How to use your own dataset? HOT 2
- How to get predictions for each observation or row in my data. HOT 7
- How to produce predictions? HOT 9
- How to produce predictions?
- How to get prediction score HOT 3
- My experiments consistently underperform in comparison to the paper's reported results. HOT 2
- Any sources for the click model based relabeling mentioned in the readme HOT 2
- About the padding value HOT 2
- RuntimeError Numpy not available HOT 1
- Embeddings
- Question about positional encoding HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from allrank.