nvidia-merlin / models Goto Github PK

Aha! Link: https://nvaiinfra.aha.io/features/MERLIN-471

Aha! Link: https://nvaiinfra.aha.io/features/MERLIN-474

Currently blocked by:

• #71

Support common retrieval models:

• Two Tower
• Matrix Factorization

Aha! Link: https://nvaiinfra.aha.io/features/MERLIN-489

Aha! Link: https://nvaiinfra.aha.io/requirements/MERLIN-489-2

Aha! Link: https://nvaiinfra.aha.io/features/MERLIN-496

The issue is: How to validate models on large datasets and analyze by additional columns?

In our examples, we predict each batch and store a copy in a separate Tensor + store a copy of the targets. After we iterate over all batches, we concatenate them and calculate the performance.

• What happens, when the dataset is too large, that the predictions and targets do not fit in memory?
• What happens, if we want to calculate the performance of the models per category? (e.g. only new users)? These categories may be not used during training (not part of X).

Aha! Link: https://nvaiinfra.aha.io/features/MERLIN-497

Aha! Link: https://nvaiinfra.aha.io/requirements/MERLIN-482-2

Aha! Link: https://nvaiinfra.aha.io/features/MERLIN-517

Aha! Link: https://nvaiinfra.aha.io/features/MERLIN-482

Currently blocked by:

• #71

Aha! Link: https://nvaiinfra.aha.io/features/MERLIN-475

Aha! Link: https://nvaiinfra.aha.io/features/MERLIN-463

• DLRM
• DCN-v2

Aha! Link: https://nvaiinfra.aha.io/features/MERLIN-463

Aha! Link: https://nvaiinfra.aha.io/requirements/MERLIN-463-1

Aha! Link: https://nvaiinfra.aha.io/requirements/MERLIN-463-1

Aha! Link: https://nvaiinfra.aha.io/features/MERLIN-476

Aha! Link: https://nvaiinfra.aha.io/requirements/MERLIN-463-2

The initial version should support some common ranking models like

• DLRM
• Wide and Deep
• Factorization Machines

DeepFM - Factorization Machine - https://arxiv.org/abs/1703.04247

Learning sophisticated feature interactions behind user behaviors is critical in maximizing CTR for recommender systems. Despite great progress, existing methods seem to have a strong bias towards low- or high-order interactions, or require expertise feature engineering. In this paper, we show that it is possible to derive an end-to-end learning model that emphasizes both low- and high-order feature interactions. The proposed model, DeepFM, combines the power of factorization machines for recommendation and deep learning for feature learning in a new neural network architecture. Compared to the latest Wide & Deep model from Google, DeepFM has a shared input to its "wide" and "deep" parts, with no need of feature engineering besides raw features. Comprehensive experiments are conducted to demonstrate the effectiveness and efficiency of DeepFM over the existing models for CTR prediction, on both benchmark data and commercial data.

Aha! Link: https://nvaiinfa.aha.io/features/MERLIN-727

Aha! Link: https://nvaiinfra.aha.io/features/MERLIN-485

Extract model architectures and convert them to tf.keras.Model sub-classes.

Aha! Link: https://nvaiinfra.aha.io/requirements/MERLIN-482-3

This will fail silently in this branch if hidden_dims and activations are different lengths.

https://github.com/NVIDIA-Merlin/models/blob/feature/retrieval-dnn/merlin_models/tensorflow/models/retrieval/youtube_dnn.py#L39

from merlin_models.tensorflow.models.retrieval import YouTubeDNN


YouTubeDNN(continuous_columns=[], 
           categorical_columns=[], 
           embedding_dims=[], 
           hidden_dims=[128, 128], 
           activations=None)

Solution:

import tensorflow as tf

from merlin_models.tensorflow.layers import DenseFeatures


class YouTubeDNN(tf.keras.Model):
    """
    https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/45530.pdf
    See model architecture diagram in Figure 3
    """

    def __init__(self, continuous_columns, categorical_columns, embedding_dims, hidden_dims=None, activations=None):

        super().__init__()

        hidden_dims = hidden_dims or []
        activations = activations or []

        if len(hidden_dims) != len(activations):
            raise ValueError('"hidden_dims" and "activations" must be the same length.')

        channels = self.channels(continuous_columns, categorical_columns, embedding_dims)

        self.input_layer = DenseFeatures(channels["categorical"] + channels["continuous"])

        self.hidden_layers = []
        for dim, activation in zip(hidden_dims, activations):
            self.hidden_layers.append(
                tf.keras.layers.Dense(
                    dim,
                    activation=activation,
                    activity_regularizer="l2",
                )
            )

    def channels(self, continuous_columns, categorical_columns, embedding_dims):
        if not isinstance(embedding_dims, dict):
            embedding_dims = {col.name: embedding_dims for col in categorical_columns}

        embedding_columns = [
            tf.feature_column.embedding_column(col, embedding_dims[col.name])
            for col in categorical_columns
        ]

        return {"categorical": embedding_columns, "continuous": continuous_columns}

    def call(self, inputs, training=False):
        x = self.input_layer(inputs)
        for layer in self.hidden_layers:
            x = layer(x)
        return x

Commit to solve: 1c7bc3d

Aha! Link: https://nvaiinfra.aha.io/requirements/MERLIN-463-2

From this paper: https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/45530.pdf

• [] MF
• Two-tower

Aha! Link: https://nvaiinfra.aha.io/features/MERLIN-473

Aha! Link: https://nvaiinfra.aha.io/requirements/MERLIN-489-1

We should have pip and conda packages for merlin models.

I think this has something to do with the way input dimensions are handled and the particular format (involving *__values fields) that the DenseLayer implementation requires.

Aha! Link: https://nvaiinfra.aha.io/features/MERLIN-464