reinaldncku / recsys

References and resources for neural recommender systems research.

References for Neural Recommender Systems Research

This is a repository containing reference materials that may be helpful in jumpstarting your research on neural, review-based recommender systems, with a special focus on collaborative filtering. Be sure to click the text in blue (like this) so that you will be redirected to the appropriate reference!

Prelude: Types of Recommender Systems

• Content-Based Filtering
• Collaborative Filtering (CF)
• Matrix Factorization (may be considered a special case of CF)

Kindly read this PPT presentation that details their differences and traditional (non-neural) implementations.

Foundational Neural Recommender Models

NCF

This time, we will talk about implementing collaborative filtering using deep learning and neural networks, AKA neural collaborative filtering (NCF). Its differences with matrix factorization are highlighted in this paper by Rendle et al. (2020).

Designing a CF model involves two crucial steps or procedures. The first step is learning user and item representations, and the second step is modeling user-item interactions based on those representations. One of the fundamental works in the utilization of NN in CF is called neural collaborative filtering (NCF) (He et al., 2017). NCF, originally implemented for implicit feedback data-driven CF, learns non-linear, flexible, and more abstractive interactions between users and items by employing MLP layers as its interaction function. An MLP-based interaction function overcomes the limitations of an inner product-based interaction function. The latter is said to be sub-optimal to learn rich yet complicated patterns from real-world data.

This PPT file contains a TL;DR version of the NCF paper. Moreover, this link provides helpful codes (basic implementations).

Before continuing any further, it is essential that you carefully understand the concepts and principles surrounding NCF.

DeepCoNN and NARRE

Arguably, DeepCoNN and NARRE are the first review-based recommender systems to effectively utilize neural networks, thereby leading to state-of-the-art performances.

DeepCoNN is the first deep learning-based model representing users and items from reviews in a joint manner (Zheng, Noroozi, & Yu, 2017). The model consists of two parallel networks powered by convolutional neural networks (CNN). One network learns user behavior by examining all reviews he has written, and the other network models item properties by exploring all reviews it has received. A shared layer connects these two networks, and factorization machines capture user-item interactions.

Another significant model is NARRE, which shares several similarities with DeepCoNN. NARRE is also composed of two parallel CNN-based networks that are uniquely incorporated with the review-level attention mechanism (Chen et al., 2018). The said mechanism distinguishes each review's usefulness or contribution based on attention weights. As a side-effect, this also leads to review-level explanations; reviews with the highest attention scores are presented as explanations. These weights are then incorporated into the representations of users and items to enhance embedding quality.

Attention-Based Recommender Models

Before you proceed, make sure that you have a grasp of attention mechanisms. The full paper on attention (Vasmani et al., 2017) can be accessed here while a more "digestible" version is here.

Here are some of the attention-based recommenders:

• D-Attn integrates global and local attention to score each word to determine its relevance in a review text (Seo et al., 2017).
• DAML utilizes CNN's local and mutual attention to learn review features (Liu et al., 2019).
• HUITA incorporates a hierarchical, three-tier attention network (Wu et al., 2019).
• MPCN is similar to NARRE, but the former does not rely on convolutional layers. It introduces a review-by-review pointer-based mechanism that is co-attentive in nature to model user-item relationships (Tay, Luu, & Hui, 2018).

BERT-Based Recommender Models

Before you continue, ensure that you are familiar with BERT. There are some helpful video links that discuss BERT by CodeEmporium and Henry AI. Keep in mind that BERT is generally used as automatic feature extractors from reviews.

Notwithstanding, CNN-based approaches fail to consider global context and word frequency information. These two factors are crucial because they can affect recommendation performance. In this regard, NCEM (Feng & Zeng, 2019) and BENEFICT (Pugoy & Kao, 2020) replaces the CNN with a pre-trained BERT model in the parallel user/item networks. BERT's advantage lies in its full retention of global context and word frequency information.