Music30 Recommendation System

Introduction

Data exploration

• 30music dataset downloaded from RecSys Lab official website.
• This dataset contains 31M play events from 45K users over 5.6M tracks collected via Last-FM public API in 1 year, from 20 Jan 2014 to 20 Jan 2015 (about 4GB of uncompressed CSV file).
• Play events are grouped into user play sessions, where each session is an ordered list of play events that are assumed to have consequently listened without interruption. Two consecutive play events belong to the same session if the gap is not larger than 800 seconds. The dataset contains 2.7M user sessions, with an average of 11 sessions per user, each one, in turn, composed of 11 tracks, on average.

  

• The 30Music dataset also provides 4.1M user love preferences and 57K user-created playlists. Despite being of undeniable interest for recommender systems, we have not considered explicit feedback from users in our experiments. We focused our attention on implicit feedbacks only - i.e., the user play events and play sessions - since they are the most challenging factors in terms of size.

Methods

Factorized Personalized Markov Chain (FPMC) Recommender

Here we fit the recommedation algorithm over the sessions in the training set.This recommender is based on the following paper:

Rendle, S., Freudenthaler, C., & Schmidt-Thieme, L. (2010). Factorizing personalized Markov chains for next-basket recommendation. Proceedings of the 19th International Conference on World Wide Web - WWW ’10, 811

In short, FPMC factorizes a personalized order-1 transition tensor using Tensor Factorization with pairwise loss function akin to BPR (Bayesian Pairwise Ranking).

TF allows to impute values for the missing transitions between items for each user. For this reason, FPMC can be used for generating personalized recommendations in session-aware recommenders as well.

In this notebook, you will be able to change the number of latent factors and a few other learning hyper-parameters and see the impact on the recommendation quality.

The class FPMCRecommender has the following initialization hyper-parameters:

• n_factor: (optional) the number of latent factors
• learn_rate: (optional) the learning rate
• regular: (optional) the L2 regularization coefficient
• n_epoch: (optional) the number of training epochs
• n_neg: (optional) the number of negative samples used in BPR learning

Prod2Vec Recommender

Here we fit the recommedation algorithm over the sessions in the training set.

This is simplified implementation of the following:

Grbovic, Mihajlo, Vladan Radosavljevic, Nemanja Djuric, Narayan Bhamidipati, Jaikit Savla, Varun Bhagwan, and Doug Sharp. "E-commerce in your inbox: Product recommendations at scale." In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1809-1818. ACM, 2015.

This implementation uses the gensim implementation of Word2Vec to compute item embeddings using the skip-gram model

Recommendations are generated by returning the k-nearest neighbors of the last items in the user profile, whose relevance is weighted using a simple exponential decay (the last item in the user profile is the most relevant one, and the first item the least relevant).

The original paper contains other variants of this algorithm (namely bagged-prod2vec and prod2vec-cluster) which are not subject of this tutorial.

The class Prod2VecRecommender has the following initialization hyper-parameters:

• min_count: the minimum item frequency. Items less frequent that min_count will be pruned
• size: the size of the embeddings
• window: the size of the context window
• decay_alpha: the exponential decay factor used to discount the similarity scores for items back in the user profile. Lower values mean higher discounting of past user interactions. Allows values in [0-1]
• workers: the number of threads used for training

Session-based RNN

Here we fit the recommedation algorithm over the sessions in the training set.

This is a simplified interface to Recurrent Neural Network models for Session-based recommendation. Based on the following two papers:

• Recurrent Neural Networks with Top-k Gains for Session-based Recommendations, Hidasi and Karatzoglou, CIKM 2018
• Personalizing Session-based Recommendation with Hierarchical Recurrent Neural Networks, Quadrana et al, Recsys 2017

we will consider the session-based (non-personalized) version of the algorithm.

Each item in the current user session is first encoded either using 1-hot encoding or a dense embedding vector. The item representation is then forwarded to one or more Gated Reucurrent Unit (GRU) layers, which "mix" the information coming from the past steps of the sequence with the representation of the current item. The last hidden state of the network is finally use to compute the likelihood scores for the next items by using one out of several loss functions (e.g. cross-entropy, BPR, TOP1, BPR-max, TOP1-max, etc.).

For simplicity, we only support 1-hot encoded inputs and the BPR-max loss function here.

The hyper-parameters of the model are:

• session_layers: number of units per layer used at session level. It has to be a list of integers for multi-layer networks, or a integer value for single-layer networks.
• user_layers: number of units per layer used at user level. Required only by personalized models. (None in this case)
• batch_size: the mini-batch size used in training
• learning_rate: the learning rate used in training (Adagrad optimized)
• momentum: the momentum coefficient used in training
• dropout: it's a float value for the hidden-layer(s) dropout.
• epochs: number of training epochs
• personalized: whether to train a personalized model using the HRNN model (False in this case).

NOTE: GRU4Rec originally has many more hyper-parameters. Going through all of them is out, but we suggest to check-out the original source code here if you are interested.

Personalized RNN-based Recommender

Here we fit the recommedation algorithm over the sessions in the training set.

This is a simplified interface to Recurrent Neural Network models for Session-based recommendation. Based on the following two papers:

• Recurrent Neural Networks with Top-k Gains for Session-based Recommendations, Hidasi and Karatzoglou, CIKM 2018
• Personalizing Session-based Recommendation with Hierarchical Recurrent Neural Networks, Quadrana et al, Recsys 2017

we will consider the session-aware (personalized) version of the algorithm.

Each user session goes through a session RNN, which models short-term user preferences. At the end of each session, the state of the session RNN is used to update a user RNN, which models more long-term user preferences. It's state is passed forward to the next session RNN, which can now personalize recommendations depending on both short-term and long-term user interests.

The hyper-parameters of the model are:

• session_layers: number of units per layer used at session level. It has to be a list of integers for multi-layer networks, or a integer value for single-layer networks.
• user_layers: number of units per layer used at user level. Required only by personalized models. It has to be a list of integers for multi-layer networks, or a integer value for single-layer networks.
• batch_size: the mini-batch size used in training
• learning_rate: the learning rate used in training (Adagrad optimized)
• momentum: the momentum coefficient used in training
• dropout: it's a 3-tuple with the values for the dropout of (user hidden, session hidden, user-to-session hidden) layers.
• epochs: number of training epochs
• personalized: whether to train a personalized model using the HRNN model (True in this case).

NOTE: HGRU4Rec originally has more hyper-parameters. Going through all of them is out from the scope, but we suggest to check-out the original source code here in case you are interested.

Evaluation

• the user profile, used to compute recommendations, is composed by the first k events in the sequence
• the ground truth, used for performance evaluation, is composed by the remainder of the sequence.

Evaluation with sequentially revealed user-profiles

TOPN=100
GIVEN_K = 1
LOOK_AHEAD = 1
STEP=1

test_sequences = get_test_sequences(test_data, GIVEN_K)
print('{} sequences available for evaluation'.format(len(test_sequences)))

results = evaluation.sequential_evaluation(recommender,
                                          test_sequences=test_sequences,
                                          given_k=GIVEN_K,
                                          look_ahead=LOOK_AHEAD,
                                          evaluation_functions=METRICS.values(),
                                          top_n=TOPN,
                                          scroll=True,  # scrolling averages metrics over all profile lengths
                                          step=STEP)

print('Sequential evaluation (GIVEN_K={}, LOOK_AHEAD={}, STEP={})'.format(GIVEN_K, LOOK_AHEAD, STEP))
for mname, mvalue in zip(METRICS.keys(), results):
  print('\t{}@{}: {:.4f}'.format(mname, TOPN, mvalue))

Evaluation with "static" user-profiles

TOPN=100
GIVEN_K = 1
LOOK_AHEAD = 'all'
STEP=1

test_sequences = get_test_sequences(test_data, GIVEN_K)
print('{} sequences available for evaluation'.format(len(test_sequences)))

results = evaluation.sequential_evaluation(recommender,
                                          test_sequences=test_sequences,
                                          given_k=GIVEN_K,
                                          look_ahead=LOOK_AHEAD,
                                          evaluation_functions=METRICS.values(),
                                          top_n=TOPN,
                                          scroll=False) 

print('Sequential evaluation (GIVEN_K={}, LOOK_AHEAD={}, STEP={})'.format(GIVEN_K, LOOK_AHEAD, STEP))
for mname, mvalue in zip(METRICS.keys(), results):
  print('\t{}@{}: {:.4f}'.format(mname, TOPN, mvalue))

Next-item Recommendation evaluation

• the length of the recommendation list

GIVEN_K = 1
LOOK_AHEAD = 1
STEP = 1
topn_list = [1, 5, 10, 20, 50, 100]

test_sequences = get_test_sequences(test_data, GIVEN_K)
print('{} sequences available for evaluation'.format(len(test_sequences)))

res_list = []

for topn in topn_list:
    print('Evaluating recommendation lists with length: {}'.format(topn))
    res_tmp = evaluation.sequential_evaluation(recommender,
                                              test_sequences=test_sequences,
                                              given_k=GIVEN_K,
                                              look_ahead=LOOK_AHEAD,
                                              evaluation_functions=METRICS.values(),
                                              top_n=topn,
                                              scroll=True,  # here we average over all profile lengths
                                              step=STEP)
    mvalues = list(zip(METRICS.keys(), res_tmp))
    res_list.append((topn, mvalues))

# show separate plots per metric
fig, axes = plt.subplots(nrows=1, ncols=len(METRICS), figsize=(15,5))
res_list_t = list(zip(*res_list))
for midx, metric in enumerate(METRICS):
    mvalues = [res_list_t[1][j][midx][1] for j in range(len(res_list_t[1]))]
    ax = axes[midx]
    ax.plot(topn_list, mvalues)
    ax.set_title(metric)
    ax.set_xticks(topn_list)
    ax.set_xlabel('List length')

• the length of the user profile.

given_k_list = [1, 2, 3, 4]
LOOK_AHEAD = 1
STEP = 1
TOPN = 20

# ensure that all sequences have the same minimum length 
test_sequences = get_test_sequences(test_data, max(given_k_list))
print('{} sequences available for evaluation'.format(len(test_sequences)))

res_list = []

for gk in given_k_list:
    print('Evaluating profiles having length: {}'.format(gk))
    res_tmp = evaluation.sequential_evaluation(recommender,
                                              test_sequences=test_sequences,
                                              given_k=gk,
                                              look_ahead=LOOK_AHEAD,
                                              evaluation_functions=METRICS.values(),
                                              top_n=TOPN,
                                              scroll=False,  # here we stop at each profile length
                                              step=STEP)
    mvalues = list(zip(METRICS.keys(), res_tmp))
    res_list.append((gk, mvalues))

# show separate plots per metric
fig, axes = plt.subplots(nrows=1, ncols=len(METRICS), figsize=(15,5))
res_list_t = list(zip(*res_list))
for midx, metric in enumerate(METRICS):
    mvalues = [res_list_t[1][j][midx][1] for j in range(len(res_list_t[1]))]
    ax = axes[midx]
    ax.plot(given_k_list, mvalues)
    ax.set_title(metric)
    ax.set_xticks(given_k_list)
    ax.set_xlabel('Profile length')

References

1. https://github.com/mquad/sars_tutorial
2. https://arxiv.org/pdf/1802.08452.pdf
3. https://recsys.acm.org/wp-content/uploads/2019/09/recsys-19-material-predictability.pdf
4. https://mquad.github.io/static/papers/2015-large-scale-music-rec.pdf
5. http://ceur-ws.org/Vol-1441/recsys2015_poster13.pdf
6. Evaluation of Session-based Recommendation Algorithms [arXiv:1803.09587]

FSM