A tutorial to demonstrate the process of training and evaluating various recommender models on a online retail store data. Along with the positive feedbacks like view, add-to-cart, we also have a negative event âremove-from-cartâ.
The complete tutorial (+code) can be found here.
In e-commerce, the number of products in the shelf space are practically infinite. Thus, the users have to navigate through a plethora of options in any category before making a purchase and they often get disinterested in the process soon. E-commerce recommendation algorithms often operate in a challenging environment. For example:
- A large retailer might have huge amounts of data, tens of millions of customers and millions of distinct catalog items.
- Many applications require the results set to be returned in realtime, in no more than half a second, while still producing high-quality recommendations.
- New customers typically have extremely limited information, based on only a few purchases or product ratings.
- Older customers can have a glut of information, based on thousands of purchases and ratings.
- Customer data is volatile: Each interaction provides valuable customer data, and the algorithm must respond immediately to new information.
An essential characteristic in many e-commerce settings is that website visitors can have very specific short-term shopping goals when they browse the site. Relying solely on long-term user models that are pre-trained on historical data can therefore be insufficient for a suitable next-basket recommendation. Simple âreal-timeâ recommendation approaches based, e.g., on impersonalized co-occurrence patterns, on the other hand do not fully exploit the available information about the userâs long-term preference profile. On Amazon.com, for example, each product page contains multiple personalized recommendation lists with different purposes. They comprise complements or alternatives to the currently viewed product, remind the user of recently viewed items, or represent recommendations that should appeal to the general taste of the user. Therefore, the displayed content not only depends on the features of the currently viewed article, like the product category, but can also be influenced, e.g., by a combination of the userâs past shopping behavior (long-term preferences) and his most recent navigation actions (short-term shopping goals).
A good product recommendation engine shall easily use the below data to display a solid list of recommended products:
- Clickstream behaviour: Views, likes, shopper behaviors like âadd to favoriteâ and âadd to cartâ
- Transactions: Date, time, amount, price of the order along with the customer ID
- Stock data: Size, color, model etc. based stock movements
- Social media data: In the case that unstructured data can be matched with a single user
- Customer reviews data: If product reviews are present can be boiled down to product specs
- Retailerâs commercial priorities: Brands/models that should be displayed in the product recommendation set
- Customer lifetime value: Recency, frequency and monetary value of customers
- Popular products: Products with high turnover rate
Problem Statement and Data
Objective is to build a recommender system using implicit feedback data of an online retail store.
User Journey
Data Snapshot
System Design
Data Transformation
Many times there are no explicit ratings or preferences given by users, that is, the interactions are usually implicit. This information may reflect usersâ preference towards the items in an implicit manner.
Option 1 â Simple Count: The most simple technique is to count times of interactions between user and item for producing affinity scores.
Option 2 â Weighted Count: It is useful to consider the types of different interactions as weights in the count aggregation. For example, assuming weights of the three different types, âclickâ, âaddâ, and âpurchaseâ, are 1, 2, and 3, respectively.
Option 3 â Time-dependent Count: In many scenarios, time dependency plays a critical role in preparing dataset for building a collaborative filtering model that captures user interests drift over time. One of the common techniques for achieving time dependent count is to add a time decay factor in the counting.
Train Test Split
Option 1 â Random Split: Random split simply takes in a data set and outputs the splits of the data, given the split ratios
Option 2 â Chronological split: Chronologically splitting method takes in a dataset and splits it on timestamp
Evaluation Metrics
Model Comparison
Deployment
As medium still doesnât support Jupyter notebooks in its native form, so we started using fastpages to share notebook based tutorials on GitHub hosted pages. The complete tutorial (+code) can be found here.