The Expanse in the Multiverse of Recommendations

A picture I took of the night-sky in Auckland

“The only thing to do with good advice is to pass it on. It is never of any use to oneself.” — Oscar Wilde

And what are recommendations, but some good advices of the pundits of AI, by the pundits of AI, for the pundits of AI.
Did I just lay down the constitution of AI? 😆

Now, if I were to explain Recommendations formally, but to a lay-man?

How would I really recommend?

Imagine having a friend who is like your bosom buddy and really understands your taste in movies. This friend has known you for so long that she knows which genres you enjoy, your favorite actors, and the types of plots that keep you glued to the screen. One day you are out with this friend, and she suggests a movie she thinks you’d love based on what she knows about your preferences. That’s essentially how a recommendation system works in the real world too. The world of machines and algorithms.

In the digital world, we have machines that learn about our preferences by looking at the things we’ve liked or interacted with before. For instance, if you’ve enjoyed certain movies on a streaming platform, the recommendation system takes note of that and figures out patterns in your choices. It then suggests other movies it believes you might enjoy based on those patterns.

My first experience with Recommendation Systems, was with the concepts and technologies of Collaborative & Content-based Filtering, while designing a Recommendation Engine / Framework for one of my previous employers — Cashify.

The model envisaged was a hybrid model, wherein, the end recommendation would have been a combination of simple recommendations, collaborative-filtering and content-based filtering approaches, like what you see below :-

Types of recommendations

Assuming that Simple Recommenders would be easy for you to understand, let me skip to Collaborative Filtering and explain it a bit to those who may not know much about it :-

Collaborative Filtering

is a famous technique which is used in recommendation systems to make self-regulating / push-button predictions about the preferences of a user by collecting preferences from many users (collaborating). The fundamental idea behind collaborative filtering can be explained as follows :-

• There are two users A and B.
• The user A has similar preferences to a user B on certain items.
• By the logic of Collaborative Filtering, A is likely to have similar preferences to B on other items as well.

There are two main types of Collaborative Filtering :-

• User-based Collaborative Filtering : In this approach, recommendations are made to the user, based on the preferences and behaviors of the users who are similar to them. In short, this approach relies on the similarity between the users.
  Learning the details from https://www.geeksforgeeks.org/user-based-collaborative-filtering/, I tried to pen down the concepts as seen below (these notes are there for you, just in case you are not a member on Medium, and may not be able to access these notes) :-

Please read the notes in the following order.

This can be achieved using a library in Python called surprise.
The library surprise is like a Python Scikit for Recommenders. We shall get into the code and practical explanation of it, a little later.

• Item-based Collaborative Filtering : In item-based collaborative filtering, recommendations are made based on the similarity between items. It is the recommendation system to establish the similarity between the items using the ratings by the user.
  Here’s a very beautiful write up by Yohan Jeong — https://towardsdatascience.com/item-based-collaborative-filtering-in-python-91f747200fab on Item-based Collaborative filtering.
  Long time back I took notes from his article and just in case you aren’t able to access his article, you can see the notes below (and excuse me for bad handwriting).

Please read the notes in the following order.

The conversational AI product “Prajna”, that I have been building, uses item-based collaborative filtering at the backend. Here’s an official video that I had created to demonstrate the concept :-

The dataset I have used is MovieLens

The MovieLens 100K dataset is a widely used dataset for collaborative filtering and recommendation system tasks. It contains 100,000 ratings from approximately 1,000 users on around 1,700 movies. Each user has rated at least 20 movies on a scale from 1 to 5.

Here’s a sample preview of the dataset :-

And what will interest you is the code, isn’t it?
Of course, it will.
Since am using this code for my product, I can’t publish it here.
But please do feel free to DM me or comment on this piece here for any tips or code snippets too 🍭.

GODSPEED, MATE :)