We might not be aware of it, but recommendation systems influence everything we read, watch and buy online. Every website has a recommendation system built in the background whether it’s to suggest an item based on what we have bought previously, suggest the next song to play, all the way to suggested jobs based on our qualifications on LinkedIn, or even dating websites matching us with potential partners.
It’s not an exaggeration to say that recommendation systems are everywhere. Like it or not, we can’t change how the internet works, but we can learn how they work and understand that they exist for the benefit of websites and users.
Hervé Schnegg, a former Principal Data Scientist at the Telegraph, showcased the use and the need for recommendation systems online and in The Telegraph, but also covered how recommendation systems work, how we can evaluate them and choose the most suitable recommendation system at Data Innovation Summit 2019.
Why recommendation systems are useful
If the most successful companies in the world are using them, that means recommender systems are worth the investment. Hervé provides two examples where recommendations account for an impressive percentage of consumption. Namely, 35% of all purchased items on Amazon and 75% of al media watched on Netflix comes from product recommendation.
As he points out, usually people don’t know what to buy or what they want to watch until it’s recommended to them. Recommender systems provide customers with what they want to find, and as a result, increase the customer lifetime value and the probability of additional purchase.
Risks related to recommendation systems
Recommendation systems are certainly improving interaction with websites. But at the same have underlying risks that have been frequently talked about in the media.
Echo chamber refers to the concept of supporting and amplifying similar views by communication in a closed circle. Simply put, the clubs that a person visits, where they study, what kind of books they read, where they work makes – all these influence people to feel like everyone around them having the same thoughts and views as them.
This phenomenon is further amplified by filter bubbles, Hervé says. They can result from personalised searches when algorithms selectively guess what information a user would like to see based on their location, past click-behaviour and search history.
How recommendation systems work
There are many different recommendation algorithms, but most of them belong to one of the three main approaches to content recommendation Hervé describes.
- Collaborative filtering is the most widely used and documented. It’s used to implement the “people like you” recommendation. Collaborative filtering uses other people’s ratings of products in the past to infer our own preferences. Perse, if two different users rate the same book positively, next time one of the users is buying a book, there is a chance the other user might like that book too.
- Content-based recommendation extracts similar features from pieces of content or products to recommend related products or content with the same topic. This recommendation approach is often seen in news and media websites under “Related content”.
- User profile is preferences-based recommendation approach. An example is Netflix request to rate movies so they know your genre preferences and can recommend relevant movies from the same genre.
Are recommendation systems AI, machine learning or deep learning?
Hervé admits that he often gets the question if recommender systems are AI. And the short answer he gives is: “it depends”. If the editor suggests a list of articles, of course, that is human intelligence and not artificial. But there are also rule-based approaches for recommendation systems where users are suggested articles related to the topic they most frequently read about. These approaches are similar to the expert systems built in the 70s which present a trivial form of AI, asserts Hervé. Further, collaborative filtering and more complex algorithms belong to the world of machine learning. Also, the latest research on building recommender systems includes deep learning methods. So we can clearly say that recommender systems definitely belong in the AI family.
Why having a single recommendation system doesn’t cut it
We’ve seen that there are different types of recommendation systems and each serves a different purpose. Depending on what we want to achieve, different types of recommendation systems promote different behaviours, states Hervé.
When organisations start considering investing in a recommender system, they should first take into consideration three important factors:
- Their objectives and what they are trying to achieve
- The type of content
- The user journey.
Combining all these elements will provide a clear idea of which combination of recommendation systems will provide the best results.
For example, a newspaper with a section in gardening and a section in sports would require two different strategies and recommendation systems, because the gardening piece can be consumed in the next 20 years and it would still be relevant. Whereas the sports article on the game results from last night might not be relevant in a week.
Another consideration is the type of readers frequenting the website. We can be certain that an avid reader of the Telegraph has read all top stories, thus recommending content based on popularity to them would be a complete miss, explains Hervé.
Why newspapers need recommender systems
Returning to the main question of why online newspapers actually need recommendation systems, Hervé lists the Telegraph’s objectives, among which are to:
- engage their users,
- help them find relevant content,
- provide a personalised experience and
- guide them to content with a higher commercial value as a way of monetisation.
How to evaluate recommender systems
When it comes to choosing and evaluating a recommendation system, many people incorrectly believe that accuracy (page views or click-through rate) is the only measurement for a good system. But according to Hervé, it might simply mean is that the website uses clickbait to attract users, and for that, accuracy alone is not relevant. There are several measures for effective evaluation of recommendation systems that should be taken into account:
- Accuracy – Are readers visiting the recommended articles?
- Diversity – How different are the recommended articles if a reader comes back tomorrow?
- Novelty – Had the user seen the recommended articles before?
- Coverage – Is the recommendation system able to suggest articles from different sections or is it specialised in one section?; Percentage of recommended article and percentage of users that a recommendation can be made.
- Serendipity – Recommendations that are unexpected, content that readers wouldn’t have found without the recommendation, but at the same time is useful.
Where are recommendation systems taking us?
Thanks to recommendation systems, commerce sites know people’s habit, offer convenience to them and make their lives easier.
But still, media websites have a role in people’s behaviour that they have to be careful where they push people, having in mind the risks we mentioned earlier relating to filter bubbles, states Hervé. What’s more, every website that implements a recommendation system has a responsibility and has to make sure it is guided by the values of the organisation. On the other hand, thanks to recommendation systems, commerce sites know people’s habit, offer convenience to them and make their lives easier.
The bottom line is, websites have to be aware that there is a certain limit to recommendations and respect their and other organisations values.
Add comment