Just two decades ago, if you wanted to discover new artists and bands to listen to, you had to go to to your local music store to rummage through the heaps of CDs or wait for your favourite music TV show to start. With the rise of the internet, you spent hours searching, downloading songs and meticulously creating your dance summer hits playlists.
But today we like to get our music just like all other information – with one touch or click. You just tap your Spotify app and dive in the endless choice of tracks. Spotify has revolutionised the way we discover music in the digital era. The music streaming app has become an association for listening and discovering new music.
Spotify’s music recommendation feature provides music lovers with fresh songs and artists based on their taste, and at the same time helps new artists get their music in front of audiences. In order to match a billion users with millions of artists and at the same time keep everyone satisfied with the results, Spotify relies on machine learning for music suggestions.
Spotify has revolutionised the way we discover music in the digital era.
Matching music fans to music creators
We were interested to know how it all works in the background and invited Oskar StĂĄl, in charge of VP Personalisation at Spotify, to share his knowledge at the Nordic Data Science and Machine Learning Summit last year.
Oskar and his team of 230 people specialised in music recommendation are focused on 3 main things:
- Make it easier to listen to music – helping users who don’t have the time, energy or skill to create their own playlists.
- Help in music discovery – helping users discover new artists, genres, podcasts they wouldn’t be able to find on their own.
- Help artists – helping creators find audiences all over the world and live off their music.
To achieve this, Spotify has several music recommendation products:
- Home page playlist recommendations as soon as the user opens the app
- Discover Weekly – weekly playlist recommendations based on the user’s taste
- Daily Mix – a daily playlist of user’s favourite songs
- Time Capsule – personalised retro playlists that take users back in time.
How music recommendation works
As Oskar is a technical guy, he is enthusiastic about explaining the technical side of the story. “The Spotify stack is a stack with 3 different layers, and there are machine learning and data science are going on on all 3 layers,” explains Oskar.
The 3 layers that constitute the Spotify stack are:
- Data is the bottom layer that contains all user data (such as age, listening information, playlists), content (i.e. data collected from the labels and downloaded information from the web) and the audio profile of the music.
- Shared models are the middle layer and are used by all recommendation features. This middle layer contains info such as user affinity (how much a user likes a particular artist or what are their favourite artist), embedding spaces for similarities (how similar are two artists or tracks, lists of 10 similar artists, or tracks), and clustering of items.
- Features are the top level. This level contains machine learning models that are targeting the satisfaction of specific features. The Features layer uses data from the previous two layers.
Machine learning techniques Spotify uses
Oskar emphasises three examples of machine learning techniques that Spotify uses.
Embeddings
The embedding training process is performed every day with 667,762,166 playlists. It’s a simple technique that helps Oskar’s team guess the missing track from a list. The result of the process is a Cloud of Similarities, which is practically an embedding space. In this space, they can ask questions regarding similarities between artists, tracks and albums. Users are also integrated into the embedding space based on their playlist and song choices. Spotify uses the Cloud of Similarities to calculate similarities between users, artists and tracks.
Embedding Space is one of the important building blocks of all Spotify’s recommendation products. One steps of the many in the process of all recommendation features is looking into the Cloud of Similarities and figure out an interesting combination of user, artist and track that are close to each other in the Cloud, but at the same time result in a suggestion the user has never heard before.
The secret sauce, as Oskar names it, of their successful Embedding Space is:
- Large volume of playlists
- The passion put into creating the playlists
- 9 years of iteration spent in tweaking and optimising the embedding space.
The homepage
When the user opens Spotify, they see the homepage where they get various suggestions about playlists and albums they think are relevant for the user. Spotify recently deployed a recommendation system named BaRT in their homepage. It’s a machine learning algorithm which gives users a personalised homepage. What’s special about BaRT is that it not only takes into account users’ music taste and favourite artists, but also considers which device they are on, the time of the day, current trends, and so on. The goal is to make the system context-aware.
In the BaRT system, Spotify implemented an Exploration-Exploitation type of solution. This solution help explore pieces of content that it’s not clear whether they should or shouldn’t be recommended. For pieces that Oskar says they are less sure they should be recommended, they use exploration so they can learn more about them. By doing exploration, Spotify learns about other potential content candidates and avoids the recommendation system bias.
Objective function in Discover Weekly
Every week Spotify creates a massive amount of daily and weekly playlists for all their users. In order to assess whether a user is satisfied with the recommendation, they looked at various signals such as if the user looked at the album of the track, the time they spent on the album page, if they added tracks from the Discover Weekly to their personal playlists, the overall streaming time and if users came back to Discover Weekly.
We are working on the idea of how we can understand on a deeper level if a user is satisfied or not.
But they found out that these metrics were not a reliable assessment for user satisfaction. What happened was users usually took tracks from Discover Weekly and transferred to their personal playlists. So although streaming time was low, users were actually satisfied with the recommendation. Or the other way around, high weekly retention rate didn’t necessarily mean users were satisfied, as many were not happy with the previous week recommendations and came back to see if the next ones were better.
Spotify realised they needed to change the way they measure user satisfaction. Oskar After a lot of hard work and analysis, they discovered 4 main user behaviours (goals), but none of them indicated that the user was truly satisfied, because it varied depending on the goal. To solve this challenge, Spotify built a machine learning model to try to assess user satisfaction with Discover Weekly more accurately. The model took into account 3 things: the user interaction data with Discover Weekly, historical data of how each user’s behaviour changed over the weeks and clustering data that classifies which goal the user belongs to depending on their behaviour.
Next level for music recommendation
On a broader perspective and future steps with music recommendation, Spotify is focused on understanding more clearly what constitutes user satisfaction. “We are working on the idea of how we can understand on a deeper level if a user is satisfied or not,” emphasises Oskar. This basically means finding out tighter causation between user satisfaction and the goal the model is optimising for.
They are also working to improve their voice interaction function. Spotify are trying to find ways how to satisfy more vague requests and how the user can control the listening sessions once they are started.
Podcast recommendations are also on Spotify’s agenda. They are finding out ways to better understand, classify and categorise the content of podcasts with the NLP technologies.
Add comment