Airbnb has millions of homes: treehouses, castles, city lofts, houseboats. When you look at one, Airbnb has to guess which others you'll like, not by price or city alone, but by feel. Their answer was to turn every home into a list of just 32 numbers. Those numbers lifted clicks on their recommendations by 21%, and now help rank Airbnb's search results in real time.
This post walks through how, from zero. The animated version:
Two doors to a booking
Guests find the home they book through two doors. Search: you type a city and dates, and a model ranks the results using more than a hundred signals. And the Similar Listings carousel on every listing page. Together, search ranking and similar listings drive 99% of Airbnb's booking conversions.
Both need the same thing: a way to say two homes are similar. Same city, same price, entire home versus private room: those are database columns. But guests also care about style, architecture and the feel of the place, and no column captures that. A cosy cabin and a glass penthouse can cost the same, sit in the same city, and appeal to completely different people.
What an embedding is
Imagine a map where every home is a dot, and homes that are alike sit close together. On a flat map each dot is two numbers, an x and a y. Now give the map 32 directions instead of two. Every home becomes a list of 32 numbers: its embedding. You can't picture 32 dimensions, but you can still measure how close two homes are, with cosine similarity.
Search sessions are sentences
Nobody types those numbers in. They're learned, with a trick borrowed from language models: words that keep showing up near each other tend to be related.
Airbnb treats a guest's trail of clicks as a sentence, and each listing ID as a word. A session is one uninterrupted run of clicks; a gap of more than 30 minutes starts a new one. When someone clicks three beach cottages in a row, that's a hint those cottages have something in common.
Learning by nudging
Start every home at a random spot on the map. Slide a window along each session. The home in the middle is the central listing; the homes clicked just before and after it, up to five on each side, are its positive context: pull them closer. Then grab a few homes completely at random and push them away. That's negative sampling.
Airbnb made two changes that fit its business:
- The booking is the point. Clicks are cheap; a booking is what the guest actually wanted. In sessions that end in a booking, the booked home stays in the context of every step. They call it global context.
- The same-city trap. Guests search within one market, so positives come from one city, but random negatives come from the whole world. The model learns "Paris is not Tokyo" and never learns which Paris flats differ from each other. The fix: also sample negatives from the same market.
Training at Airbnb scale
Airbnb trained this on more than 800 million search click sessions, producing an embedding for 4.5 million active listings. Why 32 numbers and not 300? Every vector has to live in the memory of the search machines for real-time comparisons, and 32 was the sweet spot between quality and RAM. A brand-new listing has no clicks yet, so it borrows: the average of the three geographically closest listings of the same type and price range.
Nobody told the model about location, price or style, but it learned all three. California listings clustered by embedding land in real map areas. Same type and price score higher cosine similarity. And starting at a treehouse, its nearest neighbours are treehouses; castles find castles.
Similar Listings, rebuilt
The carousel became pure geometry: take the home's vector, compare it with every home in the same market that's free on your dates, and keep the 12 closest, a k-nearest-neighbours search with k = 12. In an A/B test, the carousel's click-through rate rose 21%, and 4.9% more guests found the home they booked right there.
Search that learns while you browse
For every user, Airbnb keeps two short lists, updated in real time with Kafka: every listing you clicked in the last two weeks, and every listing you skipped (ranked high, but you clicked something lower down). Each search gives every candidate two new scores, embedding click similarity and embedding skip similarity, both features in the ranking model. High click similarity pushes a home up; high skip similarity pushes it down. It launched in summer 2017.
Takeaways
- Learn from behaviour, not labels.
- Shape the training around the outcome you care about: the booking.
- Keep vectors small enough to use in real time.
The same recipe works for almost any marketplace: products, songs, videos, jobs.
Watch the full animated breakdown: https://youtu.be/-NK9hxCb4Sk
Source: Mihajlo Grbovic, Haibin Cheng, Qing Zhang, Lynn Yang, Phillippe Siclait and Matt Jones, "Listing Embeddings for Similar Listing Recommendations and Real-time Personalization in Search Ranking", The Airbnb Tech Blog (March 2018).
Practice designing systems like this yourself at https://systemdesignlab.in
Top comments (0)