What is a recommendation system and how does it work?
A recommendation system is a machine learning system that predicts which items (products, content, features) a specific user is most likely to find relevant or engage with, and presents those items in ranked order. The prediction is based on: the user's historical interaction patterns (what they have clicked, purchased, or consumed), the patterns of similar users (collaborative filtering users who liked what you liked also liked this), and the properties of items themselves (content-based this item is similar to others you've engaged with). In production, recommendation systems have two stages: candidate generation (efficiently retrieve a shortlist of potentially relevant items from a large catalogue too many items to score all of them in real time) and re-ranking (score and rank the candidate shortlist using a more precise model with additional context the final ranked list served to the user).
What is the cold start problem in recommendation systems?
The cold start problem refers to the difficulty of making recommendations for new users or new items that have no interaction history. For new users (user cold start), the system cannot rely on their personal interaction history it must fall back to popularity-based recommendations, onboarding questions that capture explicit preferences, or content-based recommendations based on item features. For new items (item cold start), collaborative filtering cannot recommend the item until enough users have interacted with it content-based approaches using item metadata (description, category, tags) are used to recommend new items alongside established ones. Two-tower neural models mitigate item cold start by representing items through their features rather than learned interaction embeddings.
How do you measure recommendation system quality?
Recommendation quality is measured offline (using held-out interaction data) and online (through A/B testing on real users). Offline metrics: Precision@K (of the top-K recommendations, what fraction did the user actually engage with?), Recall@K (of all items the user engaged with, what fraction appeared in the top-K recommendations?), NDCG@K (Normalised Discounted Cumulative Gain weights hits higher when they appear earlier in the ranked list), and Coverage (what fraction of the item catalogue is recommended to at least one user low coverage means the model only recommends popular items). Online metrics: CTR (click-through rate on recommendations), conversion rate (purchases from recommendations), and revenue lift (measured against a control group in an A/B test). Offline metrics are fast and cheap; online metrics are the business-relevant ground truth.
How many users and interactions do I need for a recommendation system?
Collaborative filtering requires sufficient user-item interaction density to learn meaningful patterns. Practical minimums: at least 1,000 active users (users with at least 5-10 interactions each) and at least 10,000-50,000 total interactions for a matrix factorisation model to produce reliable recommendations. Below these thresholds, content-based or LLM-embedding-based recommendations (which require no interaction data) are more reliable. For a content-based system based on item metadata, there is no minimum interaction requirement the system can recommend from day one using item features. ClickMasters assesses interaction data density as part of the scoping engagement and recommends the approach that fits the current data state, with a migration path to hybrid or collaborative as interaction data grows.
