Recommendation Systems Shift from Similarity to Predicting User Intent

At the Urban ML conference, held on August 2 in Moscow by MTS True Tech and VTB, experts discussed how recommendation systems are evolving. Traditionally based on collaborative filtering and finding similar users, they now aim to predict user behavior, focusing on probabilities of specific actions. Danil Bugrienko from MWS noted that instead of determining user preferences, developers now create algorithms that forecast actions like opening, watching, or buying. However, the cold start problem remains unsolved; new users and items lack interaction history. Real-time recommendations help, analyzing even initial clicks, and using aggregate audience behavior. Sergey Kuznetsov, head of MWS recommendation platform development, highlighted challenges with using LLMs: latency (200-300 ms is desired, but LLMs take seconds), cost due to expensive GPUs, and reliability issues. Three viable LLM use cases were presented: offline feature enrichment with embeddings, offline candidate generation to address cold start, and online conversational scenarios where the LLM acts as an agent. MWS chose Gemma for conversational scenarios, trading some quality for speed and cost to handle more queries per second (RPS). Key metrics for success include retention, stickiness factor, and reduction in support tickets, but ultimately it's about money. LLM adoption often requires a leap of faith as the economics may not immediately work, but costs are expected to decrease. The industry is moving towards systems that converse with users and understand their immediate intentions, evaluating success by how well they save user time.