From CTR to Deals: How Avito Builds ML Monetization Models
Avito's data science team explains how they estimate expected revenue from user actions and use it in ranking. They use CTR, CVR, and bid adjustment models, evaluate quality with stratified ROC-AUC, and address attribution and position bias.
The article, written by Alina Babenko, acting DS-manager at Avito, describes how the company's team builds monetization models for search and recommendations. The core task is to estimate the expected revenue from user interactions with listings and incorporate it into ranking. The main metric is revenue growth, but the team must balance this with search quality for users. Three types of models are used: CTR models predict the probability of a click, CVR models estimate the chance of a target action after a click, and a correction model adjusts bids based on traffic quality. To keep the average cost per click stable, they use query cross-entropy loss. For evaluating ranking quality, they use stratified ROC-AUC instead of global ROC-AUC, because users compare listings within a single query. Calibration is assessed with the RIG metric. They face challenges with attribution, using last-click, first-click, or multi-touch approaches, and are exploring giving less weight to early clicks. To combat position bias, they use either three CatBoost models or inverse propensity weighting. They are also transitioning from CatBoost to deep learning, which already shows better CTR prediction quality, but note that modeling deals is more complex than clicks due to longer user journeys.
- Abbreviations
- CTR = Click-Through Rate — Кликабельность, частота кликов
- CVR = Conversion Rate — Коэффициент конверсии
- ROC-AUC = Receiver Operating Characteristic - Area Under the Curve — Площадь под ROC-кривой
- RIG = Regression Information Gain — Информационный выигрыш регрессии
Source: Habr — хаб ML —
original