How VK's Neural User Profile Works in Recommendations

VK AI has developed a neural user profile that forms a unified representation of a user's interests from all interactions across VK services. The core is a transformer model analyzing up to 1,024 recent actions as a sequence, using a two-tower architecture: a user tower with self-attention and an item tower encoding content. Tokens combine content embeddings from multimodal models, event embeddings, and source embeddings, with time encoded via Time2Vec. Training involves pre-training with Sampled Softmax on implicit signals and fine-tuning with causal sessions to avoid data leakage. The cross-domain model 'Heimdall' balances data across domains and predicts the next content object, improving recommendations. A/B tests show a 5.5% increase in watch time in VK Clips and +1.1% TVT on VK Video's main page.