St. Petersburg University Scientists Improve Neural Network's Dialogue Ability

Linguists at Saint Petersburg State University have improved the dialogue ability of a neural network by embedding an intonation model, which makes the AI's pronunciation more natural and familiar to humans. This development aims to improve text-to-speech systems used in voice assistants and anthropomorphic robots. The research results were published in the proceedings of the International Conference on Speech and Computer. AI-based systems can already fully or partially replace humans in many tasks, especially routine ones, but intonation has been difficult to implement. Intonation is crucial for expressing emotions, attitude, and communicative meaning, such as distinguishing a question from a statement. Traditional language models often produce monotonous or inappropriate intonation. The SPbU linguists used the Tacotron2 model for text-to-speech synthesis and trained it with a specialized intonation classification developed at SPbU. The classification includes a wide range of intonation variants, accounting for melodic range, speed, and sentence type. These modifications allowed the neural network to overcome limitations of unnatural or monotonous intonation. In an evaluation, 42 native Russian speakers listened to synthesized audio and generally found it natural, with some minor artifacts. In a second experiment, participants could identify question phrases more easily. The researchers suggest further fine-tuning with emotional and stylistic variants to improve quality.