Custom Reward Functions for Multi-Turn Reinforcement Learning with Amazon Nova Forge

In multi-turn reinforcement learning (RL), the custom reward function determines what the model learns, and a subtly wrong reward can teach the wrong thing despite healthy training curves. Amazon Nova Forge runs reward logic in your own environment through Bring Your Own Orchestration (BYOO), with a serverless multi-turn RL option now generally available. The post focuses on designing a composite multi-turn reward that Group Relative Policy Optimization (GRPO) can learn from, including a worked example of teaching Amazon Nova Lite 2.0 to ask before coding in a multi-turn collaborative-coding task. The example uses a four-component reward: correctness, asked_before_coding, guessed_immediately, and loop_penalty. It also covers executing model-generated code safely, with precautions like not exposing credentials or network, applying resource limits, and validating the number of tests. The pitfalls section discusses reward hacking, training instability, and reward collapse, emphasizing that a reward signal influences learning only through variation within a group, so a term that takes the same value for every completion contributes nothing to the gradient. The importance of un-gating desired behaviors and penalizing failure modes is highlighted, along with the need to instrument each component to trust what training is learning.