We Beat the Kalman Filter on a Single Camera, but First Spent Three Weeks Measuring a Corpse
MoranaLabs engineers developed a neural tracker that outperforms the classic Kalman filter on a monocular camera, keeping the target blind in 50% of episodes versus -3% for Kalman. The hybrid of KalmanNet and neural network achieves an 80% return-to-frame rate. However, the project was plagued by measurement bugs, including a broken probe that read zeros for weeks.
The project at MoranaLabs aimed to keep a visual lock on a target that disappears from the camera frame for 0.75 seconds, using only a monocular camera with no rangefinder. The classic Kalman filter loses the target in 84% of episodes, while the hybrid KalmanNet-neural tracker loses it in 18% and returns it to the center in 80%. The team initially wrote a teacher policy with a sign error, causing it to miss all 60 attempts. They spent three weeks debugging measurement metrics, discovering that the environment wrote some metrics to a nested dictionary, causing the probe and curriculum callback to read zeros. After fixing the bugs, they implemented an asymmetric critic with a privileged vector from the simulator, and trained a world-model auxiliary task to predict that vector, which forced the LSTM to integrate dynamics. The final model achieved 50% cosine similarity between the nose and true target direction during the blind period, compared to -3% for Kalman and 41% for a pure neural pilot.
- Abbreviations
- LSTM = Long Short-Term Memory — Долгая краткосрочная память
- MSE = Mean Squared Error — Среднеквадратичная ошибка
- PPO = Proximal Policy Optimization — Оптимизация проксимальной политики
Source: Habr — хаб ИИ —
original