A Comparative Evaluation of Robust Loss Functions for Learning with Noisy Labels: From Synthetic to Real-World Annotations

Abstract

Label noise is a pervasive challenge in supervised learning, as deep neural networks are prone to memorizing corrupted annotations during training. Robust loss functions have emerged as a principled approach to mitigate this issue without requiring auxiliary clean data or multi-network training pipelines. While numerous noise-tolerant and composite loss functions have been proposed in recent years, existing evaluations typically compare against only a limited subset of baselines under heterogeneous experimental protocols, making cross-method comparisons unreliable. This paper presents a systematic and unified comparative evaluation of eight representative robust loss functions—Cross-Entropy, Mean Absolute Error, Generalized Cross-Entropy, Symmetric Cross-Entropy, Active Passive Loss, Early-Learning Regularization, Generalized Jensen-Shannon divergence, and Active Negative Loss—across four distinct noise conditions: symmetric, asymmetric, instance-dependent, and real-world human annotation noise. All experiments are conducted on CIFAR-10 and CIFAR-100 with their corresponding CIFAR-N human-annotated noisy label sets (available at noisylabels.com), using a unified PreAct-ResNet-18 architecture and identical training protocol. The results reveal that composite and regularization-based losses consistently outperform simpler noise-tolerant alternatives, with the performance gap widening at elevated noise rates. Notably, method rankings observed under synthetic symmetric noise do not reliably transfer to real-world instance-dependent noise scenarios, highlighting the importance of evaluating under diverse noise conditions.