Label noise
Label noise refers to errors or inaccuracies in the class labels of data instances. This is a widespread issue in machine learning datasets, arising from human annotator mistakes, unclear labeling instructions, automated labeling methods, or adversarial attacks in supervised learning.[1] Label noise can be roughly divided into random noise, where labels are flipped independently of input features, and systematic noise, where mislabeling is dependent on certain patterns or biases in the data.[2] Label noise can be damaging to model performance, especially for complex models that may overfit to noisy labels rather than generalizable patterns.[3]
Many approaches have been proposed to deal with the effects of label noise, including robust loss functions, noise-tolerant algorithms, data cleaning methods, and semi-supervised learning approaches.[4] To reduce the impact of wrong labels during training, techniques like label smoothing, sample reweighting and using trusted validation sets are used.[5] The role of noise-robust training paradigms and curriculum learning strategies to improve resilience against mislabeled data is also explored in recent research.[6]
See also
References
- ↑ Frenay, Benoît; Verleysen, Michel (2014). "Classification in the presence of label noise: a survey". IEEE Transactions on Neural Networks and Learning Systems 25 (5): 845–869. doi:10.1109/TNNLS.2013.2292894.
- ↑ Zhu, Xiaojin (2005). Semi-Supervised Learning Literature Survey. University of Wisconsin-Madison.
- ↑ Zhang, Chiyuan; Bengio, Samy; Hardt, Moritz; Recht, Benjamin; Vinyals, Oriol (2017). "Understanding deep learning requires rethinking generalization". International Conference on Learning Representations.
- ↑ Song, Hwanjun; Kim, Minseok; Park, Dongmin; Shin, Jae-Gil (2022). "Learning from noisy labels with deep neural networks: A survey". IEEE Transactions on Neural Networks and Learning Systems. doi:10.1109/TNNLS.2020.3012597.
- ↑ Szegedy, Christian; Vanhoucke, Vincent; Ioffe, Sergey; Shlens, Jonathon; Wojna, Zbigniew (2016). "Rethinking the Inception Architecture for Computer Vision". Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.
- ↑ Han, Bo; Yao, Quanming; Yu, Xingrui; Niu, Gang; Tsang, Ivor W.; Sugiyama, Masashi (2018). "Co-teaching: Robust training of deep neural networks with extremely noisy labels". Advances in Neural Information Processing Systems.
 |  |
