Pseudo-Supervised Defect Detection Using Robust Deep Convolutional Autoencoders

Mahmut Nedim Alpdemir

doi:10.35377/saucis...1196381

EN

Pseudo-Supervised Defect Detection Using Robust Deep Convolutional Autoencoders

Abstract

Robust Autoencoders separate the input image into a Signal(L) and a Noise(S) part which, intuitively speaking, roughly corresponds to a more stable background scene (L) and an undesired anomaly (or defect) (S). This property of the method provides a convenient theoretical basis for divorcing intermittent anomalies that happen to clutter a relatively consistent background image. In this paper, we illustrate the use of Robust Deep Convolutional Autoencoders (RDCAE) for defect detection, via a pseudo-supervised training process. Our method introduces synthetic simulated defects (or structured noise) to the training process, that alleviates the scarcity of true (real-life) anomalous samples. As such, we offer a pseudo-supervised training process to devise a well-defined mechanism for deciding that the defect-normal discrimination capability of the autoencoders has reached to an acceptable point at training time. The experiment results illustrate that pseudo supervised Robust Deep Convolutional Autoencoders are very effective in identifying surface defects in an efficient way, compared to state of the art anomaly detection methods.

Keywords

Supporting Institution

TUBİTAK BİLGEM

Thanks

This work was partially supported by TUBİTAK BİLGEM we are grateful for that support.

References

[1] V. J. Hodge, J. Austin, A survey of outlier detection methodologies, Artificial Intelligence Review 22 (2) (2004) 85–126. doi:10.1007/s10462-004-4304-y .
[2] V. Chandola, A. Banerjee, V. Kumar, Anomaly detection: A survey, ACM Comput. Surv. 41 (3) (Jul. 2009).
[3] M. A. Pimentel, D. A. Clifton, L. Clifton, L. Tarassenko, A review of novelty detection, Signal Processing 99 (2014) 215–249. doi:10.1016/j.sigpro.2013.12.026 .
[4] Y. LeCun, Y. Bengio, G. Hinton, Deep learning, Nature 521 (2015) 436–44. doi:10.1038/nature14539 .
[5] K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition (2015). arXiv:1512.03385 .
[6] S. Bulusu, B. Kailkhura, B. Li, P. K. Varshney, D. Song, Anomalous instance detection in deep learning: A survey (2020). arXiv:2003.06979 .
[7] R. Chalapathy, S. Chawla, Deep learning for anomaly detection: A survey (2019). arXiv:1901.03407 .
[8] D. Bank, N. Koenigstein, R. Giryes, Autoencoders (2020). arXiv:2003.05991 .

[9] M. Schreyer, T. Sattarov, D. Borth, A. Dengel, B. Reimer, Detection of anomalies in large scale accounting data using deep autoencoder networks, CoRR abs/1709.05254 (2017). arXiv:1709.05254 .
[10] D. Zimmerer, F. Isensee, J. Petersen, S. Kohl, K. Maier-Hein, Unsupervised anomaly localization using variational auto-encoders, in: D. Shen, T. Liu, T. M. Peters, L. H. Staib, C. Essert, S. Zhou, P.-T. Yap, A. Khan (Eds.), Medical Image Computing and Computer Assisted Intervention – MICCAI 2019, Springer International Publishing, Cham, 2019, pp. 289–297.
[11] C. Zhou, R. C. Paffenroth, Anomaly detection with robust deep autoencoders, in: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’17, ACM, New York, NY, USA, 2017, pp. 665–674. doi:10.1145/3097983.3098052 .
[12] R. Chalapathy, A. K. Menon, S. Chawla, Robust, deep and inductive anomaly detection, in: M. Ceci, J. Hollmén, L. Todorovski, C. Vens, S. Džeroski (Eds.), Machine Learning and Knowledge Discovery in Databases, Springer International Publishing, Cham, 2017, pp. 36–51.
[13] M. Ribeiro, A. E. Lazzaretti, H. S. Lopes, A study of deep convolutional auto-encoders for anomaly detection in videos , Pattern Recognition Letters 105 (2018) 13 – 22, machine Learning and Applications in Artificial Intelligence. doi:https://doi.org/10.1016/j.patrec.2017.07.016 .
[14] H. Y. Ngan, G. K. Pang, N. H. Yung, Automated fabric defect detection?a review , Image and Vision Computing 29 (7) (2011) 442–458. doi:https://doi.org/10.1016/j.imavis.2011.02.002 .
[15] K. Hanbay, M. F. Talu, Ömer Faruk Özgüven, Fabric defect detection systems and methods?a systematic literature review, Optik 127 (24) (2016) 11960–11973.
[16] M. F. Nisha, P. Vasuki, S. M. M. Roomi, Survey on various defect detection and classification methods in fabric images, Journal of Environmental Nano Technology (JENT) 6 (2) (2017) 20–29.
[17] C. Li, J. Li, Y. Li, L. He, X. Fu, J. Chen, Fabric defect detection in textile manufacturing: A survey of the state of the art, Security and Communication Networks 2021 (2021) 9948808.
[18] T. Czimmermann, G. Ciuti, M. Milazzo, M. Chiurazzi, S. Roccella, C. M. Oddo, P. Dario, Visual-based defect detection and classification approaches for industrial applications?a survey, Sensors 20 (5) (2020).
[19] B. Schölkopf, J. C. Platt, J. C. Shawe-Taylor, A. J. Smola, R. C. Williamson, Estimating the support of a high-dimensional distribution, Neural Comput. 13 (7) (2001) 1443–1471
[20] F. T. Liu, K. M. Ting, Z.-H. Zhou, Isolation-based anomaly detection, ACM Trans. Knowl. Discov. Data 6 (1) (Mar. 2012). doi:10.1145/2133360.2133363 .
[21] S. Hariri, M. Carrasco Kind, R. J. Brunner, Extended isolation forest, IEEE Transactions on Knowledge and Data Engineering (2019) 1–1 doi:10.1109/TKDE.2019.2947676 .
[22] D. M. Tax, R. P. Duin, Support vector data description, Machine Learning 54 (1) (2004) 45–66.
[23] L. Ruff, R. A. Vandermeulen, N. Görnitz, L. Deecke, S. A. Siddiqui, A. Binder, E. Müller, M. Kloft, Deep one-class classification, in: Proceedings of the 35th International Conference on Machine Learning, Vol. 80, 2018, pp. 4393–4402.
[24] R. Chalapathy, A. K. Menon, S. Chawla, Anomaly detection using one-class neural networks (2018). arXiv:1802.06360 .
[25] E. Parzen, On estimation of a probability density function and mode , The Annals of Mathematical Statistics 33 (3) (1962) 1065–1076. URL http://www.jstor.org/stable/2237880
[26] D. Gong, L. Liu, V. Le, B. Saha, M. Mansour, S. Venkatesh, A. Hengel, Memorizing normality to detect anomaly: Memory-augmented deep autoencoder for unsupervised anomaly detection (10 2019). doi:10.1109/ICCV.2019.00179 .
[27] S. Edwards, M. S. Lee, Using convolutional neural network autoencoders to understand unlabeled data, in: T. Pham (Ed.), Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications, Vol. 11006, International Society for Optics and Photonics, SPIE, 2019, pp. 444 – 458. doi:10.1117/12.2518459 .
[28] P. Wu, J. Liu, F. Shen, A deep one-class neural network for anomalous event detection in complex scenes, IEEE Transactions on Neural Networks and Learning Systems 31 (7) (2020) 2609–2622.
[29] P. Schlachter, Y. Liao, B. Yang, Deep one-class classification using intra-class splitting , 2019 IEEE Data Science Workshop (DSW) (Jun 2019). doi:10.1109/dsw.2019.8755576 . URL http://dx.doi.org/10.1109/DSW.2019.8755576
[30] E. Plaut, From principal subspaces to principal components with linear autoencoders (2018). arXiv:1804.10253 .
[31] E. J. Candès, X. Li, Y. Ma, J. Wright, Robust principal component analysis?, J. ACM 58 (3) (Jun. 2011).
[32] F. De la Torre, M. J. Black, Robust principal component analysis for computer vision, in: Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001, Vol. 1, 2001, pp. 362–369 vol.1.
[33] H. Akrami, A. A. Joshi, J. Li, S. Aydore, R. M. Leahy, Robust variational autoencoder (2019). arXiv:1905.09961 .
[34] P. Vincent, H. Larochelle, Y. Bengio, P.-A. Manzagol, Extracting and composing robust features with denoising autoencoders , in: Proceedings of the 25th International Conference on Machine Learning, ICML ?08, Association for Computing Machinery, New York, NY, USA, 2008, p. 1096?1103. doi:10.1145/1390156.1390294 .
[35] A. Collin, C. D. Vleeschouwer, Improved anomaly detection by training an autoencoder with skip connections on images corrupted with stain-shaped noise, 2020 25th International Conference on Pattern Recognition (ICPR) (2021) 7915–7922.
[36] P. Liznerski, L. Ruff, R. A. Vandermeulen, B. J. Franks, M. Kloft, K.-R. Müller, Explainable deep one-class classification , in: International Conference on Learning Representations, 2021. URL https://openreview.net/forum?id=A5VV3UyIQz
[37] S. Mosci, L. Rosasco, M. Santoro, A. Verri, S. Villa, Solving structured sparsity regularization with proximal methods, in: J. L. Balcázar, F. Bonchi, A. Gionis, M. Sebag (Eds.), Machine Learning and Knowledge Discovery in Databases, Springer Berlin Heidelberg, Berlin, Heidelberg, 2010, pp. 418–433.
[38] A. Douady, Julia Sets and the Mandelbrot Set, Springer, Berlin, Heidelberg, 1986, pp. 161–174. doi:10.1007/978-3-642-61717-1_13 .
[39] G.-L. Zhang, M.-Q. Cai, X.-L. He, X.-Z. Gao, M.-D. Zhao, D. Wang, Y. Li, C. Tu, H.-T. Wangrmark, Pseudo-topological property of julia fractal vector optical fields, Opt Express 27 (9) (2019) 13263–13279.
[40] D. P. Kingma, J. Ba, Adam: A method for stochastic optimization (2017). arXiv:1412.6980 .
[41] Y. Lecun, L. Bottou, Y. Bengio, P. Haffner, Gradient-based learning applied to document recognition, Proceedings of the IEEE 86 (11) (1998) 2278–2324. doi:10.1109/5.726791 .
[42] A. Paszke, et al. Pytorch: An imperative style, high-performance deep learning library, in: H. Wallach, H. Larochelle, A. Beygelzimer, F. dÁlché-Buc, E. Fox, R. Garnett (Eds.), Advances in Neural Information Processing Systems 32, Curran Associates, Inc., 2019, pp. 8024–8035.
[43] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, E. Duchesnay, Scikit-learn: Machine learning in Python, Journal of Machine Learning Research 12 (2011) 2825–2830.
[44] J. Silvestre-Blanes, T. Albero-Albero, I. Miralles, R. Pérez-Llorens, J. Moreno, A public fabric database for defect detection methods and results, Autex Research Journal 19 (4) (2019) 363 – 374. doi:https://doi.org/10.2478/aut-2019-0035 .
[45] J. Božič, D. Tabernik, D. Skočaj, Mixed supervision for surface-defect detection: from weakly to fully supervised learning, Computers in Industry (2021).
[46] Zhou Wang, A. C. Bovik, H. R. Sheikh, E. P. Simoncelli, Image quality assessment: from error visibility to structural similarity, IEEE Transactions on Image Processing 13 (4) (2004) 600–612.
[47] S. Van der Walt, J. L. Schönberger, J. Nunez-Iglesias, F. Boulogne, J. D. Warner, N. Yager, E. Gouillart, T. Yu, scikit-image: image processing in python, PeerJ 2 (2014) e453.

Details

Primary Language

English

Subjects

Artificial Intelligence , Computer Software

Journal Section

Research Article

Authors

Mahmut Nedim Alpdemir ^*
0000-0001-6411-1453
Türkiye

Publication Date

December 31, 2022

Submission Date

October 29, 2022

Acceptance Date

November 22, 2022

Published in Issue

Year 1970 Volume: 5 Number: 3

DOI

https://doi.org/10.35377/saucis...1196381

IZ

https://izlik.org/JA98EL34UC

Cite

RIS / Bibtex

APA

Alpdemir, M. N. (2022). Pseudo-Supervised Defect Detection Using Robust Deep Convolutional Autoencoders. Sakarya University Journal of Computer and Information Sciences, 5(3), 385-403. https://doi.org/10.35377/saucis...1196381

AMA

1.Alpdemir MN. Pseudo-Supervised Defect Detection Using Robust Deep Convolutional Autoencoders. SAUCIS. 2022;5(3):385-403. doi:10.35377/saucis.1196381

Chicago

Alpdemir, Mahmut Nedim. 2022. “Pseudo-Supervised Defect Detection Using Robust Deep Convolutional Autoencoders”. Sakarya University Journal of Computer and Information Sciences 5 (3): 385-403. https://doi.org/10.35377/saucis. 1196381.

EndNote

Alpdemir MN (December 1, 2022) Pseudo-Supervised Defect Detection Using Robust Deep Convolutional Autoencoders. Sakarya University Journal of Computer and Information Sciences 5 3 385–403.

IEEE

[1]M. N. Alpdemir, “Pseudo-Supervised Defect Detection Using Robust Deep Convolutional Autoencoders”, SAUCIS, vol. 5, no. 3, pp. 385–403, Dec. 2022, doi: 10.35377/saucis...1196381.

ISNAD

Alpdemir, Mahmut Nedim. “Pseudo-Supervised Defect Detection Using Robust Deep Convolutional Autoencoders”. Sakarya University Journal of Computer and Information Sciences 5/3 (December 1, 2022): 385-403. https://doi.org/10.35377/saucis. 1196381.

JAMA

1.Alpdemir MN. Pseudo-Supervised Defect Detection Using Robust Deep Convolutional Autoencoders. SAUCIS. 2022;5:385–403.

MLA

Alpdemir, Mahmut Nedim. “Pseudo-Supervised Defect Detection Using Robust Deep Convolutional Autoencoders”. Sakarya University Journal of Computer and Information Sciences, vol. 5, no. 3, Dec. 2022, pp. 385-03, doi:10.35377/saucis. 1196381.

Vancouver

1.Mahmut Nedim Alpdemir. Pseudo-Supervised Defect Detection Using Robust Deep Convolutional Autoencoders. SAUCIS. 2022 Dec. 1;5(3):385-403. doi:10.35377/saucis. 1196381

Öz

Pseudo-Supervised Defect Detection Using Robust Deep Convolutional Autoencoders

Abstract

Keywords

Supporting Institution

Thanks

References

Details

Primary Language

Subjects

Journal Section

Authors

Publication Date

Submission Date

Acceptance Date

Published in Issue

DOI

IZ

Cite