Research Article
BibTex RIS Cite
Year 2021, Volume: 4 Issue: 1, 11 - 25, 30.04.2021
https://doi.org/10.35377/saucis.04.01.834048

Abstract

References

  • V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra and M. Riedmiller, “Playing Atari with Deep Reinforcement Learning,” pp. 1–9, 2013. Retrieved from http://arxiv.org/abs/1312.5602
  • M. Lai, “Giraffe: Using Deep Reinforcement Learning to Play Chess,” September, 2015. Retrieved from http://arxiv.org/abs/1509.01549
  • L. Wang, D. Zhang, L. Gao, J. Song, L. Guo and H. T. Shen, “MathDQN: Solving arithmetic word problems via deep reinforcement learning,” 32nd AAAI Conference on Artificial Intelligence, pp. 5545–5552, 2018.
  • S. Nemati, M. M. Ghassemi and G. D. Clifford, “Optimal medication dosing from suboptimal clinical examples: A deep reinforcement learning approach,” Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 2016.
  • Z. Zhou, X. Li and R. N. Zare, “Optimizing Chemical Reactions with Deep Reinforcement Learning,” ACS Central Science, vol. 3, no. 12, pp. 1337–1344, 2017.
  • M. Mahmud, M. S. Kaiser, A. Hussain and S. Vassanelli, “Applications of Deep Learning and Reinforcement Learning to Biological Data,” IEEE Transactions on Neural Networks and Learning Systems, vol. 29, no. 6, pp. 2063–2079, 2018.
  • T. Yen, A. Oprea and K. Onarlioglu, "Beehive: large-scale log analysis for detecting suspicious activity in enterprise networks,". Proc. 29th Annual Computer Security Applications Conference, pp. 199–208, 2013.
  • A. Razaq, H. Tianfield and P. Barrie, "A big data analytics based approach to anomaly detection," Proc. - 2016 IEEE/ACM 3rd International Conference on Big Data Computing Applications and Technologies (BDCAT), pp. 187–193, 2016.
  • A. O. Balogun and R. G. Jimoh, "Anomaly intrusion detection using a hybrid of decision tree and K-nearest neighbor," Journal of Advances in Scientific Research & Applications (JASRA), vol. 2, no. 1, pp. 67-74, 2015.
  • A. Hariharan, A. Gupta and T. Pal, "CAMLPAD: Cybersecurity Autonomous Machine Learning Platform for Anomaly Detection," Proc. Future of Information and Communication Conference (FICC), San Francisco, CA, USA, pp. 705-720, 2020.
  • M. S. Pervez and D. M. Farid, “Feature selection and intrusion classification in NSL-KDD cup 99 dataset employing SVMs,” SKIMA 2014 - 8th International Conference on Software, Knowledge, Information Management and Applications, pp. 1–6, 2014.
  • B. Ingre and A. Yadav, "Performance analysis of NSL-KDD dataset using ANN," 2015 International Conference on Signal Processing and Communication Engineering Systems, pp. 92-96, 2015.
  • S.O.M. Kamel, N. Hegazi, H. Harb, A. ElDein and H. ElKader, "AdaBoost Ensemble Learning Technique for Optimal Feature Subset Selection," International Journal of Computer Networks and Communications Security vol. 4, no. 1, pp. 1–11, 2016.
  • W. Hu, W. Hu, and S. Maybank, "AdaBoost-Based Algorithm for Network Intrusion Detection," IEEE Transactions on Systems, Man, and Cybernetics - Part B: Cybernetics, vol. 38, no. 2, pp. 577-583, 2008.
  • A. H. Engly, A. R. Larsen, and W. Meng, "Evaluation of Anomaly-Based Intrusion Detection with Combined Imbalance Correction and Feature Selection," Proc. 14th International Conference on Network and System Security, Melbourne, Australia, pp. 277-291, 2020.
  • N. Moustafa and J. Slay, "A hybrid feature selection for network intrusion detection systems: central points and association rules," arXiv:1707.05505, (2017) [cs.CR].
  • J. Kim and H. Kim, “Applying Recurrent Neural Network to Intrusion Detection with Hessian Free Optimization,” In: Kim H., Choi D. (eds) Information Security Applications. WISA 2015. Lecture Notes in Computer Science, vol. 9503, 2016, Springer, Cham.
  • Y. Chuan-long, Z. Yue-fei, F. Jin-long and H. Xin-zheng, “A Deep Learning Approach for Intrusion Detection using Recurrent Neural Networks,” IEEE Access, vol. 5, pp. 21954 - 2196, 2017.
  • C. Yin, Y. Zhu, J. Fei, and X. He, ‘‘A deep learning approach for intrusion detection using recurrent neural networks,’’ IEEE Access, vol. 5, pp. 21954–21961, 2017.
  • Z. Li, A. L. G. Rios, G. Xu, and L. Trajkovic, ‘‘Machine learning techniques for classifying network anomalies and intrusions,’’ in Proc. IEEE Int. Symp. Circuits Syst. (ISCAS), pp. 1–5, 2019.
  • S. Behera, A. Pradhan, and R. Dash, “Deep Neural Network Architecture for Anomaly Based Intrusion Detection System,” 5th International Conference on Signal Processing and Integrated Networks (SPIN 2018), pp. 270– 274, 2018.
  • Z. Li, Z. Qin, K. Huang, X. Yang, and S. Ye, ‘‘Intrusion detection using convolutional neural networks for representation learning,’’ in Proc. Int. Conf. Neural Inf. Process. pp. 858–866, 2017.
  • M. Lopez-Martin, B. Carro, A. Sanchez-Esguevillas, and J. Lloret, ‘‘Conditional variational autoencoder for prediction and feature recovery applied to intrusion detection in IoT,’’ Sensors, vol. 17, no. 9, p. 1967, Aug. 2017.
  • F. A. Khan, A. Gumaei, A. Derhab, and A. Hussain, ‘‘TSDL: A twostage deep learning model for efficient network intrusion detection,’’ IEEE Access, vol. 7, pp. 30373–30385, 2019.
  • T. Ma, F. Wang, J. Cheng, Y. Yu, and X. Chen, ‘‘A hybrid spectral clustering and deep neural network ensemble algorithm for intrusion detection in sensor networks,’’ Sensors, vol. 16, no. 10, p. 1701, Oct. 2016.
  • R. Vinayakumar, M. Alazab, K. P. Soman, P. Poornachandran, A. Al-Nemrat, and S. Venkatraman, ‘‘Deep learning approach for intelligent intrusion detection system,’’ IEEE Access, vol. 7, pp. 41525–41550, 2019.
  • N. Gao, L. Gao, Q. Gao, and H. Wang, "An Intrusion Detection Model Based on Deep Belief Networks," Proc. 2nd International Conference on Advanced Cloud and Big Data, Huangshan, China, pp. 247-252, 2014.
  • B. Deokar and A. Hazarnis, “Intrusion Detection System using Log Files and Reinforcement Learning,” International Journal of Computer Applications, vol. 45, no. 1919, pp. 28–35, 2012.
  • A. Servin and D. Kudenko, “Multi-agent reinforcement learning for intrusion detection: A case study and evaluation,” Frontiers in Artificial Intelligence and Applications, vol. 178, pp. 873–874, 2008.
  • R. Elderman, L. J. J. Pater, A. S. Thie, M. M. Drugan and M. A. Wiering, “Adversarial reinforcement learning in a cyber security simulation,” ICAART 2017- Proceedings of the 9th International Conference on Agents and Artificial Intelligence, pp. 559–566, 2017.
  • I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning. Cambridge, MA, USA: MIT Press, 2016.
  • L. P. Kaelbling, M. L. Littman, and A. W. Moore, “Reinforcement Learning: A Survey,” Journal of Artificial Intelligence Research, vol. 4, 1996.
  • N. Moustafa, J. Slay, "UNSW-NB15: A Comprehensive Data Set for Network i Intrusion Detection Systems (UNSW-NB15 Network Data Set)," Proceedings of the 2015 IEEE Military Communications and Information Systems Conference (MilCIS), pp. 1–6, 2015.
  • KDD Cup 1999. Avaliable online: https://kdd.ics.uci.edu/databases/kddcup99/kddcup99.html (Accessed on 20 November 2020).
  • 1998 DARPA Intrusion Detection Evaluation Dataset. Available online: https://www.ll.mit.edu/r-d/datasets/1998-darpa-intrusion-detection-evaluation-dataset (Accessed on 20 November 2020).
  • M. Tavallaee, E. Bagheri, W. Lu, and A.A. Ghorbani, “A detailed analysis of the KDD CUP 99 data set,” IEEE Symposium on Computational Intelligence for Security and Defense Applications, pp. 1–6, 2009.
  • H. Koduvely, “Github repository, gym-network_intrusion,” Retrieved from https://github.com/harik68/gym-network_intrusion, 2018.
  • Y. Sun, B. Xue, M. Zhang, and G. G. Yen, “An Experimental Study on Hyper-parameter Optimization for Stacked Auto-Encoders,” Proc. IEEE Congress on Evolutionary Computation, Rio de Janeiro, Brazil, pp. 1-8, 2018.
  • M. Lopez-Martin, B. Carro, A. Sanchez-Esguevillas, and J. Lloret, “Shallow neural network with kernel approximation for prediction problems in highly demanding data networks,” Expert Systems with Applications, vol. 124, pp. 196-208, 2019.
  • Y. Yang, K. Zheng, B. Wu, Y. Yang, and X. Wang, “Network intrusion detection based on supervised adversarial variational auto-encoder with regularization,” IEEE Access, vol. 8., pp. 42169-42184, 2020.

A Study on the Efficacy of Deep Reinforcement Learning for Intrusion Detection

Year 2021, Volume: 4 Issue: 1, 11 - 25, 30.04.2021
https://doi.org/10.35377/saucis.04.01.834048

Abstract

The world has witnessed a fast-paced digital transformation in the past decade, giving rise to all-connected environments. While the increasingly widespread availability of networks has benefited many aspects of our lives, providing the necessary infrastructure for smart autonomous systems, it has also created a large cyber attack surface. This has made real-time network intrusion detection a significant component of any computerized system. With the advances in computer hardware architectures with fast, high-volume data processing capabilities and the developments in the field of artificial intelligence, deep learning has emerged as a significant aid for achieving accurate intrusion detection, especially for zero-day attacks. In this paper, we propose a deep reinforcement learning-based approach for network intrusion detection and demonstrate its efficacy using two publicly available intrusion detection datasets, namely NSL-KDD and UNSW-NB15. The experiment results suggest that deep reinforcement learning has significant potential to provide effective intrusion detection in the increasingly complex networks of the future.

References

  • V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra and M. Riedmiller, “Playing Atari with Deep Reinforcement Learning,” pp. 1–9, 2013. Retrieved from http://arxiv.org/abs/1312.5602
  • M. Lai, “Giraffe: Using Deep Reinforcement Learning to Play Chess,” September, 2015. Retrieved from http://arxiv.org/abs/1509.01549
  • L. Wang, D. Zhang, L. Gao, J. Song, L. Guo and H. T. Shen, “MathDQN: Solving arithmetic word problems via deep reinforcement learning,” 32nd AAAI Conference on Artificial Intelligence, pp. 5545–5552, 2018.
  • S. Nemati, M. M. Ghassemi and G. D. Clifford, “Optimal medication dosing from suboptimal clinical examples: A deep reinforcement learning approach,” Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 2016.
  • Z. Zhou, X. Li and R. N. Zare, “Optimizing Chemical Reactions with Deep Reinforcement Learning,” ACS Central Science, vol. 3, no. 12, pp. 1337–1344, 2017.
  • M. Mahmud, M. S. Kaiser, A. Hussain and S. Vassanelli, “Applications of Deep Learning and Reinforcement Learning to Biological Data,” IEEE Transactions on Neural Networks and Learning Systems, vol. 29, no. 6, pp. 2063–2079, 2018.
  • T. Yen, A. Oprea and K. Onarlioglu, "Beehive: large-scale log analysis for detecting suspicious activity in enterprise networks,". Proc. 29th Annual Computer Security Applications Conference, pp. 199–208, 2013.
  • A. Razaq, H. Tianfield and P. Barrie, "A big data analytics based approach to anomaly detection," Proc. - 2016 IEEE/ACM 3rd International Conference on Big Data Computing Applications and Technologies (BDCAT), pp. 187–193, 2016.
  • A. O. Balogun and R. G. Jimoh, "Anomaly intrusion detection using a hybrid of decision tree and K-nearest neighbor," Journal of Advances in Scientific Research & Applications (JASRA), vol. 2, no. 1, pp. 67-74, 2015.
  • A. Hariharan, A. Gupta and T. Pal, "CAMLPAD: Cybersecurity Autonomous Machine Learning Platform for Anomaly Detection," Proc. Future of Information and Communication Conference (FICC), San Francisco, CA, USA, pp. 705-720, 2020.
  • M. S. Pervez and D. M. Farid, “Feature selection and intrusion classification in NSL-KDD cup 99 dataset employing SVMs,” SKIMA 2014 - 8th International Conference on Software, Knowledge, Information Management and Applications, pp. 1–6, 2014.
  • B. Ingre and A. Yadav, "Performance analysis of NSL-KDD dataset using ANN," 2015 International Conference on Signal Processing and Communication Engineering Systems, pp. 92-96, 2015.
  • S.O.M. Kamel, N. Hegazi, H. Harb, A. ElDein and H. ElKader, "AdaBoost Ensemble Learning Technique for Optimal Feature Subset Selection," International Journal of Computer Networks and Communications Security vol. 4, no. 1, pp. 1–11, 2016.
  • W. Hu, W. Hu, and S. Maybank, "AdaBoost-Based Algorithm for Network Intrusion Detection," IEEE Transactions on Systems, Man, and Cybernetics - Part B: Cybernetics, vol. 38, no. 2, pp. 577-583, 2008.
  • A. H. Engly, A. R. Larsen, and W. Meng, "Evaluation of Anomaly-Based Intrusion Detection with Combined Imbalance Correction and Feature Selection," Proc. 14th International Conference on Network and System Security, Melbourne, Australia, pp. 277-291, 2020.
  • N. Moustafa and J. Slay, "A hybrid feature selection for network intrusion detection systems: central points and association rules," arXiv:1707.05505, (2017) [cs.CR].
  • J. Kim and H. Kim, “Applying Recurrent Neural Network to Intrusion Detection with Hessian Free Optimization,” In: Kim H., Choi D. (eds) Information Security Applications. WISA 2015. Lecture Notes in Computer Science, vol. 9503, 2016, Springer, Cham.
  • Y. Chuan-long, Z. Yue-fei, F. Jin-long and H. Xin-zheng, “A Deep Learning Approach for Intrusion Detection using Recurrent Neural Networks,” IEEE Access, vol. 5, pp. 21954 - 2196, 2017.
  • C. Yin, Y. Zhu, J. Fei, and X. He, ‘‘A deep learning approach for intrusion detection using recurrent neural networks,’’ IEEE Access, vol. 5, pp. 21954–21961, 2017.
  • Z. Li, A. L. G. Rios, G. Xu, and L. Trajkovic, ‘‘Machine learning techniques for classifying network anomalies and intrusions,’’ in Proc. IEEE Int. Symp. Circuits Syst. (ISCAS), pp. 1–5, 2019.
  • S. Behera, A. Pradhan, and R. Dash, “Deep Neural Network Architecture for Anomaly Based Intrusion Detection System,” 5th International Conference on Signal Processing and Integrated Networks (SPIN 2018), pp. 270– 274, 2018.
  • Z. Li, Z. Qin, K. Huang, X. Yang, and S. Ye, ‘‘Intrusion detection using convolutional neural networks for representation learning,’’ in Proc. Int. Conf. Neural Inf. Process. pp. 858–866, 2017.
  • M. Lopez-Martin, B. Carro, A. Sanchez-Esguevillas, and J. Lloret, ‘‘Conditional variational autoencoder for prediction and feature recovery applied to intrusion detection in IoT,’’ Sensors, vol. 17, no. 9, p. 1967, Aug. 2017.
  • F. A. Khan, A. Gumaei, A. Derhab, and A. Hussain, ‘‘TSDL: A twostage deep learning model for efficient network intrusion detection,’’ IEEE Access, vol. 7, pp. 30373–30385, 2019.
  • T. Ma, F. Wang, J. Cheng, Y. Yu, and X. Chen, ‘‘A hybrid spectral clustering and deep neural network ensemble algorithm for intrusion detection in sensor networks,’’ Sensors, vol. 16, no. 10, p. 1701, Oct. 2016.
  • R. Vinayakumar, M. Alazab, K. P. Soman, P. Poornachandran, A. Al-Nemrat, and S. Venkatraman, ‘‘Deep learning approach for intelligent intrusion detection system,’’ IEEE Access, vol. 7, pp. 41525–41550, 2019.
  • N. Gao, L. Gao, Q. Gao, and H. Wang, "An Intrusion Detection Model Based on Deep Belief Networks," Proc. 2nd International Conference on Advanced Cloud and Big Data, Huangshan, China, pp. 247-252, 2014.
  • B. Deokar and A. Hazarnis, “Intrusion Detection System using Log Files and Reinforcement Learning,” International Journal of Computer Applications, vol. 45, no. 1919, pp. 28–35, 2012.
  • A. Servin and D. Kudenko, “Multi-agent reinforcement learning for intrusion detection: A case study and evaluation,” Frontiers in Artificial Intelligence and Applications, vol. 178, pp. 873–874, 2008.
  • R. Elderman, L. J. J. Pater, A. S. Thie, M. M. Drugan and M. A. Wiering, “Adversarial reinforcement learning in a cyber security simulation,” ICAART 2017- Proceedings of the 9th International Conference on Agents and Artificial Intelligence, pp. 559–566, 2017.
  • I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning. Cambridge, MA, USA: MIT Press, 2016.
  • L. P. Kaelbling, M. L. Littman, and A. W. Moore, “Reinforcement Learning: A Survey,” Journal of Artificial Intelligence Research, vol. 4, 1996.
  • N. Moustafa, J. Slay, "UNSW-NB15: A Comprehensive Data Set for Network i Intrusion Detection Systems (UNSW-NB15 Network Data Set)," Proceedings of the 2015 IEEE Military Communications and Information Systems Conference (MilCIS), pp. 1–6, 2015.
  • KDD Cup 1999. Avaliable online: https://kdd.ics.uci.edu/databases/kddcup99/kddcup99.html (Accessed on 20 November 2020).
  • 1998 DARPA Intrusion Detection Evaluation Dataset. Available online: https://www.ll.mit.edu/r-d/datasets/1998-darpa-intrusion-detection-evaluation-dataset (Accessed on 20 November 2020).
  • M. Tavallaee, E. Bagheri, W. Lu, and A.A. Ghorbani, “A detailed analysis of the KDD CUP 99 data set,” IEEE Symposium on Computational Intelligence for Security and Defense Applications, pp. 1–6, 2009.
  • H. Koduvely, “Github repository, gym-network_intrusion,” Retrieved from https://github.com/harik68/gym-network_intrusion, 2018.
  • Y. Sun, B. Xue, M. Zhang, and G. G. Yen, “An Experimental Study on Hyper-parameter Optimization for Stacked Auto-Encoders,” Proc. IEEE Congress on Evolutionary Computation, Rio de Janeiro, Brazil, pp. 1-8, 2018.
  • M. Lopez-Martin, B. Carro, A. Sanchez-Esguevillas, and J. Lloret, “Shallow neural network with kernel approximation for prediction problems in highly demanding data networks,” Expert Systems with Applications, vol. 124, pp. 196-208, 2019.
  • Y. Yang, K. Zheng, B. Wu, Y. Yang, and X. Wang, “Network intrusion detection based on supervised adversarial variational auto-encoder with regularization,” IEEE Access, vol. 8., pp. 42169-42184, 2020.
There are 40 citations in total.

Details

Primary Language English
Subjects Computer Software
Journal Section Articles
Authors

Halim Görkem Gülmez 0000-0003-0355-8790

Pelin Angın 0000-0002-6419-2043

Publication Date April 30, 2021
Submission Date November 30, 2020
Acceptance Date December 26, 2020
Published in Issue Year 2021Volume: 4 Issue: 1

Cite

IEEE H. G. Gülmez and P. Angın, “A Study on the Efficacy of Deep Reinforcement Learning for Intrusion Detection”, SAUCIS, vol. 4, no. 1, pp. 11–25, 2021, doi: 10.35377/saucis.04.01.834048.

29070    The papers in this journal are licensed under a Creative Commons Attribution-NonCommercial 4.0 International License