Research Article
BibTex RIS Cite

Year 2024, , 156 - 172, 31.08.2024

Gülyeter Öztürk , Osman Eldoğan

https://doi.org/10.35377/saucis...1404116

Abstract

References

  • A. L. Mrgole and D. Sever, “Incorporation of Duffing Oscillator and Wigner-Ville Distribution in Traffic Flow Prediction,” Promet - Traffic&Transportation, vol. 29, no. 1, pp. 13–22, Feb. 2017, doi: 10.7307/ptt.v29i1.2116.
  • J. Runge and R. Zmeureanu, “A Review of Deep Learning Techniques for Forecasting Energy Use in Buildings,” Energies, vol. 14, no. 3, Art. no. 3, Jan. 2021, doi: 10.3390/en14030608.
  • O. B. Sezer, M. U. Gudelek, and A. M. Ozbayoglu, “Financial Time Series Forecasting with Deep Learning : A Systematic Literature Review: 2005-2019.” arXiv, Nov. 29, 2019. doi: 10.48550/arXiv.1911.13288.
  • I. Yazici, O. F. Beyca, and D. Delen, “Deep-learning-based short-term electricity load forecasting: A real case application,” Engineering Applications of Artificial Intelligence, vol. 109, p. 104645, Mar. 2022, doi: 10.1016/j.engappai.2021.104645.
  • M. Murat, I. Malinowska, M. Gos, and J. Krzyszczak, “Forecasting daily meteorological time series using ARIMA and regression models,” International Agrophysics, vol. 32, no. 2, pp. 253–264, Apr. 2018, doi: 10.1515/intag-2017-0007.
  • P. Kavianpour, M. Kavianpour, E. Jahani, and A. Ramezani, “A CNN-BiLSTM model with attention mechanism for earthquake prediction,” J Supercomput, May 2023, doi: 10.1007/s11227-023-05369-y.
  • T. Ouyang, H. Huang, Y. He, and Z. Tang, “Chaotic wind power time series prediction via switching data-driven modes,” Renewable Energy, vol. 145, pp. 270–281, Jan. 2020, doi: 10.1016/j.renene.2019.06.047.
  • C. Cheng et al., “Time series forecasting for nonlinear and non-stationary processes: a review and comparative study.” IIE Transactions, vol. 47, no. 10, pp. 1053–1071, Oct. 2015, doi: 10.1080/0740817X.2014.999180.
  • H. V. Dudukcu, M. Taskiran, Z. G. C. Taskiran, and T. Yildirim, “Temporal Convolutional Networks with RNN approach for chaotic time series prediction,” Applied Soft Computing, vol. 133, p. 109945, Jan. 2023, doi: 10.1016/j.asoc.2022.109945.
  • D. S. K. Karunasinghe and S.-Y. Liong, “Chaotic time series prediction with a global model: Artificial neural network,” Journal of Hydrology, vol. 323, no. 1, pp. 92–105, May 2006, doi: 10.1016/j.jhydrol.2005.07.048.
  • H. Yuxia and Z. Hongtao, “Chaos Optimization Method of SVM Parameters Selection for Chaotic Time Series Forecasting,” Physics Procedia, vol. 25, pp. 588–594, Jan. 2012, doi: 10.1016/j.phpro.2012.03.130.
  • Y. Xiu and W. Zhang, “Multivariate Chaotic Time Series Prediction Based on NARX Neural Networks,” in 2017 2nd International Conference on Electrical, Automation and Mechanical Engineering (EAME 2017), vol. 86. Paris: Atlantis Press, 2017, pp. 164–167.
  • S. Siami-Namini and A. S. Namin, “Forecasting Economics and Financial Time Series: ARIMA vs. LSTM.” arXiv, Mar. 16, 2018. doi: 10.48550/arXiv.1803.06386.
  • R. Khaldi, A. E. Afia, R. Chiheb, and S. Tabik, “What is the best RNN-cell structure to forecast each time series behavior?,” Expert Systems with Applications, vol. 215, p. 119140, Apr. 2023, doi: 10.1016/j.eswa.2022.119140.
  • G. Alkhayat and R. Mehmood, “A review and taxonomy of wind and solar energy forecasting methods based on deep learning,” Energy and AI, vol. 4, p. 100060, Jun. 2021, doi: 10.1016/j.egyai.2021.100060.
  • H. Liu, G. Yan, Z. Duan, and C. Chen, “Intelligent modeling strategies for forecasting air quality time series: A review,” Applied Soft Computing, vol. 102, p. 106957, Apr. 2021, doi: 10.1016/j.asoc.2020.106957.
  • J. L. Elman, “Finding structure in time,” Cognitive Science, vol. 14, no. 2, pp. 179–211, Apr. 1990, doi: 10.1016/0364-0213(90)90002-E.
  • K. Cho, B. V. Merrienboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk, and Y. Bengio, “Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation.” arXiv, Sep. 02, 2014. doi: 10.48550/arXiv.1406.1078.
  • R. Chandra and M. Zhang, “Cooperative coevolution of Elman recurrent neural networks for chaotic time series prediction,” Neurocomputing, vol. 86, pp. 116–123, Jun. 2012, doi: 10.1016/j.neucom.2012.01.014.
  • G. Yanan, C. Xiaoqun, L. Bainian, and P. Kecheng, “Chaotic Time Series Prediction Using LSTM with CEEMDAN.” Journal of Physics: Conferences Series, vol. 1617, no. 1, p. 012094, Aug. 2020, doi: 10.1088/1742-6596/1617/1/012094.
  • H. V. Dudukcu, M. Taskiran, and Z. G. C. Taskiran, “Comprehensive Comparison of LSTM Variations for the Prediction of Chaotic Time Series,” in 2021 International Conference on INnovations in Intelligent SysTems and Applications (INISTA), Aug. 2021, pp. 1–5. doi: 10.1109/INISTA52262.2021.9548647.
  • K. Fu, H. Li, and P. Deng, “Chaotic time series prediction using DTIGNet based on improved temporal-inception and GRU,” Chaos, Solitons & Fractals, vol. 159, p. 112183, Jun. 2022, doi: 10.1016/j.chaos.2022.112183.
  • W. Cheng et al., “High-efficiency chaotic time series prediction based on time convolution neural network,” Chaos, Solitons & Fractals, vol. 152, p. 111304, Nov. 2021, doi: 10.1016/j.chaos.2021.111304.
  • S. Hochreiter and J. Schmidhuber, “Long Short-Term Memory,” Neural Computation, vol. 9, no. 8, pp. 1735–1780, Nov. 1997, doi: 10.1162/neco.1997.9.8.1735.
  • E. N. Lorenz, “Deterministic Nonperiodic Flow,” Journal of the Atmospheric Sciences, vol. 20, no. 2, pp. 130–141, Mar. 1963, doi: 10.1175/1520-0469(1963)020<0130:DNF>2.0.CO;2.
  • G. Chen and T. Ueta, “Yet another chaotic attractor.” International Journal of Bifurcation and Chaos, vol. 09, no. 07, pp. 1465–1466, Jul. 1999, doi: 10.1142/S0218127499001024.
  • K. Ito, “Chaos in the Rikitake two-disc dynamo system,” Earth and Planetary Science Letters, vol. 51, no. 2, pp. 451–456, Dec. 1980, doi: 10.1016/0012-821X(80)90224-1.
  • T. Rikitake, “Oscillations of a system of disk dynamos,” Mathematical Proceedings of the Cambridge Philosophical Society, vol. 54, no. 1, pp. 89–105, Jan. 1958, doi: 10.1017/S0305004100033223.
  • L. Wang and L. Dai, “Chaotic Time Series Prediction of Multi-Dimensional Nonlinear System Based on Bidirectional LSTM Model,” Advanced Theory and Simulations, vol. 6, no. 8, p. 2300148, 2023, doi: 10.1002/adts.202300148.

Prediction of Multivariate Chaotic Time Series using GRU, LSTM and RNN

Year 2024, , 156 - 172, 31.08.2024

Gülyeter Öztürk , Osman Eldoğan

https://doi.org/10.35377/saucis...1404116

Abstract

Chaotic systems are identified as nonlinear, deterministic dynamic systems that are exhibit sensitive to initial values. Some chaotic equations modeled from daily events involve time information and generate chaotic time series that are sequential data. Through successful prediction studies conducted on the generated chaotic time series, forecasts can be made about events displaying unpredictable behavior in nature, which have not yet been modeled. This enables preparation for both favorable and unfavorable situations that may arise. In this study, chaotic time series were generated using Lorenz, Chen, and Rikitake multivariate chaotic systems. To enhance prediction accuracy on the generated data, GRU, LSTM and RNN models were trained with different hyperparameters. Subsequently, comprehensive test studies were conducted to evaluate their performance. Predictions were calculated using evaluation metrics, including MSE, RMSE, MAE, MAPE, and R2. In the experimental study, each chaotic system was trained with different hyperparameter combinations on six network models. The experimental results indicate that the utilized models exhibited greater success in predicting chaotic time series compared to some other models in the literature.

Keywords

Chaotic time series Multivariate Time series prediction GRU LSTM RNN

References

  • A. L. Mrgole and D. Sever, “Incorporation of Duffing Oscillator and Wigner-Ville Distribution in Traffic Flow Prediction,” Promet - Traffic&Transportation, vol. 29, no. 1, pp. 13–22, Feb. 2017, doi: 10.7307/ptt.v29i1.2116.
  • J. Runge and R. Zmeureanu, “A Review of Deep Learning Techniques for Forecasting Energy Use in Buildings,” Energies, vol. 14, no. 3, Art. no. 3, Jan. 2021, doi: 10.3390/en14030608.
  • O. B. Sezer, M. U. Gudelek, and A. M. Ozbayoglu, “Financial Time Series Forecasting with Deep Learning : A Systematic Literature Review: 2005-2019.” arXiv, Nov. 29, 2019. doi: 10.48550/arXiv.1911.13288.
  • I. Yazici, O. F. Beyca, and D. Delen, “Deep-learning-based short-term electricity load forecasting: A real case application,” Engineering Applications of Artificial Intelligence, vol. 109, p. 104645, Mar. 2022, doi: 10.1016/j.engappai.2021.104645.
  • M. Murat, I. Malinowska, M. Gos, and J. Krzyszczak, “Forecasting daily meteorological time series using ARIMA and regression models,” International Agrophysics, vol. 32, no. 2, pp. 253–264, Apr. 2018, doi: 10.1515/intag-2017-0007.
  • P. Kavianpour, M. Kavianpour, E. Jahani, and A. Ramezani, “A CNN-BiLSTM model with attention mechanism for earthquake prediction,” J Supercomput, May 2023, doi: 10.1007/s11227-023-05369-y.
  • T. Ouyang, H. Huang, Y. He, and Z. Tang, “Chaotic wind power time series prediction via switching data-driven modes,” Renewable Energy, vol. 145, pp. 270–281, Jan. 2020, doi: 10.1016/j.renene.2019.06.047.
  • C. Cheng et al., “Time series forecasting for nonlinear and non-stationary processes: a review and comparative study.” IIE Transactions, vol. 47, no. 10, pp. 1053–1071, Oct. 2015, doi: 10.1080/0740817X.2014.999180.
  • H. V. Dudukcu, M. Taskiran, Z. G. C. Taskiran, and T. Yildirim, “Temporal Convolutional Networks with RNN approach for chaotic time series prediction,” Applied Soft Computing, vol. 133, p. 109945, Jan. 2023, doi: 10.1016/j.asoc.2022.109945.
  • D. S. K. Karunasinghe and S.-Y. Liong, “Chaotic time series prediction with a global model: Artificial neural network,” Journal of Hydrology, vol. 323, no. 1, pp. 92–105, May 2006, doi: 10.1016/j.jhydrol.2005.07.048.
  • H. Yuxia and Z. Hongtao, “Chaos Optimization Method of SVM Parameters Selection for Chaotic Time Series Forecasting,” Physics Procedia, vol. 25, pp. 588–594, Jan. 2012, doi: 10.1016/j.phpro.2012.03.130.
  • Y. Xiu and W. Zhang, “Multivariate Chaotic Time Series Prediction Based on NARX Neural Networks,” in 2017 2nd International Conference on Electrical, Automation and Mechanical Engineering (EAME 2017), vol. 86. Paris: Atlantis Press, 2017, pp. 164–167.
  • S. Siami-Namini and A. S. Namin, “Forecasting Economics and Financial Time Series: ARIMA vs. LSTM.” arXiv, Mar. 16, 2018. doi: 10.48550/arXiv.1803.06386.
  • R. Khaldi, A. E. Afia, R. Chiheb, and S. Tabik, “What is the best RNN-cell structure to forecast each time series behavior?,” Expert Systems with Applications, vol. 215, p. 119140, Apr. 2023, doi: 10.1016/j.eswa.2022.119140.
  • G. Alkhayat and R. Mehmood, “A review and taxonomy of wind and solar energy forecasting methods based on deep learning,” Energy and AI, vol. 4, p. 100060, Jun. 2021, doi: 10.1016/j.egyai.2021.100060.
  • H. Liu, G. Yan, Z. Duan, and C. Chen, “Intelligent modeling strategies for forecasting air quality time series: A review,” Applied Soft Computing, vol. 102, p. 106957, Apr. 2021, doi: 10.1016/j.asoc.2020.106957.
  • J. L. Elman, “Finding structure in time,” Cognitive Science, vol. 14, no. 2, pp. 179–211, Apr. 1990, doi: 10.1016/0364-0213(90)90002-E.
  • K. Cho, B. V. Merrienboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk, and Y. Bengio, “Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation.” arXiv, Sep. 02, 2014. doi: 10.48550/arXiv.1406.1078.
  • R. Chandra and M. Zhang, “Cooperative coevolution of Elman recurrent neural networks for chaotic time series prediction,” Neurocomputing, vol. 86, pp. 116–123, Jun. 2012, doi: 10.1016/j.neucom.2012.01.014.
  • G. Yanan, C. Xiaoqun, L. Bainian, and P. Kecheng, “Chaotic Time Series Prediction Using LSTM with CEEMDAN.” Journal of Physics: Conferences Series, vol. 1617, no. 1, p. 012094, Aug. 2020, doi: 10.1088/1742-6596/1617/1/012094.
  • H. V. Dudukcu, M. Taskiran, and Z. G. C. Taskiran, “Comprehensive Comparison of LSTM Variations for the Prediction of Chaotic Time Series,” in 2021 International Conference on INnovations in Intelligent SysTems and Applications (INISTA), Aug. 2021, pp. 1–5. doi: 10.1109/INISTA52262.2021.9548647.
  • K. Fu, H. Li, and P. Deng, “Chaotic time series prediction using DTIGNet based on improved temporal-inception and GRU,” Chaos, Solitons & Fractals, vol. 159, p. 112183, Jun. 2022, doi: 10.1016/j.chaos.2022.112183.
  • W. Cheng et al., “High-efficiency chaotic time series prediction based on time convolution neural network,” Chaos, Solitons & Fractals, vol. 152, p. 111304, Nov. 2021, doi: 10.1016/j.chaos.2021.111304.
  • S. Hochreiter and J. Schmidhuber, “Long Short-Term Memory,” Neural Computation, vol. 9, no. 8, pp. 1735–1780, Nov. 1997, doi: 10.1162/neco.1997.9.8.1735.
  • E. N. Lorenz, “Deterministic Nonperiodic Flow,” Journal of the Atmospheric Sciences, vol. 20, no. 2, pp. 130–141, Mar. 1963, doi: 10.1175/1520-0469(1963)020<0130:DNF>2.0.CO;2.
  • G. Chen and T. Ueta, “Yet another chaotic attractor.” International Journal of Bifurcation and Chaos, vol. 09, no. 07, pp. 1465–1466, Jul. 1999, doi: 10.1142/S0218127499001024.
  • K. Ito, “Chaos in the Rikitake two-disc dynamo system,” Earth and Planetary Science Letters, vol. 51, no. 2, pp. 451–456, Dec. 1980, doi: 10.1016/0012-821X(80)90224-1.
  • T. Rikitake, “Oscillations of a system of disk dynamos,” Mathematical Proceedings of the Cambridge Philosophical Society, vol. 54, no. 1, pp. 89–105, Jan. 1958, doi: 10.1017/S0305004100033223.
  • L. Wang and L. Dai, “Chaotic Time Series Prediction of Multi-Dimensional Nonlinear System Based on Bidirectional LSTM Model,” Advanced Theory and Simulations, vol. 6, no. 8, p. 2300148, 2023, doi: 10.1002/adts.202300148.
There are 29 citations in total.

Details

Primary Language English
Subjects Control Engineering, Mechatronics and Robotics (Other)
Journal Section Articles
Authors

Gülyeter Öztürk 0000-0001-9157-3709

Osman Eldoğan 0000-0001-9236-8985

Early Pub Date August 23, 2024
Publication Date August 31, 2024
Submission Date December 12, 2023
Acceptance Date July 22, 2024
Published in Issue Year 2024

Cite

IEEE G. Öztürk and O. Eldoğan, “Prediction of Multivariate Chaotic Time Series using GRU, LSTM and RNN”, SAUCIS, vol. 7, no. 2, pp. 156–172, 2024, doi: 10.35377/saucis...1404116.

29070    The papers in this journal are licensed under a Creative Commons Attribution-NonCommercial 4.0 International License