ВЛИЯНИЕ СТАТИСТИКИ ИНИЦИАЛИЗАЦИЙ  НА ТОЧНОСТЬ РЕЗЕРВУАРНЫХ ВЫЧИСЛЕНИЙ

Павел Абрамович Головинский; Станислав Валерьевич Перелыгин

doi:10.17308/sait/1995-5499/2025/4/145-154

Authors

Pavel A. Golovinski Voronezh State University https://orcid.org/0000-0002-7527-0297 (unauthenticated)
Stanislav V. Perelygin Voronezh State University https://orcid.org/0009-0007-2709-4261 (unauthenticated)

DOI:

https://doi.org/10.17308/sait/1995-5499/2025/4/145-154

Keywords:

machine learning, statistical models, reservoir computers, Monte Carlo method, Mackay — Glass model

Abstract

Modern machine learning-based solutions usually employ rather complex and timeconsuming computer algorithms. For many practical applications designed for use in mobile devices remote from high-performance computer systems, it is preferable to have more easily implemented and fast-acting methods and procedures that allow achieving results comparable to large and resource-intensive models. One of the solutions to this problem is reservoir computers that allow training based on efficient linear optimization procedures. The paper considers the influence of statistics of random initializations the reservoir weights on the quality of time series forecasting and the possibility of additional tuning the reservoir parameters with the Monte Carlo method. The test problem was data generated by the nonlinear dynamic MacKay — Glass model. Large-scale numerical experiments allowed us to found the boundaries of the leakage hyperparameters and the spectral radius of the weight matrix that provide the best reproduction of the simulated time series. Further improvements to reservoir computers can be achieved by evolutionary optimization algorithms to quickly adjust reservoir weights.

Author Biographies

Pavel A. Golovinski, Voronezh State University

DSc in Physics and Mathematics, Professor, Professor of the Department of Information Processing and Protection Technologies
Stanislav V. Perelygin, Voronezh State University

2nd year master’s student of the Department of Information Processing and Protection Technologies

References

Nakajima K. Reservoir Computing: Theory, Physical Implementations, and Applications / K. Nakajima, I. Fischer. – Singapore : Springer Nature Singapore Pte Ltd., 2021. – 458 p. – DOI: 10.1007/978-981-13-1687-6.

Jaeger H. The “echo state” approach to analyzing and training recurrent neural networks / H. Jaeger // Technical Report GMD Report 148. – Bonn : German National Research Center for Informational Technology, 2001. – 47 p.

Jaeger H. Optimization and applications of echo state networks with leaky-integrator neurons / H. Jaeger, M. Lukosevicius, D. Popovici, U. Siewert // Neural Networks. – 2007. – Vol. 20. – P. 335–352. – DOI: 10.1016/j.neunet.2007.04.016.

Stefenon S. F. Echo state network applied for classification of medium voltage insulators / S. F. Stefenon, L. O. Seman, N. F. S. Neto, L. H. Meyer, A. Nied, K.-C. Yow // International Journal of Electrical Power & Energy Systems. – 2022. – Vol. 134. – Art. 107336. – DOI: 10.1016/j.ijepes.2021.107336.

Kim J. Z. A neural machine code and programming framework for the reservoir computer / J. Z. Kim, D. S. Bassett // Nature Machine Intelligence. – 2023. – Vol. 5. – P. 622–630. – DOI: 10.1038/s42256-023-00668-8.

Sun J. Sequence Prediction and Classification of Echo State Networks / J. Sun, L. Li, H. Peng // Mathematics. – 2023. – Vol. 11, No 22. – Art. 4640. – DOI: 10.3390/math11224640.

Пугавко М. М. Динамика сети дискретных модельных нейронов при контролируемом обучении системы резервуарных вычислений / М. М. Пугавко, О. В. Масленников, В. И. Некоркин // Известия вузов. Прикладная нелинейная динамика. – 2020. – Т. 28, вып. 1. – С. 77–89. – DOI: 10.18500/0869-66322020-28-1-77-89.8. Yan M. Emerging opportunities and challenges for the future of reservoir computing / M. Yan, C. Huang, P. Bienstman [et al.] // Nature Communications. – 2024. – Vol. 15. – Art. 2056. – DOI: 10.1038/s41467-024-45187-1.

Grigoryeva L. Echo state networks are universal / L. Grigoryeva, J.-P. Ortega // Neural Networks. – 2018. – Vol. 108. – P. 495–508. – DOI: 10.1016/j.neunet.2018.08.025.

Li Z. Universality and Approximation Bounds for Echo State Networks With Random Weights / Z. Li, Y. Yang // IEEE Transactions on Neural Networks and Learning Systems. – 2025. – Vol. 36, No 2. – P. 2720–2732. – DOI: 10.1109/TNNLS.2023.3339512.

Lukosevicius M. A Practical Guide to Applying Echo State Networks / M. Lukosevicius // Neural Networks: Tricks of the Trade : Lecture Notes in Computer Science. – 2012. – Vol. 7700. – P. 659–686. – DOI: 10.1007/978-3-642-352898_36.

Buehner M. A Tighter Bound for the Echo State Property / M. Buehner, P. Young // IEEE Transactions on Neural Networks. – 2006. – Vol. 17, No 3. – P. 820–824. – DOI: 10.1109/TNN.2006.872357.

Cucchi M. Hands-on reservoir computing: a tutorial for practical implementation [Электронный ресурс] / M. Cucchi, S. Abreu, G. Ciccone et al. // Neuromorphic Computingand Engineering. – 2022. – Vol. 2, No 3. – Art. 032002. – DOI: 10.1088/2634-4386/ac7db7.

Mackey M. C. Oscillation and chaos in physiological control systems / M. C. Mackey, L. Glass // Science. – 1977. – Vol. 197, No 4300. – P. 287–289. – DOI: 10.1126/science.267326.

Gómez-Gil P. A Neural Network Scheme for Long-Term Forecasting of Chaotic Time Series / P. Gómez-Gil, J. M. Ramírez-Cortes, S. E. Pomares Hernández, J. A. Rangel-Magdaleno // Neural Processing Letters. – 2011. – Vol. 33. – P. 215–233. – DOI: 10.1007/s11063011-9174-0.

Михайлов Г. А. Статистическое моделирование. Методы Монте-Карло : учебное пособие для вузов [Электронный ресурс] / Г. А. Михайлов, А. В. Войтишек. – Москва : Юрайт, 2025. – 321 с. – (Высшее образование). – URL: https://urait.ru/bcode/559245 (дата обращения: 02.06.2025).

Hua Z. Statistical Analysis on Random Matrices of Echo State Network in PEMFC System’s Lifetime Prediction / Z. Hua, Z. Zheng, M.-C. Péra, F. Gao // Applied Sciences. – 2022. – Vol. 12, No 7. – Art. 3421. – DOI: 10.3390/app12073421.18. Горелик М. Высокопроизводительные приложения. Практическое руководство по эффективному программированию / М. Горелик, Й. Освальд. – Москва : Эксмо, 2022. – 528 с.

López-Ortiz E. J. Exploring deep echo state networks for image classification: a multi-reservoir approach / E. J. López-Ortiz, M. Perea-Trigo, L. M. Soria-Morillo, J. A. Álvarez-García // Neural Computing and Applications. – 2024. – Vol. 36. – P. 11901–11918. – DOI: 10.1007/s00521-024-09656-4.