Method for convolutional neural network hardware implementation based on a residue number system. (English) Zbl 1517.68441
Summary: Convolutional Neural Networks (CNN) show high accuracy in pattern recognition solving problem but have high computational complexity, which leads to slow data processing. To increase the speed of CNN, we propose a hardware implementation method with calculations in the residue number system with moduli of a special type \({{2}^{\alpha }}\) and \({{2}^{\alpha }} - 1\). A hardware simulation of the proposed method on Field-Programmable Gate Array for LeNet-5 CNN is trained with the MNIST, FMNIST, and CIFAR-10 image databases. It has shown that the proposed approach can increase the clock frequency and performance of the device by 11–12%, compared with the traditional approach based on the positional number system.
MSC:
| 68W35 | Hardware implementations of nonnumerical algorithms (VLSI algorithms, etc.) |
| 68T05 | Learning and adaptive systems in artificial intelligence |
References:
| [1] | Ashiq, F., CNN-based object recognition and tracking system to assist visually impaired people, IEEE Access, 10, 14819-14834 (2022) · doi:10.1109/ACCESS.2022.3148036 |
| [2] | Moon, C. I.; Lee, O., Skin microstructure segmentation and aging classification using CNN-based models, IEEE Access, 10, 4948-4956 (2022) · doi:10.1109/ACCESS.2021.3140031 |
| [3] | Mondal, A.K., Bhattacharjee, A., Singla P., and Prathosh, A.P., xViTCOS: explainable vision transformer based COVID-19 screening using radiography, IEEE J. Trans. Eng. Health Med., 2022, vol. 10, p. 1100110. doi:10.1109/JTEHM.2021.3134096 |
| [4] | Elharrouss, O.; Almaadeed, N.; Abualsaud, K.; Al-Maadeed, S.; Al-Ali, A.; Mohamed, A., FSC-set: counting, localization of football supporters crowd in the stadiums, IEEE Access, 10, 10445-10459 (2022) · doi:10.1109/ACCESS.2022.3144607 |
| [5] | Vieira, J. C.; Sartori, A.; Stefenon, S. F.; Perez, F. L.; De Jesus, G. S.; Leithardt, V. R.Q., Low-cost CNN for automatic violence recognition on embedded system, IEEE Access, 10, 25190-25202 (2022) · doi:10.1109/ACCESS.2022.3155123 |
| [6] | Wong, C.-C.; Chien, M.-Y.; Chen, R.-J.; Aoyama, H., andWong, K.-Y., Moving object prediction and grasping system of robot manipulator, IEEE Access, 10, 20159-20172 (2022) · doi:10.1109/ACCESS.2022.3151717 |
| [7] | Krizhevsky, A., Sutskever, I., and Hinton, G.E., ImageNet classification with deep convolutional neural networks, Adv. Neural Inf. Process. Syst., 2012, vol. 25, no. 2. |
| [8] | Nakahara, H. and Sasao, T., A deep convolutional neural network based on nested residue number system, Proc. 25th Int. Conf. on Field Programmable Logic and Applications (FPL), London, 2015, pp. 1-6. doi:10.1109/FPL.2015.7293933 |
| [9] | Nakahara, H. and Sasao, T., A high-speed low-power deep neural network on an FPGA based on the nested RNS: applied to an object detector, Proc. IEEE Int. Symp. on Circuits and Systems (ISCAS), Florence, 2018, pp. 1-5. doi:10.1109/ISCAS.2018.8351850 |
| [10] | Salamat, S., Imani, M., Gupta, S., and Rosing, T., RNSnet: in-memory neural network acceleration using residue number system, Proc. IEEE Int. Conf. on Rebooting Computing (ICRC), McLean, VA, 2018, pp. 1-12. doi:10.1109/ICRC.2018.8638592 |
| [11] | Omondi, A.; Premkumar, B., Residue Number Systems: Theory and Implementationi (2007), London: Imperial College Press, London · Zbl 1149.68019 · doi:10.1142/p523 |
| [12] | Chervyakov, N. I.; Lyakhov, P. A.; Deryabin, M. A.; Nagornov, N. N.; Valueva, M. V.; Valuev, G. V., Residue number system-based solution for reducing the hardware cost of a convolutional neural network, Neurocomputing, 407, 439-453 (2020) · doi:10.1016/j.neucom.2020.04.018 |
| [13] | Parhami, B., Computer Arithmetic: Algorithms and Hardware Designs (2010) |
| [14] | Vergos, H. T.; Dimitrakopoulos, G., On modulo 2^n+1 adder design, IEEE Trans. Comput., 61, 173-186 (2012) · Zbl 1365.65320 · doi:10.1109/TC.2010.261 |
| [15] | Kogge, P. M.; Stone, H. S., A parallel algorithm for the efficient solution of a general class of recurrence equations, IEEE Trans. Comput., C-22, 786-793 (1973) · Zbl 0262.68015 · doi:10.1109/TC.1973.5009159 |
| [16] | Chervyakov, N. I.; Lyakhov, P. A.; Valueva, M. V., Increasing of convolutional neural network performance using residue number system, Proc. Int. Multi-Conf. on Engineering, Computer and Information Sciences (2017), Novosibirsk-Yekaterinburg: SIBIRCON, Novosibirsk-Yekaterinburg |
| [17] | Tung, C.; Huang, C., A high-performance multiply-accumulate unit by integrating additions and accumulations into partial product reduction process, IEEE Access, 8, 87367-87377 (2020) · doi:10.1109/ACCESS.2020.2992286 |
| [18] | Habibi Aghdam, H.; Jahani Heravi, E., Guide to Convolutional Neural Networks (2017), Cham: Springer Int. Publ., Cham · doi:10.1007/978-3-319-57550-6 |
| [19] | Valueva, M., Construction of residue number system using hardware efficient diagonal function, Electronics, 8, 694 (2019) · doi:10.3390/electronics8060694 |
| [20] | Chervyakov, N. I., Residue-to-binary conversion for general moduli sets based on approximate Chinese remainder theorem, Int. J. Comput. Math., 94, 1833-1849 (2017) · Zbl 06905133 · doi:10.1080/00207160.2016.1247439 |
| [21] | Haykin, S. S., Neural Networks: a Comprehensive Foundation (1999) · Zbl 0934.68076 |
| [22] | LeCun, Y.; Bottou, L.; Bengio, Y.; Haffiner, P., Gradient-based learning applied to document recognition, Proc. IEEE, 86, 2278-2324 (1998) · doi:10.1109/5.726791 |
| [23] | Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G.S., Davis, A., Dean, J., Devin, M., Ghemawat, S., Goodfellow, I., Harp, A., Irving, G., Isard, M., Jozefowicz, R., Jia, Y., Kaiser, L., Kudlur, M., Levenberg, J., Mane, D., Schuster, M., Monga, R., Moore, S., Murray, D., Olah, C., Shlens, J., Steiner, B., Sutskever, I., Talwar, K., Tucker, P., Vanhoucke, V., Vasudevan, V., Viegas, F., Vinyals, O., Warden, P., Wattenberg, M., Wicke, M., Yu, Y., and Zheng, X., TensorFlow: Large-scale machine learning on heterogeneous systems, 2015. http://tensorflow.org. |
| [24] | Xiao, H., Kashif, R., and Vollgraf, R., Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms, 2017. arXiv:1708.07747. |
| [25] | Krizhevsky, A., et al., Learning multiple layers of features from tiny images, Tech. Rep. TR-2009, Univ. of Toronto, 2009. |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
