×

An efficient initialization mechanism of neurons for winner takes all neural network implemented in the CMOS technology. (English) Zbl 1410.92013

Summary: The paper presents a new initialization mechanism based on a Convex Combination Method (CCM) for Kohonen self-organizing Neural Networks (NNs) realized in the CMOS technology. A proper selection of initial values of the neuron weights exhibits a strong impact on the quality of the overall learning process. Unfortunately, in case of real input data, e.g. biomedical data, proper initialization is not easy to perform, as an exact data distribution is usually unknown. Bad initialization causes that even 70%–80% of neurons remain inactive, which increases the quantization error and thus limits the classification abilities of the NN. The proposed initialization algorithm has a couple of important advantages. Firstly, it does not require a knowledge of data distribution in the input data space. Secondly, there is no necessity for an initial polarization of the neuron weights before starting the learning process. This feature is very convenient in case of transistor level realizations. In this case the programming lines, which in other approaches occupy a large chip area, are not required. We proposed a modification of the original CCM algorithm. A new parameter which in the proposed analog CMOS realization is represented by an external current, allows to fit the behavior of the mechanism to NNs containing different numbers of neurons. The investigations show that the modified CCM operates properly for the NN containing even 250 neurons. A single CCM block realized in the CMOS 180nm technology occupies an area of \(300\operatorname{\mu}m^{2}\) and dissipates an average power of \(20\operatorname{\mu}W\) and at data rate of up to 20MHz.

MSC:

92B20 Neural networks for/in biological studies, artificial life and related topics
Full Text: DOI

References:

[1] Gatet, L.; Tap-Bteille, H.; Bony, F., Comparison between analog and digital neural network implementations for range-finding applications, IEEE Trans. Neural Netw., 20, 3, 460-470 (March 2009)
[2] Oike, Y.; Ikeda, M.; Asada, K., A high-speed and low-voltage associative co-processor with exact Hamming/Manhattan-distance estimation using word-parallel and hierarchical search architecture, IEEE J. Solid-State Circuits, 39, 8, 1383-1387 (Aug. 2004)
[4] Długosz, R.; Talaśka, T.; Pedrycz, W.; Wojtyna, R., Realization of the conscience mechanism in CMOS implementation of winner-takes-all self-organizing neural networks, IEEE Trans. Neural Netw., 21, Iss. 6, 961-971 (June 2010)
[5] Długosz, R.; Talaśka, T.; Pedrycz, W., Current-mode analog adaptive mechanism for ultra-low power neural networks, IEEE Trans. Circuits Syst.-II: Express Briefs, 58, Iss. 1, 31-35 (January 2011)
[6] Długosz, R.; Talaśka, T., Low power current-mode binary-tree asynchronous min/max circuit, Microelectron. J., Elsevier, 41, 1, 64-73 (2010)
[7] Kolasa, M.; Długosz, R.; Pedrycz, W.; Szulc, M., A programmable triangular neighborhood function for a Kohonen self-organizing map implemented on chip, Neural Netw., 25, 146-160 (2012)
[8] Becker, T.; Kluge, M.; Schalk, J.; Tiplady, K.; Paget, C.; Hilleringmann, U.; Otterpohl, T., Autonomous sensor nodes for aircraft structural health monitoring, IEEE Sensors J., 9, 11 (2009)
[10] Latré, B.; Braem, B.; Moerman, I.; Blondia, C.; Demeester, P., A survey on wireless body area networks, Wireless Netw., 17, 1, 1-18 (2011)
[15] Ahalt, S. C.; Krishnamurthy, A. K.; Chen, P.; Melton, D. E., Competitive learning algorithms for vector quantization, Neural Netw., 3, 131-134 (1990)
[16] DeSieno, D., Adding a conscience to competitive learning, IEEE Conf. Neural Netw., 1, 117-124 (1988)
[17] Yam, Jim Y. F.; Chow, Tommy W. S., A weight initialization method for improving training speed in feedforward neural network, Neurocomput. Issues, 30, 14, 219232 (Jan. 2000)
[18] Yam, Y. F.; Chow, Tommy W. S.; Leung, C. T., A new method in determining initial weights of feedforward neural networks for training enhancement, Neurocomputing, 16, 1, 2332 (1 1997)
[21] Kohonen, T., Self-Organizing Maps (2001), Springer Verlag: Springer Verlag Berlin · Zbl 0957.68097
[22] Okabe, A.; Boots, B.; Sugihara, K.; Nok Chiu, S., Spatial Tessellations Concepts and Applications of Voronoi Diagrams (2000), John Wiley, (ISBN 0-471-98635-6) · Zbl 0946.68144
[23] Graupe, D., Principles of artificial neural networks, Advanced Series on Circuits and Systems, vol. 6 (2007), World Scientific · Zbl 1139.68399
[24] Vora, N.; Tambe, S. S.; Kulkarni, B. D., Counterpropagation neural networks for fault detection and diagnosis, Comput. Chem. Eng., 21, 2, 177-185 (1997)
[25] Lehtokangas, M.; Saarinen, J., Weight initialization with reference patterns, Neurocomputing, 20, 1-3, 265-278 (1998)
[26] Nielsen, H., Counterpropagation networks, Appl. Opt., 26, 4979-4984 (1987)
[28] Gopalan, A.; Titus, A. H., A new wide range euclidean distance circuit for neural network hardware implementations, IEEE Trans. Neural Netw., 14, 5 (Sep. 2003)
[29] Cauwenberghs, G.; Pedroni, V., A low-power CMOS analog vector quantizer, IEEE J. Solid-State Circuits, 32, 8, 1278-1283 (1997)
[30] Liu, Bin-Da; Chen, Chuen-Yau; Tsao, Ju-Ying, A modular current-mode classifier circuit for template matching application, IEEE Trans. Circuits Syst. II: Analog Digital Signal Processing, 47, 2, 145-151 (2000)
[31] Vlassis, S.; Fikos, G.; Siskos, S., A floating gate CMOS Euclidean distance calculator and its application to hand-written digit recognition, Int. Conf. Image Processing, 3, 350-353 (2001)
[32] Bult, K.; Wallinga, H., A class of analog CMOS circuits based on the square-law characteristic of an MOS transistors in saturation, IEEE J. Solid-State Circuits, sc-22, 3, 357-365 (1987)
[33] Długosz, R.; Kolasa, M.; Talaśka, T.; Pauk, J.; Wojtyna, R.; Szulc, M.; Gargua, K.; Farine, P-A, Power, low chip area, digital distance calculation circuit for self-organizing neural networks realized in the CMOS technology, Solid State Phenom., 199 (2013)
[35] Długosz, R.; Iniewski, K., Flexible architecture of ultra-low-power current-mode interleaved successive approximation analog-to-digital converter for wireless sensor networks, VLSI Design J., Hindavi Publishing, vol. 2007 (2007), (Article ID 45269)
[37] Osowski, S.; Linh, T. H., ECG beat recognition using fuzzy hybrid neural network, IEEE Trans. Biomed. Eng., 48, 11, 1265-1271 (2001)
[38] Valenza, G.; Lanata, A.; Ferro, M.; Scilingo, E. P., Real-time discrimination of multiple cardiac arrhythmias for wearable systems based on neural networks, Comput. Cardiology, 35, 1053-1056 (2008)
[39] Lagerholm, M.; Peterson, G., Clustering ECG complexes using hermite functions and self-organizing maps, IEEE Trans. Biomed. Eng., 47, 7, 838-848 (2000)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.