Pipelining of Fuzzy ARTMAP without matchtracking: Correctness, performance bound, and Beowulf evaluation. (English) Zbl 1158.68442
Summary: Fuzzy ARTMAP neural networks have been proven to be good classifiers on a variety of classification problems. However, the time that Fuzzy ARTMAP takes to converge to a solution increases rapidly as the number of patterns used for training is increased. In this paper we examine the time Fuzzy ARTMAP takes to converge to a solution and we propose a coarse grain parallelization technique, based on a pipeline approach, to speed-up the training process. In particular, we have parallelized Fuzzy ARTMAP without the match-tracking mechanism. We provide a series of theorems and associated proofs that show the characteristics of Fuzzy ARTMAP’s, without matchtracking, parallel implementation. Results run on a BEOWULF cluster with three large databases show linear speedup as a function of the number of processors used in the pipeline. The databases used for our experiments are the Forrest CoverType database from the UCI Machine Learning repository and two artificial databases, where the data generated were 16-dimensional Gaussian distributed data belonging to two distinct classes, with different amounts of overlap (5% and 15%).
MSC:
| 68T05 | Learning and adaptive systems in artificial intelligence |
| 68T10 | Pattern recognition, speech recognition |
References:
| [1] | Agrawal, R.; Srikant, R., Fast algorithms for mining association rules in large databases, (Bocca, J. B.; Jarke, M.; Zaniolo, C., Proceedings of the twentieth international conference on very large databases (1994), Morgan Kaufmann: Morgan Kaufmann Santiago, Chile), 487-499 |
| [2] | Anagnostopoulos, G. (2000). Novel approaches in adaptive resonance theory for machine learning. Unpublished doctoral dessertation. Computer Engineering, UCF; Anagnostopoulos, G. (2000). Novel approaches in adaptive resonance theory for machine learning. Unpublished doctoral dessertation. Computer Engineering, UCF |
| [3] | Anagnostopoulos, G. C.; Georgiopoulos, M., (Ellipsoid ART and ARTMAP for incremental unsupervised and supervised learning. Ellipsoid ART and ARTMAP for incremental unsupervised and supervised learning, Proceedings of the IEEE-INNS-ENNS, Vol. 2 (2001), IEEE-INNS-ENNS: IEEE-INNS-ENNS Washington, DC), 1221-1226 |
| [4] | Anagnostopoulos, G.C., & Georgiopoulos, M. (2003). Putting the utility of match tracking in Fuzzy ARTMAP training to the test. In Seventh international conference on knowledge-based intelligent information and engineering systems; Anagnostopoulos, G.C., & Georgiopoulos, M. (2003). Putting the utility of match tracking in Fuzzy ARTMAP training to the test. In Seventh international conference on knowledge-based intelligent information and engineering systems |
| [5] | Asanović, K.; Beck, J.; Kingsbury, B.; Morgan, N.; Johnson, D.; Wawrzynek, J., Training neural networks with SPERT-II, (Editors, P. S.; Sundararajan, N., Parallel architectures for artificial neural networks: Paradigms and implementations (1998), IEEE Computer Society Press and John Wiley and Sons) |
| [6] | Blackard, J. A. (1999). Comparison of neural networks and discriminant analysis in predicting forest cover types. Unpublished doctoral dessertation. Department of Forest Sciences, Colorado State University; Blackard, J. A. (1999). Comparison of neural networks and discriminant analysis in predicting forest cover types. Unpublished doctoral dessertation. Department of Forest Sciences, Colorado State University |
| [7] | Carpenter, G. A.; Grossberg, S.; Markuzon, N.; Reynolds, J. H.; Rosen, D. B., Fuzzy ARTMAP: A neural network architecture for incremental learning of analog multidimensional maps, IEEE Transactions on Neural Networks, 3, 5, 698-713 (1992) |
| [8] | Carpenter, G. A.; Grossberg, S.; Reynolds, J. H., (Fuzzy ART: An adaptive resonance algorithm for rapid, stable classification of analog patterns. Fuzzy ART: An adaptive resonance algorithm for rapid, stable classification of analog patterns, International joint conference on neural networks, IJCNN’91, Vol. II (1991), IEEE-INNS-ENNS: IEEE-INNS-ENNS Seattle, Washington), 411-416 |
| [9] | Carpenter, G. A.; Markuzon, N., ARTMAP-IC and medical diagnosys: Instance counting and inconsistent cases, Neural Networks, 11, 793-813 (1998) |
| [10] | Carpenter, G. A.; Ross, W. D., ART-EMAP: A neural network architecture for object recognition by evidence accumulation, IEEE Transactions on Neural Networks, 6, 5, 805-818 (1995) |
| [11] | Caudell, T. P.; Healy, M. J., (Studies of generalization for the LAPART-2 architecture. Studies of generalization for the LAPART-2 architecture, International joint conference on neural networks, Vol. 3 (1999), IEEE-INNS-ENNS: IEEE-INNS-ENNS Washington, DC), 1979-1982 |
| [12] | Kasuba, T., Simplified Fuzzy ARTMAP, AI Expert, 18-25 (1993) |
| [13] | King, R.; Feng, C.; Shutherland, A., STATLOG: Comparison of classification algorithms on large real-world problems, Applied Artificial Intelligence, 9, 3, 259-287 (1995) |
| [14] | Kirkpatrick, S.; Stoll, E., A very fast shift-register sequence random number generator, Journal of Computational Physics, 40, 517-526 (1981) · Zbl 0458.65003 |
| [15] | Malkani, A.; Vassiliadis, C. A., Parallel implementation of the Fuzzy ARTMAP neural network paradigm on a hypercube, Expert Systems, 12, 1, 39-53 (1995) |
| [16] | Mangasarian, O.; Solodov, M., Serial and parallel backpropagation convergence via nonmonotone perturbed minimization, Optimization Methods and Software, 4, 2, 103-116 (1994) |
| [17] | Manolakos, E. S., Parallel implementation of ART1 Neural Networks on processor ring architectures, (Sundararajan, N.; Saratchandran, P., Parallel architectures for neural networks: Paradigms and implementations (1998), IEEE Computer Society Press and John Wiley and Sons) |
| [18] | Mehta, M.; Agrawal, R.; Rissanen, J., SLIQ: A fast scalable classifier for data mining, (Extending Database Technology (1996), Springer: Springer Avignon, France), 18-32 |
| [19] | Petridis, V.; Kaburlasos, V. G.; Fragkou, V. G.; Kehagais, A., (Text classification using the \(\sigma \)-FLNMAP neural network. Text classification using the \(\sigma \)-FLNMAP neural network, Proceedings of the international joint conference on neural networks, Vol. 2 (2001), IEEE-INNS-ENNS: IEEE-INNS-ENNS Washington, DC), 1362-1367 |
| [20] | Quinlan, J. R., C4.5: Programs for machine learning (1993), Morgan Kaufmann: Morgan Kaufmann San Mateo, California |
| [21] | Rumelhart, D. E.; Hinton, G. E.; Williams, R. J., Learning internal representations by error propagation, (Parallel distributed processing: Explorations in the microstructure of cognition: Vol. 1: Foundations (1986)), 318-362 |
| [22] | Shafer, J. C.; Agrawal, R.; Mehta, M., VLDB, SPRINT: A scalable parallel classifier for data mining, (Vijayaraman, T. M.; Buchmann, A. P.; Mohan, C.; Sarda, N. L., Proc. 22nd int. conf. very large databases (1996), Morgan Kaufmann: Morgan Kaufmann Bombay, India), 544-555 |
| [23] | Simpson, P. K., Fuzzy Min-Max neural networks-Part 1: Classification, IEEE Transactions on Neural Networks, 3, 5, 776-786 (1992) |
| [24] | Taghi, M.; Baghmisheh, V.; Pavesic, N., A fast simplified Fuzzy ARTMAP network, Neural Processing Letters, 17, 273-316 (2003) |
| [25] | Torresen, J., Nakashima, H., Tomita, S., & Landsverk, O. (1995). General mapping of feed-forward neural networks onto an MIMD computer. In Proceedings of the IEEE international conference on neural networks; Torresen, J., Nakashima, H., Tomita, S., & Landsverk, O. (1995). General mapping of feed-forward neural networks onto an MIMD computer. In Proceedings of the IEEE international conference on neural networks |
| [26] | Torresen, J.; Tomita, S., (Sundararajan, N.; Saratchandran, P., Parallel architectures for artificial neural networks: Paradigms and implementations (1998), IEEE Computer Society Press and John Wiley Sons), 41-118 |
| [27] | University of California, Irvine (2003). UCI machine learning repository. http://www.icf.uci.edu/mlearn/MLRepository.html; University of California, Irvine (2003). UCI machine learning repository. http://www.icf.uci.edu/mlearn/MLRepository.html |
| [28] | Williamson, J. R., Gaussian ARTMAP: A neural network for fast incremental learning of noisy multidimensional maps, Neural Networks, 9, 5, 881-897 (1996) |
| [29] | Zhang, D., Parallel VLSI neural systems design (1998), Springer · Zbl 0928.68089 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
