Toward unified hybrid simulation techniques for spiking neural networks. (English) Zbl 1416.92009
Summary: In the field of neural network simulation techniques, the common conception is that spiking neural network simulators can be divided in two categories: time-step-based and event-driven methods. In this letter, we look at state-of-the art simulation techniques in both categories and show that a clear distinction between both methods is increasingly difficult to define. In an attempt to improve the weak points of each simulation method, ideas of the alternative method are, sometimes unknowingly, incorporated in the simulation engine. Clearly the ideal simulation method is a mix of both methods. We formulate the key properties of such an efficient and generally applicable hybrid approach.
MSC:
| 92B20 | Neural networks for/in biological studies, artificial life and related topics |
References:
| [1] | Abbott, L., & Nelson, S. (2000). Synaptic plasticity: Taming the beast. Nature Neuroscience, 3, 1178-1183. , |
| [2] | Baker, W., Mahmood, A., & Carlson, B. (1996). Parallel event-driven logic simulation algorithms: Tutorial and comparative evaluation. In IEE Proc.-Circuits Devices Syst. (Vol., pp. 177-185). IEE. |
| [3] | Brette, R. (2006). Exact simulation of integrate-and-fire models with synaptic conductances. Neural Computation, 18(8), 2004-2027. , · Zbl 1121.92012 |
| [4] | Brette, R., Rudolph, M., Carnevale, T., Hines, M., Beeman, D., Bower, J. M., …Harris, F., Jr. (2007). Simulation of networks of spiking neurons: A review of tools and strategies. Journal of Computational Neuroscience, 23, 349-398. , |
| [5] | Brown, R. (1988). Calendar queues: A fast O(1) priority queue implementation for the simulation event set problem. Communications of the ACM, 31, 1220-1227. , |
| [6] | Carnevale, N., & Hines, M. (2002). Efficient discrete event simulation of spiking neurons in NEURON. SfN abstract 312.10. 2002 SFN Meeting, Yale University, New Haven, CT. |
| [7] | Carrillo, R., Ros, E., Ortigosa, E., Barbour, B., & Agis, R. (2005). Lookup table powered neural event-driven simulator. In Proceedings of the 8th International Work-Conference on Artificial Neural Networks (pp. 168-175). New York: Springer. , |
| [8] | Cessac, B., & Viéville, T. (2008). On dynamics of integrate-and-fire neural networks with conductance based synapses. Frontiers in Computational Neuroscience, 2(2), 1-20. , |
| [9] | Chandy, K., & Misra, J. (1979). Distributed simulation: A case study in design and verification of distributed programs. IEEE Transactions on Software Engineering, 5(5), 440-452. , · Zbl 0416.68014 |
| [10] | Chatterjee, D., Deorio, A., & Bertacco, V. (2011). Gate-level simulation with GPU computing. ACM Trans. Des. Autom. Electron. Syst., 16, 30:1-26. , |
| [11] | Delorme, A. (2001). SpikeNET: A simulator for modeling large networks of integrate and fire neurons. Neurocomputing, 38, 539-545. , |
| [12] | de Ruyter van Steveninck, R., & Laughlin, S. (1996). The rate of information transfer at graded-potential synapses. Nature, 379, 642-645. , |
| [13] | Destexhe, A., Rudolph, M., & Paré, D. (2003). The high-conductance state of neocortical neurons in vivo. Neuroscience, 4, 739-751. |
| [14] | D’Haene, M. & Schrauwen, B. (2010). Fast and exact simulation methods applied on a broad range of neuron models. Neural Computation, 22(6), 1468-1472. , · Zbl 1193.92015 |
| [15] | D’Haene, M., Schrauwen, B., Van Campenhout, J., & Stroobandt, D. (2009). Accelerating event-driven simulation of spiking neurons with multiple synaptic time constants. Neural Computation, 21(4), 1068-1099. , · Zbl 1158.92006 |
| [16] | Dolan, R., & DeSouza, G. (2009). GPU-based simulation of cellular neural networks for image processing. In Proceedings of the 2009 International Joint Conference on Neural Networks. Piscataway, NJ: IEEE Press. , |
| [17] | Fujimoto, R. (1990). Parallel discrete event simulation. Communications of the ACM, 33(10), 30-53. , |
| [18] | Goodman, D., & Brette, R. (2008). Brian: A simulator for spiking neural networks in Python. Frontiers in Neuroinformatics, 2(5). |
| [19] | Govindaraju, N., Larsen, S., Gray, J., & Manocha, D. (2006). A memory model for scientific algorithms on graphics processors. In Proceedings of the 2006 ACM/IEEE Conference on Supercomputing. Piscataway, NJ: IEEE Press. , |
| [20] | Hansel, D., Mato, G., Meunier, C., & Neltner, L. (1998). On numerical simulations of integrate-and-fire neural networks. Neural Computation, 10, 467-483. , |
| [21] | Hanuschkin, A., Kunkel, S., Helias, M., Morrison, A., & Diesmann, M. (2010). A general and efficient method for incorporating precise spike times in globally time-driven simulations. Frontiers in Neuroinformatics, 4. |
| [22] | Henker, S., Partzsch, J., & Schüffny, R. (2012). Accuracy evaluation of numerical methods used in state-of-the-art simulators for spiking neural networks. Journal of Computational Neuroscience, 32(2), 309-326. , · Zbl 1446.92052 |
| [23] | Hines, M., & Carnevale, N. (2001). Neuron: A tool for neuroscientists. Neuroscientist, 7, 123-135. , |
| [24] | Huang, S., Xiao, S., & Feng, W. (2009). On the energy efficiency of graphics processing units for scientific computing. In Proceedings of the 2009 IEEE International Symposium on Parallel & Distributed Processing (pp. 1-8), Washington, DC: IEEE Computer Society. , |
| [25] | Jefferson, D., & Sowizral, H. (1985). Fast concurrent simulation using the time warp mechanism. In SCS Conf. Distributed Simulation (pp. 63-69). Santa Monica, CA: RAND. |
| [26] | Kaabi, M. G., Tonnelier, A., & Martinez, D. (2011). On the performance of voltage stepping for the simulation of adaptive, nonlinear integrate-and-fire neuronal networks. Neural Computation, 23(5), 1187-1204. , · Zbl 1216.68329 |
| [27] | Kirst, C., & Timme, M. (2009). How precise is the timing of action potentials?Frontiers in Neuroscience, 3(1), 2-3. , |
| [28] | Lee, G., & Farhat, N. (2001). The double queue method: A numerical method for integrate-and-fire neuron networks. Neural Networks, 14, 921-932. , |
| [29] | Lipton, R., & Mizell, D. (1990). Time warp vs. Chandy-Misra: A worst-case comparison. In D. Nicol (Ed.), Distributed simulation (pp. 137-143). Marina del Rey, CA: Information Sciences Institute. |
| [30] | Lobb, C., Chao, Z., Fujimoto, R., & Potter, S. (2005). Parallel event-driven neural network simulations using the Hodgkin-Huxley neuron model. In Proceedings of the 19th Workshop on Principles of Advanced and Distributed Simulation (pp. 16-25). Piscataway, NJ: IEEE Press. , |
| [31] | Lytton, W., & Hines, M. (2005). Independent variable time-step integration of individual neurons for network simulations. Neural Computation, 17(4), 903-921. , |
| [32] | Maass, W. (1996). Lower bounds for the computational power of networks of spiking neurons. Neural Computation, 8(1), 1-40. , · Zbl 0843.68104 |
| [33] | Maass, W., & Markram, H. (2002). Synapses as dynamic memory buffers. Neural Networks, 15, 155-161. , |
| [34] | Mainen, Z., & Sejnowski, T. (1995). Reliability of spike timing in neocortical neuron. Science, 268(5216), 1503-1506. , |
| [35] | Marian, I., Reilly, R., & Mackey, D. (2002). Efficient event-driven simulation of spiking neural networks. In Proceedings of the 3rd WSEAS International Conference on Neural Networks and Applications. Cambridge, MA: MIT Press. |
| [36] | Markram, H. (2006). The blue brain project. Nature Reviews Neuroscience, 7, 153-160. , |
| [37] | Mattia, M., & Del Giudice, P. (2000). Efficient event-driven simulation of large networks of spiking neurons and dynamical synapses. Neural Computation, 12, 2305-2329. , |
| [38] | Mihalaş, Ş., & Niebur, E. (2009). A generalized linear integrate-and-fire neural model produces diverse spiking behaviors. Neural Computation, 21, 704-718. , · Zbl 1156.92008 |
| [39] | Morrison, A., Mehring, C., Geisel, T., Aertsen, A., & Diesmann, M. (2005). Advancing the boundaries of high-connectivity network simulation with distributed computing. Neural Computation, 17(8), 1776-1801. , · Zbl 1112.68494 |
| [40] | Morrison, A., Straube, S., Plesser, H., & Diesmann, M. (2007). Exact subthreshold integration with continuous spike times in discrete-time neural network simulations. Neural Computation, 19(1), 47-79. , · Zbl 1116.92016 |
| [41] | Mouraud, A., Puzenat, D., & Paugam-Moisy, H. (2006). DAMNED: A distributed and multithreaded neural event driven simulation framework. In IASTED-PDCN, International Conference on Parallel and Distributed Computing and Networks (pp. 212-217). Calgary: ACTA Press. |
| [42] | Mutch, J., Knoblich, U., & Poggio, T. (2010). CNS: A GPU-based framework for simulating cortically-organized networks (Tech. Rep.). Cambridge, MA: MIT. |
| [43] | Nageswaran, J. M., Felch, A., Chandrasekhar, A., Dutt, N., Granger, R., Nicolau, A., & Veidenbaum, A. V. (2009). Brain derived vision algorithm on high performance architectures. International Journal of Parallel Programming, 37(4), 345-369. , · Zbl 1192.68012 |
| [44] | Nasse, F., Thurau, C., & Fink, G. (2009). Face detection using GPU-based convolutional neural networks. In Lecture Notes in Computer Science (pp. 83-90). Berlin: Springer. , |
| [45] | Pecevski, D., Natschläger, T., & Schuch, K. (2009). PCSIM: A parallel simulation environment for neural circuits fully integrated with Python. Front. Neuroinformatics, 3. |
| [46] | Press, W., Teukolsky, S., Vetterling, W., & Flannery, B. (2009). Integration of ordinary differential equations. In L. Cowles & A. Harvey (Eds.), Numerical recipes in C (pp. 899-955). Cambridge: Cambridge University Press. |
| [47] | Raina, R., Madhavan, A., & Ng, A. (2009). Large-scale deep unsupervised learning using graphics processors. In Proceedings of the 26th Annual International Conference on Machine Learning (pp. 873-880). New York: ACM. , |
| [48] | Rangan, A., & Cai, D. (2007). Fast numerical methods for simulating large-scale integrate-and-fire neuronal networks. Journal of Computational Neuroscience, 22, 81-100. , |
| [49] | Reich, D., Mechler, F., & Victor, J. (2001). Formal and attribute-specific information in primary visual cortex. Journal of Neurophysiology, 85, 305-318. |
| [50] | Reynolds, P. (1988). A spectrum of options for parallel simulation. In Proceedings of the 1988 Winter Simulation Conference. |
| [51] | Richert, M., Nageswaran, J. M., Dutt, N., & Krichmar, J. L. (2011). An efficient simulation environment for modeling large-scale cortical processing. Frontiers in Neuroinformatics, 5. |
| [52] | Rochel, O. (2004). Une approche événementielle pour la modélisation et la simulation de réseaux de neurones impulsionnels. Doctoral dissertation, Université Henri Poincaré, Nancy 1. |
| [53] | Ros, E., Carillo, R., Ortigosa, E., Barbour, B., & Agís, R. (2006). Event-driven simulation scheme for spiking neural networks using lookup tables to characterize neuronal dynamics. Neural Computation, 18(12), 2959-2993. , · Zbl 1107.92012 |
| [54] | Rudolph, M., & Destexhe, A. (2006). Analytical integrate-and-fire neuron models with conductance-based dynamics for event-driven simulation strategies. Neural Computation, 18(9), 2146-2210. , · Zbl 1099.68698 |
| [55] | Rudolph-Lilith, M., Dubois, M., & Destexhe, A. (2012). Analytical integrate-and-fire neuron models with conductance-based dynamics and realistic postsynaptic potential time course for event-driven simulation strategies. Neural Computation, 24(6), 1426-1461. , |
| [56] | Sedgewick, R. (1992). Algorithms in C++. Reading, MA: Addison-Wesley. · Zbl 0826.68025 |
| [57] | Shelley, M., & Tao, L. (2001). Efficient and accurate time-stepping schemes for integrate-and-fire neuronal networks. Journal of Computational Neuroscience, 11, 111-119. , |
| [58] | Shmoys, D. S. (1995). Cut Problems and their application to divide and conquer. In D. S. Hochba (Ed.), Approximation algorithms for NP-hard problems. Boston: PWS Publishing. |
| [59] | Softky, W. (1994). Sub-millisecond coincidence detection in active dentritic trees. Neuroscience, 58, 13-41. , |
| [60] | Softky, W. R. (1995). Simple codes versus efficient codes. Current Opinion in Neurobiology, 5, 239-247. , |
| [61] | Tang, W., Goh, R., & Thng, I. (2005). Ladder queue: An O(1) priority queue structure for large-scale discrete event simulation. ACM Transactions on Modeling and Computer Simulation, 15(3), 175-204. , · Zbl 1390.90260 |
| [62] | Tonnelier, A., Belmabrouk, H., & Martinez, D. (2007). Event-driven simulations of nonlinear integrate-and-fire neurons. Neural Computation, 19(12), 3226-3238. , · Zbl 1144.68049 |
| [63] | van Elburg, R. A. J., & van Ooyen, A. (2009). Generalization of the event-based Carnevale-Hines integration scheme for integrate-and-fire models. Neural Computation, 21, 1913-1930. , · Zbl 1172.92006 |
| [64] | Wang, B., Zhu, Y., & Deng, Y. (2010). Distributed time, conservative parallel logic simulation on GPUs. In Proceedings of the 47th Design Automation Conference (pp. 761-766). New York: ACM. , |
| [65] | Watts, L. (1993). Event-driven simulation of networks of spiking neurons. In Sixth Neural Information Processing Systems Conference (pp. 927-934). San Francisco: Morgan Kaufmann. |
| [66] | Zheng, G., Tonnelier, A., & Martinez, D. (2009). Voltage-stepping schemes for the simulation of spiking neural networks. Journal of Computational Neuroscience, 26(3), 409-423. , |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
