Gradient Scaling on Deep Spiking Neural Networks with Spike-Dependent Local Information
Authors:
Seongsik Park,
Jeonghee Jo,
Jongkil Park,
Yeonjoo Jeong,
Jaewook Kim,
Suyoun Lee,
Joon Young Kwak,
Inho Kim,
Jong-Keuk Park,
Kyeong Seok Lee,
Gye Weon Hwang,
Hyun Jae Jang
Abstract:
Deep spiking neural networks (SNNs) are promising neural networks for their model capacity from deep neural network architecture and energy efficiency from SNNs' operations. To train deep SNNs, recently, spatio-temporal backpropagation (STBP) with surrogate gradient was proposed. Although deep SNNs have been successfully trained with STBP, they cannot fully utilize spike information. In this work,…
▽ More
Deep spiking neural networks (SNNs) are promising neural networks for their model capacity from deep neural network architecture and energy efficiency from SNNs' operations. To train deep SNNs, recently, spatio-temporal backpropagation (STBP) with surrogate gradient was proposed. Although deep SNNs have been successfully trained with STBP, they cannot fully utilize spike information. In this work, we proposed gradient scaling with local spike information, which is the relation between pre- and post-synaptic spikes. Considering the causality between spikes, we could enhance the training performance of deep SNNs. According to our experiments, we could achieve higher accuracy with lower spikes by adopting the gradient scaling on image classification tasks, such as CIFAR10 and CIFAR100.
△ Less
Submitted 1 August, 2023;
originally announced August 2023.
Frequency-tunable nano-oscillator based on Ovonic Threshold Switch (OTS)
Authors:
Seon Jeong Kim,
Seong Won Cho,
Hyejin Lee,
Jaesang Lee,
Tae Yeon Seong,
Inho Kim,
Jong-Keuk Park,
Joon Young Kwak,
Jaewook Kim,
Jongkil Park,
YeonJoo Jeong,
Gyu Weon Hwang,
Kyeong Seok Lee,
Suyoun Lee
Abstract:
Nano-oscillator devices are gaining more and more attention as a prerequisite for developing novel energy-efficient computing systems based on coupled oscillators. Here, we introduce a highly scalable, frequency-tunable nano-oscillator consisting of one Ovonic threshold switch (OTS) and a field-effect transistor (FET). It is presented that the proposed device shows an oscillating behavior with a n…
▽ More
Nano-oscillator devices are gaining more and more attention as a prerequisite for developing novel energy-efficient computing systems based on coupled oscillators. Here, we introduce a highly scalable, frequency-tunable nano-oscillator consisting of one Ovonic threshold switch (OTS) and a field-effect transistor (FET). It is presented that the proposed device shows an oscillating behavior with a natural frequency (f_{nat}) adjustable from 0.5 to 2 MHz depending on the gate voltage applied to the FET. In addition, under a small periodic input, it is observed that the oscillating frequency (f_{osc}) of the device is locked to the frequency (f_{in}) of the input when f_{in} ~ f_{nat}, demonstrating the so-called synchronization phenomenon. It also shows the phase lock of the combined oscillator network using circuit simulation, where the phase relation between the oscillators can be controlled by the coupling strength. These results imply that the proposed device is promising for applications in oscillator-based computing systems.
△ Less
Submitted 28 September, 2020;
originally announced September 2020.
