Leaky Integrate and Fire Neuron by Charge-Discharge Dynamics in Floating-Body MOSFET

Abstract

Neuro-biology inspired Spiking Neural Network (SNN) enables efficient learning and recognition tasks. To achieve a large scale network akin to biology, a power and area efficient electronic neuron is essential. Earlier, we had demonstrated an LIF neuron by a novel 4-terminal impact ionization based n+/p/n+ with an extended gate (gated-INPN) device by physics simulation. Excellent improvement in area and power compared to conventional analog circuit implementations was observed. In this paper, we propose and experimentally demonstrate a compact conventional 3-terminal partially depleted (PD) SOI- MOSFET (100 nm gate length) to replace the 4-terminal gated-INPN device. Impact ionization (II) induced floating body effect in SOI-MOSFET is used to capture LIF neuron behavior to demonstrate spiking frequency dependence on input. MHz operation enables attractive hardware acceleration compared to biology. Overall, conventional PD-SOI-CMOS technology enables very-large-scale-integration (VLSI) which is essential for biology scale (~10¹¹ neuron based) large neural networks.

Introduction

Spiking neural network (SNN) is an attempt to understand and mimic human brain functionalities – a key challenge of next generation computing. SNN demonstrates energy efficiency advantages over von-Neumann architecture for recognition and classification tasks¹. To construct SNN in hardware, an efficient analog to the biological neuron is essential. Primarily, Si CMOS technology is used for analog implementation of electronic neurons. The dynamic nature of neuronal cell has been successfully captured by analog circuits^2,3,4,5,6,7. Also, analog neuron circuits provide area and power benefit⁵ compared to the digital¹ implementation. But the high neuronal density (10¹¹ neurons in the human brain compared to 10⁹ transistors/chip) and connectivity (10⁴ neurons connected to each neuron compared to a typical fan out of 8 in CMOS) imposes two major constraints. First, individual components (e.g. neurons and synapses) must be highly area and power efficient. Second, the technology must be sufficiently matured to enable extreme integration of numerous (~10¹¹) neuron. Recently, our group has proposed a highly power and area efficient neuron on impact ionization based n+/p/n+ diode (I-NPN) device with an extended gate driven by a small reset circuit in a device simulations study⁸. Excellent area (60x) and power improvement (5x) is demonstrated compared to previously reported analog circuits⁸. Further, record low bias (sub-0.2 V) impact ionization in I-NPN is also experimentally demonstrated by our group⁹. Gate-I-NPN requires unconventional process integration that is challenging for experimental realization and 4 terminals that involve layout and interconnection challenges. Hence, the neuronal function still remains to be experimentally demonstrated. In this paper, we propose to replace the 4-terminal, novel gated I-NPN device with a conventional 3-terminal, highly manufacturable PD-SOI MOSFET. We experimentally demonstrate neuronal behavior based on the intrinsic charge dynamics of the device.

Background

A simplified step-wise picture of SNN algorithm¹⁰ is shown in Fig. 1. First, pre-synaptic neuronal driver D1 and D2 provide the input voltage spikes (where i^th spike occurs at time t = t _i ). Second, these input spikes are converted to a gently varying current signal proportional to the synaptic weight (w_j). Third, the current from synapses (w₁and w₂) is summed into the input of LIF neuron N3 by the network. Fourth, the LIF neuron (described in detail next) integrates the input current across a capacitor, which raises its potential. N3 resets immediately (i.e. loses stored charge) once the potential reaches/exceeds a threshold. Fifth, every time N3 reaches threshold, a driver neuron D3 produces a spike. The detail of this architecture is discussed in ref. 10. Among various neuronal models, the leaky integrate and fire (LIF) model can mimic the behavior of the biological neuron with minimum number of circuit element unlike other models^11,12,13.

Figure 1

(a) Biological neuronal network is related to (b) algorithmic SNN analog. (c) The related signal timing (d) with step-wise signal evolution is shown. The SNN algorithm requires input spikes at times t _i from j^th pre-synaptic neuron driver e.g. D1, D2 (step 1). The spikes are converted into currents at synapse which depend upon synaptic strengths w _j (step 2). The currents are summed by the networks (step 3) and input into LIF neuron that determines neuronal firing instants (step 4) while post-neuronal driver (D3) produces spikes at the firing instants.

LIF Neuron Model

Leaky integrate and fire (LIF) model represents neuron as a parallel combination of a “leaky” resistor (conductance, g _L ) and a capacitor (C) as shown in Fig. 2(a). A current source I(t) is used as synaptic current input to charge up the capacitor to produce a potential V(t).

Figure 2

(a) LIF neuron circuit model, (b) For input (I_dc < I_crit), V(t) never exceeds V_th- hence neuron never spikes. However, for I_dc ≥ I_crit, neuron will fire when V(t) ≥ V_th and immediately reset i.e. V(t) = E_L, (c) With higher input (e.g. I_dc ≥ I_crit), firing rate or the frequency increases like a biological neuron while for low input (I_dc < I_crit), frequency is zero. The output frequency (f_o) vs. input is the signature neuronal function to be mimicked artificially.

When potential exceeds threshold (V(t) ≥ V _th ), the capacitor discharges to a resting potential E _L using the voltage-controlled switch, like a biological neuron. Each time LIF neuron voltage exceeds threshold, a separate driver circuit (say, D3) issues a spike as mentioned in the last section. Thus, LIF model is governed by the following differential equation:

$$C\frac{dV}{dt}=-{g}_{L}(V(t)-{E}_{L})+I(t)$$

(1)

At low input current (I(t)), V(t) never exceeds threshold V _th - which produces no spikes (i.e. t _i  → ∞). For example, if a dc input (i.e. I(t) = I _dc ) does not exceed a critical current (I _crit  = g _L (V _th  − E _L )), V(t) will always be less than V _th (i.e. V(t) < V _th ), by simple steady state analysis of Eqn. 1. When I(t) is high (e.g. I _dc  > I _crit ), the charge up time to V _th reduces (Fig. 2b). This essentially increases the output frequency (f _o ) with increase in input I _dc (Fig. 2c).

Device Structure, Concept and Operation

In this work, a simple PD-SOI MOSFET (schematic shown in Fig. 3a) is used to demonstrate LIF neuron like behavior. As in conventional CMOS, voltage input and current output are used. This is converse of biology. However, we focus on producing an input (V _in (t)) vs. output frequency (f_o) mapping (Fig. 2c). Voltage to current conversion and vice versa is trivial and maybe done based on system implementation needs. Figure 3b shows the biasing scheme where the input bias is applied on source (S) i.e. V _SG (t) = V _in (t). The drain (D) bias i.e. V _DG (t) is used to select “integrate” vs. “reset” modes. Both V _SG and V _DG are referenced to a grounded gate (G). To explain the physics of operation, the output i.e. drain current (I _D (t)) is annotated at four time instants (i–iv). At instant (i), the device is under equilibrium (Fig. 3(c-i)). At instant (ii), a low.

Figure 3

(a) Device schematic with biased terminals, (b) Input biasing scheme (V_SG(t),V_DG(t)) with expected output (I_D(t)), (c-i) Equilibrium band diagram. (ii) Electron hole pair generation due to impact ionization (“integrate”) increases holes in well, (iii) Barrier lowering due to stored holes in the potential well. Also, the holes start to escape through the source junction (“leak”), (iv) Once threshold is reached (“fire”), removal of V_DG makes the holes escape through both the junctions bringing the barrier to its original position (“reset”). Thus, the charge dynamics enables the Leaky Integrate and Fire neuron in a PD-SOI n-MOSFET.

V _SG (~0.2 V) and a high V _DG (say 2.8 V to enable integrate) are applied to initiate impact ionization (II). The II generated electron (e) current (I _II−e ) escape through the drain while the hole (h) current (I _II−h ) flows into the channel potential well. Some fraction of hcurrent leaks through the source barrier (I _leak−h i.e. equivalent to leaky integrate function in LIF) (Fig. 3(c-ii)). The net h current (I _II−h  − I _leak−h ) builds up h-charge in the channel potential well over time (equivalent to integrate function in LIF). Increasing h-charge, electrostatically reduce the source e-injection barrier, allowing more electron injection for stronger II and set up a positive feedback (Fig. 3(c-iii)). The e-injection barrier reduction is coupled with h-well depth reduction which increases I _leak−h . Steady state is achieved when I _leak−h balances I _II−h preventing additional h-storage. However, before steady state is reached, the current exceeds pre-set threshold (I _th ) (equivalent to fire function in LIF). The I(t) ≥ I _th condition sets off a small “reset circuit” that removes drain bias (V _DG  = 0) disabling II (i.e. I _II−h  → 0)⁸. Hence, all the stored holes leak (I _leak−h ) through both the source and the drain junction (Fig. 3(c-iv)) and resets the neuron back to the initial condition. After reset is complete, the drain is set back to high (V _DG  = 2.8V) after a typical timescale (called “refractory period”) to re-initiate the LIF process (Fig. 3(c-ii)). Thus, the dc input V _in (t) produces cycles of leakage, integration, firing and resetting (LIF process) to produce firing frequency (f _o ) in the output drain current (I _D ).

Experimental Validation

The typical kink-effect is observed in the I _D  − V _D (Fig. 4(a)) to confirm impact ionization¹⁴. Next, transient measurement shows the integration function of the LIF neuron where the applied source bias (V _SG ) represents input V _in (t) (Fig. 4(b)). First, when only V _SG  = −0.25V is applied (V _DG  = 0), the device is still off i.e. current remains negligible. When V _DG  = 2.8V is applied, an instantaneous increase in current is observed akin to Fig. 3(c-ii). A slower rise in current follows, as impact ionization produces a build-up of h-charge (i.e. integration). This, in turn, reduces the e-injection barrier to increase current to reach a steady state akin to Fig. 3(c-iii). We observe that the rate of current rise increases with input i.e. V _SG . For lower V _SG , the device is unable to initiate impact ionization due to lack of e-current supply. At high V _SG , the rate of current rise increases and reaches steady state at a higher current (I _D ) level. To add the fire and reset, a current threshold I _th is set such that when I _D exceeds I _th , we set V _DG  = 0 manually. This can be automatically performed by an external circuit as explained earlier⁸. Figure 5(a) shows the reset effect where V _DG is set to zero if I ≥ I _th  = 500 μA manually. For Fig. 5(a-i), V _SG  = −0.24 V, threshold is achieved once in <1 μs duration followed by reset. For Fig. 5(a-ii–iv), V _SG is increased. The drain current rises faster (“integration”) to exceed threshold (i.e. I(t) > I _th ) followed by effective reset. Such cycles of integration and reset occurs naturally. We observe increasing frequency of reaching current threshold (fire) with increase in input V _SG . Figure 5(b) shows output frequency vs. input V _in  = V _SG akin to Fig. 2 (c). A threshold is observed such that |V _in | < 0.26 V produces no spikes while above threshold, a linear dependence of spiking frequency (f _o ) on V _in is observed. This is the signature of LIF neuron. Further, this device offers higher frequency (in the range of MHz) compared to biology (~1–10 Hz), which enables attractive hardware acceleration¹⁵.

Figure 4

(a) I_D − V_D curve for different V_G shows “kink effect” as a signature of impact ionization induced floating body effect, (b) Transient measurement showing Leaky, Integrate functions for different input bias.

Figure 5

(a) The LIF function for different input (V_SG) is shown with a V_DG based reset. At threshold (I_D ≥ I_th = 500 μA), V_DGis grounded to reset the neuron for 100 ns, then set back to V_DG = 2.8 V. The output current starts from the same initial point after each reset – essentially make each LIF cycle identical. As V_SG is increased from (i–iv) more frequent fire & reset is observed which is akin to faster “spiking” with higher input bias (b) Output frequency vs. input shows the occurrence of input threshold i.e. |V_th| = 0.26 V. If |V_in| < |V_th|, frequency is zero while for |V_in| ≥ |V_th|, f_o increases with input bias. Quantitatively, this device can provide 100000× higher frequency compared to a biological neuron, which typically fires at ~10 Hz.

Performance & Benchmarking (New Section)

To evaluate the area and power performance, we have implemented the neuron (i.e. SOI device with a reset circuit) to demonstrate spiking, evaluate energy per spike and estimate layout area (Supplementary Information 1). Further, we use 12 input and 3 output neurons based spiking neural network (SNN) for Fisher Iris classification to show state-of-the-art recognition (~95%) (Supplementary Information 2). Such an SNN algorithm has software-equivalent hardware implementation²¹. In Table 1, we benchmark SOI neuron with literature. First, the conventional current–driven CMOS analog implementations provide power benefit at the cost of large area consumption^{5, 16, 17}. Novel voltage-driven neurons have been proposed^18,19,20, though the materials used here are not standard CMOS compatible. Among several voltage-driven neurons, phase change memory (PCM) based neuron could be a power and area efficient counterpart to analog neurons. Integration of PCM material in an array has also been demonstrated²⁰. Our proposed PD-SOI technology does not require any new materials integration. SOI technology is highly mature for VLSI implementation of numerous neurons. Additionally, SOI neuron has its capability of “leaky” integration without any extra circuitry unlike other neurons. The leakiness is an essential feature of biological neuron – which adds a time-dependent memory due to ion channel dynamics²². This is realized at the device level in SOI neuron owing to its inherent charge dynamics (i.e. recombination of excess carriers leakage through source/drain region). Also SOI neuron requires ~260 mV for firing compared to these novel neurons, which require higher threshold (>1 V)^18,19,20. Our demonstration requires a smaller voltage swing (of the order of few hundred mV), which is closer to biological neuron (100 mV)²³. This voltage range can be further reduced by SOI device engineering. The energy per spike is calculated to be 35 pJ for SOI device including the external circuitry, which is comparable to the phase change neuron (30 pJ only at the device level). However, phase change neuron requires digital implementation (including a global clock).Global clocks are power inefficient compared to asynchronous implementation. In fact, analog asynchronous implementations maybe as high as 10× more energy efficient²⁴. Our implementation is asynchronous, which is energy efficient at the systems level and closer to biology.

Table 1 Benchmarking with state-of-the-art electronic neurons.

Conclusion

To summarize, a highly manufacturable Si based SOI-MOSFET is experimentally shown to demonstrate LIF neuron functionality. Intrinsic carrier dynamics of the device produces “Leak Integrate and Fire” functionality. This experimentally validated approach is noted for significant area and power efficiency compared to analog circuit implementation (Supplementary Information 1). By modeling the output characteristics of the SOI neuron, a MATLAB based spiking neuron network is shown to perform classification task with reasonable accuracy (~95%) (Supplementary Information 2).Also, this device offers higher frequency (in the order of MHz), than a biological neuron (~1–10Hz) to enable attractive hardware acceleration. CMOS based 32 nm SOI technology provides excellent maturity and very large scale integration (VLSI), which is essential for biology equivalent large scale spiking neural networks.

Method

The devices used in this study were fabricated using the 32 nm SOI High-k Metal Gate (HKMG) CMOS technology^{25, 26}. The gate dielectric stack is composed of 1.7 nm HfO₂ (ALD) and 0.8 nm interfacial SiO₂ layer which is chemically grown. Lanthanum is used as the capping layer between HfO₂ and TiN metal gate for V _T adjust. Excellent CMOS performance and manufacturability is demonstrated earlier²⁷. Figure 6 shows TEM image of the fabricated PD-SOI MOSFET at 32 nm technology node. The devices of 100 nm channel length and 1μm channel width are characterized in DC and transient modes by Keysight B1500 DC and Waveform Generation and Fast Measurement Unit (WGFMU) system. All the measurements are performed at room temperature.

Figure 6

TEM image of the fabricated PD-SOI MOSFET at 32 nm technology node with 1.7 nm HfO₂ gate oxide.