Woong Choi
ABSTRACT
Binarized neural network (BNN) is one of the most promising solution for low-cost convolutional neural network acceleration. Since BNN is based on binarized bit-level operations, there exist great opportunities to reduce power-hungry data transfers and complex arithmetic operations. In this paper, we propose a content addressable memory (CAM) based BNN accelerator. By using time-domain signal processing, the huge convolution operations of BNN can be effectively replaced to the CAM search operation. In addition, thanks to fully parallel search of CAM, the parallel convolution operations for non-overlapped filtering window is enabled for high throughput data processing. To verify the effectiveness of the proposed CAM based BNN accelerator, the convolutional layer of LeNet-5 model has been implemented using 65nm CMOS technology. The implementation results show that the proposed BNN accelerator achieves 9.4% and 38.5% of area and energy savings, respectively. The parallel convolution operation of the proposed approach also shows 2.4x improved processing time.
- Y. H. Chen et al. 2017. Eyeriss: An Energy-Efficient Reconfigurable Accelerator for Deep Convolutional Neural Networks. IEEE JSSC, 52, 1, (2017), 127--138.Google Scholar
Cross Ref
- S. Han et al. 2016. EIE: Efficient Inference Engine on Compressed Deep Neural Network. (2016). arXiv:1602.01528. Google ScholarDigital Library
- M. Courbariaux et al. 2016. Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1. (2016). arXiv:1602.02830.Google Scholar
- M. Rastegari et al. 2016. XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks. In 2016 Computer Vision, 525-542.Google Scholar
- S. Zhou et al. 2016. DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients. (2016). arXiv:1606.06160.Google Scholar
- W. Tang et al. 2017. Wang, "How to Train a Compact Binary Neural Network with High Accuracy?. in 2017 AAAI, 2625--2631.Google Scholar
- D. Miyashita et al. 2017. A Neuromorphic Chip Optimized for Deep Learning and CMOS Technology With Time-Domain Analog and Digital Mixed-Signal Processing," IEEE JSSC. 52, 10, (2017), 2679--2689.Google ScholarCross Ref
- K. Pagiamtzis et al. 2006. Content-Addressable Memory (CAM) Circuits and Architectures: A Tutorial and Survey. IEEE JSSC, (2006). 41, 3, 712--727.Google ScholarCross Ref
- S. Ioffe et al. 2015. Batch Normalization: Accelerating Deep Network Training By Reducing Internal Covariate Shift. (2015). arXiv:1502.03167Google ScholarDigital Library
- H. Yonekawa et al. 2017. On-Chip Memory Based Binarized Convolutional Deep Neural Network Applying Batch Normalization Free Technique on an FPGA. in 2017 IPDPSW, 98--105.Google Scholar
- A. M. Abas et al. 2002. Time Difference Amplifier. Electron. Lett. 38, 23, (2002). 1437--1438.Google ScholarCross Ref
Recommendations
Content Addressable Memory Based Binarized Neural Network Accelerator Using Time-Domain Signal Processing
2018 55th ACM/ESDA/IEEE Design Automation Conference (DAC)Binarized neural network (BNN) is one of the most promising solution for low-cost convolutional neural network acceleration. Since BNN is based on binarized bit-level operations, there exist great opportunities to reduce power-hungry data transfers and ...
A convolutional result sharing approach for binarized neural network inference
DATE '20: Proceedings of the 23rd Conference on Design, Automation and Test in EuropeThe binary-weight-binary-input binarized neural network (BNN) allows a much more efficient way to implement convolutional neural networks (CNNs) on mobile platforms. During inference, the multiply-accumulate operations in BNNs can be reduced to XNOR-...
Towards Demystifying Adversarial Robustness of Binarized Neural Networks
Applied Cryptography and Network Security WorkshopsAbstractQuantized neural networks are proposed for reduced computation and memory costs. When quantized neural networks are designed for edge or terminal devices, they may be more vulnerable to adversarial perturbations. We focus on the extreme cases, ...
Comments