Syed Asad Alam
James Garland
David Gregg
Skip Abstract Section
Abstract
Logarithmic number systems (LNS) are used to represent real numbers in many applications using a constant base raised to a fixed-point exponent making its distribution exponential. This greatly simplifies hardware multiply, divide, and square root. LNS with base-2 is most common, but in this article, we show that for low-precision LNS the choice of base has a significant impact.
We make four main contributions. First, LNS is not closed under addition and subtraction, so the result is approximate. We show that choosing a suitable base can manipulate the distribution to reduce the average error. Second, we show that low-precision LNS addition and subtraction can be implemented efficiently in logic rather than commonly used ROM lookup tables, the complexity of which can be reduced by an appropriate choice of base. A similar effect is shown where the result of arithmetic has greater precision than the input. Third, where input data from external sources is not expected to be in LNS, we can reduce the conversion error by selecting a LNS base to match the expected distribution of the input. Thus, there is no one base that gives the global optimum, and base selection is a trade-off between different factors. Fourth, we show that circuits realized in LNS require lower area and power consumption for short word lengths.
- S. A. Alam and D. Gregg. 2020. Beyond base-2 logarithmic number systems (WiP Paper). In Proceedings of the ACM SIGPLAN/SIGBED International Conference on Language, Compilers, and Tools for Embedded Systems. ACM.Google Scholar
- S. A. Alam and O. Gustafsson. 2014. Design of finite word length linear-phase FIR filters in the logarithmic number system domain. VLSI Des. 2014 (2014), 14.Google ScholarDigital Library
- M. Arnold, E. Chester, and C. Johnson. 2020. Training neural nets using only an approximate tableless LNS ALU. In Proceedings of the IEEE International Application-Specific System Architecture Processors Conference. 69–72.Google Scholar
- M. G. Arnold. 2002. Reduced power consumption for MPEG decoding with LNS. In Proceedings of the IEEE International Application-Specific System Architecture Processors Conference.65–67.Google ScholarDigital Library
- M. G. Arnold, T. A. Bailey, J. R. Cowles, and M. D. Winkel. 1998. Arithmetic co-transformations in the real and complex logarithmic number systems. IEEE Trans. Comput. 47, 7 (1998), 777–786.Google ScholarDigital Library
- C. Basetas, I. Kouretas, and V. Paliouras. 2007. Low-power digital filtering based on the logarithmic number system. In Proceedings of the International Workshop on Power Timing Modeling Optimization Simulation. Vol. 4644. 546–555.Google Scholar
- BLIF 1992. Berkeley logic interchange format (BLIF). University of California, Berkeley. Retrieved from http://www.cs.columbia.edu/ cs6861/sis/blif/index.html.Google Scholar
- N. Bruschi, A. Garofalo, F. Conti, G. Tagliavini, and D. Rossi. 2020. Enabling mixed-precision quantized neural networks in extreme-edge devices. In Proceedings of the ACM International Conference on Computing Frontiers. 217–220.Google Scholar
- A. Capotondi, M. Rusci, M. Fariselli, and L. Benini. 2020. CMix-NN: Mixed low-precision CNN library for memory-constrained edge devices. IEEE Trans. Circ. Syst. II: Express Briefs 67, 5 (2020), 871–875.Google ScholarCross Ref
- D. V. Satish Chandra. 1998. Error analysis of FIR filters implemented using logarithmic arithmetic. IEEE Trans. Circ. Syst. II 45, 6 (June 1998), 744–747.Google Scholar
- D. V. S. Chandra. 1998. Error analysis of FIR filters implemented using logarithmic arithmetic. IEEE Trans. Circ. Syst. II 45, 6 (1998), 744–747.Google Scholar
- J. Chen and X. Ran. 2019. Deep learning with edge computing: A review. Proc. IEEE 107 (2019), 1655–1674. Issue 8.Google ScholarCross Ref
- Wai-Kai Chen (Ed.). 2003. Memory, Microprocessor, and ASIC. CRC Press, Boca Raton, FL.Google Scholar
- E. Chung and J. Fowers et al.2018. Serving DNNs in real time at datacenter scale with project brainwave. IEEE Micro 38, 2 (Mar. 2018), 8–20.Google ScholarCross Ref
- F. de Dinechin and A. Tisserand. 2001. Some improvements on multipartite table methods. In Proceedings of the IEEE Symposium on Computer Arithmetic. 128–135.Google Scholar
- V. S. Dimitrov, G. A. Jullien, and W. C. Miller. 1999. Theory and applications of the double-base number system. IEEE Trans. Comput. 48, 10 (1999), 1098–1106. DOI:https://doi.org/10.1109/12.805158Google ScholarDigital Library
- Z. G. Feng and K. F. C. Yiu. 2010. A new formulation for the design of FIR filters with reduced hardware implementation complexity. In Proceedings of the IEEE International Conference on Green Circuits Systems.242–246.Google ScholarCross Ref
- A. Garofalo, M. Rusci, F. Conti, D. Rossi, and L. Benini. 2019. PULP-NN: accelerating quantized neural networks on parallel ultra-low-power RISC-V processors. Philos. Trans. Royal Soc. A 378, 2164 (Dec. 2019). DOI:https://doi.org/10.1098/rsta.2019.0155Google Scholar
- I. Hubara, M. Courbariaux, D. Soudry, R. El-Yaniv, and Y. Bengio. 2017. Quantized neural networks: Training neural networks with low-precision weights and activations. ACM J. Mach. Learn. Res. 18, 1 (Jan. 2017), 6869–6898.Google Scholar
- K. Johansson, O. Gustafsson, and L. Wanhammar. 2008. Implementation of elementary functions for logarithmic number systems. IET Comput. Digit. Tech. 2, 4 (June 2008), 295–304.Google ScholarCross Ref
- N. G. Kingsbury and P. J. W. Rayner. 1971. Digital filtering using logarithmic arithmetic. Electron. Lett. 7, 2 (Jan. 1971), 56–58.Google Scholar
- I. Kouretas, Ch. Basetas, and V. Paliouras. 2013. Low-power logarithmic number system addition/subtraction and their impact on digital filters. IEEE Trans. Comput. 62, 11 (2013), 2196–2209.Google ScholarDigital Library
- I. Kouretas and V. Paliouras. 2018. Logarithmic number system for deep learning. In Proceedings of the International Conference on Modern Circuits and Systems Technologies. 1–4.Google Scholar
- V. B. Lawrence and A. C. Salazar. 1980. Finite precision design of linear-phase FIR filters. Bell Syst. Tech. J. 59, 9 (Nov. 1980), 1575–1598. DOI:https://doi.org/10.1002/j.1538-7305.1980.tb03051.xGoogle ScholarCross Ref
- E. H. Lee, D. Miyashita, E. Chai, B. Murmann, and S. S. Wong. 2017. LogNet: Energy-efficient neural networks using logarithmic computation. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing.5900–5904.Google Scholar
- A. M. Mansour, A. M. El-Sawy, M. S. Aziz, and A. T. Sayed. 2015. A new hardware implementation of base 2 logarithm for FPGA. Int. J. Signal Proc. Syst. 3, 2 (2015), 171–181.Google Scholar
- Alan Mishchenko. 2005. ABC: A System for Sequential Synthesis and Verification. Retrieved from https://people.eecs.berkeley.edu/ alanmi/abc/.Google Scholar
- J. N. Mitchell. 1962. Computer multiplication and division using binary logarithms. IRE Trans. Electron. Comp. EC-11, 4 (1962), 512–517.Google ScholarCross Ref
- Sanjit K. Mitra. 2006. Digital Signal Processing. TATA McGraw-Hill, University of California, Santa Barbara, CA.Google Scholar
- D. Miyashita, E. H. Lee, and B. Murmann. 2016. Convolutional neural networks using logarithmic data representation. Retrieved from https://arXiv:cs.NE/1603.01025v2.Google Scholar
- B. Nam, H. Kim, and H. Yoo. 2008. Power and area-efficient unified computation of vector and elementary functions for handheld 3D graphics systems. IEEE Trans. Comput. 57, 4 (2008), 490–504.Google ScholarDigital Library
- V. Paliouras and T. Stouraitis. 2000. Logarithmic number system for low-power arithmetic. In Proceedings of the International Workshop on Power Timing Modeling Optimization Simulation, Vol. 1918. 285–294.Google Scholar
- M. Rastegari, V. Ordonez, J. Redmon, and A. Farhadi. 2016. XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks. Retrieved from https://arXiv:cs.CV/1603.05279v4.Google Scholar
- M. Satyanarayanan. 2017. The emergence of edge computing. Computer 50, 1 (Jan. 2017), 30–39.Google ScholarCross Ref
- A. S. Sedra, K. C. Smith, T. C. Carusone, and V. Gaudet. 2019. Microelectronic Circuits (8th ed.). Oxford University Press.Google Scholar
- W. Shi, J. Cao, Q. Zhang, Y. Li, and L. Xu. 2016. Edge computing: Vision and challenges. IEEE Internet Things J. 3 (2016), 637–646. Issue 5. DOI:https://doi.org/10.1109/JIOT.2016.2579198Google ScholarCross Ref
- G. Sicuranza. 1982. On the accuracy of 2-D digital filter realizations using logarithmic number systems. In Proc. IEEE Int. Conf. Acoust. Speech Signal Process., Vol. 7. 48–51. DOI:https://doi.org/10.1109/ICASSP.1982.1171386Google ScholarCross Ref
- T. Stouraitis and V. Paliouras. 2002. Considering the alternatives in low-power design. IEEE Circ. Syst. Mag. 17, 4 (Aug. 2002), 22–29.Google Scholar
- X. Sun, J. Choi, C. Y Chen, N. Wang, S. Venkataramani, V. Srinivasan, X. Cui, W. Zhang, and K. Gopalakrishnan. 2019. Hybrid 8-bit floating point (HFP8) training and inference for deep neural networks. In Proceedings of the Conference on Advances in Neural Information Processing Systems.4901–4910.Google Scholar
- E.E. Swartzlander Jr. and A.G. Alexopoulos. 1975. The sign/logarithm number system. IEEE Trans. Comput. C-24, 12 (Dec. 1975), 1238–1242.Google ScholarDigital Library
- J. N. Coleman et al.2008. The european logarithmic microprocesor. IEEE Trans. Comput. 57, 4 (Apr. 2008), 532–546.Google ScholarDigital Library
- Sebastian Vogel, Mengyu Liang, Andre Guntoro, Walter Stechele, and Gerd Ascheid. 2018. Efficient hardware acceleration of CNNs using logarithmic data representation with arbitrary log-base. In Proceedings of the IEEE/ACM Conference on Computer-Aided Design (ICCAD’18). ACM, New York, NY, Article 9, 8 pages.Google ScholarDigital Library
- N. Wang, J. Choi, D. Brand, C.-Y. Chen, and K. Gopalakrishnan. 2018. Training deep neural networks with 8-bit floating point numbers. In Proceedings of the Conference on Advances in Neural Information Processing Systems.7675–7684.Google Scholar
- Y. Wang, J. Lin, and Z. Wang. 2018. An energy-efficient architecture for binary weight convolutional neural networks. IEEE Trans. VLSI Syst. 26, 2 (Feb. 2018), 280–293.Google ScholarDigital Library
- B-.D. Yang and L.-S. Kim. 2003. A low-power charge-recycling ROM architecture. IEEE Trans. VLSI Syst. 4, 4 (2003), 590–600.Google ScholarDigital Library
- M. Zhu, Y. Ha, C. Gu, and L. Gao. 2016. An optimized logarithmic converter with equal distribution of relative errors. IEEE Trans. Circ. Syst. II: Express Briefs 63, 9 (2016), 848–852.Google Scholar
Index Terms
- Low-precision Logarithmic Number Systems: Beyond Base-2
Recommendations
Beyond Base-2 Logarithmic Number Systems (WiP Paper)
LCTES '20: The 21st ACM SIGPLAN/SIGBED Conference on Languages, Compilers, and Tools for Embedded SystemsLogarithmic number systems (LNS) reduce hardware complexity for multiplication and division in embedded systems, at the cost of more complicated addition and subtraction. Existing LNS typically use base-2, meaning that representable numbers are some (...
A complex-number multiplier using radix-4 digits
ARITH '95: Proceedings of the 12th Symposium on Computer ArithmeticThis paper describes the design of a 16/spl times/16 complex-number multiplier developed as part of the arithmetic datapath of a complex-number digital signal processor. The complex-number multiplier internally uses binary signed digits for fast ...
A 20 Bit Logarithmic Number System Processor
The architecture and performance of a 20-bit arithmetic processor based on the logarithmic number system (LNS) is described. The processor performed LNS multiplication and division rapidly and with a low hardware complexity. Addition and subtraction in ...
Comments