nach oben

Erschienen in:

2021 | OriginalPaper | Buchkapitel

Introducing Temporal Behavior to Computing Science

verfasst von : János Végh

Erschienen in: Advances in Software Engineering, Education, and e-Learning

Verlag: Springer International Publishing

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

The abstraction introduced by von Neumann correctly reflected the state of the art 70 years ago. Although it omitted data transmission time between components of the computer, it served as an excellent base for classic computing for decades. Modern computer components and architectures, however, require to consider their temporal behavior: data transmission time in contemporary systems may be higher than their processing time. Using the classic paradigm leaves a growing number of issues unexplained, from enormously high power consumption to days-long training of artificial neural networks to failures of some cutting-edge supercomputer projects. The paper introduces the up to now missing temporal behavior (a temporal logic) into computing, while keeps the solid computing science base. The careful analysis discovers that with considering the temporal behavior of components and architectural principles, the mystic issues have a trivial explanation. Some classic design principles must be revised, and the temporal logic enables us to design a more powerful and efficient computing.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Vorheriges Kapitel Formal Specification and Verification of Timing Behavior in Safety-Critical IoT Systems

Nächstes Kapitel Evaluation of Classical Data Structures in the Java Collections Framework

The explanations are quite different: Gyoukou was withdrawn after its first appearance; Aurora failed: retargeted and delayed; Despite the failure of SpiNNaker1, the SpiNNaker2 is also under construction [8]; “Chinese decision-makers decided to withhold the country’s newest Shuguang supercomputers even though they operate more than 50 percent faster than the best current US machines”.

https://www.scmp.com/tech/policy/article/3015997/china-has-decided-not-fan-flames-super-computing-rivalry-amid-us.

https://ec.europa.eu/newsroom/dae/document.cfm?doc_id=60156.

https://www.nextplatform.com/2019/10/30/cray-revamps-clusterstor-for-the-exascale-era/: artificial intelligence, …it’s the most disruptive workload from an I/O pattern perspective.

Reference [6] discusses this phenomenon in details.

It can be a crucial factor of inefficiency of general-purpose chips [18].

The reconfigurable computing, with its customized processors and non-processor-like processing units, does not change significantly the landscape.

The biology can change the conduction velocity, that needs energy, so finding an optimum is not as simple.

For examples see the architectures of supercomputers Taihulight and Summit.

K. Asanovic, R. Bodik, J. Demmel, T. Keaveny, K. Keutzer, J. Kubiatowicz, N. Morgan, D. Patterson, K. Sen, J. Wawrzynek, D. Wessel, K. Yelick, A view of the parallel computing landscape. Commun. ACM 52(10), 56–67 (2009)CrossRef

US National Research Council, The Future of Computing Performance: Game Over or Next Level? (2011). [Online]. Available: http://science.energy.gov/~/media/ascr/ascac/pdf/meetings/mar11/Yelick.pdf

I. Markov, Limits on fundamental limits to computation. Nature 512(7513), 147–154 (2014)CrossRef

J.P. Singh, J.L. Hennessy, A. Gupta, Scaling parallel programs for multiprocessors: Methodology and examples. Computer 26(7), 42–50 (1993)CrossRef

J. Végh, Which scaling rule applies to Artificial Neural Networks, in Computational Intelligence (CSCE) The 22nd Int’l Conf on Artificial Intelligence (ICAI’20) (IEEE, 2020). Accepted ICA2246, in print. [Online]. Available: http://arxiv.org/abs/2005.08942

J. Végh, Finally, how many efficiencies the supercomputers have? J. Supercomput. (2020). [Online]. Available: https://doi.org/10.1007%2Fs11227-020-03210-4

J.L. Gustafson, Reevaluating Amdahl’s law. Commun. ACM 31(5), 532–533 (1988)

C. Liu, G. Bellec, B. Vogginger, D. Kappel, J. Partzsch, F. Neumäker, S. Höppner, W. Maass, S.B. Furber, R. Legenstein, C.G. Mayr, Memory-efficient deep learning on a SpiNNaker 2 prototype. Frontiers Neurosci. 12, 840 (2018). [Online]. Available: https://www.frontiersin.org/article/10.3389/fnins.2018.00840

Top500.org, Retooled Aurora Supercomputer Will Be America’s First Exascale System (2017). https://www.top500.org/news/retooled-aurora-supercomputer-will-be-americas-first-exascale-system/

10.

J. Keuper, F.-J. Preundt, Distributed training of deep neural networks: Theoretical and practical limits of parallel scalability, in 2nd Workshop on Machine Learning in HPC Environments (MLHPC) (IEEE, 2016), pp. 1469–1476. [Online]. Available: https://www.researchgate.net/publication/308457837

11.

J. Végh, How deep the machine learning can be, ser. A Closer Look at Convolutional Neural Networks (Nova, In press, 2020), pp. 141–169. [Online]. Available: https://arxiv.org/abs/2005.00872

12.

US DOE Office of Science, Report of a Roundtable Convened to Consider Neuromorphic Computing Basic Research Needs (2015). https://science.osti.gov/-/media/ascr/pdf/programdocuments/docs/Neuromorphic-Computing-Report_FNLBLP.pdf

13.

G. Bell, D.H. Bailey, J. Dongarra, A.H. Karp, K. Walsh, A look back on 30 years of the Gordon Bell Prize. Int. J. High Performance Comput. Appl. 31(6), 469–484 (2017). [Online]. Available: https://doi.org/10.1177/1094342017738610

14.

S(o)OS project, Resource-independent execution support on exa-scale systems (2010). http://www.soos-project.eu/index.php/related-initiatives

15.

Machine Intelligence Research Institute, Erik DeBenedictis on supercomputing (2014). [Online]. Available: https://intelligence.org/2014/04/03/erik-debenedictis/

16.

J. Végh, A. Tisan, The need for modern computing paradigm: Science applied to computing, in Computational Science and Computational Intelligence CSCI The 25th Int’l Conf on Parallel and Distributed Processing Techniques and Applications (IEEE, 2019), pp. 1523–1532. [Online]. Available: http://arxiv.org/abs/1908.02651

17.

J. Végh, How Amdahl’s Law limits the performance of large artificial neural networks. Brain Informatics 6, 1–11 (2019). [Online]. Available: https://braininformatics.springeropen.com/articles/10.1186/s40708-019-0097-2/metrics

18.

R. Hameed, W. Qadeer, M. Wachs, O. Azizi, A. Solomatnikov, B.C. Lee, S. Richardson, C. Kozyrakis, M. Horowitz, Understanding sources of inefficiency in general-purpose chips, in Proceedings of the 37th Annual International Symposium on Computer Architecture, ser. ISCA ’10 (ACM, New York, NY, USA, 2010), pp. 37–47. [Online]. Available: http://doi.acm.org/10.1145/1815961.1815968

19.

A. Haidar, P. Wu, S. Tomov, J. Dongarra, Investigating half precision arithmetic to accelerate dense linear system solvers, in Proceedings of the 8th Workshop on Latest Advances in Scalable Algorithms for Large-Scale Systems, ser. ScalA ’17 (ACM, New York, NY, USA, 2017), pp. 10:1–10:8

20.

J. Backus, Can programming languages be liberated from the von Neumann Style? A functional style and its algebra of programs. Commun. ACM 21, 613–641 (1978)MathSciNetMATH

21.

E. Chicca, G. Indiveri, A recipe for creating ideal hybrid memristive-CMOS neuromorphic processing systems. Appl. Phys. Lett. 116(12), 120501 (2020). [Online]. Available: https://doi.org/10.1063/1.5142089

22.

Building brain-inspired computing. Nature Communications 10(12), 4838 (2019). [Online]. Available: https://doi.org/10.1038/s41467-019-12521-x

23.

P. Cadareanu, et al., Rebooting our computing models, in Proceedings of the 2019 Design, Automation & Test in Europe Conference & Exhibition (DATE) (IEEE Press, 2019), pp. 1469–1476

24.

S. Williams, A. Waterman, D. Patterson, Roofline: An insightful visual performance model for multicore architectures. Commun. ACM 52(4), 65–76 (2009)CrossRef

25.

S.J. van Albada, A.G. Rowley, J. Senk, M. Hopkins, M. Schmidt, A.B. Stokes, D.R. Lester, M. Diesmann, S.B. Furber, Performance comparison of the digital neuromorphic hardware SpiNNaker and the neural network simulation software NEST for a full-scale cortical microcircuit model. Frontiers Neurosci. 12, 291 (2018)CrossRef

26.

F. Akopyan, Design and tool flow of IBM’s TrueNorth: An ultra-low power programmable neurosynaptic chip with 1 million neurons, in Proceedings of the 2016 on International Symposium on Physical Design, ser. ISPD ’16 (ACM, New York, NY, USA, 2016), pp. 59–60. [Online]. Available: http://doi.acm.org/10.1145/2872334.2878629

27.

M. Davies, et al, Loihi: A neuromorphic manycore processor with on-chip learning. IEEE Micro 38, 82–99 (2018)CrossRef

28.

L. de Macedo Mourelle, N. Nedjah, F.G. Pessanha, Reconfigurable and Adaptive Computing: Theory and Applications, ch. 5: Interprocess Communication via Crossbar for Shared Memory Systems-on-chip (CRC press, 2016)

29.

S. Moradi, R. Manohar, The impact of on-chip communication on memory technologies for neuromorphic systems. J. Phys. D Appl. Phys. 52(1), 014003 (2018)

30.

S.B. Furber, D.R. Lester, L.A. Plana, J.D. Garside, E. Painkras, S. Temple, A.D. Brown, Overview of the SpiNNaker system architecture. IEEE Trans. Comput. 62(12), 2454–2467 (2013)MathSciNetCrossRef

31.

J. Végh, How to extend the Single-Processor Paradigm to the Explicitly Many-Processor Approach, in 2020 CSCE, Fundamentals of Computing Science (IEEE, 2020). Accepted FCS2243, in print. [Online]. Available: https://arxiv.org/abs/2006.00532

32.

M. Hutson, Core progress in AI has stalled in some fields. Science 368, 6494/927 (2020)CrossRef

33.

Y. Shi, Reevaluating Amdahl’s Law and Gustafson’s Law (1996). https://www.researchgate.net/publication/228367369_Reevaluating_Amdahl's_law_and_Gustafson's_law

34.

V. Weaver, D. Terpstra, S. Moore, Non-determinism and overcount on modern hardware performance counter implementations, in 2013 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS), pp. 215–224 (April 2013)

35.

F. Ellen, D. Hendler, N. Shavit, On the inherent sequentiality of concurrent objects. SIAM J. Comput. 43(3), 519–536 (2012)MathSciNetCrossRef

36.

G.M. Amdahl, Validity of the single processor approach to achieving large-scale computing capabilities,” in AFIPS Conference Proceedings, vol. 30, pp. 483–485 (1967)

37.

K. Hwang, N. Jotwani, Advanced Computer Architecture: Parallelism, Scalability, Programmability, 3rd edn. (McGraw Hill, 2016)

38.

P. Molnár, J. Végh, Measuring performance of processor instructions and operating system services in soft processor based systems, in 18th Internat. Carpathian Control Conf. ICCC, pp. 381–387 (2017)

39.

G. Buzsáki, X.-J. Wang, Mechanisms of gamma oscillations. Ann. Rev. Neurosci. 3(4), 19:1–19:29 (2012)

40.

J. Sawada et al., TrueNorth ecosystem for brain-inspired computing: Scalable systems, software, and applications, in SC ’16: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, pp. 130–141 (2016)

Titel: Introducing Temporal Behavior to Computing Science
verfasst von: János Végh
Verlag: Springer International Publishing
Buch: Advances in Software Engineering, Education, and e-Learning
Print ISBN: 978-3-030-70872-6

Electronic ISBN: 978-3-030-70873-3

Copyright-Jahr: 2021
DOI: https://doi.org/10.1007/978-3-030-70873-3_33

Neuer Inhalt

Bildnachweise

Smart-Manufacturing Dashboard Banner/© AdobeStock_583269095, VDI-Icon, Profil Icon, inhalt2, Springer Professional Modul/© Springer Fachmedien Wiesbaden GmbH, chassis.tech plus/© TÜV Süd Product Service GmbH, Search Icon, Banner Hanser, Digitalisierung im Marketing/© Fotolia/alphaspirit, Unternehmensnachfolge/© Ulrich/stock.adobe.com, Dirk Wolters/© Netec GmbH, Zeitschrift Wissensmanagement Cover, PatentFit-Logo/© Springer Fachmedien Wiesbaden GmbH, Sustainibility Finance/© Robert Kneschke / stock.adobe.com / Springer Fachmedien Wiesbaden GmbH, Zukunftswerkstatt Sales Excellence 2024/© AndreyPopov / Getty Images / iStock, 2023_Antrieb/© supervisuell

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Neuer Inhalt

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.