nach oben

Erschienen in:

2017 | OriginalPaper | Buchkapitel

9. Custom Hardware Versus Cloud Computing in Big Data

verfasst von : Gaye Lightbody, Fiona Browne, Valeriia Haberland

Erschienen in: Understanding Information

Verlag: Springer International Publishing

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

The computational and data handling challenges in big data are immense yet a market is steadily growing traditionally supported by technologies such as Hadoop for management and processing of huge and unstructured datasets. With this ever increasing deluge of data we now need the algorithms, tools and computing infrastructure to handle the extremely computationally intense data analytics, looking for patterns and information pertinent to creating a market edge for a range of applications. Cloud computing has provided opportunities for scalable high-performance solutions without the initial outlay of developing and creating the core infrastructure. One vendor in particular, Amazon Web Services, has been leading this field. However, other solutions exist to take on the computational load of big data analytics. This chapter provides an overview of the extent of applications in which big data analytics is used. Then an overview is given of some of the high-performance computing options that are available, ranging from multiple Central Processing Unit (CPU) setups, Graphical Processing Units (GPUs), Field Programmable Gate Arrays (FPGAs) and cloud solutions. The chapter concludes by looking at some of the state of the art solutions for deep learning platforms in which custom hardware such as FPGAs and Application Specific Integrated Circuits (ASICs) are used within a cloud platform for key computational bottlenecks.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Vorheriges Kapitel Distributed and Connected Information in the Internet

Nächstes Kapitel Information Overload in a Data-Intensive World

Quantified Self. http://quantifiedself.com/. Accessed: 2017-02-03.

IGSR: The International Genome Sample Resource. http://www.internationalgenome.org/home. Accessed: 2017-02-03.

EMOTIV. https://www.emotiv.com/. Accessed: 2017-02-03.

BrainWaveBank. https://www.brainwavebank.com/. Accessed: 2017-02-03.

ThingSpeak. https://thingspeak.com/pages/learn_more. Accessed: 2017-02-03.

ioBridge. http://connect.iobridge.com/. Accessed: 2017-02-03.

CeNSE. http://www8.hp.com/us/en/hp-information/environment/cense.html#.WJCsHbaLR0K. Accessed: 2017-02-03.

IBM Smarter Planet. http://www.ibm.com/smarterplanet/us/en/. Accessed: 2017-02-03.

Google Brain Team. https://research.google.com/teams/brain/. Accessed: 2017-02-03.

NVIDIA. http://www.nvidia.co.uk/page/home.html. Accessed: 2017-02-03.

NVIDIA: Artificial Intelligence and Deep Learning. http://www.nvidia.co.uk/object/deep-learning-uk.html. Accessed: 2017-02-03.

Amazon About AWS. https://aws.amazon.com/about-aws/. Accessed: 2017-02-03.

Amazon EC2. https://aws.amazon.com/ec2/. Accessed: 2017-02-03.

Microsoft Azure. https://azure.microsoft.com/. Accessed: 2017-02-03.

TensorFlow^TM. https://www.tensorflow.org/. Accessed: 2017-02-03.

Nervana. https://www.nervanasys.com/intel-nervana/. Accessed: 2017-02-03.

Human Brain Project. https://www.humanbrainproject.eu/en_GB. Accessed: 2017-02-03.

23andMe (2015) DNA genetic testing & analysis. 23andme. Available via https://www.23andme.com/. Accessed 06 Feb 2017

Abadi M, Agarwal A, Barham P, Brevdo E, Chen Z, Citro C, Corrado GS, Davis A, Dean J, Devin M, (30 additional authors not shown) (2015) Tensorflow: large-scale machine learning on heterogeneous distributed systems. arXiv:1603.04467 [cs.DC]. Available via https://arxiv.org/abs/1603.04467

Aldridge I (2009) High-frequency trading: a practical guide to algorithmic strategies and trading systems, 2nd edn. Wiley, Somerset

Almalki M, Gray K, Sanchez FM (2013) The use of self-quantification systems for personal health information: big data management. Health Inf Sci Syst 3(Suppl 1):1–11

AncestryDNA^TM (2016) DNA tests for ethnicity & genealogical DNA testing. AncestryDNA^TM. Available via https://www.ancestry.co.uk/. Accessed 06 Feb 2017

Armbrust M, Fox A, Griffith R, Joseph AD, Katz R, Konwinski A, Lee G, Patterson D, Rabkin A, Stoica I, Zaharia M (2010) A view of cloud computing. Commun ACM 53(4):50CrossRef

Azoff M (2015) Machine learning in business use cases: artificial intelligence solutions that can be applied. NVIDIA. Available via http://www.nvidia.com/. Accessed 06 Feb 2017

Baker M (2010) Next-generation sequencing: adjusting to data overload. Nat Methods 7:495–499CrossRef

Batty M, Axhausen KW, Fosca G, Pozdnoukhov A, Bazzani A, Wachowicz M, Ouzounis GK, Portugali J (2012) Smart cities of the future. European Phys J Spec Top 214(1):481–518CrossRef

Blayney J, Haberland V, Lightbody G, Browne F (2015) Biomarker discovery, high performance and cloud computing: a comprehensive review. In: Proceedings of 2015 IEEE International conference on bioinformatics and biomedicine (BIBM), pp 1514–1519

British Gas (2017) How data can personalise your energy. British gas. Available via https://www.britishgas.co.uk/. Accessed 06 Feb 2017

Chen CLP, Zhang CY (2014) Data-intensive applications, challenges, techniques and technologies: a survey on big data. Infor Sci 275:314–347CrossRef

Chen M, Mao S, Liu Y (2014) Big data: a survey. Mob Netw Appl 19(2):171–209CrossRef

Di S, Kondo D, Cirne W (2012) Characterization and comparison of cloud versus grid workloads. In: Proceedings of 2012 IEEE International conference on cluster computing (CLUSTER’12), pp 230–238

Dydel S, Bała P (2004) Large scale protein sequence alignment using FPGA reprogrammable logic devices. In: Proceedings of 14th International conference field programmable logic and application (FPL’04), pp 23–32

Elton D (2016) Neuromorphic chips: a path towards human-level AI. Singularity. Available via https://www.singularityweblog.com/. Accessed 06 Feb 2017

Erlich Y (2015) A vision for ubiquitous sequencing. Genome Res 25(10):1411–1416CrossRef

Fan Z, Qiu F, Kaufman A, Yoakum-Stover S (2004) GPU cluster for high performance computing. In: Proceedings of 2004 ACM/IEEE conference on supercomputing (SC’04), pp 47–47

Fan J, Han F, Liu H (2014) Challenges of big data analysis. Natl Sci Rev 1(2):293–314CrossRef

Feldman M (2016) Microsoft goes all in for FPGAs to build out AI cloud. TOP500 supercomputer sites. Available via https://www.top500.org/. Accessed 06 Feb 2017

Foster I, Kesselman C, Tuecke S (2001) The anatomy of the grid: enabling scalable virtual organizations. Int J High Perform Comput Appl 15(3):200–222CrossRef

Gannon D, Fay D, Green D, Takeda K, Yi W (2014) Science in the cloud: lessons from three years of research projects on Microsoft Azure. In: Proceedings of 5th ACM workshop on scientific cloud computing (ScienceCloud’14), pp 1–8

Gubbi J, Buyya R, Marusic S, Palaniswami M (2013) Internet of things IoT: a vision, architectural elements, and future directions. Future Gener Comput Syst 29(7):1645–1660CrossRef

Hancke GP, de Carvalho e Silva B, Hancke GP Jr (2013) Sensors. Role Adv Sens Smart Cities 13(1):393–425

Hashem IAT, Yaqoob I, Anuar NB, Mokhtar S, Gani A, Ullah KS (2015) The Rise of “big data” on cloud computing: review and open research issues. Inform Syst 47:98–115CrossRef

Hruska J (2016) Intel announces major AI push with upcoming Knights Mill Xeon Phi, custom silicon. Extreme Tech. Available via https://www.extremetech.com/. Accessed 06 Feb 2017

Intel (2012) Big data analytics Intel’s IT manager survey on how organizations are using big data. Intel. Available via http://www.intel.me/. Accessed 06 Feb 2017

Jia Y, Shelhamer E, Donahue J, Karayev S, Long J, Girshick R, Guadarrama S, Darrell T (2014) ‘Caffe’. In: Proceedings of 22nd ACM International conference on multimedia (MM’14), pp 675–678

Kitchin R (2014) The real-time city? Big data and smart urbanism. GeoJournal 79(1):1–14CrossRef

Klus P, Lam S, Lyberg D, Cheung MS, Pullan G, McFarlane I, Yeo GS, Lam BY (2012) BarraCUDA – a fast short read sequence aligner using graphics processing units. BMC Res Notes 5(1):27CrossRef

Krizhevsky A, Sutskever I, Hinton GE (2012) ImageNet classification with deep convolutional neural networks. In: Proceedings of advances in neural information processing systems 25 (NIPS’12), pp 1097–1105

Kühnemund M, Wei Q, Darai E, Wang Y, Hernández-Neuta I, Yang Z, Tseng D, Ahlford A, Mathot L, Sjöblom T, Ozcan A, Nilsson M (2017) Targeted DNA sequencing and in situ mutation analysis using mobile phone microscopy. Nat Commun 8:13913CrossRef

Leong L, Petri G, Gill B, Dorosh M (2016) Magic quadrant for cloud infrastructure as a service, worldwide. Gartner. Available via https://www.gartner.com. Accessed 06 Feb 2017

Li G, Bankhead P, Dunne PD, O’Reilly PG, James JA, Salto-Tellez M, Hamilton PW, McArt D (2016) Embracing an integromic approach to tissue biomarker research in cancer: perspectives and lessons learned. Briefings Bioinf bbw044, pp 1–13. Available via https://academic.oup.com/bib/article-lookup/doi/10.1093/bib/bbw044

Lightbody G, Browne F, Zheng H, Haberland V, Blayney J (2016) The role of high performance, grid and cloud computing in high-throughput sequencing. In: Proceedings of 2016 IEEE International conference on bioinformatics and biomedicine (BIBM), pp 890–895

Liu Y, Schmidt B, Maskell DL (2012) Cushaw: a CUDA compatible short read aligner to large genomes based on the Burrows-Wheeler transform. Bioinformatics 28(14):1830–1837CrossRef

Liu Y, Wirawan A, Schmidt B (2013) CUDASW++ 3.0: accelerating Smith-Waterman protein database search by coupling CPU and GPU SIMD instructions. BMC Bioinf 14(1):117

Loman NJ, Misra RV, Dallman TJ, Constantinidou C, Gharbia SE, Wain J, Pallen MJ (2012) Performance comparison of benchtop high-throughput sequencing platforms. Nat Biotechnol 30(5):434–439CrossRef

Melanakos J (2008) Parallel computing on a personal computer. Biomedical Computation Review. Available via http://www.biomedicalcomputationreview.org/. Accessed 06 Feb 2017

Metzker ML (2009) Sequencing technologies – the next generation. Nat Rev Genet 11(1):31–46CrossRef

Miller NA, Farrow EG, Gibson M, Willig LK, Twist G (16 additional authors not shown) (2015) A 26-hour system of highly sensitive whole genome sequencing for emergency management of genetic diseases. Genome Med 7(1):100

Naccache SN, Federman S, Veeeraraghavan N, Zaharia M, Lee D (21 additional authors not shown) (2014) A cloud-compatible bioinformatics pipeline for ultrarapid pathogen identification from next-generation sequencing of clinical samples. Genome Res 24(7):1180–1192

Najafabadi MM, Villanustre F, Khoshgoftaar TM, Seliya N, Wald R, Muharemagic E (2015) Deep learning applications and challenges in big data analytics. J Big Data 2(1):1CrossRef

O’Driscoll A, Daugelaite J, Sleator RD (2013) ‘Big data’, Hadoop and cloud computing in genomics. J Biomed Infor 46(5):774–781CrossRefMATH

O’Dwyer E, De Tommasi L, Kouramas K, Cychowski M, Lightbody G (2016) Modelling and disturbance estimation for model predictive control in building heating systems. Energy Build 130:532–545CrossRef

Osborne J (2016) Google’s tensor processing unit explained: this is what the future of computing looks like. TechRadar. Available via http://www.techradar.com/. Accessed 06 Feb 2017

Papadopoulos A, Kirmitzoglou I, Promponas VJ, Theocharides T (2012) FPGA-based hardware acceleration for local complexity analysis of massive genomic data. Integr VLSI J 46(3):230–239CrossRef

PR Newswire (2016) $1.77 billion deep learning market 2016 – global forecasts to 2022: Google is among the market. PR Newswire. Available via http://www.prnewswire.com/. Accessed 06 Feb 2017

Qiao D, Yip WK, Lange C (2012) Handling the data management needs of high-throughput sequencing data: speedgene, a compression algorithm for the efficient storage of genetic data. BMC Bioinf 13(1):100CrossRef

Raghupathi W, Raghupathi V (2014) Big data analytics in healthcare: promise and potential. Health Infor Sci Syst 2(3):1–10. Available via https://link.springer.com/article/10.1186/2047-2501-2-3/fulltext.html

Ramdas T, Egan G (2005) A survey of FPGAs for acceleration of high performance computing and their application to computational molecular biology. In: Proceedings of TENCON 2005 – 2005 IEEE region 10 conference, pp 1–6

Riggins FJ, Wamba SF (2015) Research directions on the adoption, usage, and impact of the Internet of Things through the use of big data analytics. In: Proceedings of 2015 48th Hawaii International conference on system sciences, pp 1531–1540

Rodellar V, Díaz F, Belean B, Malutan R, Stetter B, Gomez P, Martínez-Olalla R, García-Rico E, Pelaez J (2007) Genomic microarray processing on a FPGA for portable remote applications. In: Proceedings of 2007 3rd Southern conference on programmable logic (SPL’07), pp 13–17

Sagiroglu S, Sinanc D (2013) Big data: a review. In: Proceedings of 2013 International conference on collaboration technologies and systems (CTS), pp 42–47

Schneider D (2017) Deeper and cheaper machine learning [Top Tech 2017]. IEEE Spectr 54(1):42–43CrossRef

Seddon JJJM, Currie WL (2017) A model for unpacking big data analytics in high-frequency trading. J Bus Res 70:300–307CrossRef

Shen J, Ma D, Gu Z, Zhang M, Zhu X, Xu X, Xu Q, Shen Y, Pan G (2016) Darwin: a neuromorphic hardware co-processor based on spiking neural networks. Sci China Infor Sci 59(2):1–5CrossRef

Singh D, Reddy CK (2014) A survey on platforms for big data analytics. J Big Data 2(1):8CrossRef

Starostenkov V (2013) Hadoop + GPU: boost performance of your big data project by 50x-200x? Network World. Available via http://www.networkworld.com/. Accessed 06 Feb 2017

Sugiarto I, Liu G, Davidson S, Plana LA, Furber SB (2016) High performance computing on SpiNNaker neuromorphic platform: a case study for energy efficient image processing. In: Proceedings of 2016 IEEE 35th International performance computing and communications conference (IPCCC), pp 1–8

Swan M (2013) The quantified self: fundamental disruption in big data science and biological discovery. Big Data 1(2):85–99CrossRef

Tan G, Zhang C, Tang W, Zhang P, Sun N (2016) Accelerating irregular computation in massive short reads mapping on FPGA co-processor. IEEE Trans Parallel Distrib Syst 27(5):1253–1264CrossRef

Terrell J (2015) Test-driving the brain could reveal early signs of Alzheimer’s. The Conversation. Available via http://theconversation.com/. Accessed 06 Feb 2017

Townsend AM (2014) Smart Cities – Big Data, Civic Hackers, and the Quest for a New Utopia. W. W. Norton & Company, Edition Reprint

Tractica (2015) Deep learning software market to surpass $10 billion by 2024. Tractica. Available via https://www.tractica.com/. Accessed 06 Feb 2017

Van Dijk EL, Auger H, Jaszczyszyn Y, Thermes C (2014) Ten years of next-generation sequencing technology. Trends Genet 30(9):418–426CrossRef

Venieris SI, Bouganis CS (2016) fpgaConvNet: a framework for mapping convolutional neural networks on FPGAs. In: Proceedings of 2016 IEEE 24th annual International symposium on field-programmable custom computing machines (FCCM), pp 40–47

Vicent J (2016) Qualcomm’s deep learning SDK will mean more AI on your smartphone. The Verge. Available via http://www.theverge.com/. Accessed 06 Feb 2017

Wang Y, McCleary D, Wang CW, Kelly P, James J, Fennell DA, Hamilton PW (2010) Ultra-fast processing of gigapixel tissue microarray images using high performance computing. Anal Cell Pathol (Amsterdam) 33(5):271–285CrossRef

Woods L, Alonso G (2011) Fast data analytics with FPGAs. In: Proceedings of 2011 IEEE 27th International conference on data engineering workshops, pp 296–299

Xu Z, Liua Y, Meia L, Hua C, Chen L (2015) Semantic based representing and organizing surveillance big data using video structural description technology. J Syst Softw 102:217–225

Xu Z, Mei L, Hu C, Liu Y (2016) The big data analytics and applications of the surveillance system using video structured description technology. Clust Comput 19(3):1283–1292CrossRef

Zhu M, Liu L, Wang C, Xie Y (2016) CNNLab: a novel parallel framework for neural networks using GPU and FPGA – a practical study with trade-off analysis. arXiv:1606.06234 [cs.LG]. Available via https://arxiv.org/abs/1606.06234. Accessed 06 Feb 2017

Zikopoulos PC, Eaton C (2012) Understanding big data: analytics for enterprise class Hadoop and streaming data. McGraw-Hill Osborne Media, New York

Titel: Custom Hardware Versus Cloud Computing in Big Data
verfasst von: Gaye Lightbody
Fiona Browne
Valeriia Haberland
Verlag: Springer International Publishing
Buch: Understanding Information
Print ISBN: 978-3-319-59089-9

Electronic ISBN: 978-3-319-59090-5

Copyright-Jahr: 2017
DOI: https://doi.org/10.1007/978-3-319-59090-5_9

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Premium Partner