Paul G. Brown
ABSTRACT
SciDB [4, 3] is a new open-source data management system intended primarily for use in application domains that involve very large (petabyte) scale array data; for example, scientific applications such as astronomy, remote sensing and climate modeling, bio-science information management, risk management systems in financial applications, and the analysis of web log data. In this talk we will describe our set of motivating examples and use them to explain the features of SciDB. We then briefly give an overview of the project 'in flight', explaining our novel storage manager, array data model, query language, and extensibility frameworks.
- P. A. Boncz, M. L. Kersten, and S. Manegold. Breaking the memory wall in monetdb. Commun. ACM, 51(12):77--85, 2008. Google Scholar
Digital Library
- http://public.web.cern.ch/public/en/lhc/computing-en. html.Google Scholar
- P. Cudre-Mauroux, H. Kimura, K.-T. Lim, J. Rogers, R. Simakov, E. Soroush, P. Velikhov, D. L. Wang, M. Balazinska, J. Becla, D. DeWitt, B. Heath, D. Maier, S. Madden, J. Patel, M. Stonebraker, and S. Zdonik. A demonstration of scidb: a science-oriented dbms. Proc. VLDB Endow. 2(2):1534--1537, 2009. Google ScholarDigital Library
- P. Cudre-Mauroux, H. Kimura, K.-T. Lim, J. Rogers, R. Simakov, E. Soroush, P. Velikhov, D. L. Wang, M. Balazinska, J. Becla, D. DeWitt, B. Heath, D. Maier, S. Madden, J. Patel, M. Stonebraker, and S. Zdonik. Scidb at cidr. Proc. CIDR. 2009.Google Scholar
- J. Dean and S. Ghemawat. Mapreduce: Simplified data processing on large clusters. In OSDI, pages 137--150, 2004. Google ScholarDigital Library
- http://www.hdfgroup.org/documentation/.Google Scholar
- M. Stonebraker, D. J. Abadi, A. Batkin, X. Chen, M. Cherniack, M. Ferreira, E. Lau, A. Lin, S. Madden, E. O'Neil, P. O'Neil, A. Rasin, N. Tran, and S. Zdonik. C-store: a column-oriented dbms. In VLDB '05: Proceedings of the 31st international conference on Very large data bases, pages 553--564. VLDB Endowment, 2005. Google ScholarDigital Library
- M. Stonebraker and L. A. Rowe. The design of postgres. In IEEE Transactions on Knowledge and Data Engineering, pages 340--355, 1986.Google Scholar
- http://www.unidata.ucar.edu/software/netcdf/docs/.Google Scholar
- Hubble space telescope servicing mission 4 fact sheet, 2007.Google Scholar
Index Terms
- Overview of sciDB: large scale array storage, processing and analysis
Recommendations
SciDB: A Database Management System for Applications with Complex Analytics
A description and discussion of the SciDB database management system focuses on lessons learned, application areas, performance comparisons against other solutions, and additional approaches to managing data and complex analytics.
Selective Scan for Filter Operator of SciDB
SSDBM '16: Proceedings of the 28th International Conference on Scientific and Statistical Database ManagementRecently there has been an increasing interest in analyzing scientific data generated by observations and scientific experiments. For managing these data efficiently, SciDB, a multi-dimensional array-based DBMS, is suggested. When SciDB processes a ...
Big data analytics in Cloud computing: an overview
AbstractBig Data and Cloud Computing as two mainstream technologies, are at the center of concern in the IT field. Every day a huge amount of data is produced from different sources. This data is so big in size that traditional processing tools are unable ...
Comments