W. Alexander
Abstract
In dataflow architectures, each dataflow operation is typically executed on a single physical node. We are concerned with distributed data-intensive systems, in which each base (i.e., persistent) set of data has been declustered over many physical nodes to achieve load balancing. Because of large base set size, each operation is executed where the base set resides, and intermediate results are transferred between physical nodes. In such systems, each dataflow operation is typically executed on many physical nodes. Furthermore, because computations are data-dependent, we cannot know until run time which subset of the physical nodes containing a particular base set will be involved in a given dataflow operation. This uncertainty creates several problems.
We examine the problems of efficient program loading, dataflow—operation activation and termination, control of data transfer among dataflow operations, and transaction commit and abort in a distributed data-intensive system. We show how these problems are interrelated, and we present a unified set of mechanisms for efficiently solving them. For some of the problems, we present several solutions and compare them quantitatively.
- Acc86 M Accetta, R Baron, W Bolosky, D Golub, R Rashld, A Tevaman and M Young, "Mach A New Kernel Foundatton For UNIX Development," Summer Usentx Conference (July 1986)Google Scholar
- Ale87 W Alexander and G Copeland, "Process And Dataflow Control In D~stnbuted Data-Intenstve Systems," MCC Techmcal Report No ACA-ST-281-87 (September 1987)Google Scholar
- Ale88 W Alexander and G Copeland, "Comparison Of Dataflow Control Teehntques In D~strtbuted Data-Intensive Systems," A CM SIGMETRICS Conference, Santa Fe, New Mexico (May 1988) Google ScholarDigital Library
- DeW86 D J DeW~tt, R H Gerber, G Graefe, M H Heytens, K B Kumar and M Murallknshna, "GAMMA---A H~gh Performance Dataflow Database Machine," VLDB Conference, Japan (August 1986) Google ScholarDigital Library
- Gra87 J Gray and F Putzolu, "The 5 Minute Rule For Trading Memory For Disc Accesses And The 10 Byte Rule For Trading Memory For CPU T~me," ACM SIGMOD Conference, San Francisco (May 1987) Google ScholarDigital Library
- Hud84 P Hudak and B Goldberg, "Experiments In Diffused Combmator Reduction," A CM Symposmm on LISP and Functtonal Programmmg, Austin, Texas (1984) Google ScholarDigital Library
- Kho87 S Khoshafian and P Valdunez, "Parallel Executton Strategies for Deelustered Databases", Internattonal Workshop on Database Machmes, Japan (October 1987)Google Scholar
- Liv87 M Livny, S Khoshafian and H Boral, "Multl-D~sk Management Algorithms," ACM SIGMETRICS Conference, Alberta, Canada (May 1987) Google ScholarDigital Library
- Sel79 P G Selhnger, M M Astrahan, D D Chamberhn, R A Lone and T G Price, "Access Path Selection In A Relational Database Management System," ACM SIGMOD Conference, Boston (May 1979) Google ScholarDigital Library
- Sto86 M Stonebreaker, "The Case For Shared Nothing," Database Engmeermg, Vol 9, No 1 (1986)Google Scholar
- Tan87 The Tandem Database Group, "NonStop SQL, A Distributed, High-Performance, Hlgh-Avadablhty Implementation of SQL," 2nd International Workshop on Htgh Performance Transactton Systems, Asdomar, Calffornta (September 1987) Google ScholarDigital Library
- Ter85 "DBC/1012 Data Base Computer System Manual, Release 1 3," C10-0001-01, Teradata Corp , Los Angeles (February 1985)Google Scholar
- Tig85 S Tighe, "A Study Of The Parallelism Inherent In Combmator Reduction," MCC Techmcal Report No PP-140-85 (November 1985)Google Scholar
Index Terms
- Process and dataflow control in distributed data-intensive systems
Recommendations
Process and dataflow control in distributed data-intensive systems
SIGMOD '88: Proceedings of the 1988 ACM SIGMOD international conference on Management of dataIn dataflow architectures, each dataflow operation is typically executed on a single physical node. We are concerned with distributed data-intensive systems, in which each base (i.e., persistent) set of data has been declustered over many physical nodes ...
Comparison of dataflow control techniques in distributed data-intensive systems
SIGMETRICS '88: Proceedings of the 1988 ACM SIGMETRICS conference on Measurement and modeling of computer systemsIn dataflow architectures, each dataflow node (i.e., operation) is typically executed on a single physical node. We are concerned with distributed data-intensive systems, in which each base (i.e., persistent) set of data has been declustered over many ...
Comparison of dataflow control techniques in distributed data-intensive systems
In dataflow architectures, each dataflow node (i.e., operation) is typically executed on a single physical node. We are concerned with distributed data-intensive systems, in which each base (i.e., persistent) set of data has been declustered over many ...
Comments