Architecture of Parallel and Distributed Systems

Parallelism is nowadays in all levels of computer architectures. The first level is the processor itself, in which we can find enhancements that probably represent the most spectacular breakthroughs of these last ten years. This chapter begins with a detailed description of superscalar processor features which are intended to increase instruction-level parallelism. Mechanisms for tolerating the latency of the memory hierarchy like speculative execution, speculative disambiguation or fine grain multithreading are then presented. A quantitative analysis of the current and future needs in terms of memory bandwidth and latency shows the problems that must be solved in the memory hierarchy and introduces the part dedicated to the memory hierarchy. This part gives the state of the art of available and future memory chips. Multibanked memories are a good introduction to shared-bus multiprocessors, which are the most commercially popular. Physically shared-memory multiprocessors are then analysed, through some classical processor-memory networks. A synthesis of data coherency algorithms is developped and the data consistency problems are shown. Again, a quantitative evaluation of performance needs introduces physically-distributed memory multiprocessors and the analysis of available multiprocessors systems allows us to point out the main advantages and drawbacks of the various possible options, including logically-shared memory systems and message passing systems Finally, the state of the art of I/O systems is assessed, using a performance analysis of available disk systems and current trends in interconnecting peripherals to processors and memories.