Scalable Shared Memory MIMD Computers

Why are shared memory MIMD computers with many processors difficult to implement? The answer depends in part on the definition of the term “shared memory”, but there is probably broad agreement that any sort of shared memory system with hundreds of processors is a challenge. The main problem is the latency associated with memory access and its consequences for processor performance. There are two possible solutions to this problem, namely latency avoidance and latency tolerance. Latency avoidance is accomplished by arranging a processor’s memory accesses so that most of them are to locations that are both spatially and temporally nearby. Latency tolerance is brought about through the use of additional parallelism. Both ideas have been used in shared memory systems, with varying success.