Large-Scale Diversity Minimization via Parallel Genetic Algorithms

Diversity minimization is a class of large-scale combinatorial optimization problems arising from the assignment of processors to database fragments in a parallel database environment. These problems may be formulated as nonconvex assignment problems, but they are very difficult to solve via standard optimization techniques because of their large size and because they have many alternative optima. The method described here utilizes theoretical results regarding the form of optimal solutions as the basis for the development of a high-level parallel genetic algorithm. In effect, the genetic operators serve to produce good starting points and neighborhoods for exploration by a heuristic that uses knowledge of the small number of alternatives for desirable processor assignment patterns. Computational results from a parallel implementation of the algorithm on a Connection Machine CM-5 are reported, including the computation of optimal solutions to a million-variable diversity minimization problem.