Our parallel FEM package NuscaS allows us to solve adaptive FEM problems with 3D unstructured meshes on distributed-memory parallel computers such as PC-clusters. For solving sparse systems of equations, NuscaS uses the message-passing paradigm to implement the PCG method with geometric multigrid as a preconditioner.
For the mesh adaptation, the 8-tetrahedra longest-edge partition is used as a refinement mesh algorithm. In this paper, a new method for parallelizing this algorithm is presented. It was developed for the message-passing model, and implemented using the MPI standard. The new solution is based on a decentralized approach. So it is more scalable in comparison to previous implementations, where a centralized synchronizing node (coordinator processor or gateway node) is required.
Both the sequential and parallel versions of the mesh adaptation are carefully optimized to maximize performance. One of key solutions is the usage of suitable data structures, such as hash tables. They allow for high performance while preserving modest memory requirements.