1 Introduction
-
Application of the speculative fine-grained scheme to a family of metric-aware mesh operations applicable to volume and boundary adaptation (Sect. 3). As seen in previous work [9] this combination is highly effective. As of today and to the best of the authors' knowledge, it offers on shared memory machines the highest strong scaling speedup among scalability-first approaches and better weak scaling speedup than state-of-the-art functionality-first approaches. This is achieved via the low startup cost of our methods and the nature of the optimistic/speculative approach that exploits parallelism as soon as possible. Moreover, the ability to adapt both surface and volume elements at the same time yields better robustness and offers more than 3 orders of magnitude improvement when compared to adapting only the volume.
-
Extension of the above operations in order to support for geometrical (CAD-based) models (Sect. 4). Access to a geometrical kernel allows to interrogate the underlying domain an recover information about its curvature and local feature size which could be absent in a coarse mesh representation which is often the starting point of the adaptation procedure. Handling of CAD data gives access to well-known verification examples like the ONERA-M6 wing and the JAXA highlift model.
-
Evaluation of the method in an adaptive pipeline that includes a CFD solver and real-world geometry cases and comparison of the computed quantities with values present in the literature (Sect. 5). In contrast to our previous work that focused mainly on element quality measures, we compiled a suite of publicly available cases and bring our attention to the main goal of mesh adaptation: capturing features of the underlying simulation. At the same time, we test how well CDT3D operates as a part of an adaptive pipeline composed of open-source components.
2 Related work
3 Metric-based adaptation within the CDT3D library
3.1 Parallel speculative local reconnection in metric spaces
3.2 Parallel point creation for anisotropic mesh adaptation
3.3 Speculative edge collapse for metric-based adaptation
#pragma omp parallel for schedule(guided)
OpenMP construct. Each thread picks and locks (speculatively) the vertex (a in Fig. 3) corresponding to the iterator value. Then it speculatively locks its adjacent vertices (blue in Fig. 3). If any of the locks fail, the thread will release any acquired locks and it will proceed to the next vertex. Notice that locking the vertices implicitly grants exclusive access to all their adjacent tetrahedra (red elements of Fig. 3). Once the required locks have been acquired, the edge lengths between the vertex a and the rest of the vertices of its cavity are evaluated. If an edge with a length less than a user-defined value is found then the edge will be collapsed. Additional criteria such as not applying edge collapse in cases that will increase the surface deviation (see Sect. 4) are also applied for edges on the surface of the mesh. Finally, the edge is collapsed by moving the second point to the first. The guided
scheduler was selected because it performs on average better for cases that require different levels of coarsening [9]. More sophisticated parallel iteration and tasking schemes are out the scope of this work but have been explored in [47].3.4 Vertex smoothing
3.5 Effect of the complete set of operations
4 Handling geometry through metric spaces
5 Evaluation
5.1 Experimental setup
ref bootstrap
which is part of the refine mesh mechanics suite [56]. ref bootstrap
uses the EGADS [57] kernel in order to generate an initial surface triangulation of the input CAD file. The surface mesh is then adapted based on the curvature and other geometrical features. Adapting the surface in the absence of a volume mesh gives greater flexibility since the software is not constrained by the requirement of conformity to a volume after each operation. A volume mesh is then generated using an external tool such as TetGen [61] or AFLR [62] and finally the volume mesh is adapted based on a metric field derived by the geometrical features of the CAD input. SU2
will then produce a solution file that holds values of the discrete solution at each vertex of the input mesh. dat2solb
is used to convert the solution to a libMeshb-compatible file [63]. The extracted Mach field (solution-mach.solb
) is then passed to ref_metric_test
that creates a multiscale metric field based on it (solution-metric.solb
). The multiscale metric field can be optionally intersected with a curvature- and feature-based metric built based on the geometrical features of the input model. CDT3D will then use the metric field along with the mesh used by SU2
to generate an adapted mesh (new_mesh.meshb
). If solution interpolation is utilized, we pass the new mesh along with a .solb
version of the SU2
solution to ref_intrep_test
which we then convert using solb2dat
to an SU2-compatible file (interpolated_solution.dat
). The values of the previous solution are interpolated using linear interpolation. Finally, the adapted mesh is passed to SU2
after being converted to a .su2
mesh file along with the interpolated solution if this was generated.5.2 Analytic scalar fields
sinfun3
, tanh3
and sinatan3
fields. Bottom: Corresponding CDT3D adapted meshes at 256,000 target complexity5.3 Laminar subsonic flow over a delta wing
turing
cluster8 that houses nodes with a variety of different specifications. The number of cores used by the solver was set so that it corresponds to about 10,000 vertices per core and it was constrained to 300 to reduce the waiting time in the job scheduler queue of the cluster. CDT3D is using one of turing
’s nodes with two sockets each one with a Intel® Xeon® CPU E5-2698 v3 @ 2.30GHz (16 cores) for a total of 32 cores.iter | vertices | tetrahedra | solver | solver | CDT3D | CDT3D |
---|---|---|---|---|---|---|
(s) | core-hours | (s) | core-hours | |||
0 | 901 | 3,444 | 57.55 | 0.16 | - | - |
5 | 97,896 | 563,930 | 2,833.51 | 7.87 | 63.53 | 0.56 |
10 | 192,098 | 1,114,412 | 3,301.11 | 18.34 | 49.26 | 0.44 |
15 | 377,569 | 2,203,660 | 2,897.73 | 32.20 | 111.95 | 1.0 |
20 | 749,290 | 4,391,974 | 3,865.75 | 85.91 | 207.08 | 1.84 |
25 | 1,467,922 | 8,641,694 | 3,476.85 | 154.53 | 392.55 | 3.49 |
30 | 2,897,903 | 17,108,219 | 2,777.01 | 232.96 | 808.27 | 7.18 |
35 | 5,726,724 | 33,883,975 | 3,281.82 | 273.49 | 1622.15 | 14.32 |
5.4 Inviscid Onera M6 case
ref bootstrap
.5.5 Inviscid flow over the JAXA Standard Model
ref bootstrap
of the refine mesh mechanics suite [56]. The metric is constructed based on the Mach field of the solution and the metric gradation value is set to 1.5. The complexity of the metric (see equation (10))is doubled every 5 iterations. The solution-based metric is intersected also with a curvature- and feature-based metric built based on the geometrical features of the model. We considered 8 metric complexity values for this study: \([50\,000, 100\,000, 200\,000, 400\,000, 800\,000, 1\,600\,000, 3\,200\,000, 6\,400\,000]\). The final mesh contains 13, 227, 952 vertices, 478, 518 triangles and 78, 479, 450 tetrahedra.5.6 Parallel evaluation
wahab
cluster of Old Dominion University using dual-socket nodes equipped with two Intel® Xeon® Gold 6148 CPU @ 2.40GHz (20 slots) and 368 GB of memory. The compiler is gcc 7.5.0
and the compiler flags -O3 -DNDEBUG -march=native
. Each run was repeated 5 times and the results were averaged using the geometrical mean [77]. For the base case, we ran the parallel code using one core.