AI-aided exploration of lunar arch forms under in-plane seismic loading

Machine learning is a powerful tool that is capable of detecting and learning patterns in data, and, in turn, creating models for forecasting and aiding decision making under uncertain conditions [1, 2]. In the last decade, machine learning techniques have been integrated into many various structural engineering applications that include monitoring structural performance, determining structural damage, and identifying recovery methods [2]. These machine learning applications in structural engineering and many more can be categorised into four groups: (a) predicting structural response and performance, (b) interpreting experimental data and formulating prediction models for component-level structural properties, (c) retrieving information using images and written text, and (d) recognising patterns in structural health monitoring data.

Burton, Huang, and Sun [2] investigated the studies conducted in each of these four categories in the past decade. It is evident from their paper that the machine learning models developed in prior studies focused on areas such as building performance, component-level design, seismic damage, damage control, and structural health. A limited number of studies have been conducted on the application of machine learning for the design and optimisation of whole structural systems, and some are outlined in the overview papers by Salehi & Burgueno [3] and Málaga-Chuquitaype [1]. However, there is no apparent research on the use of machine learning for the design of extraterrestrial habitats, making this a novel area of study. This paper will make use of the third of the options outlined above (c) to build an unsupervised machine learning model which extracts necessary features from image data sets of arch shapes and contour plots in order to find an alternative to traditional structural design for the purpose of designing extra-terrestrial structures. A design proposal of lunar outposts can be seen in Fig. 1.

A large benefit of shifting focus towards machine learning-based structural design is the large size of data that can be fed into the model [5]. The data set could be in the form of stress or displacement contour plots, giving rise to the potential of producing complex models that have an abundance of structural knowledge for better and more accurate optimisation results. This is particularly useful for extra-terrestrial design where environmental conditions are not well understood and, therefore, would require the consideration of a large set of design scenarios to compensate. In addition, machine learning models are able to easily identify patterns that would not be apparent to a human designer, spotting efficient designs that would not have been traditionally thought of from a structural viewpoint. This could be useful when coming up with creative structural shapes that could be easily 3D printed using extra-terrestrial soil.

Finally, a regression model is built in order to determine and investigate the relationship between the input geometric variables and the latent space representation of the arch shapes and contours. Alongside the convolutional autoencoder, the regression model will allow the visualisation of the effect on the arch shape and contours when moving towards a certain direction in the input space.

Autoencoders are unsupervised machine learning models that consist of two components: the encoder and the decoder [6]. As seen from Fig. 2, the encoder is a network that compresses the input data, \({\bar{x}}\), into an encoded feature vector, \({\bar{z}}\), whose dimensionality is smaller than that of the input’s. The decoder then reconstructs the original input using the feature vector to form the output, \({\hat{x}}\). As visualised, the model forms a bottleneck between the encoder and decoder components, which serves an important purpose for the autoencoder: it ensures that the model does not simply copy the input and passes it along the network to the output, i.e. it forces the model to constrain the input data and use its own understanding to reconstruct it.

Convolutional autoencoders are a type of autoencoders which exploit the observation that an input signal can be decomposed as a sum of other signals [7]. A convolutional autoencoder takes an input, encodes it in a set of simple signals, and then attempts to reconstruct the input signal from them. An advantage of using convolutional autoencoders as opposed to conventional autoencoders is that they do not ignore the structure of the input 2D image [8]. Conventional autoencoders introduce a redundancy by only accepting 2D input images in the form of 1D vectors (i.e. 2D images must be unrolled) and have networks that are constrained to the number of inputs. Convolutional autoencoders, on the other hand, avoid the introduction of any redundancy in the parameters and prevent the features from becoming global, i.e. spanning the entire visual field. In addition to retaining a 2D structure, convolutional autoencoders increase a third dimension that corresponds to the number of filters that are necessary to represent the input image.

The Python library TensorFlow with the toolkit Keras is used to build, model, and train the convolutional autoencoder. Keras is able to construct and train complex machine learning architectures easily and, for this reason, is used [6].

The code for the encoder starts with an input layer that acts as an entry point to the network. Following this, a block of code is repeated several times (seven in this case) consecutively. Each block of code is made up four functions or layers in the following order: After the encoder reduces the dimensionality of the input images as seen in Fig. 3, the decoder attempts to reconstruct the input images using only the latent space vector by reversing the encoder’s operations [9]. The decoder’s input is thus the encoder’s output.

As seen in Fig. 4, the decoder starts with creating an input layer of size equal to the latent space dimension. Following this, a block of code that starts with batch normalisation and ends with a transposed convolution is repeated several times (again, seven in this case), consecutively. The purpose of these consecutive code blocks is to reverse the operations performed by the encoder, in hopes to reconstruct the input image from the latent space. Each block of code is made up four functions:

The process of convolutional layers comprises of the matrix multiplication between the input image and multiple learning matrices or kernels (filters) to produce feature maps [10]. This produces as many feature maps as there are filters (i.e. adding to the third dimension of the input image). Each of these feature maps corresponds to the feature that has been detected by a particular filter.

Since the types of images that are inputted into the built convolutional autoencoder are images of structures with complex shapes and detailed contours that represent stress and displacement variations, a deep network is required. Deeper models have higher capacity to learn complex patterns present in the input data and have a higher redundancy, which is suitable for the design optimisation. The number of layers is also affected by the size of input data, as the learning of the latent features is done gradually through convolutional operations. Increasing the number of convolutional layers, as well as the number of filters in each layer, would increase the number of features the model will detect, since more feature maps would be generated, and would allow the model to learn more complex features.

In order to find a suitable number of convolutional and max pooling layers to use, an input image was fed into the autoencoder and the quality of the reconstructed image (i.e. a measure of how well the encoder was able to pick up the complex and detailed features from the input image) was observed. To avoid the model overfitting the data and becoming computationally expensive, a relatively simple model (a model with three convolutional and maxpooling layers) was used as a starting point and additional layers were added as needed. It was found that using seven convolutional and max pooling layers (in conjunction with a suitable latent space dimension) resulted in reconstructed images that highly resembled the input image.

When choosing an activation function for the model, three of the most popular activation functions were considered (see Fig. 5): rectified linear units (ReLU), sigmoid, and hyperbolic Tangent. The ReLU activation function was used in this model due to its simple mathematical interpolation \((y = \max (0, x))\) and its efficiency in training deep machine learning models.

The arch shapes investigated using thrust line analysis under lunar environments in Málaga-Chuquitaype et al. [11] paper act as motivation for the arch shapes that the machine learning model trains on (details on this are covered in Sect. 3.2). In order to fully understand and make use of the convolutional autoencoder, three arch image data sets are set up:

When creating the different sets of inputs for the arches, it is important to use methods that produce arches that cover the full range of the input geometric variable space. This is to maximise the range of arch shapes that are fed into the autoencoder model and to prevent skewing the data in the direction of specific arch shapes. Latin hypercube sampling was used as it is useful for complex models with many variable inputs. It is a random sampling method that increases computational efficiency by reducing the need for very large input data sets, which is achieved by ensuring full coverage of the input space [12].

In order to build a better understanding on how the input geometric variables that define the arch shapes affect the latent space representation, which, in turn, affects the output arch image, a regression neural network is built using Keras. In the context of this paper, the regression model is able to determine the relationship between the input geometric variables (independent variables) and the latent space representation (dependent variables) to obtain the effects on the latent space vectors when the input variables are moved in a specific direction, e.g. increasing a thickness at a certain section in the arch. Figure 9 displays the architecture of the built regression model.

Since the relationship that the regression model was attempting to find is simple, i.e. a relationship between two small sets of numbers, a wide (network with more nodes in each layer) and shallow (networks with less layers) neural network was used as a starting point. According to Chollet [13], simple networks can learn simpler patterns more efficiently, but they are also more prone to overfitting. In order to avoid overfitting, a very simple model (a model with only one fully connected layer and 10 neurons) was used as a starting point and additional layers and neurons have been added as required. In order to measure whether the model has successfully detected a pattern between the input geometric variables and the latent space representation of the arches, the output from the regression model (i.e. the predicted latent space representation) is fed into the decoder part of the convolutional autoencoder in order to visualise the arch that the predicted latent space vector represents. The model’s architecture was not augmented with additional layers or neurons when the decoder generated arches with shapes that exhibit conformity with the input geometric variables (i.e. the predicted latent space representation was accurate enough for the decoder to reproduce the input geometric variables in its output).

The final regression model starts with an input layer of size equal to the number of input geometric variables for Data Set 1 (five variables t1 t2, r1, r2, and z as shown in Fig. 11 in Sect. 3.4), continues with two fully connected neural layers with 20 neurons each, and ends with an output layer of size equal to the latent space dimension. For similar reasons to the convolutional autoencoder, the ReLU activation function is used for this regression neural network. The regression model is then trained on the input geometric variables (t1, t2, r1, r2, and z) that have created the data set and the latent space representation of the corresponding arches.

Kalapodis et al. [14] discussed the seismic aspects of the Moon by referring to the data gathered by the seismometers installed during the Apollo 11, 12, 14, 15, and 16 missions. Table 1 summarises the corresponding findings.

From Table 1, it is evident that shallow moonquakes will govern the seismic design of the arches due to their large maximum body wave magnitude (4.8 \(m_{b}\)). Given the very small observation period of these motions (only 8 years), it is anticipated that larger events should occur during the life cycle of a lunar structure; hence, a design level acceleration of 0.2 g (where g is the acceleration of Earth’s gravity) is considered. This level of acceleration is consistent with previous assumptions (i.e. Málaga-Chuquitaype et al. [11], Kalapodis et al. [15]) and is in accordance with the observations of Oberst and Nakamura [16] who compared the magnitude–recurrence relationships for shallow moonquakes with interpolate earthquake in Central-Eastern USA. Therefore, static forces equivalent to a peak acceleration of 0.2 g were used in the structural analysis of Data Set 3 as seen in Sect. 3.3 for direct comparison purposes with Málaga-Chuquitaype et al. [11] case study.

The definition of the range of arch shapes used as input for this study was done with two principles in mind: (i) first, to be able to include wide variations of geometry that would enable the exploration of shapes and structural responses beyond what would be intuitively obvious for a human engineer so that the benefits of AI could be materialised; and (ii) the need to include arch shapes that had been previously identified as optimal. To this end, McLean et al. [17] and Málaga-Chuquitaype et al. [11] have developed a form finding algorithm that obtains an optimal arch shape given any standard geometry of minimum thickness for a given loading. In brief, this algorithm finds the limit and admissible thrust lines of the arch shape, offsets them by a required finite strength offset, and uses these offset lines to form a new arch shape envelope. The algorithm then attempts to find the most optimal shape that has the least area compared to the original input arch. Figure 10 shows the progression of an input semi-circular arch to the optimal form-found arch using limit thrust lines. It is evident that the final (optimal) shape is associated with a significant reduction in structural material, forming a ‘pinch point’ at the crown of the arch and two symmetrical ones to the sides of the arch.

As seen in Sect. 2.3, the algorithm used for creating the input data is able to produce shapes that resemble the general shape of an optimal form-found arch from the limit thrust-line analysis based procedure devised by McLean et al. [17]. By doing so, we prevent the machine learning model from working with extremely large data sets of shapes that equate to low structural capacities and make the model training process much more efficient.

To produce the stress and displacement contours using the commercial finite element analysis software Abaqus [18] for Data Set 3, material properties, loading conditions, and boundary conditions need to be defined.

To create all the input training images (shapes and contours), the structural analysis software Abaqus is used. In order to create thousands of input images, three Python-Abaqus scripts have been created:

As mentioned above, the arches comprising Data Set 3 were subjected to detailed numerical analyses using Abaqus. To this end, the arches were discretised into reduced integration 4-noded bilinear plane stress quadrilateral elements. A preliminary mesh sensitivity analysis indicated that a maximum element dimension of 0.2 m was enough to ensure stability in the response of our models. A geometric nonlinear static analysis was carried out with a one second time period.

Special care was taken to ensure that the stress and displacement contour scales remained consistent between all analyses in order for them to be comparable and suitable for input into our convolutional autoencoder network. To this end, the stress contours produced range from the assumed tensile (0.42 MPa) and compressive (\(-\) 4.2 MPa) strengths of the material, as described previously in Sect. 3.3. Likewise, the displacement contours vary from \(+\) 1 to − 1% of the largest arch thickness considered (20 m), i.e. from \(-\) 0.2 to 0.2 m. The contour bar legend that corresponds to the contour plots in Data Set 3 can be seen in Fig. 12. Any stresses or displacements above or below that contour range are represented by a solid grey or black contours, respectively.

After the convolutional autoencoder has been trained on the three data sets, it is important to ensure that the model was able to pick up and differentiate the different features of the arches. In order to do so, the unsupervised machine learning algorithm K-means clustering is implemented on the latent space representation of the data. By doing so, the arches will be grouped according to the features that the encoder was able to extract from the data set.

After using the Elbow method (a mathematical method in which the optimal number of clusters is sought for a specific data set) to find the correct number of clusters (k), the data are then reduced to two components using the t-distributed stochastic neighbour embedding (t-SNE) method, which is a statistical tool that allows better visualisation of high-dimensional data by reducing it into fewer components.

Figure 17 (left) shows the reduced data with six clusters (\(k=6\)) for Data Set 1. In order to ensure the model was able to categorise the images correctly, ten random arch images from each of the six clusters are visualised in Fig. 17 (right). Each row represents a different cluster, and it is clear that the model was able to group very similar looking arch shapes with each other. Because all arches in a specific row contain very similar looking shapes and no anomalies were present, the value chosen for the number of clusters, \(k = 6\), is ideal. The model was also able to distinguish between similar arch shapes with different thicknesses, for example, rows 1, 3, 4, and 6 all have similar looking arches but have different bulging thicknesses near the apex.

In order to be able to use the model for optimisation, it is crucial to understand how the arch shapes and contours are varying from one point in the latent space to another. For this purpose, the latent space representation of a certain data set is clustered using the K-means clustering algorithm and the cluster centroids are found. The clustered data are then reduced to two components and plotted alongside their corresponding cluster centroids. Straight lines are then fitted between the centroids in order to define candidate points along these lines. These two-dimensional candidate points are then transformed back to their latent space representation form, which are, in turn, reconstructed using the decoder to visualise how the arches vary from one centroid to another. It should be noted that a different unsupervised technique called the principal component analysis (PCA) is used for the reduction and inverse-reduction of this data as opposed to t-SNE. The PCA, as opposed to t-SNE, is able to perform inverse-reductions, enabling the transformation of the reduced data back to a latent space representation form.

Figure 18 shows the 2D representations of the clusters and the variation of arches along the lines connecting the cluster centroids for different data sets and number of clusters (k values). Each row in Fig. 18b, d, and f represents the variation along a linear line between two centroids in their corresponding 2D visualisations (a), (c), and (e), respectively. The starting point of the lines, and thus the first row of arch images, is the red cluster for all three data sets, and the order continues along the lines to the end point at the brown cluster for Data Set 1 and the blue cluster for Data Sets 2 and 3.

It is evident that when moving along a line or a row of images, one or more arch features are changing to form a new arch. For example, along the first row in Fig. 18d (i.e. from the red to the orange cluster in (c)), the thicknesses of the two bulging zones are progressively increasing, as well as the side thicknesses in the last row in (d). As for the contour data set, (e) and (f), the shape along with the contours of the arches progressively changes along the lines that connect the cluster centroids, which can be clearly seen in rows one and two in (f) and the two lines connecting the red, green, and orange clusters in (e), in that order.

Optimisation is a function of the latent space representation and, therefore, understanding where the optimal arches, i.e. ones that have green contours, are situated in this latent space is important. Referring to Fig. 18e, f, when moving from the red cluster to the green cluster, to the orange cluster, and ending at the blue cluster, the contours of arches are turning from green (zero horizontal displacement) to red (limit horizontal displacement) to grey (failure). This indicates that optimisation in this case should include an objective function that moves towards the red cluster that contains the green contoured arches.

After the regression model has been trained and has determined a relationship between the input geometric variables and the latent space variables, the effect of moving towards one direction in the input space on the latent space can be determined. This is done by creating input vectors and increasing one input variable from its minimal value up to its maximum value and feeding these vectors into the regression model. The regression model then outputs the equivalent latent space vectors for these input vectors, which can be fed into the decoder in order to visualise the effect on the arch images. For example, \(t_{1}\) in Data Set 1’s input variables is varied as seen in Table 3 (seen as the grey cells) to produce five input vectors (i.e. five arches) with increasing thicknesses at \(30^{o}\) from the horizontal. The definitions of the geometric variables can be seen in Fig. 11 (Left).

In order to better visualise the effect of the input geometric variables on the latent representation, the output latent space vectors from the regression model are reduced using t-SNE into two components and plotted on top of the reduced latent space representation of the whole data set. A line is then drawn between the points from the regression model in order to visualise the movement in the latent space as a result of movement in the input space. As seen from this Fig. 19, it is expected that the greater the impact of the input variable on the overall shape of the arch, the greater the distance traversed in the latent space (rows 1 and 3), and vice versa (rows 2, 4, and 5). The start and end points are represented as the green and red dots, respectively.