Revealing the structural behaviour of Brunelleschi’s Dome with machine learning techniques

The Italian artistic and architectural heritage represents a cultural and economic resource of inestimable value that must be safeguarded and conserved. The city of Florence reached its economic and cultural zenith during the 15th and 16th centuries under the rule of the Medici family. That was a period of exceptional artistic activity, thanks to which Florence is nowadays known throughout the world as the cradle of the Italian Renaissance.

One of its most representative and famous monuments is the cathedral of Santa Maria del Fiore (Fig. 1), whose history is rich and complex but not of interest to the purposes of this article.

Suffice it to know that by 1418 the only remaining task for the cathedral’s completion was the construction of the dome. However, the challenge was considerable as no one at the time knew how to build a dome of that size, given that it was to be even larger than the Pantheon’s Dome in Rome and that no dome of that size had been built since antiquity. On August 20th, 1418, a special competition was announced for the construction of the dome: the project presented by Filippo Brunelleschi won the competition.

Composed of over 4 million bricks, standing 116 ms tall, and weighing 37,000 tons, Brunelleschi’s Dome was hailed as the greatest architectural achievement in the Western world upon its completion. Because of its octagonal supporting tambour, the dome presents eight "slice" webs rising in height and joined at the center by a summit lantern, also designed by Brunelleschi. Even today, it remains the largest masonry dome ever constructed, with a special curvature which differed from the hemispherical domes that had been built up to that time.

The left panel of Fig. 2 reports the plan of the cathedral with the webs of the Dome traditionally numbered counter-clockwise starting from the one facing the nave. In the right panel of the same figure a vertical section of the Dome is represented. From an architectural point of view, the vault consists of two nested domes (i.e., an internal dome and an external dome) separated by a walkway leading to the lantern. The maximum diameter of the internal dome is 45.5 m, while that of the external one is 54.8 m. A series of ingenious intuitions allowed Brunelleschi to close the dome building site in just sixteen years. The main one was definitely the brick laying strategy known as “Brunelleschi’s herringbone pattern”, a self-balanced construction technique that allowed him to set up a construction site suspended from the ground without shoring (Corazzi and Conti 2011; Paris et al 2020).

Unfortunately, cracks began to appear in the masonry of the Dome a few decades after its completion, at the end of the 15th century. In 1695, the Grand Duke of Tuscany established a first commission with the task of investigating the stability of the Dome: in its final relation, Vincenzo Viviani stated that “the weight of the upper dome and of the lantern exceeds the resistance of the (tambour at the) base of the monument” (Galluzzi 1977). Over the centuries, there have been other interpretations regarding the origin of the crack patterns that today involves all webs, but mainly webs 4 and 6, both opposite the nave (Corazzi and Conti 2011).

Therefore, the presence of cracks in the Dome has always been a source of concern for the preservation of the monument. For this reason, among the other world records attributed to it, Brunelleschi’s Dome is also one of the most monitored architectural monuments in the world. Currently, the Dome is endowed with two main monitoring networks: a detection system based on 22 mechanical devices installed in 1955, and another detection system based on 166 electronic instruments installed in 1987, recently integrated by other instruments (including an anemometer and a seismograph). In particular, all of the mechanical devices and most of the electronic instruments are deformometers installed across the cracks to measure their width. Other devices such as thermometers and piezometers complete the monitoring system (Ottoni et al 2010).

Managing and analysing sensor data is often challenging (Nieto et al 2021). This is particularly true in the present case, because over time the monitoring system has produced a huge amount of data highlighting a sort of “breathing mechanism” of the vault: namely, the cracks tend to expand and contract cyclically according to seasons and their relative temperatures, some in a harmonious relationship, others in the opposite way (Ottoni and Blasi 2014; Bertaccini 2015). Therefore, the Dome may be assimilated to what in physics is defined as a “closed system”: the structural constraints define the relationship of forces among the various cracks, which in turn are affected by the action of the “surrounding” environment (Bertaccini 2015). Nevertheless, the statistical analyses carried out to date have generally been conducted on a single or on a limited set of devices, showing a progressive increase in the size of the main cracks and, at the same time, a clear relationship with meteorological and seismic variables (Bartoli et al 1996; Ottoni et al 2010).

Since the behaviour of the Dome as a whole (or of its wide masonry surfaces) cannot be fully understood from single sensor measurements alone, a more comprehensive analysis than what has been carried out so far is necessary. Recently, Bertaccini (2015) and Bertaccini et al (2020) – fully aware that the structural behaviour of the Dome and its response to the surrounding environment are "hidden" within the data collected by the sensors – tried to model the complex relationships between endogenous and exogenous variables involved in the monitoring process in order to describe the underlying structural (latent) behaviour of the Dome. In the light of the nature of the variables involved (mainly, multivariate time series data with exogenous and endogenous variables), both Bertaccini (2015) and Bertaccini et al (2020) relied on the estimation of latent variables models (Bartholomew et al 2011), as structural equation models (Hox and Bechger 1998; Bollen et al 2008) and dynamic factor models (Pena and Yohai 2016; Asparouhov and Muthén 2018). Both of these studies aimed to assess the possibility of developing software tools for predicting the structural evolution of the monument. Unfortunately, the vast amount of data and the numerous model parameters led the authors to limit their proposals, both in space (a single web; Bertaccini 2015) and time (measurements gathered in a specific year; Bertaccini et al 2020).

The need to develop a forecasting software that may help in suggesting possible improvement policies for the safety and stability of the Dome represents the main objective of a project funded by the National Research Center in High Performance Computing, Big Data and Quantum Computing foreseen within Mission 4 (Education and Research) of the “National Recovery and Resilience Plan” (NRRP) that is part of the Next Generation EU (NGEU) program (https://​www.​italiadomani.​gov.​it/​en/​). Within this project, the present contribution aims at evaluating the potentialities of some machine learning (ML) techniques to overcome the limits of the above mentioned studies. In the intention of the authors, these ML algorithms should form the core of a specific software for the prediction of the structural evolution of the monument both as a whole and in its specific elements (i.e., its eight slice webs). Such software will integrate the current monitoring system through a real-time acquisition of the measurements of its sensors.

The following two main Research Goals drive the selection of the best set of ML techniques:

Among the various ML approaches, we focus on some dimensionality reduction techniques (Schölkopf et al 1997; Tenenbaum et al 2000; van der Maaten and Hinton 2008) to reach Goal1 and on some recurrent and convolutional neural network models (Hochreiter and Schmidhuber 1997) to achieve Goal2.

To the best of our knowledge, this is the first time that ML techniques are combined to analyse multiple sensor data in the context of historic-architectural heritage protection, with explanatory and predictive purposes. Indeed, the application of ML in the field of preserving cultural and historical heritage is a research area that is yet to mature, primarily due to the scarcity of large datasets. Among the first contribution, Zhu et al (2011) introduced a novel distance measure and algorithms which allow efficient and effective data mining on human-made carved markings on stone, known as petroglyphs. Recently, Fiorucci et al (2020) proposed a wide review of works involving application of ML to cultural heritage data, outlining how the most active areas of study relate to archaeological artefacts (e.g., inferring the intensity of human use of ancient potteries, classifying ceramic artefacts based on their chemical composition) and paintings (e.g., detecting fake artworks, attributing authorship to various artworks, classification of artworks into different artistic categories). Another interesting review contribution is due to Mishra (2021) that focused on applications of ML techniques for the structural health monitoring of heritage buildings. This review discusses several applications (see references therein) of ML approaches for detecting and forecasting cracks in masonry structures and stone monuments, as well as evaluating their remaining lifespan. Such studies relied on various data sources, including laboratory testing, non-destructive testing, high-resolution images, and simulation models. A different stream of the literature focuses on statistical methods and/or ML techniques used to analyse sensor data collected to monitor the health of an architectural feat. Vespier et al (2011) modelled the structural health of a major Dutch highway bridge by clustering time series data collected by the monitoring system installed on it. More recently, Xu et al (2023) and Gomez-Cabrera and Escamilla-Ambrosio (2022) proposed an interesting review of the current machine-learning algorithms implemented respectively in concrete and steel bridge and in buildings structural health monitoring systems, evaluating their effectiveness in detecting damages caused by material deterioration. Abbas et al (2023) presented a study for underground metro shield tunnels, using a deep learning auto-encoder to detect structural damages by incorporating raw vibration signals.

The rest of the paper is structured as follows. Section 2 describes the data and provides insights into the engineering activity necessary to make the dataset more suitable for data visualization and for the ML techniques being used. Section 3 illustrates results obtained through the application of some dimensionality reduction techniques. A particular focus is devoted to describe the structural behaviour of wide masonry surfaces starting from the measurements collected by the sensors installed on them. In Sect. 4 these results are used to predict the future behaviour of each web based on the application of some ML time series forecasting techniques. In Sect. 5, some concluding remarks end the contribution.

All the analyses conducted in this contribution have been performed with R (version 4.2.1) and Python (version 3.7.8) languages integrated with the ScikitLearn (version 1.0.2; Pedregosa et al 2011) and TensorFlow (version 2.9.1; Abadi et al 2015) libraries.

Data used in this work (kindly provided by the “Opera del Duomo” Foundation) have been acquired by the electronic sensors composing the Brunelleschi’s Dome monitoring system, which were installed in 1987 and started recording in January 8th, 1988.

The main part of such instruments is represented by 57 deformometers, which are special sensors for highly repeatable automated measures, generally used in the field of deformability tests. On the Dome, deformometers are installed on the walls across cracks to measure their width, mostly on the even webs due to their worst crack patterns. As shown in Fig. 3, during installation this type of sensor is calibrated on value 0; subsequently, it records negative values when the crack widens, and positive values when the crack narrows.

The width of the cracks is clearly related to the action of the environment; in particular, the masonry volume tends to expand and shrink cyclically according to seasons and related temperatures: the higher the temperature, the greater the masonry volume (and vice-versa). For most cracks, an increase in the volume of the masonry corresponds to a contraction of their width (which, as mentioned above, causes the relative deformometer to register a positive value). However, not all cracks exhibit the same behaviour over time. Some cracks show a contrary pattern, widening when temperatures rise. This is due to the fact that the Dome is comparable to a “closed system”, where structural constraints define the relationships of forces between the various cracks (Bertaccini 2015). For this reasons, a set of thermometers are installed on each web of the Dome to measure the air and the masonry temperature, for a total of 13 air thermometers and 47 masonry thermometers; see Table 1 for the distribution of masonry thermometers and deformometers on each web. The other sensors that complete the monitoring system are piezometers, plumb lines, telecoordinometers and (but only since a few years) an anemometer and a seismograph.

Since its installation, the electronic part of the Dome monitoring system has been recording data at least every 6 h, resulting in a large database of measurements. Moreover, some sensors have broken and have been replaced (resetting their initial values), others have been added over time. For this reason, all the analyses conducted and illustrated in this paper are limited to the data acquired from January 1st, 1997 (when all the current deformometers were active) to February 28th, 2017. Data were subsequently reduced computing the average daily measurements for each sensor. This approach was taken because the fluctuations in crack amplitude and temperature within the 24 h were considered irrelevant (Bertaccini 2015). This process resulted in a conclusive dataset comprising 7364 observations per sensor.

Unfortunately, electronic sensors are susceptible to temporary faults, mainly due to storms and blackouts. Deformometers, in particular, can sometimes be thrown out of calibration by thunderbolts, which can cause anomalous oscillations or full-scale values. Such outliers, in view of their lack of information power, were deleted and treated as missing data. A specially designed statistical method based on the estimation of a quadratic-sinusoidal regression model per sensor was used to impute missing values (Bertaccini 2015) and complete the data matrix.

In this section, for the first time ever, we are able to describe the (latent) structural evolution of the Dome by using the average daily measurements of cracks. We start by exploring the existence of seasonal periodicity in the measurements acquired by deformometers, and we then investigate how to suitably synthesize these measures to characterise the overall structural behaviour of the eight webs. More specifically: Sect. 3.1 is devoted to compare the ability of alternative unsupervised ML techniques to synthesize the dynamics of the entire Dome on the basis of all 57 deformometers; the approach selected in this section is then applied in Sect. 3.2 to synthesize the dynamics of each web.

The dimensionality reduction analysis carried out in the previous section provided insights into the behavioural mechanism of the Dome. By the synthesis provided by the sum of the first two principal components of the eight webs, we gained a better understanding of the relationship between the observed cracks and season-related exogenous variables. Building on this understanding, we now shift our focus to predicting the future movements of each web.

The time series forecasting is a well-documented and extensively studied issue: scholars have developed numerous models and methodologies to deal with this goal, focusing on accuracy, efficiency, and adaptability to various types of data and forecasting horizons. In the ML literature, time series forecasting is typically approached pursuing single-step or multi-step forecasting. In single-step forecasting, the aim is predicting the value at the next time based on the preceding observations. This approach is used when the primary interest is in predicting the immediate future. In multi-step forecasting, the goal is instead predicting values relative to many future times ahead. This approach is applied when one is interested in forecasting a longer-term outlook or anticipate future patterns over an extended period. Both strategies have their own sets of techniques and algorithms tailored to their respective objectives. In both cases, models typically use a sliding window approach, where consecutive samples are used for training. By learning from these sequences, the models can identify patterns and relationships within the data, enabling them to make accurate predictions.

In this work, we address forecasting by employing Deep Neural Networks (DNNs, Goodfellow et al 2016). We emphasize that the aim of this research is not to introduce new ML models or techniques, but, rather, to optimize existing models for a particular application domain, specifically the preservation of cultural heritage. In what follows, we outline the process of optimizing the DNN models to effectively address the unique challenges and requirements of this application domain and then present the results obtained.

In this study, we employed various Machine Learning (ML) techniques to analyse data collected from multiple sensors within the domain of historical-architectural heritage preservation. By applying ML techniques to the monitoring data obtained from Brunelleschi’s Dome, we have successfully achieved two significant objectives.

First and foremost, our research has made a departure from previous studies that presented limitation in both spatial and temporal aspects. For the first time, we have been able to unveil the comprehensive structural behaviour of the Dome as a whole. This has allowed us to gain a more holistic understanding of the Dome’s structural dynamics, surpassing the limitations of previous investigations. This comprehensive understanding could never have been fully achieved solely through the analysis of individual sensors. By employing ML techniques, we were able to synthesize the movements of individual cracks, thereby highlighting the significant role played by the webs in maintaining the structural stability of the entire monument, through a compensative “breathing mechanism” between the webs in even position relative to those in odd position. Furthermore, the analysis has also outlined a close relationship between the masonry temperatures and the movements of the webs over time. Secondly, we used various DNNs to forecast the behaviour of each web, both single- and multi-step ahead. The results obtained from these approaches serve as foundation for the development of an active real-time monitoring and forecasting system. This system will undergo continuous training with new data and will have got the capability to identify anomalies in the Dome’s movements over the medium term. It will provide valuable insights into the overall structural behaviour of the Dome, as well as its individual components, such as the webs.

Authors acknowledge some limitations of the present contribution that will be addressed in future works. The first limitation is that we did not consider the spatial location of the cracks on the surface of the Dome. Factors such as whether the cracks are on the internal or external surface, internal or external dome, or their proximity to the tambour or lantern were not taken into account. The second limitation is that we did not incorporate information related to exogenous variables beyond temperatures, such as wind or seismic movements happened in the past. Following the proposal of Palet et al (2023), an arbitrarily-high number of historical context variables could be integrated in the model to guide the learning task. Currently, only partial information regarding these additional variables is available.

Accordingly, in future works it would be interesting to enhance the dataset by including additional features to identify new relationships with the structural movements of the Dome, consequently improving the forecasting capability of ML algorithms. Moreover, we plan to investigate the performance of recent ML techniques based on the “attention” mechanism. These techniques have shown promising results in various time series contexts, although they require a very long time series window. Unfortunately, up to this point, with the available data limited to a 20 years monitoring window, we have been unable to apply these techniques. However, in future work, we aim to explore the potential of these attention-based ML techniques once we have access to a more extensive dataset.

Finally, as an alternative approach to the analysis of multiple time series, it might be interesting to consider the structural movements of the Dome as an evolving system. This may be tracked by Time Series Chains (Zhu et al 2019) or Novelets (Mercer and Keogh 2023), an emerging technique that allows detection of initially apparent anomalies that are later discovered to be previously unknown highly conserved behaviors.