Reconstruction of historical hygrometric time series for the application of the European standard EN 15757:2010 and its comparison with current time series

Abstract

The quality and quantity of thermo-hygrometric data are essential to carry out an appropriate assessment of the microclimate from a preventive conservation standpoint in those spaces where the artefacts to be preserved are located. These analyses are fundamental for long-term preventive conservation plan to assess chemical, biological or fracture risks. However, many small and medium-sized museums as well as heritage buildings have only a limited amount of historical data, with various problems that hinder the evaluation of microclimatic conditions. Two of the most common problems are short monitoring time periods, usually less than one year, and low sampling rates of measurements. In many of these situations, guidelines such as the European standard EN 15757:2010 cannot be applied because they require a monitoring period of at least 13 months and a minimum sampling frequency of one measurement per hour. In addition to these issues, there are other drawbacks such as missing values or lack of regularity in data collection. This paper proposes a procedure for the reconstruction of historical thermo-hygrometric data using multivariate statistical methods. The methodology allows the arrangement of long historical series of sufficient quality, enabling museums to restore their datasets for further analysis regarding the application of guidelines for preventive conservation. The methodology has been validated on the basis of real data. The application of the European standard EN 15757:2010 is presented as a practical example of the procedure using historical data collected at a partner museum of the H2020 CollectionCare project, together with data currently being collected for some months by a set of wireless sensor nodes.

Introduction

In order to preserve cultural heritage properly, it is necessary to assess and control the microclimatic conditions of the place where the objects to be preserved are located. In particular, air conditions are mainly determined by temperature (T) and relative humidity (RH) [1, 2].

In recent decades, the scientific community has changed the way of approaching this challenge, moving from a static perspective to a new dynamic standpoint. In the traditional perspective, standards such as UNI 10829 [3], DM 10/2001 [4] or The National Trust Specifications for Conservation Climate Control [5, 6] established fixed limits for T and RH. The static traditional approach has been widely adopted because it is generally assumed that keeping constant the optimum thermo-hygrometric conditions, from a preservation standpoint, or with the minimum variability along the time, is a reasonable criterion to minimise the potential damages to the artworks. However, this lack of flexibility results in high costs in HVAC (heating, ventilation and air conditioning) systems and might turn into inadequate conditions for people visiting the artworks, if the optimum conservation temperature departs from the comfort range for humans. This issue has been widely studied in the literature [7,8,9]. Currently, the dynamic perspective for the microclimatic conditions advocates more flexible limits adapted to the context where the object is located [10]. Based on this idea, Michalski [11] proposed the concept “proofed fluctuation” as the largest fluctuation of RH or T to which the object had been exposed in the past [11].

The most important standards based on this approach are European standard EN 15757:2010 [12] and the United States guidelines ASHRAE [13]. These are methodological recommendations for the setting of appropriate limits for T and RH variations. The European standard concerns conservation of objects made of organic hygroscopic materials and highlights the priority of focusing on the historic climate of the object to limit mechanical damage induced by strain–stress cycles. The concept of “acclimatisation” (i.e., get used to the climate) of materials is the premise for the application of this standard; it states that if an object has been subjected to certain microclimatic conditions for a prolonged period of time, which has been found not to be harmful, it is recommended to maintain similar conditions. This idea contrasts with the universality of the fixed ranges recommended by previous standards and highlights the importance of studying the historical conditions to which the artworks have been subjected. The present study takes EN 15757:2010 as a reference and applies it to characterise the microclimate at the Fine Arts Museum of Alava in Spain.

A monitoring system is necessary in order to analyse air conditions in the context of cultural heritage. There are many studies in which multiple sensors are installed in order to characterise a given microclimate [14,15,16,17,18,19,20,21]. The present study analyses air conditions at the Alava Fine Arts Museum, which are strongly determined by the presence of HVAC systems, a very determining factor that impacts the short-term and long-term variability of T and RH as discussed by several published studies [22,23,24,25,26,27,28,29]. These systems are convenient for an appropriate control of the microclimate, but they are affected by outside conditions and can undergo failures, changes of setpoint conditions and other drawbacks.

A set of 19 dataloggers were deployed in this museum in two different time periods. Firstly, 9 dataloggers, model EL-USB-2-LCD from Lascar Electronics [30], were installed between October 24th 2017 and May 2nd 2019 to record thermo-hygrometric air conditions at the museum. Information about the accuracy of the measurements are provided in the corresponding section. In the second period, 10 dataloggers were deployed for microclimate monitoring from from January 1st, 2021 to February 1st, 2022.

In the case of multivariate monitoring systems, the straightforward application of EN 15757:2010 would require observations to be collected at the same instants for all sensors, at constant sampling rates. However, some form of pre-processing is always necessary, especially when it comes to the installation of multiple dataloggers. For example, they may be programmed to record values every 6 h, but the instant when the reading is performed might not be the same for all. Battery failures can also produce data losses. The presence of missing values is a common problem in this field [31,32,33,34], in particular when the amount of missing data is large [35, 36]. Application of the European standard requires at least 13 months of data at a minimum frequency of one record every hour. During this time period, many events causing significant data losses may occur (e.g. loss of connectivity of dataloggers, battery depletion, reduction of storage capacity, sensor failures, etc.). The choice of sampling frequency in this context has been discussed in [37].

The EN 15757:2010 standard can be applied to both RH and T. However, as in many other investigations [32, 38,39,40], it was decided to focus this study on RH for two reasons. Firstly, hygroscopic materials are much more sensitive to fluctuations of RH than of T [41] because of their ability to absorb moisture from the environment where they are located. This is the case of the paintings exhibited at the Alava Fine Arts Museum. Secondly, given the great similarity in the pattern of the time series of RH and T, the reconstruction methodology proposed here can be applied analogously to a dataset of RH or T.

This study has the following objectives: (i) to discuss the usual difficulties in the analysis of microclimatic data for the preservation of artworks, in particular regarding the application of EN 15757:2010; (ii) to propose a methodology able to tackle these problems applying the standard; (iii) to evaluate this proposal based on real data, discussing its advantages and limitations; (iv) to demonstrate its usefulness by characterising the microclimate under study, comparing past and present microclimatic conditions.

The structure of this paper is as follows: (i) materials and methods are discussed, in particular describing the proposed reconstruction methodology and the validation procedure; (ii) in results and discussion, the method is validated, applied to evaluate its advantages and, finally, the European standard is applied to the reconstructed time series to characterise the historical climate including the seasonal cycle and the historical climate variability; (iii) conclusions are drawn from the research and future research lines are pointed out.

Materials and methods

Case study: Alava Fine Arts Museum

The Alava Fine Arts Museum is located in Vitoria-Gasteiz (Basque Country) in northern Spain (geographical coordinates: 42.8417, − 2.6797). It consists of a complex of three buildings: (i) the Augustin Zulueta Palace, whose exterior appearance is shown in Fig. 1 (left), (ii) a three-storey annexe extension built in the 1960s (Fig. 2), and (iii) a visitor entrance built in 2001. Figure 3 shows the plans of the complex, excluding the third floor of Zulueta’s palace because it is not used for exhibition.

Fig. 1

Outside view of Augustin Zulueta’s palace (left) and the hall (right)

Fig. 2

Appearance of one of the rooms in the annexe building displaying artworks by Ignacio Díaz Olano

Fig. 3

Museum floor plans. From left to right (bottom, Zulueta’s palace): semi-basement, 1st floor and 2nd floor. The 3rd floor of Zulueta’s palace is not depicted because the rooms are not used for exhibition. From left to right (top, annexe building attached): 1st, 2nd and 3rd floor. The position of historical dataloggers is indicated as red circles, as well as CollectionCare (CC) sensors (blue and grey circles). The grey sensors were discarded due to poor data quality. Pairs of historical and CC sensors are indicated on the map according to proximity by means of adjacent circles or black lines joining the circles. Datalogger A is the only unpaired one

Zulueta’s palace is a four-storey building designed by the architects Julián Apraiz (1876–1962) and Francisco Javier de Luque (1871–1941), which was built in 1912 for the private residence of Elvira Zulueta and Ricardo Augustin. This historic palace is a sumptuous building, eclectic in style, inspired by Renaissance palaces and with regionalist elements. The building is centred on a large quadrangular entrance hall, which can be seen in Fig. 1 (right) and divided into a semi-basement, first floor and two more floors. The facades are made of sandstone from Fontecha (Alava), and the interior features cabinetmaking, which is a deep-rooted tradition in Vitoria. Both the chapel and the vestibule were made by Casa Ibargoitia and the original stained-glass windows by Casa Maumejean. In terms of construction, the palace (historic building) consists of masonry foundations covered with concrete, iron frame and beams, sandstone facades, wooden floors and stairs, and plaster ceilings.

A three-storey building was annexed to the palace in the 1960s with a reinforced concrete foundation and structure, hipped slate roof, brick wall enclosure with stone exterior cladding, marble and stone floors, marble staircase and false plaster ceiling.

Finally, in 2001, a visitor access was built as part of a major renovation in which, among other things, architectural barriers were removed and access was improved with additional visitor services. This paper focuses on the set of buildings with the current configuration.

The museum specialises in Basque art from 1850–1950 and Spanish art from the eighteenth and nineteenth centuries. Most of the collection consists of paintings on canvas that are arranged chronologically. The first-floor rooms are dedicated to Basque art from 1850–1950, showing the evolution of painting from pioneering artists such as Juan Ángel Sáez (1811–1873), Antonio Lecuona, Eduardo Zamacois y Zabala and José Echenagusia, among others. The exhibition continues with the works of innovative artists like Adolfo Guiard, Darío de Regoyos and Juan de Echevarría, until reaching the most outstanding artists of the period before the outbreak of the Civil War, such as Julián de Tellaeche (1884–1957).

Other oeuvres by Basque artists can be found on the second floor, including large format works by Ignacio Zuloaga, Aurelio Arteta, Elías Salaverria, Francisco Iturrino and the brothers Ramón and Valentín de Zubiaurre, creations in which traditional and modern artistic languages converge. The second floor is devoted to Fernando de Amárica, with works donated by his foundation for permanent exhibition.

Located in the area corresponding to the Augustin Zulueta Palace, the collection of Spanish art from the eighteenth to the twentieth centuries, shown through portraits, landscapes and genre scenes, illustrates the transition from classical and academic painting to the more spontaneous romantic styles and the painters of the realist movement. The portraits of Vicente López Portaña and Federico and Raimundo de Madrazo, the landscapes of Carlos de Haes, Aureliano de Beruete and the mural sketches of Josep Maria Sert are a good examples of this movement.

The climate of the city of Vitoria-Gasteiz, according to the Köppen–Geiger classification [42, 43], is classified as Cfb (oceanic climate), which is characterised by mild temperatures with small diurnal oscillations and high RH influenced by the Atlantic Ocean. In this climatic context, the conservation tasks of the important collection contained in this museum are complex because the museum inherited a historic building that was not designed specifically for this purpose and because a building with different construction characteristics was later annexed. In order to maintain appropriate environmental conditions in the museum, two different air conditioning systems are used simultaneously: air conditioning in the building constructed in the 1960s and heating by cast iron radiators in the historic building, with the addition of humidifiers to achieve an environment that meets the needs of the collection. The conservation team has been recording and monitoring the microclimate since 1987 in order to assess if thermo-hygrometric conditions remained inside the recommended intervals for appropriate long-term preservation of the paintings.

Measurement systems

As mentioned above, some of the devices used to monitor environmental conditions in the museum are Lascar Electronics dataloggers, model EL-USB-2-LCD, calibrated by the manufacturer according to ISO 10012-1 and ANSI/NCSL 2540-1-1994. Their resolution is 0.5 °C, with an accuracy of 0.55 °C between 5 and 60 °C for temperature. In the case of relative humidity, the resolution is 0.5%RH, with an accuracy of 2.25%RH between 20%RH and 80%RH, which are not in accordance with EN 16242:2012 and EN 15758:2010 metrological specifications for T and RH. Actually, it might be argued that the resolution of temperature is rather low for the purpose of microclimatic monitoring in the museum, but this model was chosen years ago because of its reasonable price, quality and other technical advantages with the purpose of routine ambient monitoring and without any application of standards in mind (e.g. minimum of one record every hour for EN 15757:2010). High data logging rate or very reliable systems leading to a minimal amount of missing data require improved technical capabilities (e.g., more memory or online storage, more battery, etc.) or further personnel to attend to them (frequent checking of the device’s proper functioning, regular data download, etc.).

Figure 3 shows in red the points where dataloggers were placed and whose data have been used in this study. Initially, a sampling frequency of 6 h was implemented to be able to adequately plan the unloading tasks with the personnel available, and these datasets are the ones used in the present work. This decision was taken because, given that no standard requiring a higher frequency was intended to be applied, recording measurements every 6 h allows the storage capacity of the dataloggers to last longer and the museum staff to download the data less frequently.

In July 2020, 10 prototypes of a wireless sensor developed in the framework of the European project CollectionCare [44] were installed. Among other parameters, these sensors measure ambient T and RH at an interval of one record per hour, precisely to comply with the minimum measurement interval required by EN 15757:2010. From the standpoint of measurement quality, the sensors also comply with the metrological specifications for T and RH sensors and measuring methods in accordance with EN 16242:2012 [45] and EN 15758:2010 [46].

Data selection and matching between former and new dataloggers

The monitoring campaigns considered comprise two very distinct periods (Fig. 4). On the one hand, the Lascar dataloggers, which have collected the time series, denoted here as historical, between October 24th 2017 and May 2nd 2019. Since a period of 13 months is required for this study to apply the EN 15757:2010 standard, data between April 1st, 2018 and May 1st, 2019 were finally chosen to minimise the time gap between the two series. On the other hand, the CollectionCare wireless sensors started gathering data on July 22nd 2020 and continue to do so at present. From this time span, a 13-month period, January 1st, 2021 to February 1st, 2022, was again selected.

Fig. 4

Historical time series (Lascar dataloggers) (since October 2017, in red) already reconstructed and CC sensors time series (since January 2021, in blue) over time. The period between May 2019 and January 2021 is not displayed to better appreciate fluctuations

A problem arises when comparing historical and CollectionCare (CC) time series. Some new dataloggers have been installed in the same position as the former ones, but in other cases there is a separation distance of up to 8 m (Fig. 3). However, in most cases there are no obstacles between the sensors and they are located in the same room, which allows us to assume small microclimate differences. In order to estimate what the conditions would have been at the same point (i.e., coincident position of sensors) and to investigate on the differences between distant sampling points, if any, experiments should be carried out by taking T and RH measurements over time in a horizontal cross section of the rooms. Unfortunately, this option is complex and currently unaffordable due to the insufficient number of dataloggers per floor. For this reason, it was decided to pair historical and CC sensors based on the criteria of proximity and the existence or not of obstacles between them. Each pair of dataloggers (coded in Fig. 3 as dNN, NN = 01 to 09) is set up as follows for each historical datalogger:

• d01: CC sensor was installed in the same position, but due to signal coverage problems, it underwent 58% of missing data, so it was discarded for this study.

• d02: The CC sensor was installed in the same position. It is located in the storeroom where artworks are kept when they are not on display.

• d03: The CC sensor was installed in the same position, but due to connectivity problems, it recorded erroneous values most of the time, so it was discarded for this study.

• d04: The CC sensor was installed about 7 m away from the historical one in the same room, with no obstacles between them.

• d05: The CC sensor was installed about 5 m away from the historical one in adjacent rooms, although there is no obstacle between them due to the entrance that connects both rooms.

• d06: The CC sensor was installed about 8 m apart in adjoining rooms.

• d07: The CC sensor was installed in the same position, but due to problems in the thermo-hygrometric sensor, it registered erroneous values most of the time, so it was discarded for this study.

• d08: The CC sensor was installed about 7 m apart in the same room with no obstacle between them.

• d09: The CC sensor was installed about 5 m apart in adjacent rooms, although there is no obstacle between them due to the entrance that connects the two rooms.

Among the 10 wireless sensors installed in the framework of the CollectionCare project (i.e., d01, … d09 and sensor A, see Fig. 2), three of the 10 time series recorded were ruled out due to various errors, mainly because of connectivity failures causing data loss, so that most of the measurements from these sensors are not available. The sensors discarded are those corresponding to pairs d01, d03 and d07 (Fig. 3). For the remaining time series, the amount of missing data is 4.39%, so that most of the gaps are small: 52% with up to 3 consecutive missing values and 74% with up to 12. This problem is mainly caused by connectivity failures with the collection system (gateway). The gateway is connected to the power grid and stops functioning when there are power cuts or mobile internet connectivity failures. Therefore, when the gateway fails, all dataloggers stop collecting data during the same period. The existence of missing data at the same time for all dataloggers poses a serious difficulty for accurate imputation of missing observations. It was decided to use linear interpolation method because previous studies [36] have shown that it is a simple method that yields reasonably good results in RH imputation for short periods of missing data. This method performed better than other approaches like cubic interpolation, according to the cited study, which has also been confirmed in the development of the present research. On the other hand, the time series from CC sensors are primarily used to compute the width of the safety bands according to EN 15757:2010 standard (as explained later in the Standard EN 15757:2010 section) and, for this calculation, a low percentage of missing data has a minimal impact compared to using other imputation methods. Thus, considering more sophisticated tools for imputation would lead to basically the same width of safety bands.

Reconstruction methodology

The data gathered by Lascar dataloggers are very useful, but they have shortcomings that prevent the direct application of EN 15757:2010. These limitations are the following:

• The interval between measurements is 6 h, but the minimum required by the standard is one measurement every hour.

• There is no synchrony in data logging between dataloggers. That is, records between different dataloggers are taken at a completely different time.

• Records are not taken at constant time intervals on the same datalogger because when the measurements are downloaded manually (approximately every 3 months), the data logging restart begins at a different time for each datalogger.

• There are periods with missing values due to run out of batteries or because some dataloggers had to be moved elsewhere temporarily for short periods of time.

These limitations are generally encountered in similar microclimate monitoring studies and, therefore, are not unique to this dataset. For example, we observed analogous problems in historical datasets from European museums partners of the CollectionCare project like the Alava Arms Museum (Spain), National Historical Museum (Greece), The Ethnographic Open Air Museum of Latvia or The Royal Danish Collection—Rosenborg (Denmark) [47]. These historical data were recorded over the years with different data recording devices and methodologies. It is necessary to address these limitations by reconstructing the time series in order to obtain a data matrix that allows application of the standard.

The proposed reconstruction methodology consists of three steps (Fig. 5). Firstly, data are interpolated to regularise the time instants of the measurements in order to obtain all synchronised records, which are arranged as a matrix of time instants in rows by dataloggers in columns. Secondly, the missing data from the regularised rows (i.e., time instants) are imputed. Thirdly, records are estimated every hour with a linear interpolation added to a Gaussian noise (i.e., normal statistical distribution). These steps have been automated by implementing them in a Python script, as described below.

Fig. 5

Methodology followed for data reconstruction. Step 1: regularisation by means of interpolation, Step 2: Imputation of missing data, Step 3: Imputation with linear interpolation and Gaussian noise

Step 1: Regularisation by means of interpolation

The initial frequency of data collection was one record every 6 h, but since the data from the dataloggers were downloaded manually, each time they were downloaded there was a change in the regularity of the records for each datalogger. In order to obtain a matrix with all records at the same time, reference times were set according to those most frequently encountered for the records. The following procedure is applied: if there is a record 180 min before or after that reference hour, it is interpolated between the two records. For example, if one of the reference times is 7:00 and the d01 datalogger has two consecutive records, one at 4:15 and the next at 10:15, since there is a record less than 180 min before the reference time, it is linearly interpolated between the two records. If this condition is not satisfied, no value is imputed in this step but in the next one by means of PCA (Principal Component Analysis). A linear interpolation was chosen because the intraday variability of the time series is low (i.e., daily cycles are poorly marked compared with seasonal variability), as shown in the results of the research. The intraday variability was checked visually by plotting the complete time series and observing the range of daily cycles, compared with seasonal variations. Moreover, the difference between one observation and the next is, at most, 6 h, most of them being at a shorter distance.

Once this procedure is completed, a matrix with records every 6 h is obtained, which contains missing values.

Step 2: Imputation of missing data

The values missing after row regularisation have to be imputed. For this task we decided to use a PCA-based technique, in particular making use of the Trimmed Scores Regression [PCA-TSR] method [48]. This approach is suited to the nature of this process because the time series are rather parallel and multivariate statistical models based on latent variables take advantage of the correlation structure to capture the underlying patterns, allowing accurate predictions in this case for the missing RH measures [15, 34, 49,50,51]. Starting from this matrix with variables (i.e., data from the different dataloggers) highly correlated, the underlying correlation structure was used for imputation, achieving a matrix without missing values, but with a low sampling frequency of one record every 6 h.

Step 3: Imputation with linear interpolation and Gaussian noise

In this last step, we convert a matrix with one record every 6 h to one every hour. The main objective of increasing the frequency of observations is to obtain time series with the appropriate characteristics for applying EN 15757:2010.

Firstly, a linear interpolation is performed between two consecutive records by imputing five new ones. For example, if there are records at 2:00 and 8:00, five new ones are imputed at 3:00, 4:00, 5:00, 6:00, and 7:00. Following the logic established in the first step, linear interpolation has been considered a better option than quadratic or cubic interpolation, given the low intraday variability of the time series. Results of the validation of the methodology have also shown that linear interpolation yielded lower predictive error than the other options.

Once the new data have been imputed by linear interpolation, a normal disturbance of mean 0 has been summed to each imputed value. The variability (standard deviation) of this normal disturbance will be different for each datalogger. This criterion is necessary because, for example, as indicated below in the “Results and discussion” section, a datalogger located in the storage room will capture less variability than one installed in a showroom, as it is subject to fewer factors that can alter microclimatic conditions (e.g., air flows, people, radiation, etc.). After considering several possibilities to determine these standard deviations σ_i, the following procedure has been chosen: firstly, for each time series, we calculated the differences in RH between one instant and the previous one. This step is usually referred to as differencing of a time series. Secondly, the standard deviation was calculated for  these differences. It has been checked that this variance is approximately constant along the time. Thirdly, a multiplying factor was established so that the disturbances are not excessively large, which has been set at 0.2 after several trials. This standard deviation was used for the normal disturbance of mean 0.

Methodology validation

The reconstruction methodology is validated using a dataset with no missing data. Thus, the results can be contrasted with real data and the fitting error of the reconstruction can be estimated. The nature of the data for validation should be as close as possible to the historical time series, i.e. the data on which the reconstruction methodology is to be subsequently applied, so that extrapolating the conclusions of the validation process is as reasonable as possible. Therefore, the dataset chosen for this purpose is the CC sensors RH time series of the same museum, which present similar variability patterns. This dataset is composed of 7 time series instead of 9, namely the sensors of pairs d02, d04, d05, d06, d08, d09, and sensor A. This could be a drawback, since having insufficient columns in the data matrix could lead to a PCA model for the reconstruction not capturing properly the latent structure of the process. Nonetheless, this is not considered to be a major problem for validation, as this reduction in columns would generally lead to worse validation results, so the results would not overestimate the model goodness-of-fit.

Prior to validating, all rows with missing data were eliminated from the dataset since it should not have a single missing value. The next steps consist of doing the reverse process of reconstructing the time series so that they can then be reconstructed. For this purpose, the first step is to obtain a dataset with one record every 6 h, eliminating the rest of rows. The second step is to eliminate values (i.e., to generate the missing data) that will then be imputed with the PCA-TSR method. Different causes of missing data give rise to different missing data structures and some imputation methods are not necessarily equally as suitable as others. To make the validation results as comparably as possible to the reconstruction of historical data, the missing data were generated (i.e., values were eliminated) by copying the missing data structure of the historical time series dataset according to the Fig. 3 correspondences. In this way the validation dataset is appropriate to apply the methodology from step 2. It should be noted that step 1 (i.e., regularisation by interpolation) cannot be applied due to the difficulty of simulating the necessary conditions to be applied because the initial dataset was already regularised with an hourly record.

Once these steps were applied, the time series were reconstructed with the proposed methodology. Validation was performed by comparing the time series reconstructed in the different steps of the methodology with the observed time series.

The first to be validated is step 2, missing data imputation using the PCA-TSR method. In the validation process of this step, we obtained the number of principal components in order to achieve the best results. Finally, step 3 of the reconstruction was validated by applying a Gaussian noise computed from the standard deviation of the differenced time series and using a multiplying factor of 0.2. In addition to the linear interpolation proposed in the methodology, the results of the adjustment with quadratic and cubic interpolation are shown in order to select the most appropriate interpolation for the methodology.

The comparison is made from two complementary perspectives. The first and most important, by means of three statistics that quantify the imputation error: mean error (ME), mean square error (MSE) and root mean square error (RMSE). These three statistics are also used for the comparison between the three proposed interpolation methods. The second is a visual analysis by superimposing the time series of the predicted and observed values superimposed. The contrasted observation of both time series facilitates the understanding of results and may indicate errors in how the PCA model captures some patterns.

Standard EN 15757:2010

As stated in the introduction, the European standard EN 15757:2010 concerns conservation of objects made of organic hygroscopic materials and recommends to focus the priority on the historic climate of the object to limit mechanical damage induced by strain–stress cycles assuming that the object has been “acclimatised” to this historic climate.

The standard establishes control bands for each time series of RH and T, so that observations outside these bands are considered warning signals that require special consideration. By default, these fluctuations comprise 14% of the data and are determined as the 7th and 93rd percentiles of the short-term fluctuations.

Due to the method used for the calculation of the bands, they might be considered too strict and of little interest if the fluctuations are small. Therefore, the standard states that “if the above procedure determines that RH fluctuations depart by less than 10% from the seasonal RH level, the calculated limit is considered unnecessarily strict and can be disregarded. The 10% RH threshold can be accepted instead under the responsibility of a qualified conservation professional.”

Taking into account that the main objective of this work is to reconstruct missing values in time series and to validate the methodology through the effect on the width of the safety bands, the more relaxed application of the 10% rule has been discarded. The primary objective of the work is not to diagnose the environment in the museum, but to demonstrate that it is possible to reconstruct the missing data in order to be able to perform, in the future, such an evaluation.

This band considered as “safe” according to the standard comprises a period of 12 months, but in order to be calculated, it requires an additional month, 13 in total. These additional 30 days are required to compute the moving average, which is necessary for the calculation of the band itself. In addition, as already noted, the minimum sampling rate of records must be one record every hour.

This safety band is established by means of an upper and a lower limit. These limits are first established by calculating the moving average of the time series. Then, the moving average is subtracted from the time series to obtain short-term adjusted time series. Finally, the limits for the seasonally adjusted time series are calculated. The lower limit corresponds to the 7th percentile and the upper limit to the 93rd percentile. Therefore, regardless of the time series, 14% of the fluctuations are considered as warning signals that need to be examined, since their status as potential conservation risk is always dependent on further analysis. To obtain the alert limits for the original time series, it is necessary to add to the moving average the value of the lower and upper limits.

The width of the safety bands, i.e., the distance between the upper and lower boundary, was regarded as a statistic used for comparison of microclimatic variability between dataloggers [31].

It is important to note that the standard EN 15757:2010 is currently under revision. A recent study [41] has discussed some limitations and proposed certain improvements. As far as we know, the same group of experts who developed this standard is currently working on a draft for a new version.

Analysis of time series reconstruction

To evaluate the results of the reconstruction of historical data, the original time series were compared at the different steps of the reconstruction. Since the main interest of the comparison is to study how the variability changes at each process step, taking into account that the objective is the application of EN 15757:2010, the safety bandwidths are used as a statistic for comparison. Therefore, in the analysis of the time series reconstruction, the purpose of applying the standard at each process step of the reconstruction is not to discuss the microclimate in the museum, but to compare the different phases of the reconstruction and to obtain conclusions about the proposed methodology.

For each datalogger there are 5 time series to which the standard has been applied:

• Raw: Raw time series without any type of pre-processing.

• Step 1: Regularisation (raw); the time series regularised by interpolation (step 1 of the reconstruction). In order to be able to apply EN 15757:2010, all observations with missing data were discarded. The frequency is one observation every 6 h, lower than the requirement of the standard.

• Step 1: Regularisation (interpolation); the time series regularised by interpolation (step 1 of the reconstruction). The difference with the previous time series is that, in order to be able to apply EN 15757:2010, in this case the missing data are imputed by linear interpolation. Again, the frequency is one observation every 6 h.

• Step 2: Imputation; the time series imputed by PCA (step 2 of the reconstruction). The frequency is one observation every 6 h.

• Step 3: Interpolation with Gaussian noise; the time series are composed of hourly observations, estimated using linear interpolation and added Gaussian noise (reconstruction step 3).

In addition to comparing the width of the safety bands, other metrics have been defined to study how the imputation of step 2 of the reconstruction affects the variability of the reconstructed time series. In particular, the aim is to assess how the proposed methodology impacts on the application of EN 15757:2010. Three additional metrics per time series are established:

The ratio of missing observations in step 1 is expressed as missing_ratio (Eq. 1).

$$mis{sing}_{ratio}=\frac{total\_missing\_observations}{total\_observations}$$

(1)

This ratio will be compared with the results of the step 2 allocation. The parameter out_ratio (Eq. 2) is the number of imputed observations in step 2 that are out-of-band divided by the total number of missing (and imputed) observations in step 2. The imputed out-of-band observations are determined by applying EN 15757:2010 with the reconstructed time series, i.e., to check how many are below the 7th percentile or above the 93rd percentile of all observations in the reconstructed time series. The aim is to assess to what extent the missing observations corresponded to out-of-band observations.

$$ou{t}_{ratio}=\frac{imputed\_out\_of\_band\_observations}{total\_missing\_observations}$$

(2)

In order to asses the impact of out_ratio in the imputation, it is necessary to relativise it in terms of the missing_ratio. This relationship is expressed by the relative_outs metric (Eq. 3), defined as the imputed out of bands fraction of the total values, as follows:

$$relative\_outs =ou{t}_{ratio}\times missin{g}_{ratio}$$

(3)

For example, let us assume that, after step 1, one of the time series is composed of 1000 observations, 50 of which are missing. Therefore, missing_ratio = 0.05 (Eq. 1). In step 2, the 50 missing values are imputed, and let us also assume that 33 of these were found to be outside the safety bands after applying EN 15757:2010 to the reconstructed time series. In this case, out_ratio = 33/50 = 0.66 (Eq. 2). To asses the impact that imputed out-of-band observations have on the entire time series, out_ratio is relativised with missing_ratio by means of relative_outs (Eq. 3). In this case, relative outs = 0.05 × 0.66 = 0.03. The same result can be obtained in abbreviated form by dividing the imputed out-of-band observations by the total observations: 33/1000 = 0.03. Thus, from this example it could be concluded that, although the imputation was made on mostly out-of-band observations given the relatively high value of out_ratio, the final impact on the application of the standard is small because the number of missing observations was low.

Regarding the impact of the presence of out-of-bands observations on the evaluation of risks for the conservation of artworks, it is always a question when applying these standards that one single transgression (i.e., a value out-of-bands) might cause physical damage, e.g., cracking of a paint on wood, in case the strain is sufficient to crack the paint [52]. Thus, in principle, there should be 100% conformity, and it is not so clear if 5% transgression is worse than 1% transgression. The 1% could be a sudden abrupt peak of RH that might result in more strain and damage than considering the set of 5% out-of-band values if they do not depart too much from the bands. By keeping in mind this consideration, regarding the reconstruction of time series, if the change from the real to reconstructed data set is from 99 to 100% conformity, this could imply that the very real risk that damaged the object is not observed. This is anyway a problem with missing data sets, and the reconstruction should improve the situation.

Results and discussion

Firstly, the results corresponding to the validation of the methodology on the time series from CC sensors are presented. Secondly, the results of the reconstruction of the historical time series are discussed. Thirdly, the microclimate is characterised and dataloggers with a similar bandwidth are clustered once the standard is applied.

Validation

To obtain a dataset without missing values, 1300 rows were eliminated from the original 9504 (13.7%), resulting in a dataset with 8204 rows. Subsequently, one record of every 6 rows was kept, eliminating the rest and leaving the dataset with 1368 rows. Next, the missing data structure of the historical time series was transferred to the CC sensors time series, generating 1051 missing data from a total of 9576 observations (1368 rows × 7 columns), i.e. 11% of the total. These missing data are distributed in 4 of the 7 dataloggers in the following proportions: d06, 35.4%; d05, 19.1%; d04, 17.0%; and d02, 5.1%.

Step 2 of the reconstruction was carried out by imputing the 1051 missing data. Results of the six imputations performed with 1 to 6 principal components are shown in Table 1. The best results correspond to the model with three principal components. Results show a very satisfactory fit of the imputation model, appreciating that the mean error is low (ME = 1.00%) and that most errors are close to the observed value (RMSE = 1.33%). This is a very good fit because RH ranges from 1 to 100%. As shown in Fig. 6, the predictions in different time series follow the observed values quite closely.

Table 1 Comparison of the three statistics used to measure  the fit of imputations with models built with different number of principal components: mean error (ME), mean square error (MSE) and root mean square error (RMSE)

Fig. 6

Imputation results from Step 2 of the reconstruction. The imputed values (orange) and observed values (blue) of different time series over the 13 months are compared. The black colour corresponds to the d08 time series, with no missing data, used in the graph only for visual continuity. Fragments in the figure correspond to: (i) d05, (ii) d06, (iii) d02, (iv) d06 and (v) d04

In step 3, data were imputed by interpolation and addition of Gaussian noise in order to move from a matrix with one record every 6 h to a matrix with one record every hour. It was checked that quadratic and cubic interpolation gave worse results than linear interpolation. Different multiplying factors were tested with the results in Table 2.

Table 2 Comparison of the three statistics used to measure the fit of imputations with different values for the multiplicative factor: mean error (ME), mean square error (MSE) and root mean square error (RMSE)

According to the result, the minimum imputation error with respect to linear interpolation is achieved by not applying Gaussian noise. However, it was decided to add the Gaussian noise and consider a factor of 0.20, even though it leads to a small increase in the error between the predicted and observed values, since it reflects the variations in RH between hours more naturally (Fig. 7). The increase in error is ME = 0.09 and 0.11 for ME and RMSE, respectively.

Fig. 7

Imputation results from step 3 of the reconstruction with zoom for detail. The imputation values from step 2 (orange), step 3 (red) and the observed values (blue) of the historical time series of the d02 pair over 24 days are shown for comparison

The statistical distribution of residuals from steps 2 and 3 is shown  in Fig. 8. Certain positive asymmetry is observed in the first case departing from the normal distribution, indicating certain bias in the imputation. This issue can also be derived from Fig. 6, where it can be observed that there are segments in the time series d04 and d05 where the imputed values are systematically lower than the observed ones. In the second graph, which corresponds to the residuals in step 3, the statistical distribution is quite close to a normal model.

Fig. 8

Histograms for the residuals from step 2 (left) and step 3 (right)

Reconstruction

Following the procedure described in the section “Reconstruction methodology”, and given that we start from time series without synchrony between them nor with records taken at regular hours, we first determined the reference hours at which to regularise the data. It was decided to establish these times at 1:00, 7:00, 13:00 and 19:00 after observing that these are the most frequent times at which there are records. Once the interpolation has been carried out, the amount of missing values in the matrix is 14%. Dataloggers yielding the largest amount of missing data are d03 (34.8%), d06 (30.6%) and d07 (23.5%). The imputation, performed with the Trimmed Scores Regression method, was applied with three principal components. This number was determined from the evaluation results.

To obtain the matrix with an hourly record, the variability of different time series was studied to assess which one was more appropriate to take for the generation of normal disturbances (Table 3). Such variability was quantified as the standard deviation (% RH) of the differenced time series (i.e., one value minus the one previously observed). These time series are those corresponding to the CC sensors with data removed to emulate 6-hourly recordings, and the Lascar dataloggers with a record every 6 h. It is observed that the variability of the CC sensors with respect to the Lascar dataloggers is much lower, discarding its use to impute the historical time series. Finally, it was decided to use the historical time series to determine the standard deviation from the differences between records. Despite the lower frequency of records (i.e., one record every 6 h), it was obtained in the time series of the current dataloggers that the frequency does not change the standard deviation of the differences considerably. The different steps of the reconstruction can be seen in Fig. 9.

Table 3 Standard deviation (% RH) of differenced time series for historical and CC sensors (frequency of one record every 1 h and every 6 h)

Fig. 9

Visualisation of the three-step reconstruction for the time series from datalogger d06: step 1, regularisation (blue); step 2, imputation (orange); step 3: interpolation with Gaussian random noise (red)

Analysis of the reconstruction process

Figure 10 shows the comparison, for each datalogger, of the band sizes corresponding to the five different time series according to the steps of the reconstruction procedure described in the Analysis section. It is observed that in all dataloggers the bandwidth increases when the reconstruction process is finished.

Fig. 10

Comparison of the band sizes resulting from the application of EN 15757:2010 for each part of the reconstruction process. The reference bandwidth is step 1: regularisation (interpolation). For this reason, in all dataloggers this step has a change ratio = 1. All other band sizes are expressed with respect to it. The y-axis has been constrained to the range 0.85 to 1.35 to better appreciate the differences in band sizes

In dataloggers d01, d08 and d09, the bandwidth barely changes in the reconstruction steps. This is due to the low amount of missing data (< 1%), so the imputation has almost no impact. The only relevant change is the increase of the band in step 3 due to the higher frequency of observations.

It turned out that the bandwidth of step 1: regularisation (interpolation), used as a reference, was the smallest of each datalogger. The use of linear interpolation as an imputation method does not increase the variability, but the number of observations in the time series is greater. Therefore, the overall variability of the process decreases as much as the ratio of missing data in the time series. This issue is reflected in Fig. 10 by comparing the bars of step 1: regularisation (raw), especially in dataloggers d03 and d06, which yield the highest amount of missing data (34% and 30%). As a result of this reduction in variability, the 7th and 93rd percentiles, which establish the upper and lower limits of the norm, generate narrower safety bands. The straightforward consequence is that the imputation of missing data by linear interpolation always implies narrower bandwidths than the actual ones. This method may be valid for time series with a low ratio of missing values, but in case of high proportion, the application of EN 15757:2010 may lead to erroneous conclusions.

Another important result is the difference between dataloggers with respect to the band size in step 1 (raw) and step 2. It can be observed that dataloggers d02 and d03 underwent a considerable shift in bandwidths in step 2 with respect to step 1 (raw). This shift is noticeably larger than in the case of d06 and d01. This is due to the periods with missing data. When no values are missing in a period of high variability (out_ratio > 0.14, i.e. higher than the ratio of out-of-band observations according to EN 15757:2010), the application of the standard to the time series with imputed data becomes in a larger width of safety bands. This is the case for d02 (Fig. 11a), d03 or d04 dataloggers. On the contrary, when the missing data occur in a period of low variability (out_ratio < 0.14), the application of the standard to the time series with imputed data leads to a smaller band size. This is the case for d06 (Fig. 11b) or d01 dataloggers. This occurs because the limits (bands) of the norm are calculated according to the 7th and 93rd percentiles, so periods with higher variability expand these limits, generating larger band sizes, while periods with lower variability reduce them, generating smaller band sizes.

Fig. 11

Reconstructed time series from datalogger d02 (a) and datalogger d06 (b). Periods highlighted in light brown correspond to the imputed segments. Below each time series, with black circles, out-of-band observations are randomly displayed on the Y-axis in order to easily visualise the density of the points as a function of time (X-axis)

Nevertheless, the ratio of out-of-band observations (out_ratio) is not the only factor explaining the change in variability after imputation. The impact of this ratio is greater or lesser depending on the ratio of missing observations (miss_ratio). To assess the relationship between these two factors and the change in the variability after the reconstruction, we used the relative outs and the difference in band size between step 1: interpolation (raw) and step 2: imputation (Table 4). The Pearson correlation coefficient between relative outs and the difference between these steps is 0.81, which implies a positive correlation between the imputed out-of-band observations and the change in band size after imputation. This relationship can also be observed in Fig. 12.

Table 4 Application of the 3 metrics miss_ratio, out_ratio and relative_outs

Fig. 12

Scatter plot showing out-of-band imputed observations versus the difference between band size (values from the table above). The dashed line represents the trend. A strong correlation is observed

The reconstruction procedure presented here can be used to compute the limits and band sizes according to EN 15757:2010 when dealing with time series that do not meet the requirements of the standard (missing data, low frequency, etc.). This approach could be used, for example, to make a comparison between years. However, constructing the charts based on the raw data can lead to smaller or larger band sizes depending on the amount and distribution of missing data. By contrast, the use of linear interpolation for data imputation leads to systematically smaller band sizes.

For many museums, the methodology proposed here can be useful aimed at managing historic thermo-hygrometric data. For the practical application, a simple automatized procedure or computer program would be of interest for handling all the difficulties arising with these varying data sets. This issue will be tackled in further developments of our research group. Anyway, the scripts developed in the present work will be available for other researchers upon request.

Comparative analysis of historical and CollectionCare sensors time series

Figure 13 compares the historical time series, already reconstructed (left), with respect to the time series from CC sensors (right). The first noticeable difference is the reduction in variability, observable through the smaller band size after applying EN 15757:2010 to the CC sensors time series. The CC sensors band sizes are 21% smaller on average than those of the historical time series. Specifically, for each datalogger, this reduction is 32% (d08), 23% (d04), 20% (d05), 20% (d09), 18% (d06), and 16% (d02).

Fig. 13

Width of safety bands (i.e., difference between upper and lower limits) for each datalogger according to EN 15757:2010. The historical dataloggers (left) are compared with the CC ones (right). Blue bars correspond to the dataloggers installed in the north (new) building; in orange, the dataloggers deployed in the south (old) building

The reasons for this reduction are somewhat uncertain. It is likely that this difference is due to the effect of external climatic conditions, which have a natural incidence on the building and because the air conditioning system obtains air from the outside. It would therefore be appropriate to check whether this difference in the two periods also exists outside the museum. Unfortunately, the data available to study this relationship are very limited. However, data from two weather stations have been used to study this issue. The first one belongs to the Spanish State Meteorological Agency (AEMET in its Spanish acronym), it is located 5.9 km away [53] and provides daily averages of T at least from 2018. However, RH data are not available. On the other hand, the second station is referenced as C040 [54] and it belongs to the Basque autonomous government, located in the same city of Alava, 2.2 km from the museum, with T and RH records every 10 min. Unfortunately, data are only available from January 1st 2020, therefore it is not possible to make a comparison between the two periods analysed in this research. The three locations (museum, AEMET station, and station C040) are not expected to present major climatological differences due to their proximity and the absence of orographic differences.

The first station was used to study whether the dispersion of the outdoor T is similar to the RH inside the museum. Data from the second station will be used to determine if the relationship between T and RH is robust enough to conclude that the outdoor variability of T can be equated to RH.

The T at AEMET station during the two analysed periods shows a higher variability in the historical time series (April 1st 2018 to May 1st 2019) than for the CollectionCare ones (January 1st 2021 to February 1st 2022): while the interquartile range of the former is 9.75 °C, that of the latter is 9.30 °C. However, this difference is relatively small. On the other hand, although the relationship between T in both stations is comparable (Pearson’s correlation coefficient is 0.98), and it could be inferred from this that the changes in RH in both stations are also expected to be very similar, it is not possible to establish a direct relationship between T and RH. In the 24-month period between December 1st 2020 and December 1st 2022, the correlation coefficient between T and RH recorded by station C040 is − 0.63.

Based on the results of the limited data available, the stated hypothesis (i.e., that the changes in RH inside the museum in the two periods is a product of external conditions) seems reasonable although it cannot be verified. This hypothesis will be investigated in further studies.

According to Fig. 13, there is also a noticeable disparity in RH variability between the dataloggers in the northern part of the museum (the new building) and the southern part (the historical palace) (Fig. 13). This discrepancy is also reflected by the standard deviation of the differenced time series (i.e., distribution of RH differences from one instant (logger recording) to the next, see Fig. 14). These two approaches (i.e., the comparison of band sizes according to the norm and the standard deviation of the differenced time series), are able to draw a clear difference between the RH variability of one building and the other, given that the dataloggers appear clustered far apart in Fig. 14 by using just the standard deviation.

Fig. 14

Scatterplot of the standard deviation of the differenced time series (i.e., differences between the RH of one instant (logger recording) and the next) for the historical Lascar dataloggers with respect to the CC sensors. The orange circle surrounds the dataloggers installed in the old building, while the blue circle highlights those deployed in the new building. The standard deviation was calculated from the reconstructed time series

The reason for this dissimilarity in variability is difficult to determine because the two parts of the building differ not only in their construction but also in the HVAC systems in operation. Nonetheless, it was observed that the difference in variability occurs over short periods of time (between one and a few days) and homogeneously throughout the year, which leads to the hypothesis that this disparity might be a result of the different characteristics in the construction of the buildings. Actually, if the reasons for the disparity would be the different performance of the HVAC systems used in each building, it is likely that such differences in variability would be more explicit in some seasons depending on their use (e.g., dehumidifiers in the historic building are used in autumn and winter, but not in summer). This difference in variability from one part of the museum to another is taken into consideration by the museum curators for the mobility of the artworks, especially when they are taken out of the storage room where the d02 dataloggers are located [55].

Conclusions

This research deals with a very common problem in the field of preventive conservation and scarcely addressed in the literature: the proper data pre-treatment of data for microclimate analysis and application of standards, in particular EN 15757:2010. By considering the main problems in the quality of the measurements collected by the monitoring systems regarding the application of EN 15757:2010 standard, a procedure is proposed for the reconstruction of the time series of the datasets. There is an interest in making such a data handling/“correction” procedure automated and available to museums in a format that they would be able to apply to their datasets, in case of having the appropriate resources. This issue is out of the scope of the present work, but it will be tackled in further deployments of our research group.

The proposed procedure has yielded satisfactory results. Firstly, the validation of the methodology on the CC sensors time series indicates a reasonable correspondence between observed and predicted values, which indicates a good performance in the method applied for the reconstruction of historical hygrometric time series. This issue has been verified both by visual analysis and with the statistics measuring the fit of the models used in steps 2 and 3 of the methodology. Secondly, by reconstructing the historical time series and applying EN 15757:2010, important general conclusions have been derived:

• The direct application of the standard on time series that do not meet the requirements of the norm may lead to erroneous conclusions that depart from reality.

• This drawback can be avoided by means of appropriate time series reconstruction that addresses the various problems of low-quality datasets, particularly in the case of long periods of missing data.

• An inappropriate method of data imputation can lead to larger or smaller bandwidths than actual when applying EN 15757:2010 depending on the amount and distribution of missing data. Particular attention should be paid to periods with noticeable changes in variability.

• In time series with long periods of missing data, imputation should not be performed by the unique application of interpolation, as it always leads to a significant reduction in band widths regarding the variability of the time series.

The reconstruction of historical time series has allowed the application of the EN 15757:2010 standard as well as the characterisation of the microclimate. It was possible to clearly identify the difference in variability between the two museum buildings, which must be taken into account in the mobility of artworks for proper conservation. On the other hand, the reconstruction of time series has also been useful to detect a significant shift in variability between the microclimatic conditions of the museum between the first and second monitoring campaigns and to establish hypotheses about this change.

In future research, the procedure should be applied to other case studies. In addition, alternative methods to the one applied in step 3 should be worked out for improving the proposed methodology.