Skip to main content

Tensor decomposition for painting analysis. Part 1: pigment characterization


Photo-sensitive materials tend to change with exposure to light. Often, this change is visible when it affects the reflectance of the material in the visible range of the electromagnetic spectrum. In order to understand the photo-degradation mechanisms and their impact on fugitive materials, high-end scientific analysis is required. In a two-part article, we present a multi-modal approach to model fading effects in the spectral, temporal (first part) and spatial dimensions (second part). Specifically, we collect data from the same artwork, namely “A Japanese Lantern” by Norwegian artist, Oda Krohg, with two techniques, point-based microfading spectroscopy and hyperspectral imaging. In this first part, we focus on characterizing the pigments in the painting based on their spectral and fading characteristics. To begin with, using microfading data of a region in the painting, we analyze the color deterioration of the measured points. Then, we train a tensor decomposition model to reduce the measured materials to a spectral basis of unmixed pigments and, at the same time, to recover the fading rate of these endmembers (i.e. pure, unmixed chemical signals). Afterwards, we apply linear regression to predict the fading rate in the future. We validate the quality of these predictions by spectrally comparing them with temporal observations not included in the training part. Furthermore, we statistically assess the goodness of our model in explaining new data, collected from another region of the painting. Finally, we propose a visual way to explore the artist’s palette, where potential matches between endmembers and reference spectral libraries can be evaluated based on three metrics at once.


Light-sensitive pigments may irreversibly change the appearance of an artwork after exposure to light. For this reason, special research and efforts are dedicated to keep in control the exhibition conditions and choose the optimal lighting policy. While traditionally, broad categories of materials were established depending on assumed light sensitivity, more recently microfading has come into use to make more individual assessments. In this way, less sensitive objects may be displayed for longer, while the most sensitive objects will be better protected.

Furthermore, the degradation of pigments might interfere with the accurate recognition of materials in paintings. This is because fading might cause the disappearance or transformation of some highly sensitive elements in the materials. In such cases, state-of-the-art non-invasive optical techniques used for the task of pigment identification, such as reflectance imaging spectroscopy can be helpful to find residual pigments, by sensing their known responses to certain incident light [1]. Nevertheless, knowledge of the pigments’ fading behaviour can contribute towards their recognition.

The sensitivity to light applies not only to pigments, but to all colorants, dyes included and other materials, such as wood. This has a practical implication and general applicability in a wide range of fields beyond cultural heritage, such as printing, automotive industries, fashion and textile, architecture, etc. Because photodegradation is such a widespread concern, it has even been approached by the computer graphics community in an attempt to synthesize the spectral aging effects of multi-layered surfaces [2].

In this first part of a two-series article, we are proposing an approach based on microfading spectroscopy, coupled with multivariate data analysis to model pigment fading effects along the spectral and temporal dimensions. We show the performance of our model on a pastel painting, where specific research questions are answered.

Related work

Spectroscopic analysis for pigment identification

Spectral response of a pigment to electromagnetic radiation represents a distinctive signature. For this reason, the spectroscopic signal is often studied for pigment identification tasks. While it is fruitful to operate in the reflectance domain [3], some works in the literature point out that the absorption spectra [4, 5] or the first-derivative of the reflectance might be more helpful for discriminating pigments with similar composition [6,7,8]. Johnston and Feller [4] used the additive property of the constituents of a mixture in the absorption domain to subtract the absorption spectra of a pigment before and after aging. The difference absorption spectrum preserved the peaks characteristic of a certain red lake pigment that were attenuated in the individual absorptions. Fonseca et al. [7] designed a decision support system to distinguish between plant (madder root) and animal (cochineal insects) red lake pigments based on inflection points of the reflectance curve and its first-order derivative. Recently, Gabrieli et al. [8] used derivative analysis to identify the palette of Rembrandt’s “Night Watch”.

Given a spectral library, i.e. a dataset of reference pigments, it is possible to assess the similarity between a standard specimen of a pigment and the measurements in an artwork, by comparing their reflectances as vectors in the spectral space. The spectral similarity can be computed for reflectance or its variants: absorption and first derivative [8]. While there are various distance metrics that can be used, with spectral angle mapper being the most common [8], there are works in the literature that argue for the higher performance of spectral correlation mapper [9] and Kullback Leibler pseudo-divergence [10]. Nevertheless, to find good matches, it is important to design the spectral libraries in such a way that they resemble the target in terms of chemical composition, concentration and binding media [11].

Pigment unmixing

In the case of image spectroscopy also known as hyperspectral imaging, where the spectra is documented at every spatial location of an artwork, it becomes feasible to map the existence of a pigment in two dimensions by thresholding the spectral similarity between a reference material and the measured painting [8]. In addition, hyperspectral imaging also allows for pigment unmixing, where traces of an endmember and their abundance are found at every spatial location. The pigment identification based on the spectral signal can be affected due to mixing mechanisms. Thus, it is a common practice to perform pigment unmixing before computing spectral similarities with a databases. Given a collection of measurements, pigment unmixing finds a set of endmembers, and the abundance of all endmembers in each sample. An endmember is a pure pigment that forms the basis of a palette in a painting, implying that all the other signals in the painting can be represented as a mixture of the endmembers. Nonetheless, the task of finding the endmembers is not a trivial one because of the nature of mixing mechanisms. There is a lot of literature for material unmixing in the remote sensing field, where hyperspectral imagery is used as well [12,13,14]. However, the spatial resolution in remote sensing applications is lower than close-range imaging as is the case in the cultural heritage field. In the former case, optical mixing occurs, at the sensor level, where a pixel is formed by a linear combination of the radiances of endmembers. Hence, linear unmixing models work acceptably for the remote sensing field [13] and examples of known linear unmixing methods are pixel purity index (PPI) [15], N-FINDR [16], vertex component analysis [17]. PPI is the default unmixing method implemented in the Spectral Hourglass Wizard functionality of the ENVI software [18], which gave good results for painting analysis while complemented by experts’ input [8] or automated algorithms to find meaningful spectral features in the endmembers’ reflectance [11].

When it comes to close-range reflectance image spectroscopy, intimate mixing and layered mixing effects are of higher concern. Intimate mixing refers to the heterogeneous chemical composition of a material, while layered mixing refers to the blended stacks of materials, where due to translucency, the material in the background has a contribution to the surface material that is recorded at pixel level. In these cases, non-linear unmixing models are more appropriate to find the endmembers. Kubelka-Munk [19] is an example of a non-linear model, that describes the physical interaction of turbid media and it characterizes a medium with two optical coefficients, absorption and scattering. Unmixing in the Kubelka-Munk space has proven especially effective for the analysis of pigments [20,21,22]. While physically-based models such as Kubelka-Munk are accurate and offer an analytical solution to the radiative transfer equation, several data-driven methods for non-linear unmixing were shown to be successful as well. For instance, Kleynhans et al. [23] trained a convolutional neural network to associate 1D spectra of pigments to their labels, given a dataset of illuminated manuscripts, scanned with a hyperspectral camera and with known chemical composition.

A common pre-processing step in the pigment unmixing pipeline is represented by dimensionality reduction [24], typically achieved by principal component analysis, non-negative matrix factorization or clustering techniques such as k-means and t-SNE [25]. In several studies, tensor decomposition [26], in particular Parallel Factor Analysis (PARAFAC) [27], also known as canonical decomposition [28], was employed to achieve dimensionality reduction, clustering and classification of hyperspectral images in a trilinear fashion without resorting to flattening to a 2D array [29, 30]. The main advantages of tensor (i.e. multidimensional array) analysis over matrix processing consist in the preservation of data structure, the ability to retrieve latent variables in the multi-linear space and the uniqueness of the base factors [31]. Thus, operations in the tensor space have a wide applicability in signal processing [32] and computer vision tasks [31]. Moreover, tensor computation was often been coupled with the sparse representations in dictionary learning methods towards speech recognition [33] and spectral unmixing [34].

In the field of chemometrics, numerous articles stand as evidence for the capabilities of PARAFAC [27] to extract unique pure spectra in a completely unsupervised manner from multi-way data such as fluorescence spectroscopy [35, 36], chromatography [37], laser-induced breakdown spectroscopy [38] and nuclear magnetic resonance spectroscopy [39]. For example, PARAFAC was effective in extracting the emission and excitation functions of the fluorophores and their concentration in a set of bispectral measurements of sugar samples [36].

Fading analysis of pigments

Beside their spectral reflectance, pigments are also characterized by their lightfastness properties. The behaviour of pigments to light exposure is used to judge the quality of a pigment by artists, paint manufacturers and art historians alike [40]. So much so, that in compendiums describing pigments [41,42,43], fugitivity to light is included to assess the permanence of a pigment. Starting as early as the second half of the nineteenth century [44], art historians together with conservation scientists designed fading experiments in a controlled environment that surpassed visual observation, and quantified the light-induced degradation of pigments in the color [45] and spectral domain [4]. For instance, Saunders et al. [45] prepared samples of organic pigments following historical recipes and aged them in an accelerated way by exposure to artificial daylight fluorescent lamps at an illuminance of 10,000 lux inside a chamber with constant temperature and relative humidity levels. They measured the samples colorimetrically in a gradual way, at every predefined interval of time during the fading procedure. The samples were exposed for 3000 h. The CIE \(\Delta E_{ab}\) color difference was computed to quantify the extent of the fading. In addition, the color change was compared to that of ISO blue wool (BW) standards 1–3 [46], that have increasing lightfastness. This controlled fading experiment showed that the most fugitive red lake is brasilwood lake followed by lac lake, cochineal, kermes madder and alizarin. Moreover, the authors discovered several important factors that impact the fading: the lake extraction method (raw, directly from insects as opposed to colored textile patches), lake precipitation methods (aluminum, aluminum with calcium carbonate and tin), pigment concentration, content of ultraviolet radiation in the light source.

Several studies investigate the relation between the photo-permanence of pigments and the spectral power distribution of the light source used in the fading process. Earlier on, Saunders et al. [45] showed that ultraviolet radiation strongly accelerates the fading pattern of organic pigments. The same finding was later confirmed by Hattori et al. [47] who noticed that ultraviolet radiation contributed to the fading of blue wool standards [46] with a higher share than visible light. As far as the visible range of the electromagnetic spectrum is concerned, the fading seems to be positively impacted by the amount of overlap between the absorption of the pigment and the spectral distribution of incident light [48]. Saunders et al. [48] were among the first to model the light-induced color change of red lake pigments as a function of the incoming wavelength. They adjusted the fading experiment in [45] by coupling the light source with seven broad-band filters with peak transmittances sampled every 50 nm between 400 and 700 nm. Because red lakes absorb more in the lower wavelengths, their photo-degradation increases when exposed to light in the blue side of the spectrum. Lerwill et al. [49] reiterated a similar experiment, with an improved setup based on narrow-band filter for separating the incident light signal and examining more pigments. While they agreed to the conclusion of [48], i.e. damage increases with increasing absorption, they found out that Prussian blue is an exception. In other words, Prussian blue is less affected by the light it absorbs and more by the light it reflects. The impact of the light source spectrum in accelerated aging experiments was acknowledged as well by Pintus et al. [50]. In an attempt to characterize the formation of cadmium soaps in oil paints, they discovered that LED light impinged more aging effects than a halogen lamp and a natural light source.

It is important to make the distinction between fading and other related terms such as light bleaching and photobleaching. In art conservation and material research, the first term is commonly adopted to refer to the process of yellowness and stain removal and it has been encountered in applications on aged paper [51], film artifacts [52], textile [53] and teeth enamel [54]. The second term is typically used in biology to refer to the process by which fluorophores are removed from a molecule through light excitation [55]. While photobleaching may relate to the light-induced chemical alteration of a colorant that affects its color attributes, its meaning is mainly tailored to the loss of fluorescence. For this reason, in the current study, we use the term fading which covers the general alteration of a colorant with exposure to any type of light.

Microfading analysis

Microfading is a fading process performed at a smaller scale, that concentrates light over spots of submillimetric size, thus minimizing the extent of the damage. For this reason, it can be considered an almost non-destructive technique and can therefore be applied on real artworks. Whitmore et al. [56] were the first to propose a microfading protocol for measuring real art objects. The non-invasiveness is ensured by terminating the procedure before a noticeable color difference or a maximum number of iterations is reached.

In the recent years, thanks to the accessibility of portable microfadeometers (MFT) such as the one proposed by Lojewski [57], there has been an increased interest to analyse the light sensitivity of pigments directly on artworks. Chan et al. [58] measured with the MFT the litographic print and the painted version (from 1910) of “The Scream”, both belonging to Munch museum. The most fugitive pigment in both artworks was revealed to be vermilion, with a light-sensitivity comparable to that of BW 1. The lightfastness analysis inspired the museum to design an intermittent display that lowers the display time for each artwork. Grimstad et al. [59] microfaded 27 paintings of Edvard Munch and similar to [58], they noticed that reds are the most sensitive colors, and all of these reds contain vermilion. Aambø et al. [60] gathered microfaded observations from 63 paper-based artworks in the Munch museum collection and discovered intricate light-induced change mechanisms for pigments with apparently the same color and/or chemical composition. In one case, for two reproductions of a woodcut plate supposedly made of the same material, two samples that appeared to have the identically same green color showed a different fading pattern. This implies the use of two different pigments.

While the study of lightfastness in the color domain gives a first intuition on whether similar materials were used in a sample, it is the knowledge of the spectral composition that adds more precision. Even when a set of measurements share one pigment, the photo-degradation can have different dynamics because of the rest of the materials in the mixture. In [60], the red colors were found to be the most fugitive, approaching BW 1 fading rate, a finding similar to that of [58 and 59]. While analytical techniques discovered traces of vermilion in all the highly sensitive reds in [60], vermilion wasn’t the only pigment in the composition. In addition, other red colors were found to be more light resistant (BW 2–3 category), even though they contained vermilion as well. Moreover, other red points, with both similar pigments (vermilion) and similar color sensitivity, manifested a slightly different change rate. In most of these studies, the microfading experiments are usually performed in the visible range of the spectrum, which better aligns with the current exhibition conditions in museums and galleries [61].

Fig. 1
figure 1

The diagram of our method. The module 1 is the core model, where we extract the endmembers and their fading rate with three-way tensor decomposition from a collection of microfading observations. Using this trained model, we verify how well we can explain new microfaded data (2). Also, we apply regression on the fading rates and extrapolate the behaviour for future temporal modelling (3). Finally, we spectrally compare our endmembers with databases of pigments to identify the materials used in the painting (4).


In many of the above mentioned related articles, the pigment’s composition is sometimes known apriori, and the microfading analysis is mainly performed with the purpose of implementing a more or less protective lighting policy for artworks displayed in museum exhibitions. To the best of our knowledge, there are no works that use the microfading analysis in the reverse way, i.e. to use the lightfastness behaviour aside from the spectroscopic features for characterizing and identifying an unknown pigment. In our work, we propose a linear unmixing method based on microfading single-point measurements, by modelling changes in the spectral domain and extracting endmembers using multivariate algebra analysis. We represent the light-induced measurements as a 3D tensor (samples*time step*spectra), where every sample is described by the spectral reflectance at different points in time during the photo-degradation process. With trilinear tensor decomposition, we are then able to recover chemically relevant endmembers that best explain the data, their concentration in every sample and the temporal change provoked by light exposure. While still a linear unmixing technique, by preserving the trilinear data structure, the tensor decomposition is able to recover more latent variables than a traditional bilinear setting [31].

Figure 1 shows an overview of our method. Given a set of spectral microfading measurements, we create a 3D array, where the first dimension corresponds to the number of observations, the second dimension corresponds to temporal changes and the third to the spectral reflectance. Then, we use PARAFAC in order to retrieve the loadings for each of the three dimensions, also called modes in our data representation. The PARAFAC analysis gives the pure spectra underlying the sample, the concentration of all pure components for each sample, and the degradation rate of every principal component. Once we get the trained model, we can test whether it can explain new samples, unseen at the calibration stage, by keeping fixed the loadings corresponding to the spectral composition and the alteration rate. Moreover, by using regression on the temporal change loadings, we can predict unmeasured future values.

Tensor decomposition with parallel factor analysis

PARAFAC is a tensor decomposition method that was initially proposed by Harshman [27] to increase interpretability of multivariate data. PARAFAC can be considered as a generalization of the bilinear PCA method [36] in the sense that both methods assume the input data can be explained as a linear combination of basis factors. These factors are also called scores and loading vectors. However, one of the most important aspects is that PARAFAC doesn’t require unfolding the data into bilinear form, thus maintaining the structure of the multivariate data. This way, there is a one-to-one mapping between the effects of a variable and a loading vectors, which is relevant for describing an underlying chemical or physical phenomena. Moreover, PCA doesn’t give an unique solution, because all rotations of the orthogonal loading vectors can be good fits of the data. Thus, while PCA scores and loadings are still meaningful for deciphering the data, it is difficult to draw a direct association with pure chemical components. On the contrary, PARAFAC loadings have the property of being unique, where the unique loadings resemble loyally the actual pure spectra in a material [62]. At the same time, the scale of the loading vectors resulting from tensor decomposition is unidentifiable and has no immediate correspondence with units of measure unless a scaling factor to a ground-truth can be determined [62].

Let \(X^{M\cdot K \cdot N}\) be a 3D tensor, where \(M\) is the number of samples, \(K\) the number of time steps and \(N\) the number of spectral bands of the input microfading data. Then, using three-way decomposition, we can model this tensor as the outer product of 3 factor matrices, A, B, C:

$$\begin{aligned} \hat{X}^{M\cdot K \cdot N} = A^{M \cdot F}\otimes B^{K \cdot F} \otimes C ^{N \cdot F}, \end{aligned}$$

where \(F\) is the user-defined number of components to split the tensor into. Using the Kathri-Rao product \(\odot\), Eq. 1 can be re-written in the following flattened form:

$$\begin{aligned} \hat{X}^{M\cdot KN} = A^{M \cdot F}\times ( C ^{N \cdot F} \odot B^{K \cdot F})^T, \end{aligned}$$

where \(\times\) denotes the conventional matrix multiplication. The flattened array can then be easily reshaped to the original size of the tensor.

The decomposition is solved with the alternating least squares algorithm [63] where the objective is to minimize the squared residuals between the actual data and the model. If \(E\) is the tensor corresponding to the residuals, defined by \(E^{M\cdot K \cdot N} = X-\hat{X}\), then the loss function is \(\underset{A,B,C}{min}E^2\). Alternating least squares algorithm implies that the factors \(A, B, C\) will be conditionally estimated on each other. To determine \(A\), \(B\) and \(C\) will be initialized and then, the minimization function will be solved for \(A\) given the \(B\) and \(C\) priors. Afterwards, the same operation is repeated to get \(B\) with the previously fitted \(A\) and initialized \(C\). Subsequently, the optimization function is run again for \(C\) with the already fitted \(A\) and \(B\) loadings. Finally, based on the values of the 3 factors obtained after the first iteration, the least-square optimization is repeated in the same conditional fashion until convergence is reached, i.e. there is negligible improvement in the newly fitted values with respect to the previous. While numerically there is no difference between scores and loadings, it is common practice to consider that the first factor, \(A\) refers to the scores of the model and \(B\) and \(C\) to the loadings. In this article, \(C\) represents the endmembers, \(A\) the concentration of each endmember \(f = \{1\ F\}\) in all the input samples and \(B\) the fading rate for each endmember. Moreover, because we are dealing with physical and chemical feasible data, the constraints of non-negativity for \(A\) and \(C\) are enforced during the decomposition process.

The choice of number of components \(F\) is not trivial. Typically, if no previous knowledge is known about the tensor’s rank, then experiments are done starting from a very small number of components in increments of 1. For each trained model, residual analysis is performed and the chemical meaningfulness of the loadings is interpreted. The model that gives the most sensible fitting from a chemical point of view and has good residual metrics is then selected.

Testing the PARAFAC model for new data

It is possible to test new data based on a trained model. Let’s consider a test tensor \(X_{test} ^{W\cdot K \cdot N}\)with different size in the \(1{st}\) dimension but identical size in the \(2{nd}\) and \(3{rd}\) dimensions with respect to the tensor used for training. Then we can fit the new data by keeping the \(B\) and \(C\) loadings fixed in Eq. 1 and solving only for the new concentration matrix \(A_{test}^{W\cdot F}\). This makes sense under the assumption that the test data has similar chemical content as the data used for training.

Future modelling

The fading rate \(R_{f}\) of each pure pigment is given by the loadings of the \(2{nd}\) mode, \(B\). Mathematically, it could be approximated by a line equation, implying that the reflectance of a pure pigment changes linearly with time:

$$\begin{aligned} R_{f} = ak + b, \end{aligned}$$

where \(k = \{1\ K\}\) are the modelled time steps, \(a\) the slope and \(b\) the intercept of the line. Once the slope and intercept are computed, we can replace \(k\) in Eq. 3 with values higher than \(K\) and get the values of the change for future times. These new values can then be input in Eq. 2 to predict the spectra of the original samples for future times other than those modelled. If there are samples in the original data, for which there are microfading spectra recorded for more than \(K\) time steps included in the training model, then the future modelling can be validated quantitatively. It is important to mention that the linear approximation in Eq. 3 is a simplification of the real change mechanisms and it assumes that the fading rate of each pigment evolves independently from that of other pigments. However, the fading rate \(R_{f}\) was extracted as one of the factors from the tensor decomposition method, which considers the tandem change of all the microfaded measured samples. Thus, it can be argued that the interaction between pigments is included to a certain extent in the fading rate curve.

Pigment identification

The endmembers extracted with our proposed method are described spectrally, which facilitates the comparison with reference datasets using established distances in the spectral domain towards pigment identification. We use spectral correlation and spectral angle as comparison metrics with databases of known pigments. Given two spectra \(s_{1},s_{2}\), the spectral angle treats them as vectors and computes the angle between these two vectors as the inverse cosine function of their dot product:

$$\begin{aligned} \text {SA} = cos^{-1}{\frac{\sum {s_{1}s_{2}}}{\sqrt{\sum {s_{1}^{2}}}\sqrt{\sum {s_{2}^{2}}}}} \end{aligned}$$

A smaller angle indicates a smaller difference between the spectra. On the other hand, spectral correlation [64] computes the similarity of two spectra as the dot product of their mean-centered signals:

$$\begin{aligned} \text {SC} = \frac{\sum {(s_{1}-\overline{s_{1}})(s_{2}}-\overline{s_{2}})}{\sqrt{\sum {(s_{1}-\overline{s_{1}})^{2}}}\sqrt{\sum {(s_{2}-\overline{s_{2}})^{2}}}} \end{aligned}$$

A higher correlation indicates higher resemblance of the two spectra.

Data collection and results

Case study: Oda Krohg’s painting “A Japanese Lantern”

In this article, we show the performance of our method on the pastel painting “A Japanese Lantern”, also known as “By the Christiania Fjord”. (see Fig. 2) created by the Norwegian painter Oda Krohg in 1886 and present in the collection of the National Museum of Norway (inventory number NG.M.00879). The painting is made of pastels applied on canvas and it is not very well documented from a scientific conservation point of view. Thus, the exact materials are unknown, and there is no record of the initial color appearance so as to make any immediate assumption about visible changes in the current version. This leads to the first research objective of our case study: to assess the light sensitivity of the painting materials. Typically, pastel sticks come in a wide range of hues and saturations [65] and are essentially composed of colored pigment powders in combination with white pigments, a white filler and binder [66]. To achieve a certain desired colour, pastel stick might contain a single pigment or a mixture [65]. While various binders can be used for the preparation of pastels [66], art conservators assume that the ones in Oda Krohg’s painting are soft pastels and so, probably mixed with aqueous binder in rather small amount. In terms of artistic technique, soft pastel paintings are often built up in layers, which combined with the pluralistic composition of pastel sticks, results in complex pigment mixing mechanisms. Therefore, the second research question for the case study is: can we surpass these complex mixing mechanisms and extract the pure pigments used in the pastel painting?

In order to address these research questions, two areas of interest in the painting were selected by the museum conservator for the analysis: the central lantern and the lantern in the left upper edge. Stylistically, the latter is actually depicting the reflection of the central lantern on the window. In particular, the area along the left edge is of special interest because it has been partially covered by the rebate of the frame, which prevented part of the pigments in the left lantern being exposed to light. Therefore, we are investigating if our approach can detect similar materials in the two lanterns.

Fig. 2
figure 2

Oda Krohg’s painting “A Japanese Lantern”. Pastel on canvas (1886). Courtesy of photographers Børre Høstland/Lathion, Jacques, National Museum

Microfading data collection

Specific locations on the central and left were faded using the microfadeometer provided by FotoNowy Institute [67]. The MFT illuminated spots of 0.5 mm diameter, with a white LED at an irradiance of 12.585 MegaLux and a power of 3.44 mW, for a 0\(^{\circ }\)/45\(^{\circ }\) geometry, as shown in Fig. 3. The light source was apriori calibrated by the manufacturers with a white standard card [68] and, as can be seen in the spectral power distribution in Fig. 4, it has a stronger emission in the middle and right parts of the electromagnetic spectrum. The measurement was stopped when the \(\Delta E_{00}\) color difference with respect to the original reached 2 units or after 600 s. For this reason, the cardinality of the observations is uneven, depending on the sensitivity of the pigment: some are measured for 77 s, others for 600, etc. The L*, a*, b* color coordinates are recorded for D65 standard illuminant and CIE 1931 2\(^{\circ }\) observer after every fading iteration. Apart from the color change, the instrument records the spectral reflectance every \(11{th}\) second with a spectral resolution of 2.5 nm in the 400–730 nm range. To reduce the noise, the spectral data was smoothed with a Savitzky-Golay filter [69] of order 2 and window size 17, and then further downsampled. In addition, the spectral range has been restricted to 440 and 680 nm due to the high noise in the data at both far ends of the measured spectrum.

Fig. 3
figure 3

Microfading measurement setup. Light is incident on the surface at 0\(^{\circ }\) and the colorimeter is collecting the signal at 45\(^{\circ }\)

Fig. 4
figure 4

Spectral power distribution of the white LED of the microfadeometer

As shown in Fig. 5, for the central lantern, 15 locations were microfaded that cover 5 apparent color groups (pink, green, red, pale blue/violet, orange). The minimum common number of recorded spectral changes is 8 (including the initial reflectance before fading), equivalent to 77 s of light exposure. As far as the left lantern is concerned, 5 locations were selected for investigation, that correspond to red and dark pink colors. The dark pink points have been protected by the painting’s frame, which was removed during the microfading measurements. Thus, one hypothesis that we try to verify with our model, is whether the dark pink and the red spots were initially applied from the same stroke, but they now differ in color because fading has occurred in areas unprotected by the frame.

Fig. 5
figure 5

Locations measured with the MFT for the left lantern and the central lantern. Courtesy of Børre Høstland, National Museum

Colorimetric analysis

Before diving into the tensor decomposition specifications and findings, we applied exploratory analysis techniques to get a first-hand overview of the microfading data. The straightforward outcome from a microfading experiment is the color change of the samples as a function of time. This is a primary hint to the stability of the pigments. Figure 6 depicts the \(\Delta {E_{00}}\) color difference for central lantern points measured with MFT and shows how the pink samples have the fastest change rate, while the green ones seems to be more stable.

Fig. 6
figure 6

\(\Delta {E_{00}}\) as a function of time for selected microfaded samples in the central lantern, representatives of each colour group

Another immediate analysis from the microfading is to assess the color change for a*, b* coordinates, where \(a^*_{+}\) corresponds to the red quadrant in the CIE L*a*b* space, \(a^*_{-}\) to green, \(b^*_{+}\) to yellow, and \(b^*_{-}\) to blue. As shown in 7, the change occurs in the negative direction. This entails that all samples lose the yellow and red chromatic components. It is interesting to note that Red 2 after fading has the same a*b* coordinates as Pink 5 before fading.

Fig. 7
figure 7

The chromatic changes of the microfaded samples in the central lantern. The black square marks the initial value, before fading, while the diamond marks the final value, after fading

Specification of PARAFAC model fitting

The microfaded observations of the central lantern were structured into a 15*8*81 (samples * time steps * spectra) tensor. The three-way array was then decomposed using parallel factor analysis (PARAFAC) with the Matlab implementation of the N-Way toolbox [70]. In Additional file 1, we inserted a code snippet to exemplify basic operations. Setting the number of components in the PARAFAC analysis is not a trivial task. We adopted a trial-and-error approach starting from 5 components and stopping where the loadings of the endmember resembled impossible reflectances. Our rationale was rooted in the typical formulation of pastels: they are manufactured as saturated color, where typically there would be a dominant pure pigment combined with others in smaller quantities. Given that samples collected in the microfading experiment roughly represent 5 colour groups, we assumed there should be at least 5 pure pigments. Figure 8 depicts the endmembers extracted when fitting the model with 5, 6 and 7 components. We can see that in the latter case, the spectra become noisier. Also, correlated spectra seem to appear (orange and cyan curves) and the orange curve resembles a composite multi-lobe signal rather than a realistic reflectance. On the other hand, the 5 and 6-component case output smooth and similar reflectances. Nonetheless, the 6-component case introduces an additional component in the red end of the spectrum (cyan curve in Fig. 8b), which is a realistic reflectance resembling a typical red lake. As a matter of fact, in a later subsection 4.7, we will prove that indeed, this endmember has a high similarity with a crimson carmine lake, when comparing with a reference database of pigments. Thus, we continued the analysis with the factors of the tensor decomposition model fitted for 6 components. The model converged in 1099 iterations, retaining 99.99% variance in the data while achieving a sum-of-squared residuals of 0.0145. Moreover, the histogram of the residuals shows a bell-shaped curve around 0 (see Fig. 9), suggesting that the variance follows a normal distribution, which is a good quality indicator of the regression process.

Fig. 8
figure 8

Endmembers obtained when fitting the tensor decomposition model with 5 (a), 6 (b) and 7 (c) components. In the 7 component case, the some spectral curves are noisy (cyan and orange curves) and seems to have a mixed rather than pure chemical composition. While both 5 and 6 component cases give smooth curves and similar members, it seems that the latter is able to extract a different pigment (cyan curve), that with the sudden growth close to 600nm, shows the characteristic of an organic red lake pigment

Fig. 9
figure 9

Histogram of the residuals for the PARAFAC model fitted with the microfaded observations in the central lantern

Figure 10 plots the three modes of the resulting model: endmembers, fading rate of each endmember, and concentration of each endmember in the input samples in the central lantern collected with the microfadeometer. In Additional file 1: Figure S1 displays the measured and modelled spectra for all the 15 samples, at time steps 1 and 8. The endmembers have the same color code in the three plots. One of the first comments that emerges is that the Pink 1–5 samples seem to be predominantly made of the endmember 2 (magenta curve), which also fades the most rapidly because it has the steepest descent in Fig. 10b. This is in conformity with the colorimetric analysis of the color degradation, where the Pink samples change most rapidly, reaching 2 \(\Delta E_{00}\) units before the other samples. Another interesting outcome is given by the concentration of the endmember in each sample. As the microfading experiment proceeds, we can infer that the underneath pastel layers get revealed as the top layer fades. This effect should be stronger for the pigments that are more fugitive. However, given the composite nature of the pastel sticks, we should also take into account that dominant concentrations in a sample hint to the multiple pigments present in a single pastel stick.

Fig. 10
figure 10

Loadings of the three factors (ABC) in our tensor decomposition model, fitted for 6 components

Future modelling

In the trained model, we included only a subset of the measured time steps, which is the minimum number of measurements common to all 15 samples. However, except Pink 1 sample, there are more measurements for the rest of the samples. Hence, another way to validate the goodness of the model is to test how well the excluded measurements can be explained by the trained model. To this purpose, we need to extend the fading rates for future time steps. As explained in subsection 3.3, we generated the fading rate for more than 8 time steps, by applying linear regression on the loadings in Fig. 10b. This gives as a slope and an intercept for the temporal change rate of each endmember, that we can use to further compute the fading rate for new time steps using Eq. 3. Table 1 summarizes the line equations for the fading rates of the 6 endmembers and Fig. 11 exemplifies the goodness of the linear fit to the changing trend of endmember 1.

Table 1 Slope, intercept and coefficient of determination for fitting the fading rate of the 6 endmembers to a line
Fig. 11
figure 11

Linear regression fit for the fading rate of endmember 1. \(y\)-axis represents the fading rate, while \(x\)-axis stands for the time step

Fig. 12
figure 12

The RMSE between measured and reconstructed spectra for future time steps. The RMSE is computed for each measured time step, hence the different cardinality of bars plotted per sample. Overall, the RMSE has small values, suggesting that our model is able to reconstruct future reflectances with good quality. However, the higher the time step values, the larger the RMSE, showing a limitation of the linear approximation of the fading rate

Using these fits, we generate the fading rate for time steps higher than 8 with Eq. 3. Then, we input these new values in Eq. 2 to recover the spectra of the samples. Afterwards, we compare the simulated reflectances with the measured ones with the root mean square error metric (RMSE). The barplot in Fig. 12 shows RMSE values for all the simulated time steps available for every sample. To be noted that the reconstructions for the first 8 time steps are done with the fading rate of the trained model, not using the linear approximation in eq. 3. Overall, the RMSE values are very low, proving a good performance of our model in estimating the future reflectance of the samples. Nevertheless, it is clear from the plot that the error increases with higher values of the time steps. This suggests a lower accuracy of the line fitting in approximating the fading rate when the gap between the modelled and simulated time steps is too high, especially for orange samples. It seems that the linear regression is most robust for the green samples. Indeed, Fig. 13 shows the measured and reconstructed spectra for Green 2 sample at time step 55 (594 s), which are very similar in shape and only slightly vary in amplitude.

Fig. 13
figure 13

Example of future spectral simulation for Green 2 sample for time step 55 (594 s). The reconstruction presents slight changes in amplitude, but follows well the spectral shape of the original. In this case, linear approximation of the fading rate seems to be more robust for high values of the time steps with respect to other samples

Fig. 14
figure 14

Example of future spectral simulation for Violet 1 sample for time steps 16 (165 s), 32 (341 s) and 55 (594 s). The reconstruction at time step 16 slightly exaggerates the decrease of the spectra at 600–650 nm, but otherwise closely follows the measured reflectance. The reconstructions worsen in this spectral range for time step 32 and for time step 55

Even though the linear regression gives the lowest coefficient of determination for endmember 4’s fading rate (see Table 1), the future simulation of the samples with a high concentration of endmember 4 (Violet 1, Violet 2) is not severely affected as the corresponding RMSEs are among the lowest numbers. The visual analysis of the reconstructed reflectance of Violet 1 at time step 16 in Fig. 14 confirms the accuracy of the simulation with respect to the measurement. However, as previously mentioned, the reconstructions further deviate from the original for time step 32 (341 s), and especially time step 55 (594 s). The deviation happens mostly in the region 600–650 nm, which might also indicate unexplained variation of the tensor decomposition model in this spectral range. In future, other approaches to model the fading rate can be implemented towards improvement, such as spline interpolation or time-series models [71].

Model validation for the left lantern

In the previous sections we proved that our proposed tensor decomposition model is statistically valid and works fine for future modelling of the samples with the same chemical composition as those included in the training stage. Now, we’ll verify the performance of our model for unseen microfaded samples, taken from the left lantern in the pastel painting (see Fig. 5). It is assumed that the same materials were used to render the red and pink colors in the central and left lantern, since actually the left lantern depicts the reflection of the central lantern. If so, then the trained model should be able to explain the variance in the left lantern data. To test this hypothesis, we arrange the 5 microfaded measurements into a three-way tensor, similar to the central lantern, which results into a 5*8*81 tensor. Then, we fitted a PARAFAC model to this new data, by keeping fixed the loadings of the \(B\) (fading rate) and \(C\) (endmembers) factors extracted with the previously trained model (for details, revise subsection 3.2). The model thus fitted achieved a sum of squared residuals of 0.0476. As a result, we obtain a new set of loadings, the \(A\) factor, (shown in Fig. 15) that gives the concentration of the already known endmembers in the left lantern samples. All 5 samples have as major constituents endmember 1, 2 and 6, but in different proportions. It appears that Red 1–3 samples in the left lantern are predominantly made of endmember 1, similar to the red samples from the central lantern. On the other hand, it seems that the two Dark Pink samples are richest in endmember 6. Additional file 1: Figure S2 displays the measured and modelled spectra at time steps 1 and 8, for the 5 samples in the left lantern.

To assess the quality of the reconstruction, we graphically analyze the residuals of the model fitted on the test data. Figure 16 shows the histogram of the residuals for each of the 5 samples, and the surface plots of the residuals distributed along the spectral and temporal dimensions. We’d expect the histograms to have a Gaussian distribution and the surface plots to be as flat as possible. The histogram for Red 1–3 samples are centered around 0 and fairly symmetric, even though they are slightly skewed towards the left. While the magnitude of the left-side skewedness and asymmetry are more evident for the Dark Pink samples, the normal distribution pattern does not get disrupted to a far extent. At the same time, while the surface plots are not completely flat, showing some variation that the model did not capture from the data, the magnitude of this unexplained structure is however of a very low order. Thus, the model has statistical relevance in explaining the left lantern samples, and we can conclude that the materials used in the central lantern are most likely the same as the ones in the left lantern.

Another research question is whether the 5 samples on the left lantern correspond to the same materials, since in the painting it looks as if they were part of the same strokes. In this regard, it is noteworthy to mention that the resulting factor \(A\) loadings point to a sensible layering pattern of the pastels. Considering that the samples Dark Pink 1 and Dark Pink 2 were collected from an area in the painting that was covered by the frame, we know they were less exposed to less light than the rest of the left lantern. Assuming that all the 5 samples represent the same material, the uneven exposure to light would explain the difference in the dominant pigment component. For instance, if the initial material was made of a layer of endmember 1 covered by a layer of endmember 6, the fact that Red 1–3 samples were exposed to light for a considerably higher extent, would explain why they have less concentration of endmember 6 as opposed to Dark Pink 1 and Dark Pink 2. In addition, this theory can be corroborated by a common artistic practice, namely that of covering a base red pigment (endmember 1) with a lake glaze (endmember 6) [45]. Even though this theory seems plausible, in order to fully validate it, we would need to complement our study with analytical data such as XRF, that would shed further light on the chemical composition of the pigments. We plan to pursue such an extended analysis in our future work.

Fig. 15
figure 15

The estimated concentration of the 6 endmembers in the left lantern samples. The estimation was achieved by using the endmembers and fading rates apriori obtained through training the PARAFAC model on the central lantern samples

Fig. 16
figure 16

Visualization of the residuals when testing the model on the left lantern. The residuals are plotted for the first factor, A, i.e. concentration of endmembers in each sample. The histogram of the residuals (top) show a fairly symmetric spread around the mean and the distribution is close to a Gaussian one. The surface plots (bottom) distribute the residuals (\(z-\) axis) along the spectral (\(x-\) axis) and temporal dimension (\(y-\) axis) hint that there’s little chemical structure in the test samples that the model is not able to explain

Pigment identification

We compared the spectra of our endmembers with three reference datasets of known pigments using spectral angle and spectral correlation. The first dataset includes spectrophotometric measurements of 54 pigments combined with various binders: gum Arabic, gum Arabic measured with a reflection probe (that discards specularities from the measurement), egg-tempera, acrylic and fresco [72]. From here on, this dataset will be called FORS-CHO. It is believed that the pastels used in the painting are soft, and most probably an aqueous binder was used in small quantities. For this reason, out of the binders in the FORS-CHO dataset, we chose the pigments bound with gum Arabic (and measured with a reflection probe for higher precision), because this pigment-binder combination is considered to be the most sensible with respect to the soft pastels in the painting. The second dataset consists of hyperspectral measurements with the HySpex camera of a pigment panel, called ENST, that was prepared by National Gallery in London for the VASARI project [73]. It contains 64 unvarnished patches of historical pigments bound with egg-tempera and applied on a panel primed with gesso. Even though the binder is egg-tempera, which is not typically used for the preparation of soft pastels, we still used this dataset as standard for comparison because it contains a higher variety of historical pigments than the first dataset. Finally, the third reference collection is the only spectral database of pastels applied on paper and measured with a spectrophotometer, and it is published by Centore [74]. While this database includes both modern brands (Blue Earth, Unison, Great American, Mount Vision) and historical pastel manufacturers (Girault, Sennelier, Schmincke), nowadays all the manufactures most probably use modern materials for the pastels’ preparation. Usually, these recipes are not revealed by the manufacturers so the chemical composition is unknown making it difficult to interpret and validate the results in a pigment identification task. Nonetheless, we deemed it interesting to explore the resemblance with our endmembers especially because it is a pastel-to-pastel comparison. To be noted that among the pastel manufacturers present in this dataset, Girault and Schmincke were already producing pastels when Oda Krohg painted “A Japanese Lantern” in the late \(19{th}\) century.

We have to keep in mind that the painting in our study has a more intricate formulation than the patches measured in the three spectral libraries. This is expected to negatively impact the quality of the spectral matches [11]. While for future work, we plan to collect a spectral reference dataset of a mockup that is carefully designed to mirror the materials and technique of the studied painting, we are still interested to see if the comparison with the currently available libraries leads to sensible matches.

The spectral sampling of the pigments in the reference databases varies with the measuring instruments, so they were interpolated to align to the sampling of the endmembers. After the interpolation, the spectral metrics in Eq. 4 and 5 were computed in a two-folded way: for the original reflectance vector and for the first derivative of the reflectance. The first derivative better captures the inflection points (minima and maxima) of the spectral reflectance curve. As a result, the spectral distance comparison will be more sensitive to big shifts in reflectance, which has more discriminative power than reflectance alone for pigment analysis, as proved in previous works [8]. In order to simultaneously visualize the similarity according to the reflectances, and both spectral metrics, we are visualizing the distances with a bubble chart in the following way. For each reference dataset, the lowest 5 spectral angles with respect to the endmembers, are chosen for the reflectance and first derivative mode, giving a total pool of maximum 10 best matches. Then, we define a 2D coordinate system of the bubble chart with the spectral angle in the reflectance mode as the \(x-\)axis and the spectral angle in the derivative mode as the \(y-\)axis. We plot each of the 10 best matches in this coordinate system. It can happen that the first derivative and reflectance comparisons give overlapping matches. For this reason, we actually have less than 10 bubbles plotted for each pigment. Furthermore, the diameter of the bubbles is scaled with the spectral correlation value between of the endmembers with the reflectances in the reference datasets. We chose to vary the size of the bubbles with the diameter instead of the area because previous research [75] showed that the human visual system better appreciates changes in the diameter of a disk as opposed to its area.

Fig. 17
figure 17

Matches of the endmembers with the pigments in the ENST dataset, visualized as a bubble chart. The x-coordinate represents the spectral angle in radians computed on reflectance spectra between endmembers and reference dataset, while the y-coordinate is the spectral angle in radians computed on the first derivative. The diameter of the bubble is proportional with the spectral correlation between the reflectance spectra. Bigger bubbles in the bottom left indicate a higher confidence of the match

Fig. 18
figure 18

Matches of the endmembers with the pigments in the FORS-CHO dataset, visualized as a bubble chart. The x-coordinate represents the spectral angle in radians computed on reflectance spectra between endmembers and reference dataset, while the y-coordinate is the spectral angle in radians computed on the first derivative. The diameter of the bubble is proportional with the spectral correlation between the reflectance spectra. Bigger bubbles in the bottom left indicate a higher confidence of the match

Hence, in the bubble chart visualization, an ideal candidate would be represented by a big size bubble located in the bottom left corner of the chart, which translates to the following attributes: big correlation, low spectral angle for the reflectance, low spectral angle for the first derivative. Figure 17 displays the similarities of our endmembers (1–6) with the pigments in the ENST dataset, while Fig. 18 features the FORS-CHO dataset. It is easy to notice that overall, the spectral angles computed for the first derivative have a higher magnitude than those computed in the reflectance mode. This is because the first derivative is sensitive to any change of growth in the reflectance curve, and so it amplifies the noise of the initial signal. At the same time, the values of the spectral metrics are lower for the comparison with the ENST library. This might be due to the use of historical recipes in the manufacturing of the ENST target. At a quick perusal of the charts, we can see that there is a certain clustering of the endmembers, meaning that some endmembers have better matches than others. For instance, endmember 1 (red) has closer matches than endmember 3 (orange) since the candidates for the former are located in the bottom left corner, while those of the latter are in the top right corner. For endmember 1, we can see that the reflectance comparison with ENST suggest the red lead pigment as the best match, because it’s situated in the left-most corner, while the first derivative spectral angle picks red ochre as having the lowest value in the \(y\) dimension. Interestingly, red lead is designated as the best candidate for endmember 1 by the FORS-CHO dataset as well, followed by vermilion and red ochre. Moreover, we can see that all candidates for the red endmember have bubbles of similar size, so in this case, spectral correlation does not bring clarification to distinguishing between candidates. ENST shows as secondary matches vermilion, cadmium red and rose madder, which are positioned very close to each other, with cadmium red being slightly better than the other two. Following the same reasoning, the most valid choices from ENST dataset for endmember 2 (pink) seem to be red lead and vermilion. Nonetheless, mercuric iodide arises as the best match according to the first derivative comparison, but it jumps more than 2 orders of magnitude along the \(x-\)axis. Out of the FORS-CHO pigments, red lead comes atop with the highest similarity.

It is interesting to note here several known aging attributes of the red lead pigment. It has been documented [41] that when exposed to sunlight, rain and carbon dioxide, red lead can cause the formation of lead carbonate, giving it a whiter, thus pinkish appearance. Hence, if we follow the theory that both endmembers are red lead, the difference in their spectra might be due to this aging phenomena. On the contrary, if we were to judge by inflection peaks only, endmember 1 and 2 have maxima at 590 nm and, respectively, 587 nm, which were associated to vermilion in a previous multi-analytical study of pigments [76]. In the same study, red lead was associated with a peak shifted towards 565 nm. However, we believe this might be to a great extent influenced by the media (the previous study was performed for an illuminated manuscript) and the binder. Actually, as we can see in Fig. 19, the reference vermilion has a peak at 600nm, while red lead reaches its maxima at 578 nm. At the same time, the peaks of endmembers 1 and 2 are approximately equally distant to the reference red lead and vermilion in the ENST dataset.

Fig. 19
figure 19

First derivative spectra of endmembers 1 and 2 together with best matches from the ENST spectral library. Offset was added for clarity

As far as endmember 3 is concerned, chrome yellow surfaces as the most reasonable option from the ENST dataset, while FORS-CHO leads to saffron and cobalt yellow as best compromise along the two dimensions. By studying the reflectance of endmember 3, we can actually see that it doesn’t have the characteristics of a pure orange, because the curve has a depression in the red region of the spectrum. This means that the color is shifting to green, which is a well known aging behaviour of the chrome yellow pigment [41]. The shift to green of chrome yellow has been encountered in other paintings, such as Van Gogh’s “Field with Irises”, as recorded by Geldof et al. [77]. Figure 20 displays the reflectance of the aged chrome yellow from Van Gogh’s painting together with the spectra of our endmember. While we can see that the two spectra are similar, it appears that in the case of our endmember, the aging is more advanced. However, we should keep in mind that it is possible for this accentuated loss of the red color to be rooted in the limitations of our model in perfectly unmixing the components. Nonetheless, chrome yellow classifies as a feasible option for endmember 3 and the shift to green would also explain why the spectral metrics give generally higher errors for this component.

Fig. 20
figure 20

Solid line: the \(3{rd}\) endmember in our model, which is found in highest concentration in the yellow samples. Dashed line: chrome yellow in an aged version, as identified in Van Gogh’s “Field with Irises near Arles” by Geldof et al. [77]. Chrome yellow has the characteristic of turning green over time and so lose the longer-wavelength components. For our endmember, which is likely a chrome yellow, this tendency is obvious

Top choices for endmember 4 (blue) from ENST seem to be cerulean blue, manganese blue and ultramarine mixed with lead-white. At the same time, the most similar pigments from FORS-CHO are Egyptian blue, cobalt blue and ultramarine. Regarding endmember 5 (green), emerald green, synthetic malachite and Scheele’ s green are nominated as the most similar pigments among ENST group. On the other hand, cobalt green and malachite are the most salient with respect to the FORS-CHO group. As for endmember (violet), it is obvious from the sudden and steep growth of the reflectance curve close to 600 nm that it is a red lake. Red lakes surface as good options from the bubble charts as well. However, a very helpful tool in identifying the actual type of lake is the protocol proposed by Fonseca et al.[7]. According to this protocol, the presence of two minima between 450–600 nm prove that it is a carmine lake. Moreover, the proximity of second minimum to 600 nm, as shown in Fig. 21 gives additional information about the preparation of the carmine lake in that it was precipitated on aluminum.

Fig. 21
figure 21

First derivative spectra of endmember 6 together with potential candidates from the ENST spectral library. Offset was used for clarity. Endmember 6 has two minima between 400–600 nm, with the second one at 578 nm. Also, the maximum overlaps with that of carmine lake, while the lac lake has the maximum more towards the right

Out of the comparison with the Centore dataset (See Fig. 22), we can see that historic pastel brands emerge among top 5 matches for the endmembers: Girault 36 and Schmincke 042B for endmember 1; Girault 196 and Sennelier 930 for endmember 2; Girault 195 for endmember 3; Sennelier 290 for endmember 4; Girault 229 and Schmincke 076 H for endmember 5; and lastly Schmnicke 048B for endmember 6. While the chemical formulation of the pastels included in the Centore database is unknown, it is still possible that historic manufacturers might have preserved a formula close to the traditional formulations.

Fig. 22
figure 22

Matches of the endmembers with the pastels in the Centore dataset, visualized as a bubble chart. The x-coordinate represents the spectral angle in radians computed on reflectance spectra between endmembers and reference dataset, while the y-coordinate is the spectral angle in radians computed on the first derivative. The diameter of the bubble is proportional with the spectral correlation between the reflectance spectra. Bigger bubbles in the bottom left indicate a higher confidence of the match

With the bubble chart visualization approach, we aim to shortlist possible pigments and to alleviate the task of conservators of assessing the potential candidates for identification. It is up to the conservator to give weights to the two spectral metrics (SA in reflectance or derivative mode, and SC in reflectance mode). We believe with this visualization, it is also easy to spot and discard anachronistic pigments (as long as there are no suspicions of forgery), albeit high votes from the spectral metrics. For example, titanium yellow could be discarded as an option for endmember 3, because it is a pigment that was created more than 60 years after the painting was created. Nonetheless, pigment identification is a complex task, hence multiple data, visualizations and knowledge about the pigment aging behaviour should all be weighed by an expert. Also, for full validation, analytical data such as XRF should be added to the spectral analysis, which we plan to collect in the future.


To summarize our results, we showed how microfading spectrometry can be useful to the analysis of an artwork beyond assessing pigments' sensitivity to light and summarizing their color degradation. Namely, we use the microfading data to perform pigment unmixing and identification. Moreover, we model the photodegradation phenomena in a spectral not only color dimension, and for all wavelengths in one go as opposed to a tedious recovery of the fading rate for each wavelength taken separately. This is achieved with multivariate data techniques, such as the tensor decomposition model that is the pillar of our work. This model is trained on the microfading data and decouples signals to get pure pigments (endmembers) on a spectral basis, together with their fading rates. As a byproduct, we can solve the pigment identification task by comparing the endmembers with reference spectral datasets of pigments.

While the results look promising, our approach is not without limitations. Above all, we lack ground-truth to fully validate our results. For the pigment identification task, we plan to solve this in the future by collecting XRF data of the painting. In addition, our method requires user input, as long as the number of components (endmembers) is concerned. This is common to other traditional pigment unmixing methods, such as PPI. Indeed, with pigment unmixing being such a complex task, we believe that art conservators and art historians should remain in the loop and ascertain the chemical meaningfulness of the endmembers. Moreover, it is important to mention that the tensor decomposition is a linear unmixing method that might not manage to capture all the non-linearities present in a pastel painting, where both intimate and layered mixing mechanisms exist. While the accuracy of tensor decomposition for spectral unmixing tasks is supported by previous findings in chemical [37] and computer vision [31] applications, in future, we plan to compare the performance of our approach with that of non-linear unmixing models. Finally, although we show results for a single case study, our method can be applied on any other type of colorant data (ink, dyes, etc.) and artworks given a set of overlapping microfading measurements and we intend to test this on mockups and additional artworks.


In this article, we presented an approach to pigment characterization in a painting. Our method is based on a tensor decomposition model that, from a set of microfading measurements, disentangles in one shot, the reflectances of pure pigments, their concentration and their fading rate as a function of time.

We statistically validate our model on the training data, as well as for unseen, test data. In addition, we compare the endmembers with reference datasets of pigments and we present our results with a bubble-chart visualization that is capable of combining the result of three spectral metrics. This way, we ease the decision of art conservators in shortlisting the best pigment matches.

In conclusion, we showed how microfading, which is traditionally used only for assessing sensitivity to light, can be useful as well for the task of pigment characterization. In this context, it appears that parallel factor analysis shows promising results for pigment unmixing.

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.



Commission Internationale de l’Éclairage (International Commission on Illumination)




Light-emitting diode


Parallel factor analysis


Principal component analysis


Pixel purity index


Root mean square error


Spectral angle


Spectral correlation


X-Ray fluorescence


  1. Dooley KA, Chieli A, Romani A, Legrand S, Miliani C, Janssens K, Delaney JK. Molecular fluorescence imaging spectroscopy for mapping low concentrations of red lake pigments: Van Gogh’s painting The Olive Orchard. Angew Chem Int Ed. 2020;59(15):6046–53.

    Article  CAS  Google Scholar 

  2. Kimmel BW, Baranoski GVG, Chen TF, Yim D, Miranda E. Spectral appearance changes induced by light exposure. ACM Trans Gr. 2013;32(1):10–11013.

    Article  Google Scholar 

  3. Bacci M, Orlando A, Picollo M, Radicati B, Laterna G. Colour analysis of historical red lakes using non-destructive reflectance spectroscopy. PACT. 2000;58:21.

    Google Scholar 

  4. Johnston RM, Feller RL. The use of differential spectral analysis in the study of museum objects. Dyestuffs. 1963;44(9):1–10.

    Google Scholar 

  5. Aceto M, Agostino A, Fenoglio G, Idone A, Gulmini M, Picollo M, Ricciardi P, Delaney JK. Characterisation of colourants on illuminated manuscripts by portable fibre optic UV-visible-NIR reflectance spectrophotometry. Anal Methods. 2014;6(5):1488–500.

    Article  CAS  Google Scholar 

  6. Bisulca, C., Picollo, M., Bacci, M., Kunzelman, D.: UV-Vis-NIR reflectance spectroscopy of red lakes in paintings. In: 9th International Conference on NDT of Art, Citeseer, Jerusalem, Israel; 2008. pp. 25–30.

  7. Fonseca B, Schmidt Patterson C, Ganio M, MacLennan D, Trentelman K. Seeing red: towards an improved protocol for the identification of madder- and cochineal-based pigments by fiber optics reflectance spectroscopy (FORS). Herit Sci. 2019;7(1):92.

    Article  CAS  Google Scholar 

  8. Gabrieli F, Delaney JK, Erdmann RG, Gonzalez V, van Loon A, Smulders P, Berkeveld R, van Langh R, Keune K. Reflectance Imaging Spectroscopy (RIS) for operation night watch: challenges and achievements of imaging rembrandt’s masterpiece in the glass chamber at the rijksmuseum. Sensors. 2021;21(20):6855.

    Article  Google Scholar 

  9. Deborah H, George S, Hardeberg JY. Pigment mapping of the scream (1893) based on hyperspectral imaging. In: Elmoataz A, Lezoray O, Nouboud F, Mammass D, editors. Image and signal processing. Cham: Springer; 2014. p. 247–56.

    Chapter  Google Scholar 

  10. Deborah H, Ferrer J, Sandu I, George S, Hardeberg J. Old Man in Warnemunde (1907) colouring palette: A case study on the use of hyperspectral imaging for pigment identification. 2017;25:339–344.

  11. Kleynhans T, Messinger DW, Delaney JK. Towards automatic classification of diffuse reflectance image cubes from paintings collected with hyperspectral cameras. Microchem J. 2020;157: 104934.

    Article  CAS  Google Scholar 

  12. Heylen R, Parente M, Gader P. A review of nonlinear hyperspectral unmixing methods. IEEE J Sel Top Appl Earth Obs Remote Sens. 2014;7(6):1844–68.

    Article  Google Scholar 

  13. Bioucas-Dias JM, Plaza A, Dobigeon N, Parente M, Du Q, Gader P, Chanussot J. Hyperspectral unmixing overview: geometrical, statistical, and sparse regression-based approaches. IEEE J Sel Top Appl Earth Obs Remote Sens. 2012;5(2):354–79.

    Article  Google Scholar 

  14. Keshava N, Mustard JF. Spectral unmixing. IEEE Signal Process Mag. 2002;19(1):44–57.

    Article  Google Scholar 

  15. Boardman JW, Kruse FA, Green RO. Mapping target signatures via partial unmixing of AVIRIS data. In: Summaries of the Fifth Annual JPL Airborne Earth Science Workshop. 1995;1:23–26.

  16. Winter ME. A proof of the n-findr algorithm for the automated detection of endmembers in a hyperspectral image. Algorithm Technol Multispectr Hyperspectr Ultraspectr Imag X. 2004;5425:31–41.

    Article  Google Scholar 

  17. Nascimento JM, Dias JM. Vertex component analysis: a fast algorithm to unmix hyperspectral data. IEEE Trans Geosci Remote Sens. 2005;43(4):898–910.

    Article  Google Scholar 

  18. L3Harris geospatial solutions: ENVI spectral hourglass wizard. 2022. Accessed 03 Feb 2023.

  19. Kubelka P, Munk F. An article on optics of paint layers. Z Tech Phys. 1931;12(593–601):259–74.

    Google Scholar 

  20. Zhao Y. Image segmentation and pigment mapping of cultural heritage based on spectral imaging. PhD thesis, Rochester Institute of Technology. 2008.

  21. Kirchner E, Ivd Lans, Ligterink F, Geldof M, Gaibor ANP, Hendriks E, Janssens K, Delaney J. Digitally reconstructing Van Gogh’s Field with Irises near Arles. Part 2: pigment concentration maps. Color Res Appl. 2018;43(2):158–76.

    Article  Google Scholar 

  22. Rohani N, Pouyet E, Walton M, Cossairt O, Katsaggelos AK. Nonlinear unmixing of hyperspectral datasets for the study of painted works of art. Angew Chem. 2018;57(34):10910–4.

    Article  CAS  Google Scholar 

  23. Kleynhans T, Schmidt Patterson CM, Dooley KA, Messinger DW, Delaney JK. An alternative approach to mapping pigments in paintings with hyperspectral reflectance image cubes using artificial intelligence. Herit Sci. 2020;8(1):84.

    Article  Google Scholar 

  24. Amigo JM. Hyperspectral imaging. Amsterdam: Elsevier; 2019.

    Google Scholar 

  25. Grabowski B, Masarczyk W, Glomb P, Mendys A. Automatic pigment identification from hyperspectral data. J Cult Herit. 2018;31:1–12.

    Article  Google Scholar 

  26. Hitchcock FL. The expression of a tensor or a polyadic as a sum of products. J Math Phys. 1927;6(1–4):164–89.

    Article  Google Scholar 

  27. Harshman RA. Foundations of the PARAFAC procedure: models and conditions for an “explanatory” multimodal factor analysis. UCLA Work Pap Phon. 1970;16:1–84.

  28. Carroll JD, Chang J-J. Analysis of individual differences in multidimensional scaling via an N-way generalization of “Eckart-Young” decomposition. Psychometrika. 1970;35(3):283–319.

    Article  Google Scholar 

  29. Bilius L-B, Pentiuc SG. Improving the analysis of hyperspectral images using tensor decomposition. In: 2020 international conference on development and application systems (DAS). 2020; pp. 173–176. IEEE .

  30. Veganzones MA, Cohen JE, Farias RC, Usevich K, Drumetz L, Chanussot J, Comon P. Canonical polyadic decomposition of hyperspectral patch tensors. In: 2016 24th European Signal Processing Conference (EUSIPCO). 2016; pp. 2176–2180. IEEE.

  31. Panagakis Y, Kossaifi J, Chrysos GG, Oldfield J, Nicolaou MA, Anandkumar A, Zafeiriou S. Tensor methods in computer vision and deep learning. Proc IEEE. 2021;109(5):863–90.

    Article  Google Scholar 

  32. Chen P, Cheng L, Zhang T, Zhao H, Li J. Tensor dictionary learning for representing three-dimensional sound speed fields. J Acoust Soc Am. 2022;152(5):2601–16.

    Article  Google Scholar 

  33. Zubair S, Wang W. Tensor dictionary learning with sparse TUCKER decomposition. In: 2013 18th international conference on Digital Signal Processing (DSP). 2013; pp. 1–6 .

  34. Zhao C, Wang M, Su N, Feng S. Dictionary learning hyperspectral target detection algorithm based on tucker tensor decomposition. In: IGARSS 2020 - 2020 IEEE international geoscience and remote sensing symposium. 2020; pp. 1763–1766.

  35. Christensen J, Becker EM, Frederiksen CS. Fluorescence spectroscopy and PARAFAC in the analysis of yogurt. Chemom Intell Lab Syst. 2005;75(2):201–8.

    Article  CAS  Google Scholar 

  36. Bro R. Multi-way analysis in the food industry. PhD thesis, University of Amsterdam. 1998

  37. Smilde AK, Geladi P, Bro R. Multi-way analysis: applications in the chemical sciences. Hoboken: John Wiley & Sons; 2005.

    Google Scholar 

  38. Castro JP, Pereira-Filho ER, Bro R. Laser-induced breakdown spectroscopy (LIBS) spectra interpretation and characterization using parallel factor analysis (PARAFAC): a new procedure for data and spectral interference processing fostering the waste electrical and electronic equipment (WEEE) recycling process. J Anal At Spectrom. 2020;35(6):1115–24.

    Article  CAS  Google Scholar 

  39. Jansen JJ, Bro R, Hoefsloot HCJ, van den Berg FWJ, Westerhuis JA, Smilde AK. Parafasca: Asca combined with parafac for the analysis of metabolic fingerprinting data. J Chemom. 2008;22(2):114–21.

    Article  CAS  Google Scholar 

  40. Feller RL. Studies on the darkening of vermilion by light. Rep Stud Hist Art. 1967;1:99–111.

    Google Scholar 

  41. Feller RL. Artist’s pigments: a handbook of their history and characteristics. National gallery publications: London, UK; vol. 1 1986.

    Google Scholar 

  42. Rutherford J, Fitzhugh West E, Kuhn H, Plesters J, Chase W, Muhlethaler B, Thissen J, Feller R. Artists’ pigments: a handbook of their history and characteristics, vol. 2. National Gallery of Art: Washington, DC; 1993.

    Google Scholar 

  43. FitzHugh EW. Artists’ pigments: a handbook of their history and characteristics. National Gallery of Art: Washington, DC; vol. 3 1997.

  44. Russell WJ, de Wiveleslie Abney W. Report to the Science and Art Department of the Committee of Council on Education on the Action of Light on Water Colours: Presented to Both Houses of Parliament by Command of Her Majesty. HM Stationery Office. 1888

  45. Saunders D, Kirby J. Light-induced colour changes in red and yellow lake pigments. Natl Gallery Tech Bull. 1994;15(1):79–97.

    Google Scholar 

  46. International Organization for Standardization: ISO 105-B02:2014 Textiles - Tests for colour fastness - Part B02: Colour fastness to artificial light. 2014. Accessed 10 Sept 2022.

  47. Hattori H, Yoshizumi K, Crews PC. Wavelength sensitivity of AATCC Blue wool lightfastness standards under light radiation. Dyes Pigments. 2012;92(3):936–41.

    Article  CAS  Google Scholar 

  48. Saunders D, Kirby J. Wavelength-dependent fading of artists’ pigments. Stud Conserv. 1994;39:190–4.

    Article  Google Scholar 

  49. Lerwill A, Brookes A, Townsend JH, Hackney S, Liang H. Micro-fading spectrometry: investigating the wavelength specificity of fading. Appl Phys A. 2015;118(2):457–63.

    Article  CAS  Google Scholar 

  50. Pintus V, Garrappa S, Baragona AJ, Hradil D, Szabo F, Sterflinger K. Different Shades of Cadmium Soaps in Light Aged Modern Oil Mock-up Paints: First Multi-Analytical Investigation. In: MUNCH2022: Understanding Munch and the Art at the Turn of the Centuries - Between the Museum and The Laboratory, Oslo. 2022; p. 104

  51. Kirschner B. Light bleaching with light emitting diodes (led): evaluation of treatment procedure and bleaching potential. J Pap Conserv. 2020;21(4):151–65.

    Article  Google Scholar 

  52. Whitmore PM, Colaluca VG, Morris HR. The light bleaching of discolored films of an acrylic artists’ medium. Stud Conserv. 2002;47:228–36.

    Article  CAS  Google Scholar 

  53. Lennox FG, King MG. Studies in wool yellowing: part XXIII: UV yellowing and blue-light bleaching of different wools. Text Res J. 1968;38(7):754–61.

    Article  CAS  Google Scholar 

  54. Zhang B, Huo S, Liu S, Zou L, Cheng L, Zhou X, Li M. Effects of cold-light bleaching on enamel surface and adhesion of Streptococcus mutans. BioMed Res Int. 2021;2021:3766641.

    Article  CAS  Google Scholar 

  55. Diaspro A, Chirico G, Usai C, Ramoino P, Dobrucki J. Photobleaching. Handbook of biological confocal microscopy. New York: Springer; 2006. p. 690–702.

    Book  Google Scholar 

  56. Whitmore PM, Pan X, Bailie C. Predicting the fading of objects: identification of fugitive colorants through direct nondestructive lightfastness measurements. J Am Inst Conserv. 1999;38(3):395–409.

    Article  Google Scholar 

  57. Lojewski T. Lightfastness studies with MFT. In: MUNCH2022: Understanding Munch and the Art at the Turn of the Centuries - Between the Museum and The Laboratory, Oslo. 2022; p. 87

  58. Chan E, Aambø M, Godzimirska M, Grimstad I, Lojewski T, Sandu ICA. Light-induced Color Changes on “The Scream” Versions in the Munch Museum Collection. In: MUNCH2022: Understanding Munch and the Art at the Turn of the Centuries - Between the Museum and The Laboratory, Oslo. 2022; p. 91

  59. Grimstad I, Lojewski T, Sandu ICA. Is the interpretation and application of collected microfading data straight forward? In: MUNCH2022: Understanding Munch and the Art at the Turn of the Centuries - Between the Museum and The Laboratory, Oslo. 2022; p. 103

  60. Aambø M, Godzimirska M, Chan E, Lojewski T, Sandu ICA. Light Sensitivity of Pigments in Edvard Munch’s Works on Paper. In: MUNCH2022: Understanding Munch and the Art at the Turn of the Centuries - Between the Museum and The Laboratory, Oslo. 2022; p. 100

  61. Beltran VL. Advancing microfading tester practice. Los Angeles: Getty Conservation Institute; 2019.

    Google Scholar 

  62. Kolda TG, Bader BW. Tensor decompositions and applications. SIAM Rev. 2009;51(3):455–500.

    Article  Google Scholar 

  63. Yates F. The analysis of replicated experiments when the field results are incomplete. Emp J Exp Agric. 1933;1(2):129–42.

    Google Scholar 

  64. De Carvalho OA, Meneses PR. Spectral correlation mapper (SCM): an improvement on the spectral angle mapper (SAM). In: Summaries of the 9th JPL Airborne Earth Science Workshop, JPL Publication 00-18, vol. 9. JPL publication Pasadena, CA, USA. 2000

  65. Shelley M. Joseph Wright’s pastel portrait of a woman part III: technique and aesthetics. Metrop Mus J. 2009;44:113–20.

    Google Scholar 

  66. Townsend JH. Analysis of pastel and chalk materials. Paper Conserv. 1998;22(1):21–8.

    Article  Google Scholar 

  67. Instytut Fotonowy: Micro fading tester. 2022. Accessed 10 Sept 2022.

  68. Instytut Fotonowy: Fotolon, white standard reference. 2022. Accessed 10 Sept 2022.

  69. Savitzky A, Golay MJ. Smoothing and differentiation of data by simplified least squares procedures. Anal Chem. 1964;36(8):1627–39.

    Article  CAS  Google Scholar 

  70. Rasmus Bro: The N-way Toolbox for Matlab. 2023. Accessed 03 Feb 2023.

  71. Ciortan I, Poulsson TG, George S, Hardeberg J. Predicting pigment color degradation with time series analysis. In: Color and Imaging Conference, vol. 2022. Society for Imaging Science and Technology. 2022.

  72. Cosentino A. FORS spectral database of historical pigments in different binders. E Conserv J. 2014;2:54.

    Article  Google Scholar 

  73. Saunders D, Cupitt J. Image processing at the National Gallery: the VASARI project. Natl Gallery Tech Bull. 1993;14(1):72–85.

    Google Scholar 

  74. Centore P. A colour survey of artist’s pastels. J Int Colour Assoc. 2016;15:42–59.

    Google Scholar 

  75. Raidvee A, Toom M, Averin K, Allik J. Perception of means, sums, and areas. Atten Percept Psychophys. 2020;82(2):865–76.

    Article  Google Scholar 

  76. Delaney JK, Ricciardi P, Glinsman LD, Facini M, Thoury M, Palmer M, Rie ER. Use of imaging spectroscopy, fiber optic reflectance spectroscopy, and X-ray fluorescence to map and identify pigments in illuminated manuscripts. Stud Conserv. 2014;59(2):91–101.

    Article  CAS  Google Scholar 

  77. Geldof M, Proano Gaibor AN, Ligterink F, Hendriks E, Kirchner E. Reconstructing Van Gogh’s palette to determine the optical characteristics of his paints. Herit Sci. 2018.

    Article  Google Scholar 

Download references


No acknowledgement.


Open access funding provided by Norwegian University of Science and Technology. This work has not received additional funding.

Author information

Authors and Affiliations



IMC: conceptualization, methodology design, software implementation, formal analysis, investigation, results’ visualization and interpretation, writing of the main manuscript; TGP: conceptualization of conservation-related research questions, collection of the microfading data, investigation and interpretation, revision of the final manuscript; SG and JYH: supervised the research, ensured the funding acquisition, revision of the final manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Irina M. Ciortan.

Ethics declarations

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Additional file 1: 

The supplementary file attached to this paper includes a code snippet for a better understanding of the implementation details, as well as plots with all the measured and modelled spectra of the samples in the central and left lantern.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ciortan, I.M., Poulsson, T.G., George, S. et al. Tensor decomposition for painting analysis. Part 1: pigment characterization. Herit Sci 11, 76 (2023).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI:


  • Multivariate analysis
  • Microfading spectroscopy
  • Color photodegradation
  • Pastel painting
  • Bubble chart