Open Access

Recovery of archaeological wall paintings using novel multispectral imaging approaches

  • Stefano Legnaioli1,
  • Giulia Lorenzetti1,
  • Gildo H Cavalcanti2,
  • Emanuela Grifoni3,
  • Luciano Marras3,
  • Anna Tonazzini4,
  • Emanuele Salerno4,
  • Pasquino Pallecchi5,
  • Gianna Giachi5 and
  • Vincenzo Palleschi1, 6Email author
Heritage Science20131:33

https://doi.org/10.1186/2050-7445-1-33

Received: 25 April 2013

Accepted: 7 October 2013

Published: 24 October 2013

Abstract

New approaches in the application of multispectral imaging to the recovery of archeological wall paintings are presented, based on statistical techniques and on a novel method of image treatment (Chromatic Derivative Imaging – ChromaDI) which offers a way of embedding information coming from four spectral bands into a standard RGB image. The methods are applied to some wall paintings from the Tomb of the Monkey, an Etruscan tomb in the necropolis of Poggio Renzo, near the city of Chiusi (Siena), Italy, dated around 480-470 BC. It is shown that the techniques described are able to highlight and enhance a number of details that cannot be perceived in either any of the original channel images or any single processed output channel.

Keywords

ArchaeologyWall paintingsMultispectral imaging

Introduction

Multispectral Infrared imaging is a simple non-invasive technique that has been widely applied in Cultural Heritage analysis and study [15]. In its simplest realization, four images of the subject under study are acquired in the spectral bands of Blue, Green, Red and Infrared; in most cases, the Infrared image carries the most relevant information, because infrared radiation penetrates under the surface, thus allowing for the visualization of otherwise invisible details such as underdrawings and ‘pentimenti’ in canvas and panel paintings [6, 7]. Infrared imaging is also important for other applications, because of the possible enhancement of features deriving from the different infrared reflectivities of the subject’s constituent materials. The improvement of readability of degraded manuscripts in the Infrared image was demonstrated, for example, in the recovery of the burnt Erculaneum scrolls [8]. However, in some cases, useful information can be derived by the analysis of the whole multispectral series; in a recent paper [9] we presented a new approach based on blind signal processing [1012] for extracting hidden information from a painting. The guiding idea is that the appearance of the object under study (in that case, a painting on canvas) results from a linear mixture of mutually independent patterns, which can be separated by specific statistical methods. The separation algorithm is fed of the multispectral data captured, and produces a new set of images, each of which should in principle contain different features of the subject under study. The separation algorithms are ‘blind’, in the sense that no prior information on either the subject or the combination coefficients is provided. When using this approach, in general just one output image is considered significant, so this is not too dissimilar from using just one spectral band (usually the infrared image). It should be noted, however, that each output image is a combination of the whole multispectral set, so that it contains information from all input channels.

A much simpler approach for enhancing hidden features in the subject under study is the so-called ‘false-colors’ method. Since only three spectral bands can be displayed in a color image, three channels are picked from the multispectral set and superimposed in the form of a (false) color image. The most usual combination is Infrared, Red and Green (IrRG), although the combination Infrared, Green and Blue (IrGB) [13] can also be used. This procedure implies that one channel is discarded, so the information it carries is lost in the final image. In this case, the information from the discarded image is not used in the final color image.

Here we show that a simple method, exploiting the reflectivity curve of the subject would guarantee the simplicity of the approach just described while maintaining the information from all the multispectral set. The output of the method, called Chromatic Derivative (ChromaD) image, is a false color image related to the first derivative of the reflectivity curve of the subject. The ChromaD image often highlights the same details obtained through statistical routines. Moreover, the resulting false color images can be obtained very easily through standard image processing software, even in the presence of very large multispectral image sets.

Methods

This research did not involve experiments on humans or animals.

Experimental setup

The Tomb of the Monkey is the most important tomb of the Etruscan necropolis of Poggio Renzo, near Chiusi (Siena), Italy. Dated around 480-470 BC, it was discovered in 1846 by the Florentine archaeologist Alexander François. The tomb lies 8 meters below the ground level, it is composed of three rooms around a main atrium (see Figure 1).
Figure 1

Map of the Tomb of the Monkey. The wall paintings zones recovered with multispectral imaging are marked in red.

The walls of the atrium are decorated with polychrome paintings, showing scenes from funerary games. The tomb itself takes his name from a monkey depicted on a tree (zone 3 in Figure 1). In the front room, two other pictures (1 and 2), apparently monochromatic, probably represent young slaves carrying unknown objects.

For imaging the paintings, a multispectral camera (Chroma C4) from DTA s.r.l. (Cascina, Pisa, Italy) was used. The camera has a resolution of 8 megapixel with 16 bits dynamic range; its CCD sensor is cooled for reducing the electronic noise during the acquisition. The illumination was provided by two halogen lamps (100 W each, color temperature 2300 K) placed about 2 meters away from the paintings, at an angle with the surface of 45 degrees. The spectral resolution is obtained through the use of interferential filters with ± 25 nm pass bands around the central wavelengths. For the analysis, images in the spectral bands of Blue (450 nm), Green (550 nm), Red (600 nm) and Infrared (1050 nm) were acquired and processed. The central wavelength of the infrared band was chosen taking into account the spectral sensitivity of the CCD sensor, which drops dramatically around 1100 nm. The exposition time at the different spectral bands was adjusted for obtaining white-balanced images using a Spectralon® reference target.

The RGB (visible) images of the four paintings analysed are shown in Figure 2a-d.
Figure 2

RGB (visible) images of the four paintings analysed. a. Monochrome picture in the front room (point 1 in Figure 1). b. Monochrome picture in the front room (point 2 in Figure 1). c. The Monkey (point 3 in Figure 1). d. Three figures in the main room (point 4 in Figure 1).

It should be noted that the state of preservation of the paintings is rather precarious, because of the particular technique used for their realization. In fact, the pigments (hematite, charcoal black and Egyptian blue) were applied on a fragile clay layer on the rock wall [14]. Moreover, a further deterioration of the painting was produced by environmental effects, including the biodegradation produced by Actinobacteria colonization (whitish spots particularly visible in pictures 2a and 2b) [15]. It’s thus important to recover details of the original images that are visually lost. This is particularly true for the two monochrome figures in the front room, whose style seems rather primitive with respect to the paintings of the main room. Figure 3a-d show the multispectral series corresponding to the four zones analysed.
Figure 3

From left to right: Blue (450 nm), Green (550 nm), Red (550 nm) and Infrared (1050 nm) images. The bandwiths of all the images are ± 25 nm around the central wavelengths. a) Picture in zone 1; b) Picture in zone 2; c) Picture in zone 3 and d) Picture in zone 4.

Image recovery

The sequence of multispectral images in Figure 3 evidences the different behavior of the four wall paintings; while for the paintings 1 and 2 the infrared image shows an amazing recovery of readability with respect to the visible images, the images in the same spectral band almost completely vanish for paintings 3 (the Monkey) and 4. It seems thus reasonable to hypothesize that different image treatment approaches would be needed for enhancing details that are not immediately apparent from the analysis of the single pictures in the multispectral series.

False color images

The simplest way to introduce the infrared band information in a color image is combining the multispectral pictures in a false color (FC) image. The FC image is usually built assigning to its R, G and B channels three of the four images in the multispectral series. In Figure 4a-d the images obtained by combining the infrared, red and green (IrRG) and infrared, green and blue (IrGB) images are compared with the infrared image alone. Note that, in both cases, one of the images in the multispectral series (the blue one in IrRG and the red one in IrGB image) is not used for building the final picture.
Figure 4

From left to right: Infrared image, IrRG false color image, IrGB false color image. a) Picture in zone 1; b) Picture in zone 2; c) Picture in zone 3 and d) Picture in zone 4.

A comparison of Figure 4 and Figure 1 shows that the false color images marginally improve the readability of the pictures only when the infrared image carries significant information (Figure 4a and 4b). For Figure 4c and 4d, the enhancement of the picture with respect to the visible image is negligible, if any. In all cases but in Figure 4d, the IrGB false color image brings the most significant information; however, especially in the case of Figure 4a and 4b the readability of the image is compromised with respect to the sole infrared information. The perturbing effect of the surface biological attack, which practically disappears in the infrared image, is brought back from the images in the visible spectral channels. It appears thus clear that, in this case, a different kind of approach is needed for the treatment of the multispectral images.

Blind separation techniques

Our present goal is to process the channels that form a multispectral image to obtain a new set of images where the different physical features of the painting under study are well distinguishable. Of course, this entails that the properties we want to distinguish have different spectral signatures in the available bands. However, the patterns of the desired features may not be spatially separated, so they cannot be perceived distinctly in either the RGB appearance or any of the channels available. This is what we mean when we talk about “invisible details”: the relevant information is all contained in the data, but needs to be disentangled from the interfering patterns to be made visible to the analyst.

What we can attempt is to combine somehow the multispectral channels to obtain a set of images each representing one of the target patterns. For example, one of the outputs could represent the original painting, while the others could represent natural or human-made alterations, such as overpainting, scratches, or several kinds of biological attacks on the substrate or the paint.

A simple assumption, often made in remote-sensed image classification [16], is that the individual patterns combine linearly to form the final appearance. In formulas, let (i,j) be a pixel index; if x(i,j) is an N-vector map representing the multispectral image, and s(i,j) is an M-vector map representing the collection of the original patterns, we have
x i , j = A s i , j
(1)
where the N × M-matrix A is called mixing matrix. We take this as a working hypothesis, whose validity can be assessed from the significance of the derived results. Once linearity is assumed, separating the individual patterns is easy if all the relevant spectral signatures are known, that is, if A is known. Since this is seldom true, we rely on blind techniques, consisting in making assumptions on the patterns s, leaving A unknown, and then estimating s from the multispectral data x alone. Common assumptions made are of statistical nature. As an example, we could assume that the elements of vector s can be modeled as mutually independent, non-Gaussian random variables. This leads to the class of separation techniques denoted as Independent Component Analysis [1012]. Enforcing uncorrelatedness rather than independence poses problems of uniqueness, but has often proved to give very useful results [9, 11]. We should just find a matrix W such that
E x T W T x = D
(2)

where D is a diagonal matrix and E is the expectation operator. In other words, we just diagonalize the data covariance matrix R x  = E{x T x} to obtain estimated patterns s ^ = W x that are mutually uncorrelated (or, equivalently, orthogonal). This result is not always physically meaningful, but (subsets of) the components of vector s ^ are often much more readable than the original multispectral image. Each component carries information from the entire data cube, and is likely to represent some pattern that shares little mutual information with the other components.

In the analysis of paintings, our tentative assumptions are reasonable, since the degradations are expected to show a small correlation to the original painting they affect. In our case, we set N = M = 4, and W is a simple rotation-scaling operator in the (Ir,R,G,B) hypercolor space. Several algorithms, described in detail in [9], can be used to find a good matrix W. Depending on the situation, a single image can be picked out, or three over four output images can be put together to form a false-color image. With respect to more advanced separation methods (see [12]), the orthogonalization technique summarized above has the advantage of a particularly simple numerical implementation, while providing a straightforward interpretation of the results as linear combinations of the original multispectral series. The results obtained by orthogonalizing our data are shown in Figure 5a-d.
Figure 5

Set of images resulting from the application of the orthogonalization algorithm. The most significant image is evidenced in red. a) Picture in zone 1; b) Picture in zone 2; c) Picture in zone 3 and d) Picture in zone 4.

Apparently, orthogonalization does not separate completely the significant details of the images from, for example, the imperfections of the substrate and the effects of biological attacks. However, whereas in Figure 5a and 5b the best image essentially corresponds to the infrared image, in Figure 5c and, especially, 5d, a definite improvement in readability is obtained. It should be noted that the last picture in each set of orthogonal images does not bring any useful information; this consideration suggests the possibility of combining the other three pictures in a false color image [17], as shown in Figure 6a-d.
Figure 6

False color images obtained from the first three images in Figure5. a) Picture in zone 1; b) Picture in zone 2; c) Picture in zone 3 and d) Picture in zone 4.

As expected, the pictures that are better separated (Figure 5a, b and d) are the less readable after merging back – although in different colors – the superfluous information from the other orthogonalized images. When the separation is less effective, as in Figure 5c, the false color approach can give an acceptable result.

It must be noted that this procedure can be applied to any number of input channels, for example, using some additional infrared or ultraviolet bands, or even a set of narrowband hyperspectral channels. Generally, each individual output contains information coming from the entire data cube, and the analyst can choose the details to be highlighted by combining wisely a subset of outputs and displaying them through false colors or other visualization strategies.

Chromatic derivative imaging (ChromaDI) method

The new method proposed for the enhancement of non-visible features is based on the observation that the information carried by the four images in the multispectral series can be exploited in a single (false) color image by using a simple transformation:
Ir R / λ Ir λ R < R ¯ R G / λ R λ G < G ¯ G B / λ G λ B < B ¯
(3)
where Ir, R, G and B are the four images in the multispectral series, λIr,R,G,B represent their central wavelengths and R ¯ , G ¯ and B ¯ are the three components of the resulting ChromaD false color image. The transformation described has a simple physical significance in terms of the derivatives of the reflected intensity ∆I/∆λ of the subject (see Figure 7), approximated by central differences at λ1 = (1050 + 600)/2 = 825 nm, λ2 = (600 + 550)/2 = 575 nm and λ3 = (550 + 450)/2 = 500 nm. Since, the derivatives can assume negative real values outside the (0,255) range of integer values allowed for the RGB components, a suitable rescaling and rounding must be performed for allowing the visualization of the Chroma D images. The rescaling can be performed associating to 0 the minimum value of the chromatic derivative in each image, while the maximum would correspond to 255. The other pixel values would be rescaled proportionally, rounding the value to the closer integer.”
Figure 7

Reflectivity curve and chromatic derivative (rounded and rescaled) for the two points marked by the arrows.

The resulting ChromaD images are shown in Figure 8a-d for the four multispectral series considered in this work. Note that ChromaDI could be interpreted as a particular case of linear composition as described in Section 3.2, with N = 4 and M = 3, and
W = 1 λ Ir λ R 1 λ Ir λ R 0 0 0 1 λ R λ G 1 λ R λ G 0 0 0 1 λ G λ B 1 λ G λ B
(4)
Figure 8

ChromaD images corresponding to the four multispectral series considered. a) Picture in zone 1; b) Picture in zone 2; c) Picture in zone 3 and d) Picture in zone 4.

In the case of orthogonalization, the coefficients in W are adaptive, and tend to reduce the statistical dependencies among the image channels. In this case, conversely, the coefficients are fixed and determined by the experimental setup, do not try to enforce any statistical property on the results but have a precise physical meaning. Each output does not contain information from all the data cube but rather from just two adjacent channels. In principle however, also this strategy can be extended to any number of input channels and any order of derivative, that is, of information content of each single output.

While being less general, ChromaDI has the advantage of being even simpler than orthogonalization.

The four ChromaD images clearly show that the readability of images 8a and 8b is better recovered with respect to the other approaches. The details of the substrate are well visible, and at the same time the information carried by the infrared image is preserved. In Figure 8b, interesting details become visible around the head of the figure, which were not appreciable in the images obtained by conventional false color imaging or using the separation techniques described in the previous subsection. On the other hand, the recovery of pictures 8c and 8d is more limited, and definitely less satisfactory with respect to the results of blind separation technique.

Conclusion

Multispectral analysis have proved to be a very useful tool to investigate object features beyond the possibilities of the human visual system. However, a full exploitation of the entire data cube gets more difficult as the number of channels gets larger. Examining the individual channels one by one can be a solution, but something better can often be done from the point of view of the image analyst. A number of mathematical techniques have been proposed to extract the relevant information from the data and present the result in a visually meaningful way. We adapted some of them to the study of several kinds of objects relevant to cultural heritage, often with satisfactory results. These techniques, however, are very general, and the hypotheses on which they are based are not physically sound in all the applications. This is why the results obtained are not always easy to interpret. In this paper, we propose a very simple strategy, the Chromatic Derivative Imaging (ChromaDI), to include the information of an infrared channel into a false-color image where, however, no visible channel has been dropped. This strategy can be extended to any number of input channels, and then associated to a strategy to choose and display the most significant outputs. The results obtained often compare favorably with the ones obtained through the more complicated statistical methods. As a case study, we assume four wall paintings from the same archaeological context (the Tomb of the Monkey in Chiusi, Italy). The multispectral analysis evidences different behaviors of the corresponding multispectral series. Depending on the spectral response of the pigments and their conservation state, the recovery of the images can be satisfactory just limiting the analysis to the infrared (Ir) band. In general, false color imaging is not particularly useful when the visible details are scarce with respect to the ones recovered in the infrared band. The same consideration applies to blind separation techniques, which typically try to extract the useful information from the multispectral series in a single image. The usefulness of such techniques decreases when the infrared image already carries most of the information. On the other hand, ChromaDI works quite well when the infrared image is significant, allowing the detection of otherwise invisible details. Considering the large variety of possible situations, no single technique can be assumed as a general approach to study and diagnosis of paintings. Rather, a rich toolbox of specific procedures should be put at the disposal of the expert analyst to cope with different data and needs. Even in the very simple situation studied here, the need of applying different methods to improve the readability of the images is apparent.

Declarations

Acknowledgment

Gildo de Holanda Cavalcanti acknowledges the support of the grant CAPES (Coordenadoria de Aperfeiçoamento de Pessoal do Ensino Superior) proc. 3877-11-6 and FAPERJ, CNPq (PRONEX proc. E-26/111.445/2010).

Anna Tonazzini and Emanuele Salerno acknowledge the partial support of EU funds, through POR Calabria FESR 2007-2013 - PIA, project ITACA.

Authors’ Affiliations

(1)
Institute of Chemistry of Organometallic Compounds, Research Area of National Research Council
(2)
Instituto de Fìsica, Universidade Federal Fluminense
(3)
Art-Test s.a.s
(4)
Institute of Sciences and Technology of Information, Research Area of National Research Council
(5)
Soprintendenza per i Beni Archeologici della Toscana
(6)
Department of Civilizations and Forms of Knowledge, University of Pisa

References

  1. Van Asperen de Boer JRJ: Infrared reflectography: a method for the examination of paintings. Applied Optics. 1968, 7: 1711-1714. 10.1364/AO.7.001711.View ArticleGoogle Scholar
  2. Fischer C, Kakoulli I: Multispectral and hyperspectral imaging technologies in conservation: current research and potential applications. Stud Conserv Suppl. 2006, 1: 3-16.View ArticleGoogle Scholar
  3. Balas C, Papadakis V, Papadakis N, Papadakis A, Vazgiouraki E, Themelis G: A novel hyper-spectral imaging apparatus for the non-destructive analysis of objects of artistic and historic value. J Cult Heritage. 2003, 4: 330-337.View ArticleGoogle Scholar
  4. Vilaseca M, Pujol J, Arjona M, de Lasarte M: Multispectral system for reflectance reconstruction in the near-infrared region. Applied Optics. 2006, 45: 4241-4253. 10.1364/AO.45.004241.View ArticleGoogle Scholar
  5. Cristoforetti G, Legnaioli S, Palleschi V, Salvetti A, Tognoni E: “Optical chemical sensors for cultural heritage” in optical chemical sensors NATO science series II: mathematics. Physics Chem. 2006, 224: 515-526.Google Scholar
  6. Fontana R, Gambino MC, Greco M, Marras L, Materazzi M, Pampaloni E, Pezzati L, Poggi P: New high resolution IR-colour reflectography scanner for painting diagnosis. Proc SPIE Int Soc Optical Eng. 2003, 5146: 108-115. 10.1117/12.501235.Google Scholar
  7. Van Asperen De Boer JRJ: A note on the use of an improved infrared vidicon for reflectography of paintings. Stud Conserv. 1974, 19: 97-99. 10.2307/1505621.View ArticleGoogle Scholar
  8. Chabries DM, Booras SW, Bearman GH: Imaging the past: recent applications of multispectral imaging technology to deciphering manuscripts. Antiquity. 2003, 77: 359-372.View ArticleGoogle Scholar
  9. Legnaioli S, Grifoni E, Lorenzetti G, Marras L, Pardini L, Palleschi V, Salerno E, Tonazzini A: Enhancement of hidden patterns in paintings using statistical analysis. J Cult Heritage. 2013, 14: S66-S70. 10.1016/j.culher.2012.11.013.View ArticleGoogle Scholar
  10. Bell AJ, Sejnowski TJ: An information-maximization approach to blind separation and blind deconvolution. Neural Comp. 1995, 7: 1129-1159. 10.1162/neco.1995.7.6.1129.View ArticleGoogle Scholar
  11. Cichocki A, Amari SI: Adaptive blind signal and image processing. 2002, New York: WileyView ArticleGoogle Scholar
  12. Tonazzini A, Gerace I, Martinelli F: Multichannel blind separation and deconvolution of images for document analysis. IEEE Trans Image Proc. 2010, 19: 912-925.View ArticleGoogle Scholar
  13. Moon T, Schilling MR, Thirkettle S: A Note on the use of false-color infrared photography in conservation. Stud Conserv. 1992, 37: 42-52. 10.2307/1506436.View ArticleGoogle Scholar
  14. Colombini MP, Giachi G, Pallecchi P, Ribechini E: Tecniche pittoriche utilizzate nelle tombe etrusche di chiusi e di sarteano, in A. Minetti (a cura di): pittura etrusca. 2003, Italian: Problemi e prospettive, 162-167.Google Scholar
  15. Diaz-Herraiz M, Jurado V, Cuezva S, Laiz L, Pallecchi P, Tiano P, Sanchez-Moral S, Saiz-Jimenez C: “The actinobacterial colonization of Etruscan paintings”. Sci Reports. 2003, 3: Article number 1440Google Scholar
  16. Li Y, Yan L, Liu J: “Remote sensing image classification development in the past decade”. Proc SPIE. 2009, 74941D: 338-343.Google Scholar
  17. Rogerio-Candelera MA, Jurado V, Laiz L, Saiz-Jimenez C: Laboratory and in situ assays of digital image analysis based protocols for biodeteriorated rock and mural paintings recording. J Archaeol Sci. 2010, 38: 2571-2578.View ArticleGoogle Scholar

Copyright

© Legnaioli et al.; licensee Chemistry Central Ltd. 2013

This article is published under license to BioMed Central Ltd. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.