A high sensitivity, low noise and high spatial resolution multi-band infrared reflectography camera for the study of paintings and works on paper

Abstract

Background

Infrared reflectography (IRR) remains an important method to visualize underdrawing and compositional changes in paintings. Older IRR camera systems are being replaced with near-infrared cameras consisting of room temperature infrared detector arrays made out of indium gallium arsenide (InGaAs) that operate over the spectral range of ~900 to 1700 nm. Two camera types are becoming prevalent. The first is staring array infrared cameras having 0.25–1 Megapixels where the camera or painting is moved to acquire tens of individual images that are later mosaicked together to create the infrared reflectogram. The second camera type is scanning back cameras in which a small InGaAs array (linear or area array) is mechanically scanned over a large image formed by the camera lens to create the reflectogram, typically 16 Megapixels. Both systems have advantages and disadvantages. The staring IR array cameras offer more flexible collection formats, provide live images, and allow for the use of spectral bandpass filters that can provide reflectograms with better contrast in some cases. They do require a mechanical system for moving the camera or the artwork and post-capture image mosaicking. Scanning back cameras eliminate or reduce the amount of mosaicking and movement of the camera, however the need to minimize light exposure to the artwork requires short integration times, and thus limits the use of spectral bandpass filters. In general, InGaAs cameras are not sensitive in the 1700 to ~2300 nm spectral region, which has been identified in prior studies as useful for examining paintings with copper green pigments or thick lead white paints. Prior studies using cameras with sensitivity from 1000 to 2500 nm have found in general the performance at wavelengths longer than 1700 nm degraded relative to the performance at shorter wavelengths. Thus, there is interest in a camera system having improved performance out to 2500 nm that can utilize spectral bandpass filters.

Methods

Design requirements for such an improved IRR camera system were determined by first re-examining the optimal spectral window for detecting underdrawing. Thus the wavelength dependence of the clarity of carbon black underdrawings on a chalk ground covered by various paint swatches was measured from 750 to 2500 nm. Second, analysis of the loss of light transmission (1000–2500 nm) and the impact of thermal radiation (3000–5000 nm) on the performance of IR arrays sensitive to the 1000–5000 nm region was analyzed. From the results of these studies, a high sensitivity, near-infrared (1000–2450 nm) indium antimonide (InSb) staring array camera with a cold filter that blocked light >2450 nm was constructed with a color-corrected macro lens capable of holding interference filters. The camera system was characterized in three spectral bands (1100–1400, 1500–1800, and 2100–2400 nm) using test targets and art objects.

Results

The experimental results of the contrast difference between the model underdrawing on the chalk ground showed the optimal spectral window for a given pigment varied over the range from 700 to 2300 nm. The contrast in all cases was found to be low from 2300 to 2600 nm and even lower from 2600 to 5000 nm. This is attributable in part to broad absorption by the drying oil paint binder. Performance testing of the IRR camera found high signal-to-noise was observed in all three spectral bands due to the optimized macro lens having high transmission from 1000 to 2600 nm, a 100% efficiency cold stop, and a cold filter that blocks light >2450 nm. The camera was found to have high light sensitivity, requiring only ~30 to 50 lx from incandescent lamps having a color temperature of ~2060 K. The images produced in each spectral band were sharp, and the modulation at Nyquist (1/2 the sampling frequency) was 20%. IRR images of Old Master paintings, works on paper, and a warped panel painting were collected to test the practical utility of the camera system.

Conclusions

The IR camera system presented here was found to produce a variety of high-resolution image products. The ability to extend image collection over the 1700–2450 nm spectral range was found in some examples to provide improved visibility of underdrawing. The ability to register the resulting multi-band IRR images with the color image offers opportunities to produce new image products such as false-color images and principal component images. These image products were found to give new insights into the construction of works of art. A collection scheme based on a ‘Z-stack’ method was found to solve the problem of producing high-resolution IRR images of highly warped panel paintings.

Background

X-radiography and infrared reflectography (IRR) [1] remain the standard tools for macroscopic examination of paintings even with the advent of advanced elemental (X-ray fluorescence) [2] and reflectance imaging spectroscopy [3]. Infrared reflectograms can provide useful information about underdrawings or paint composition changes because most artists’ pigments absorb less light and also scatter less light in the near-infrared spectral region compared to the visible spectral region. The color of materials is mainly due to electronic transitions induced by absorption of visible light. Most pigments have no or weak absorption features outside of the visible region (>700 nm). In addition, the amount of the light scattering by the pigment particles is inversely proportional to wavelength, and so the longer infrared wavelengths are scattered less than shorter visible wavelengths. Both the lower absorption and the lower amount of light scattering by the pigment particles means more incident light is transmitted through the paint layer and can reach the preparatory ground layer or underdrawing. An underdrawing can be seen when more light is reflected back through the paint layers from the ground layer than from the more absorbing underdrawing. Hence, the difference in reflectance between the underdrawing lines and the preparatory ground is necessary to produce enough contrast in an infrared reflectogram to render the underdrawing visible. Similarly, the contrast generated by differences in the absorption and reflectance of infrared light by two different paint layers helps to visualize areas of the composition that have been painted over and changed.

Several studies have summarized the results of experiments aimed at finding the ‘optimal’ spectral transmission window for pigments and paints used by artists [1, 4,5,6]. In general, such studies have found increasing transmission of light starting around ~700 nm and peaking at about ~2000 nm. Thus it is not surprising infrared reflectography is often implemented such that the reflected light is collected over a broad spectral band out to or near 2000 nm. However, the optimal window is not solely determined by the transmission of light through the paint layers [4, 5], given the variety of underdrawing materials employed by artists as well as tinted ground layers, both can affect the contrast of the underdrawing.

In the last 20 years, solid-state detectors or detector arrays have emerged as the dominant ‘cameras’ over near-infrared vidicon tubes. Since no commercial near-infrared camera system possesses a sufficient number of pixels to allow a single-capture IRR of the entire painting at a sufficient spatial resolution, multiple IRR images are collected using some sort of scanning arrangement. The simplest is a single pixel raster scanning system where a single-pixel detector is scanned across the painted surface or the image of the painting formed by a lens [7]. Other raster scanning systems use multiple detectors [7,8,9] with different spectral filters to acquire color and IRR images (960–1700 nm) [8] and some are true multispectral systems operating out to 2500 nm [9]. Such systems have spatial sampling of 0.25 mm at the painting or 16 samples per mm² and scan rates of ~1–7 h per m² have been reported [7,8,9]. Given the size of the area that can be scanned, typically ~1 m², no mosaicking is required for paintings smaller than this. Mosaicking is required for paintings larger than ~1 m². The large f-number collection lens allows for a larger depth of focus but combined with the short exposure time, bright lights are required for sufficient signal-to-noise, although the short dwell time minimizes heating. Such systems can be transported, but not readily, and do not offer a live view for the analyst.

Digital scanning back cameras utilizing small IR arrays have been constructed to improve the scan efficiency and enable the camera system to be portable [10, 11]. Such examples include the novel sensor constructed at the National Gallery, London. This IRR camera consists of a custom lens with a large field of view and a small InGaAs area array. The image from the lens is ‘scanned’ with the small IR area array (256 × 320 pixels) to create a mosaicked 5k by 5k pixel IRR image in ~20 min [10]. A commercial version of this design, the Osiris camera, utilizes a small linear photodiode InGaAs array instead of the area array [11], producing a 4k by 4k pixel IRR image. A small ‘live’ image is used for focusing. Spatial sampling varies with the size of the painting, assuming no mosaicking of the large frames is done. Assuming a 1 m² painting is imaged, the spatial sampling would be 0.25 mm (i.e. 16 samples/mm²). Since the InGaAs line array has some light leakage between pixels, the sharpness of the images as Nyquist is approached (1/2 sampling frequency) can be expected to be lower than with the raster single pixel scanners or the small framing array cameras.

Another scanning mode is step and stare, originally used in the first IRR camera systems [1, 4, 5]. This collection mode uses an IR camera with lens having a field of view that matches the size of the IR area array. In this capture mode, an IRR image frame is acquired from a small portion of the painting, and then the painting or camera is translated to acquire the next image frame. The first such digital camera systems utilized platinum silicide (PtSi) 512 × 512 or 640 × 512 pixel area arrays [4]. New IR arrays made from InGaAs or InSb having 1280 × 1024 pixels are now commercially available. Having a higher number of pixels in each capture frame allows slowing of the effective collection rate which in turn allows for longer integration times (10’s of msecs instead of a few msec). This allows for lower light levels and the collection of IR images in narrower spectral bands defined by spectral bandpass filters, rather than defined by the sensitivity of the detector being used. The spatial sampling and area covered depends on the lens and working distance as in the above case. To acquire a 4k by 4k pixel image assuming 20% overlap, 80 captures would need to be collected if using a 640 × 512 pixel detector array and 20 captures if using a 1280 × 1024 detector array. Practical collection times for manual easel systems to move the painting are ~30 s per image capture, giving total collection times of ~30 min. Automating this processing using a 2-axis motorized easel or a gantry system to move the camera reduces the capture rate to ~2 s per image, thus total collection times of ~5 min can be achieved. Since the IR array is not moved around the image plane of the lens as in the scanning back systems, IR focal planes that require cryo-cooling to extend the spectral range out to ~3000 nm can be used.

Such camera systems that employ area arrays and a step and stare scanning mode can image larger areas at fast scan rates while providing high enough signal-to-noise to utilize lower light levels and the use of spectral bandpass filters. They also provide live video, offering analysts the ability to study the artwork in real time.

The downside to collecting many image frames is that they require post-capture mosaicking. This problem increases if several spectral bands are collected. This has created interest in improved stitching software (both manual and semi-automatic routines). Fortunately, advances in computer processing power and new mosaicking software [12] have made mosaicking and registration less daunting. One useful advantage of the small field of view of the translating step and stare camera system compared to all but the single pixel scanners, is that the final IRR images produced are closer to orthophoto images. That is, each pixel is collected close to the painting normal and not from an oblique angle, reducing the amount of ‘projection’ distortion in the resulting images.

Instead of translation of the camera in step and stare scanning mode, one can rotate the camera up and down (tilt) and left to right (pan) to acquire the images needed to create the mosaic [13]. Such scanning modes work best for objects at infinity and Gargano et al. [13] have solved the mapping and focus issues in their IRR camera system through software processing.

In this paper we describe and provide case studies for a step and stare multi-band infrared camera system that overcomes many of the limitations that the IRR camera systems discussed above possess. The design was driven by the need to provide improved visualization of fine underdrawing lines and compositional paint changes. To handle paintings with thick paint layers, the decision was made to extend the spectral range beyond that of InGaAs cameras as well as to use spectral bandpass filters. To ensure the system could be used to examine works on paper and parchment, increased light sensitivity was deemed necessary as well. The resulting camera system thus took advantage of the availability of high quality, near-infrared lenses optimized for the 1000–2500 nm spectral region and newly developed image registration software tools specifically developed for mosaicking and registering IRR image frames [12]. The resulting sensor system can be used with both simple wooden easels and computer-controlled easels that provide automatic step and stare collections. With this system, high spatial resolution images in narrow spectral bands from 1000 to 2500 nm can be collected on paintings, including warped panel paintings, and also on light-sensitive works on paper. False-color IRR image products can be provided by viewing different infrared spectral regions in the red, green, and blue color channels of an RGB image.

Experimental

The infrared reflectography camera system (Fig. 1) consists of two diffuse lamps and a cryo-cooled infrared camera outfitted with a custom lens and interference spectral bandpass filters. The diffuse illumination system consists of two-150 Watt tungsten halogen (TH) lamps (Lowel Pro-light with Osram FVM bulbs) having two ground glass windows, placed 45°–60° from the painting normal. A rheostat (3PN1010B Staco Energy Products, OH) was used to adjust the light levels and color temperature. At integration times of 20–40 ms, the rheostat was used to drop the voltage to the lamps such that the light flux was 30–50 lx. Under these conditions the peak spectral radiance was 4 × 10⁻³ Watts/sr-nm-m² at 1400 nm. This corresponds to a color temperature of about 2060 K. A custom electronic easel (SmartDrive, UK) is used to move the artwork relative to the camera and lights. Several manual 2-axis easels have been used as well. For example, a traditional hand-crank easel on wheels was used to move the painting up and down and slide it left to right to capture the image frames. The camera system itself consists of a custom f/2 macro 50 mm EFL lens that is color corrected from 1000 to 2600 nm (Stingray Optics, NH). The lens has an exit pupil that is larger in diameter than the cold stop of the camera dewar. The cold stop of the dewar sets the f/d of the IR camera system with lens to 2.3. The liquid nitrogen dewar also contains a window and an interference bandpass filter that limits the spectral range from 1000 to 2450 nm and is attached to the cold shield. The focal plane array is a 640 × 512 InSb infrared detector with a 24-μm pixel pitch (SBF-193, Santa Barbara Focal Plane, CA) and is operated at 77 K using the liquid nitrogen dewar. The camera electronics and software allow for variable integration time and provide a live view for the operator. Typical integration times used are 8–50 ms. The digital frame-grabber provides 12 bit per pixel output. Pixel to pixel non-uniformity correction as well as non-uniformity illumination correction is done using a per-pixel two-point calibration method found in the software package IRWin (IRCameras, CA). A dark (lens cap) and uniform white target (99% diffuse reflectance Spectralon panel, Labsphere, NH) is collected and a table of offset and gain per pixel is generated by the camera software and stored. All of the collected images have this correction applied, thus skipping the need for a gray card correction in post processing. The overall system stability allows for the collection of images over several hours without the need to correct brightness in the mosaicking step.

Fig. 1

Diagram of the multi-spectral system, including InSb infrared camera (with window, cold stop and cold filter), custom lens, bandpass filter, and diffuse lamps. The painting is moved relative to the camera and lights using a manual wooden easel or a computer-controlled easel in the step/stare capture mode

Test panels originally made in 1993 [4] were re-examined. Each test panel contained drawing lines in different materials on a preparatory ground layer made from chalk and animal skin glue. Historic pigments ground in oil were painted on top of the underdrawing lines. The test panels have been previously characterized as to pigment particle size and paint thickness [4].

Diffuse reflectance spectra of test panels [4] were measured using a fiber-optic spectroradiometer (FS3, ASD Inc, Boulder, CO) with a leaf probe head as the light source and the Spectralon target for calibration. The contrast between the underdrawing and the preparatory ground layer was calculated by measuring the reflectance of the paint in a region with and without a lamp black underdrawing line.

The modulation transfer function (MTF) was estimated from the spatial frequency response (SFR) obtained from analysis of an image of a slanted edge in the Golden Thread Target (Image Science Associates, NY) [14]. The analysis was performed using a Matlab script (sfrmat.m, Burns Digital Imaging, LLC). The mosaicking and registration of the IRR images were done using scripts written in Matlab and described in Conover et al. [12]. The principal component analysis was done using the Minimum Noise Fraction in ENVI (Harris Geospatial Solutions, CO). The Kubelka–Munk modeling was done using Mathcad (PTC). The Z-stack software tool in Adobe Photoshop© (Adobe, CA) was used to prepare in-focus image frames prior to automatic image mosaicking.

Results and discussion

The requirements for the IRR camera were defined by the need to have a camera system that would allow for the examination of paintings and light-sensitive works on paper and parchment such as illuminated manuscripts. The camera system had to operate over the complete spectral range identified as providing the most usable information: 1000 to ~2500 nm. The selected spectral range of 1000–2500 nm is based on the prior studies of the wavelength dependence of the transmission of the paint layer [1, 6] and also a study that included the effect of the underdrawing material and the paint grounds [4]. This latter study examined only five spectral regions from 900 to 2500 nm. To confirm the findings, the test panels were re-measured with a fiber-optic spectroradiometer. The difference in reflectance measured in regions where light was transmitted through the paint layer and reflected off of the chalk ground to those regions where light was transmitted though the paint and reflected off the lampblack drawing line was calculated. A plot of the results for seven paints (Fig. 2a) shows the peak contrast or difference in reflectance varies over 1000–2500 nm depending on the pigment and its concentration. All of the paints show an abrupt decrease in contrast at ~2250 nm. The fall-off is partially attributable to broad absorption features associated with CH groups of the drying oil used as the binding medium for all of the paints, however some absorption features between 2250 and 2500 nm may also be due to the pigment. For example, the pigment azurite has absorptions from its hydroxyl and carbonate groups in this spectral region. The differences in peak contrast reflect the differences in absorbance due to electronic and vibrational transitions of the pigments and the binding medium in the near infrared, as well as the differences in Mie scattering arising from differences in particle size and indices of refraction. Such wavelength dependence has been seen in paintings using hyperspectral reflectance imaging in the near infrared [15]. The plot also shows a variation in the maximum value of the contrast or difference in reflectance with pigments, indicating the need for the camera system to image reflectance differences of a few percent.

Fig. 2

a The wavelength dependence for the difference in reflectance (i.e. contrast) for light which is transmitted through a paint layer and then is reflected off a white ground versus light which is reflected off lampblack. b The reflectance of lampblack, a common underdrawing material, and chalk, an ideal white

Measurement of the reflectance of the chalk ground from 1000 to 4500 nm shows there is lower contrast between the underdrawing material fireplace black beyond 2600 nm (Fig. 2b). The absorption features of the chalk ground are due to calcite and the binder (animal skin glue). Studies of the transparency of drying oil and egg tempera in this region finds the strong absorptions from 2650 to 4200 nm are from the CH and OH functional groups. Thus the region from ~2600 to 4200 nm is expected to provide reduced utility for IR reflectography of paintings. Thus the requirement for the spectral region of the camera was set to 1000 to ~2400 nm.

To enable mosaicking of IR image frames and aligning the image frames to a reference color image, the spatial sampling had to be sufficient to resolve fine cracks in the paint layers that are useful spatial features for image registration. Finally, the camera system had to allow for upgrades of the detector array by allowing for the same size IR area but with smaller individual pixels. Paint cracks are typically on the order of size of the diameter of a human hair, ~80 to 120 μm. A spatial sampling of ~90 μm also allows one to infer if an underdrawing is applied with a dry medium (stick) or wet medium (brush). For light sensitivity, the goal for the camera system was to keep the illumination near that of Gallery conditions, which are 200–300 lx for most paintings and 50 lx for drawings and illuminated manuscripts.

Selection of camera elements was determined by the driving requirements, operating spectral range and light sensitivity. For the detector array, an InSb array was selected over InGaAs because the latter is sensitive out to only 1700 nm. Mercury cadmium telluride (MCT) would also have been a viable choice, but the current high cost precludes it at the present time. The array described here is a commercially available 512 × 640 pixel array contained in a liquid nitrogen dewar.

Because the InSb array is sensitive to light out to ~5 μm, the focal plane needs to be cooled to cryo temperatures (<90 K). To achieve optimal performance, a cold spectral band filter limiting the light reaching the infrared detector to 1000–2500 nm was used. This minimized thermal radiation from getting to the detector array and reducing the signal to noise of the camera system. This makes the detector blind to thermal radiation (3000–5000 nm). This is a significant advantage over prior PtSi cameras and InSb systems that utilized a window that transmitted radiation from 1000 to 5000 nm. The large thermal signal (3000–5000 nm) filled the detectors and reduced the overall performance. Furthermore, these cameras used a visible camera lens that lacked an exit pupil, thus leading to strong vignetting at the edges of the field of view. Moreover, these lenses had low transmission from 2100 to 2500 nm, in part from the coatings and types of optical glass used.

To achieve the high spatial resolution, a custom lens was fabricated. The requirements for the lens were to be well corrected from 1000 to 2600 nm, to have a working distance from 20 to 100 cm, and to have an exit pupil that matched the f/d 2.3 cold stop of the dewar. The IR focal plane dewar is designed to have a small opening (cold shield) to limit thermal radiation from entering the dewar and getting to the focal plane array. Thus, the effective lens aperture stop is the cold stop in the dewar. By placing this requirement on the lens, high sensitivity, no vignetting, and lower noise is achieved compared to prior cameras. A summary of the key parameters that describe the camera system is presented in Table 1.

Table 1 Key camera parameters

The performance of the camera system has been determined using two metrics, signal-to-noise ratio and the modulation transfer function (MTF), in each of the three spectral bands. The signal-to-noise is measured in reflectance units at the art object. Many factors contribute to the overall noise of the camera system, including shot noise, residual fixed pattern noise from the pixel-to-pixel non-uniformity, and the random noise of the detector. The measurements here included all of these factors, and the signal-to-noise was determined from diffuse reflectance standards having reflectance of 2 and 99% in the spectral bands of interest. For each spectral band, the reflectance standards were imaged after calibration of the camera. The resulting mean digital count difference between the reflectance standards was used to convert digital counts into reflectance units. The noise was determined by calculating the standard deviation over small areas for the 2 and 99% reflectance targets and then converted into reflectance units and summarized in Table 2. These results show in reflectance units the noise is less than 0.2% across the spectral range and less than ~0.1% across the 1000–1800 nm spectral range, which allows small differences in reflectance to be detected.

Table 2 Performance parameters

The MTF shows the loss in modulation with increasing spatial frequency. The measured MTF includes the degradation from the optics, detector, and electronics. Of particular interest is the modulation at Nyquist, and its measured value for the different spectral bands is summarized in Table 2. The observed modulation at Nyquist in each spectral band is in the acceptable range for digital camera systems and is adequate to support super-resolution.

While the performance metrics confirm the system performance in optical terms, image examples provide the users with a more intuitive insight into the camera system’s performance. Four image examples are given representing the diversity of problems the camera system was designed to address, including (1) high resolution images of underdrawing in paintings, (2) use of IRR false-color image products to provide insight into artists’ working methods, (3) examination of light-sensitive artworks such as drawings or illuminated manuscripts, and (4) IRR of warped panel paintings.

Case studies

Improved visualization of underdrawing within paintings

The camera system was used to collect infrared reflectograms of the painting The Madonna and Child with Saint John the Baptist, attributed to Giulio Romano (c. 1522-1524), which is in the collection of the Walters Art Museum (Baltimore, MD). These series of infrared reflectograms were acquired in three spectral bands (1100–1400, 1500–800, and 2100–2400 nm) and then mosaicked and registered. Comparison of the three IR images reveal an underdrawing that shows two faces that appear to be in a close embrace (Fig. 3). The increased visibility of the drawing with increasing wavelength is attributable to the decrease in light scattering by lead white paint. While the image acquired in the 2100–2400 nm spectral band shows the clearest image of the underdrawing, the drawing related to the final composition is also present, presenting a jumbled image.

Fig. 3

Color and infrared details from The Madonna and Child with Saint John the Baptist, attributed to Giulio Romano (c. 1522-1524), Walters Art Museum. IR reflectograms taken in the spectral regions of 1100–1400, 1500–1800, and 2100 to 2400 nm show progressive improvements in the clarity of the underdrawing of a prior planned composition

Various image processing procedures can be used to better separate these two drawings and remove some of the contributions of the final paint layers. Methods based on principal component analysis, which create orthogonal images along axes of maximum variance, and even Kubelka–Munk-based models to remove the contributions from the final paint layer have been suggested [9, 16]. However, such algorithms require both low noise images and good image registration between the spectral images.

A principal component analysis using the Minimum Noise Fraction algorithm on the three registered IRR images gave an Eigenimage showing more clearly features associated with the prior underdrawing (Fig. 4). For example this transform image is free of most of the painted and drawn features of Mary’s face found in the 2100–2400 nm image. The success of principal component analysis to separate out such overlapping images depends on the statistical process of the analysis itself. A more deterministic process would be to try and separate the images using a model based on the light interaction within the paint layers.

Fig. 4

Comparison among IR reflectogram collected in the spectral band from 2100 to 2400 nm (a), image obtained from applying eq-1 (b), and the second principal component image (c) from the PCA analysis of the three IR reflectograms

Such a model to remove the contribution of a paint layer over that of underdrawing, based on approximations of the 2-flux Kubelka–Munk theory, was proposed by C. Wiener [16] and used here to see if a similar result to the principal component analysis could be obtained. The enhanced underdrawing image, I_UD (x,y), can be calculated using the visible image, I_VIS (x,y) and the IR reflectogram, I_IRR (x,y) assuming the values for the absorption coefficient (K) and the scattering coefficient (S) are known for the visible layer. Note that the terms X₁ and X₂ in Eq. 1 adjust the contrast and brightness respectively.

$$I_{UD} \left( {x,y} \right) = X_{1} \left[ {\overline{{\left( {I_{VIS} \left( {x,y} \right) + S} \right)^{K} }} \left[ {\frac{{I_{IRR} (x,y)}}{{\left( {I_{IRR} \left( {x,y} \right) + S} \right)^{K} }}} \right]} \right] + X_{2}$$

(1)

The K and S coefficients are expected to vary with the composition of the paint layer. For this case study, the detail was made up of several individual IR images. K and S were optimized for each image frame using the 1100–1400 nm IRR image as I_VIS (x,y) and the 2100–2400 nm IRR image as I_IRR (x,y), producing a further improved image (Fig. 4).

Multi-band IRR to visualize changes in the paint composition

IR reflectograms collected in different spectral bands can be used to highlight reflectance differences of pigments. In other words, multispectral imaging in the near-infrared is a useful tool to separate many pigments even though there is not sufficient reflectance information to identify them. The use of false-color images, constructed from such spectral images, provides more insight than the black and white IR reflectograms. An example of this can be seen in the false-color images constructed from three IR spectral band images of Mary Cassatt’s Little Girl in a Blue Armchair (c.1878), in the collection of Mr. and Mrs. Paul Mellon at the National Gallery of Art, Washington DC. The false-color image was constructed from three IRR mosaicked images (Fig. 5). The 1100–1400 nm IRR was used for the blue channel, 1500–1800 nm IRR for the green and 2100–2400 nm IRR for the red.

Fig. 5

Mary Cassatt, Little Girl in a Blue Armchair (c.1878). Collection of Mr. and Mrs. Paul Mellon, National Gallery of Art, Washington, DC. (Top) Color image, (Middle) IRR 1100–1400 nm, (Bottom) False–color IRR (B 1100–1400 nm, G 1500–1800 nm, R 2100–2400 nm)

Examination of the monochromatic IRR image (1100–1400 nm) reveals changes were made in the gaze of the girl and the height of the back of the sofa was increased. However, the false-color IRR image product provides additional information including the prior placement of the dog on the floor in front of the couch. As can be seen in Fig. 5, the reflectance properties of the sofa addition and the chair on the back wall are similar, as both appear yellow in the three spectral bands chosen for the false-color image. The ‘yellow’ color of the sofa addition and chair on the back wall is suggestive this furniture was painted with different materials relative to the chairs in the foreground, which appear pink in the false-color image. This information is of interest, as the painter Edgar Degas was known to have been involved in Cassatt’s decision to make changes to this painting, although the specific changes were not documented. The ‘yellow’ color of the two pieces of furniture could indicate changes made in the composition that were instigated by Degas.

Illuminated manuscripts and works on paper

The collection of high spatial resolution images of illuminations represent a challenge given their high sensitivity to light. The IRR camera system here utilizes light levels similar to Gallery conditions (Table 2).

The IR camera was used to collect IR reflectogram images in the 1100–1400 nm spectral band of Christ Giving the Keys to St. Peter by Lorenzo Monaco, in the Rosenwald Collection of the National Gallery of Art, Washington DC. Details from the IRR image (Fig. 6) show useful information that aid in the understanding of how the illumination was painted. For example, the dark blue fold at the bottom right edge of Christ’s mantle has been painted over an already decorated section of the pavement. The same image also shows the penciled note on the verso of the miniature. Fine underdrawing can also be observed in the robes, and along the left edge of St. Peter’s head, which might have been slightly repositioned. The flesh tones seem to have been painted quite loosely, then better defined by dark outlines (see especially the figures’ necks and Peter’s wrists).

Fig. 6

Detail color and IRR (1100–1400 nm) images from Christ Giving the Keys to St. Peter by Lorenzo Monaco, Rosenwald Collection, National Gallery of Art, Washington DC

Imaging warped un-cradled panel paintings

A major challenge to obtaining high spatial resolution IR reflectograms of un-cradled panel paintings is their deviation from planarity and deviations of several centimeters are not uncommon. This is beyond the depth-of-focus of most high f-number IRR camera systems. Single pixel raster scanners have incorporated auto-focus mechanisms which moves the detection system towards and away from the painting as needed [9] to keep the ‘image’ in focus.

The practical depth-of-focus for the camera presented here was measured using a slanted edge to determine the loss at modulation over a range of spatial frequencies most affected by defocus. A “Golden Target” which incorporates a slanted edge was imaged with the IR camera system as it was moved in and out of the object plane. A metric of the loss modulation due to going out of focus is to calculate the area under the MTF curve from 0.15 to 0.45 cycles per pixel for each position. Limiting the area loss to 30% of the normalized area gave a depth of focus of ±2 mm in object space. Thus, to image panel paintings having outer plane deviation in excess of ±3 mm would give unacceptable blurring of the IRR mosaic. Given the high signal-to-noise of the camera system, some adaptive sharpening can restore much of this loss in sharpness, as this limit is approached.

To ensure an in-focus IR reflectogram mosaic, a Z-stack image collection protocol was tested on Andrea del Sarto’s un-cradled panel painting Charity, c. 1530, in the Samuel H. Kress Collection at the National Gallery of Art in Washington DC (Fig. 7). The painting is 92.5 cm wide and 119.5 cm high, and deviates from planarity by ~1.5 cm on one side and 2.5 on the other side. Examination with all three spectral bandpass filters identified the 1500–1800 nm filter as providing the best overall visibility of the underdrawing. Given the detector array size (640 × 512) and a 20% overlap of successive IRR images, 11 × 18 images were required to give the final image having dimensions of 5530 × 7150 pixels.

Fig. 7

Color image (lower left) of Andrea del Sarto’s Charity, c. 1530. The Samuel H. Kress Collection, National Gallery of Art, Washington DC. Overall IRR (right) collected from 1500 to 1800 nm at 150 ppi and detail IRR images (top left) from a proximate and a distant area of the warped panel

If the IRR camera was focused on the high point of the panel, near the center, and no focus adjustments were made as the painting was scanned, the individual IRR image frames would blur near the left and right sides. Two different actions could be taken to restore focus as the IRR image frames were collected. One option would be to adjust the focus of the lens, but this would change the magnification. The other option would be to leave the focus of the lens unchanged and adjust the distance between the camera and the painting until the image is in focus again. Only the latter approach would keep the image magnification fixed while scanning, and thus this latter approach was used. The IR camera system was mounted on a translation stage with a micrometer screw, which allowed moving the camera along the optical axis of the lens, the z-axis.

To account for the 2.5 cm variation from planarity, 14 different camera positions with 2 mm spacing along the z-axis were used. For most of the 198 image frames required for the mosaic, each with a field-of-view of 10.9 × 8.7 cm, the local inclination of the painting surface was small, and a single z-position was enough to have a good focus across the whole field-of-view. At the left and right edges of the painting, on the other hand, the inclination of the painting surface was so great that a single z-position did not produce a focused image over the whole field-of-view. In these cases, for a single x, y-position, multiple images (2 or 3) were acquired with different z-positions. These digital images were arranged in layers (z-stack) and processed with an automatic processing tool that merged the stack in a single image, locally selecting the content from the sharpest layer. The resulting sharp image frames were then mosaicked with a point-based registration algorithm [12] to form a sharp high-resolution infrared reflectogram of the whole painting (Fig. 7). Comparison of the detail IRR of Mary’s face (z = 0) (Fig. 7) and the detail from the lower right edge (z = 2.5 cm) showed the approach taken here gave a high-resolution IR image that was in focus across the warped panel. Since the image mosaicking was done by registering the IR image frames to the color image no image distortion was introduced into the final refletogram.

Conclusions

A variety of scanning IRR camera systems have been explored over the past 20 years since solid state IR detectors and detector arrays have become more widely available. We have shown that a traditional step and stare system utilizing an extended spectral range and having high sensitivity can be used to produce multi-band IRR image products of use to scientists, conservators, and art historians. The system is adaptable as improvements in IR focal plane arrays become available. Recent testing with an InSb 1280 × 1024 pixel array with 12-μm pixels showed the system can acquire useful IRR images at 560 dpi, thus nearly matching the resolutions of x-radiography. The cost of commercial InSb and MCT detectors are rapidly dropping and near infrared commercial lenses are becoming more widely available worldwide. The new capabilities offered by image processing on multispectral data sets and the new information which is emerging suggest IRR camera systems like the one described here will find their place among the other novel IRR camera systems such as the digital scanning back cameras and the single pixel raster scanners.