Digital technology virtual restoration of the colours and textures of polychrome Bodhidharma statue from the Lingyan Temple, Shandong, China

This work proposes the virtual restoration of the Bodhidharma polychrome sculpture from the Lingyan Temple in China. Based on scientific analyses and simulation experiments, exterior colours and textures were virtually restored by combining 3D scanning and multi-view 3D reconstruction. At the same time, an efficient cultural relics high-fidelity information 3D modelling method was proposed. Colours and textures are essential for polychrome cultural relics, reflecting the historical appearance and technology. Due to long periods of natural ageing and the destruction of environmental factors, the colours and textures of polychrome sculptures are often altered or radically changed, making it difficult for people to appreciate the initial appearance. With the rapid development of digital technology and high-fidelity 3D modelling, virtual reality technology allows us to restore the appearance of cultural relics. This study expands the dimension of cultural relics exhibition, provides new perspectives for archaeology, art history and cultural heritage research, and provides a reference for the virtual restoration and digitalised archive of other cultural relics.


Introduction and the aims of the research
Lingyan Temple (灵岩寺) in Shandong province is an essential part of mount Tai world's natural and cultural heritage. A set of 40 polychrome arhat statues dating from Song (960-1279 CE) and Ming (1368-1644 CE) Dynasties were preserved in the Qianfo Hall (千佛殿).
These statues are skillfully sculpted with vivid and lifelike characters and have high artistic value. Due to environmental factors and natural ageing, dust and pollution, pigment discolouration, fading, an efflorescence of salts, and the loss of paint with flaking and peeling, the appearance of the arhat statues was significantly compromised. From August 2019 to May 2021, Shandong Cultural Relics Conservation and Restoration Center and the Institute of Cultural Heritage Shandong University implemented the protection and restoration project of part arhat statues in Qianfo Hall of Lingyan Temple in Changqing (长 清), Shandong (Phase I). Figure 1 is the comparison of the Bodhidharma statue before and after conservation. In the protection and conservation work, dust and pollutants on the surface of 12 arhats on the east side of Qianfo Hall have been addressed. The areas of flaking, efflorescence, cracking and peeling in the paint layer have been treated.
Colours and textures are essential features for the historical, artistic, and scientific values of polychrome cultural relics, which reflect the historical appearance and technology. Restoring the appearance state (colour and texture) of polychrome cultural relics at the initial or a specific period can enhance people's understanding of the original appearance of cultural relics. At the same time, it is also significant to art history, archaeology, and cultural heritage research. Under the principles of "no change original state of cultural relics and minimal intervention" [1], restoring the original appearance of the relics related to a specific historical period is generally   impossible. However, 3D modelling and virtual reality technology allow us to create a virtual restoration of the initial appearance of works of art without compromising the current condition and removing historically significant overpaint. The research on the virtual restoration of cultural relics is growing. As a topic with a broad research scope, it mainly focuses on five perspectives: virtual splicing of broken cultural relics [2][3][4][5], virtual matching of missing parts of cultural relics [6][7][8], virtual restoration of the shape and structure of cultural relics and historical space [9,10], virtual reconstruction of ancient human's appearance [11][12][13], and virtual restoration of the colours and textures of objects. In terms of the virtual restoration of colours and textures. Li et al. [14] designed and developed a computer prototype for the Dunhuang fresco colour restoration and image retrieval in the early twenty-first century. The colour restoration of fresco images was completed according to the fade and changed rules of pigments under different light, temperature and humidity over time. Wei et al. [15] developed a knowledge-based colour restoration system for ancient murals. The technique combines the prior knowledge of the environmental factors affecting colour change, the colour distribution knowledge specific to the period and location, and the knowledge of painting style and pigments. Pappas et al. [16] presented several techniques for digitally restoring the original appearance of old paintings. These techniques include sample mean matching, linear approximation, iterative closest point approximation, white point transformation, and RBF approximation.
They aim to find the best colour transformation from the deteriorated to the correct colour. Nemes et al. [17] proposed a nonlinear digital restoration method of painting colour based on support vector regression. The work examines the efficiency of the supervised learning method in deriving colour correction functions.
Yin et al. [18] introduced a technique for restoring the colours of ancient heritage, taking Japanese Noh masks as an example. The process involves the measurement of the object's 3D mesh and colour, the calculation of the colour variation and the original colour restoration of the 2D texture image extracted from the object's 3D surface. Ikeuchi et al. [19] established a 3D geometry model reconstruction process of the Nara Giant Buddha in combination with the digitalised archive and restoration. In this study, two texture mapping methods were developed, and virtual restoration of the original appearance of the Giant Buddha was carried out under the guidance of experts using 3D scanning data and literature. Li [20] applied computer graphics processing technology combined with texture mapping technology of cultural relics 3D model to restore the colour texture of faded objects. The restoration work was based on the partial painting remains, and the painted results were based on historical materials. Fieberg et al. [21] analysed the fading mechanism of pigments in two of Vincent Van Gogh's paintings and digitally reconstructed the original appearance of the faded flowers in the paintings. Kirchner et al. [22] performed a physical reconstruction of the pigments used in Van Gogh's famous painting "Field with Irises Near Arles" based on the raw materials and historical materials related to the production process and determined the optical properties of various pigments. The optical properties were combined with a map of pigment distribution, and the painting's original colours were digitally reconstructed. Machidon et al. [23] proposed and applied a specific set of unsupervised, pipelined image processing schemes, which digitally restored the colour of degraded reversal films heritage by performing colour cast removal and colour correction. Zhou and Ma et al. [24,25] studied colour reconstruction and costume restoration of Buddha statues unearthed from the Site of Longxing Temple in Qingzhou, Shandong Province. The study achieved scientific, artistic, and natural restoration effects based on the scientific analyses of cultural relics materials and the simulation experiment. It provided a new idea for the virtual recovery of cultural relics. However, the virtual This paper takes the Bodhidharma statue as an example of carrying out a virtual restoration of its exterior colours and textures. The Bodhidharma statue is colourful and rich, the body proportion is like a person, and the expression of the character is solemn and serene, tolerant and introspective; it is a rare masterpiece in the Realistic Buddhist statue of the Song Dynasty. Lingyan Temple polychrome arhat statues have experienced at least eight repainting events in history; the most recent large-scale repainting occurred in the 13th year of the reign of Tongzhi (1874 CE) (同治) in the Qing Dynasty (1639-1912 CE) [26]. The virtual restoration work is based on the repainting in the 13th year of the reign of Tongzhi.
The research work is divided into four parts: scientific analyses of the paint layers, simulation experiments (physical reconstruction of colours and textures), 3D modelling, and mapping of colours and textures. Firstly, multi-analysis techniques are used to reveal the materials and techniques used in the paint layer of the statue. Secondly, the analysis results are used as the basis to perform simulation experiments, physical reconstruction the original colours and textures of the statue. Finally, the texture mapping technology is used to map the reconstructed colours and textures to the surface of the statue 3D model to achieve the 3D restoration of colours and textures. The restoration process is shown in Fig. 2, in which the scientific analyses of the paint layers have been published [27]. Based on scientific analysis and simulation experiment, applying digital technology to achieve 3D restoration of original colours and textures is the difference from other restoration methodologies. The research intending to reveal and show the appearance state of the Bodhidharma statue during the 19th C. and promote people's new understanding of the statue. At the same time, the research aims to provide a methodology for the virtual restoration of the colour and texture of similar polychrome cultural relics.

Simulation experiment
According to the analysis results of the Bodhidharma statue paint layers (Table 1), the same materials and techniques were used to reconstruct the colours and textures of the statue in the 13th year of the Reign of Tongzhi in  Table 2. Raman spectrum analyses tested various mineral pigments to ensure that the pigments used in the simulation experiments were the same as those used in the Bodhidharma statue.

Pigment preparation
The granularity of pigments strongly influences colour [28] and texture [29]. Therefore, before the different colours and textures were physically reconstructed, various mineral pigments were first ground to the same particle size as the pigments corresponding to the Bodhidharma statue. For example, when preparing red cinnabar pigment, 200 mesh cinnabar pigment was placed in an agate mortar and ground until the average particle size was less than 10 μm. The pigment particle sizes were measured using a Lecia DVM6 super depth of field 3D video microscope. When measuring, a small amount of ground pigment was observed on a glass slide. Randomly selected 10-30 particles were measured under magnification, and their diameters were averaged. In some areas of the Bodhidharma statue, the pigment layer is composed of two minerals. For example, the D1-8 L1 pink pigment layer comprises red ochre and chalk. In the simulation experiment, pigments were mixed according to the semiquantitative analysis results of scanning electron microscopy coupled with energy-dispersive X-ray analysis (mass ratio of different minerals).

Gelatin alum solution preparation
Gelatin is a widely used binding media in ancient Chinese polychrome cultural relics [30,31]. Alum, a colour-fixing agent, is usually mixed with gelatin to prepare gelatin alum solution to blend pigment. Studies have shown [32] that when gelatin is used as the binding media in the pigment of polychrome statues, a suitable ratio of gelatin to water is between 1:32, 1:50 and 1:80. According to this, the physical reconstruction (painting different colours and textures planks) uses a gelatin alum solution of 2% gelatin and 0.5% alum to bind the pigments and the priming layer minerals. When preparing gelatin alum solution, gelatin and alum (mass ratio 4:1) were placed in the beaker, which was covered with distilled water (submerged in the solute) and soaked for 5 min. Distilled water was added and heated to 80 ℃ until the gelatin and alum were wholly dissolved.

Painting different colours and textures planks
The experience analyse show a white kaolin priming layer was painted on the ground layer of the Bodhidharma statue and the pigment layer was painted on the smooth white priming layer. When painting, the planks simulate the painting process of the statue. Firstly, an appropriate amount of kaolin was mixed with the gelatin alum solution, blending it repeatedly with a brush and then evenly brushing it on the planks' surface. After waiting for the white priming layers to be completely dry, paint layers are applied. It is essential to find the proportion of binding media solution and pigment; paint should not be too thick or too thin. Analyses of the paint layer show that the black pigment (lamp black) and priming layer on the surface of the Dharma statue cassock was painted on white paper, which was pasted on the ground layer. The same materials and techniques have been used to simulate painting the black plank. Tung oil was used as binding media to paste gold foil in the decoration of Dharma statue. Physical reconstruction of the gold foil's colour and texture is also applied to the same materials and techniques as the statue gilding, that is, the gold foil was pasted on the priming layer surface with boiled tung oil.

Colours and textures acquiring and processing
A Nikon D3X ® with an AF-S Micro 60 mm lens was used to photograph the completed difference colours and Fig. 7 a, d,   rection" in the lens correction bar and selecting the configuration file created by ColorChecker ® software in the camera correction bar. • 4. Selecting all images to carry out "synchronous" processing and exporting the JPG file format image after colour correction.
• 5. Cropping the image files in Adobe Photoshop ® and saving the files to the same folder for future use. Different colour and texture information after colour correction processing are shown in Fig. 3.

Statue 3D modelling
Creating a 3D model of cultural relics containing highfidelity information is essential to digitalising cultural relics. In the past, high fidelity information 3D models of cultural relics was usually built by manual texture mapping. In the texture mapping process, the position of the object and the images of cultural relics are manually adjusted within the computer software to align the colour texture in the images with the corresponding position of the model. Then, mapping the colours and textures to the model is realised through manual mouse brushing. The operation needs to expand the UV coordinates of the model in advance, and the operator needs to constantly adjust the model's position and images and adjust the brush size. This modelling process has many steps, work efficiency is extremely low, and colour inconsistency and double shadow are common problems that may develop in texture maps. In addition, because the operator manually adjusts the matching between colour texture and model position, it is difficult to avoid the mismatch between colour texture and model. Diao [33,34] and his team developed automatic texture mapping technology for 3D model and object image data using digital HD image acquisition, laser 3D scanning, multi-view 3D reconstruction, etc. Xia [35] applied the nearest point iterative algorithm to accurately align the model data from different sources and interpolated the texture coordinates of each vertex of the high-precision geometric model to create a precise  3D texture model. Fawzy [36] proposed a hybrid digital reconstruction method in the research of buildings using 3D digital documentation. The technique combines the 3D data obtained by close-range photogrammetry technology with the 3D data of ground laser scanning to realise the building's 3D digital documentation with higher accuracy. Yu et al. [37] studied and designed the FOTOMOULD 3D reconstruction system of dual-camera structured light to achieve the vivid effect of colour and texture while obtaining high-precision geometric information in the 3D reconstruction of the polychrome Bodhisattvas statue in Mogao Grottoes (莫高窟). However, the hardware structure of the 3D reconstruction system is complicated, as is the later data processing process. Based on the virtual restoration of colours and textures of the polychrome Bodhidharma statue, we proposed a modelling method of cultural relics high-fidelity information 3D model. The technique combines the highprecision geometric information of a 3D scanning model with the high-resolution colour texture information of a multi-view 3D reconstruction model. An HP Z8 G4 ® workstation equipped with an HP DreamColor Z27 G2 Studio Display ® as computer hardware was used in the modelling work. The specific process is as follows.

High-precision 3D geometric modelling
A Creaform Go!Scan 20 ® handheld structured light 3D scanner was used to obtain high-precision 3D data of the Bodhidharma statue. The raw data storage size is 10.9 GB, with 317,784,656 triangles (Additional file 1: Fig.   S1). Geomagic Wrap ® (v 2015 16.1.2) was applied to process the 3D data. They are using the manual registration tool to splice the data of different scanning sites. Global registration was further applied to reduce data deviation and delete the overlapping surface. The date standard deviation is only 0.44 mm after global registration (Fig. 4 b). Then, a series of commands such as merging, mesh doctor, filling holes, repairing, denoising, and local smoothing were carried out to build a complete highprecision geometric model. The specific modelling process is shown in Fig. 4. The geometric model comprises 19,297,538 triangular meshes, the original point cloud data spacing is ≤ 0.5 mm, and the theoretical accuracy is better than 0.5 mm. Figure 5 shows the magnified facial image of the Bodhidharma statue's high-precision geometric model and images of the model's front, right, left, and back.

High-resolution texture model modelling
The multi-view 3D reconstruction technique, called close-range photogrammetry in surveying and mapping [38], was used to build the high-resolution texture model of the Bodhidharma statue. A photographic camera, Nikon D3X ® with an AF-S Micro 60 mm lens, was used to shoot multi-view images. Two professional softlight photographic lamps (Broncolor Softbox 100 × 100) provided a uniform and stable light environment (colour temperature 5500 K). A total of 900 high-resolution multi-perspective images were taken, and the overlap area of adjacent photos was greater than 2/3. The camera positions diagram is shown in Additional file 1: Fig. S2. Agisoft Metashape Professional ® (v 1.6.0) software was applied to build a high-resolution texture model of the Bodhidharma statue. The texture model was completed with the forward workflow: aligning images (highest), generating dense point cloud (middle), generating meshes (middle) and texture mapping. The specific modelling process is shown in Fig. 6, and the processing parameters are shown in Additional file 1: Table S1. Figure 7 shows the texture images of different sides of the statue texture model (a, d, e, f ) and the facial amplification texture image (b). The texture image is clear and photorealistic, with 8192 × 8192 pixels in unfold. Since the statue's lower back surface is only 5-10 cm away from the wall, the images of the lower right back of the statue cannot be obtained. This results in meshes and textures missing at the lower right back of the texture model after modelling (Fig. 7f ). The skeleton of the texture model is composed of 8,027,721 triangles. It can be seen from the 4. Export and save the model ④ after the texture is generated. 5. Take high-precision geometric model ① as the reference model to align model ④ through matching scales (Fig. 8e) and feature points alignment (Fig. 8f ). 6. After points alignment, "fine registration" is applied to realise fine alignment between model ④ and the high-precision geometric model (Fig. 8g). 7. After fine registration, export and save the high-fidelity information model ⑤.
The texture images of the high-fidelity information model of different sides of the statue are shown in Fig. 9a, d, e and f. Table 3 lists the quantitative analysis data for each model performed by Geomagic Wrap ® . The meshes number, point spacing, dimension and surface area of the high-fidelity information model are the same as the high-precision geometric model, and the standard deviation is 0.0001; that is, the two models are nearly identical in the spatial dimension. The texture model has different geometry dimensions from the high-precision geometry model. The texture model after dimension registration has a noticeable deviation relative to the geometry model. The deviation of the edges and angular areas is more than 15 mm, the maximum deviation is 75.18 mm, and the standard deviation is 6.57 mm (Fig. 10). This shows that the geometric precision of the texture model facial image after removing the texture (Fig. 7c) that the surface of the mesh model is rough, the data noise is evident, and there are some missing phenomena.

High-fidelity information 3D modelling
The high-precision geometric model has space and dimension information, but no colour and texture.
The texture model has high-resolution colour texture and space information, but no dimension information same the real object. Through the mutual matching and alignment of two different models, mapping the high-resolution texture image from the texture model to the geometric model can result in the fusion of high-resolution texture information and high-precision spatial information. The open-source software CloudCompare ® (v 2.11.3) was used to create mutual matching and alignment of the two models. The specific modelling process is as follows: 1. Open the high-precision geometric model ① and high-resolution texture model ② in CloudCompare ® software simultaneously, take model ② as the reference model aligning model ① through matching scales (Fig. 8a) and feature points alignment (Fig. 8b). 2. Export and save the aligned model ③. 3. Import model ③ into the project file of texture model ② to replace the original model and run "Generate texture" (Fig. 8c) in Agisoft Metashape ® .
is much lower than that of the high-fidelity information model. Compared with the texture model, the high-fidelity information model also complemented all the missing texture and mesh data on the back (Fig. 9 a, within the green line is the missing data area of the texture model), and the texture area increased by 5538.40 cm 2 , among which the effective texture area increased by 2390.35cm 2 .
In addition, the high-fidelity information model has smaller point spacing and finer mesh.

Mapping of reconstruction colours and textures
The surface of the Bodhidharma statue is rich in colour, and the pattern and line decorations are various and Fig. 12 a, e, f, g texture images of the front, right, left, and back side of the Bodhidharma statue after virtual restoration, respectively; b texture unfolded image; c and d statue local photo and the details image after restoration complex. The biggest challenge in texture mapping is accurately matching the position relationship between colour texture and object model. Mapping of reconstruction colours and textures through texture replacement in texture images of high-fidelity information models can ensure accurate matching between colour texture and model. Mudbox 2020 ® (v.14.0.0.0) software was used to complete reconstruction colours and textures mapping.
Here the Bodhidharma statue high-fidelity information 3D model was imported into the software, and gypsum was selected as the preset material. The images of reconstruction colours and textures were loaded into the image browser, and selecting a texture image as the template.
In the 3D view interface, the function "Projection" was applied in the drawing tool to start mapping textures. The texture mapping process does not need to create a new image layer, directly mapping reconstruction colours and textures on the texture image layer of the statue highfidelity information 3D model to ensure that the textures exactly match the model position. When mapping the flower decorations on the surface of the statue, selects "Painting" mode to paint, so as to imitate the hand-painting as much as possible. The schematic workflow for texture mapping is shown in Fig. 11. Additional file 1: Fig. S3 shows the colour texture state of the statue in different stages of texture mapping. Figure 12a, e, f and g are the different sides texture images of the Bodhidharma statue after the virtual restoration of its exterior colours and textures. The statue, after virtual restoration, has bright colours and textures, the texture unfolded image (Fig. 12b) reaching 8192 × 8192 pixels. Table 4 lists the L*a*b* values, measured in Adobe Photoshop ® and colours difference values (ΔE) of the statue's different colours before and after virtual restoration. The results show that ΔE of all colours are greater than 1.5; that is, the changes in all colours can be discernable by human eyes. The ΔE of the pedestal edge black area and reddish-brown skin area are less than 5; colours change are relatively small. The ΔE of the pedestal reddish-brown area is greater than 60; the colour change is excellent. By comparing images of the current condition ( Fig. 12c) with the restored detail image (Fig. 12d) of the same position, the colour texture and position relationship of the decorative patterns on the statue surface have been accurately restored.

Conclusion
Virtual restoration added the missing area of the paint layers of the Bodhidharma statue. It restored the whole external appearance of the statue to create a reading of how the polychrome work may once have appeared, using analytical data from 19th Century paint layers. The same method could be used to create a virtual restoration using earlier painting schemes. The large-scale application of bright colours such as red, green, blue and yellow reflects the colours aesthetic orientation of the Chinese folk plastic arts in the Qing Dynasty. This is an important expression of the Sinicization of Buddhism and the secularisation of the colour of the statues. The virtual restoration work has enhanced the understanding of the original appearance and ageing of statues. Brought a new perspective for archaeology, art history, cultural heritage, and other related research and provided a reference scheme for the colour and texture virtual restoration of similar polychrome cultural relics. The 3D modelling method of high-fidelity information proposed in restoration work is realised based on mature commercial software and hardware. Thus it has practical significance for the efficient digitalised archiving of cultural relics. There are also shortcomings in the restoration work. For example, when mapping the decorative pattern on the surface of the statue model, it is difficult for the manual painting operation in the software to accurately represent the thick and light pigments in traditional Chinese painting techniques. Besides, it is cannot re-orient dynamically the direction of the brush strokes in the software when mapping texture. It is expected that there will be better algorithms to assist the restoration of painting techniques in the future. However, the virtual restoration opens avenue for changes and alterations that could not be possible with the real cultural relic. Artificial intelligence and digital technology are promising to solve (or assist in the task) the new challenges in the virtual restoration of cultural relics.

Additional file
High-resolution texture model modelling processing parameters.