Using Virtual Imaging Technology to Restore Cultural Heritage in Situ Virtually

The restoration of incomplete cultural heritage is important for presenting the value of cultural heritage. Usually, the restoration methods mainly include physical restoration, digital restoration, digital projection restoration, and plane patterns restoration, which all have insurmountable shortcomings. This paper presents a virtual imaging technology based on a kind of visual illusion phenomenon to restore cultural heritage in situ virtually. We proposed a virtual imaging restoration device and examined the practicability of the virtual imaging technology in restoration by applying it in restoring an incomplete bronze mirror. The results showed that the virtual imaging restoration performances better than traditional restoration methods from the perspective of keeping authenticity, non-interference, reversibility and discernibility of the cultural heritage. The potential of this technology and its encouraging results may provide a better alternative to the restoration of certain kinds of incomplete cultural heritage and a useful tool to pretest the restoration programmes.


Introduction
Cultural heritage is a kind of human treasure, due to various reasons, a considerable portion of cultural heritage is incomplete, which hinders the presentation of its values. One way to give people a complete cultural heritage is restoration, which mostly consists of four conventional methods: (1) Physical restoration, restoring incomplete cultural heritage to complete state by physical materials [1][2][3][4]. (2) Digital restoration, restoring the digitalized incomplete cultural heritage to complete state by multimedia technology [5][6][7][8]. (3) Digital projection restoration, restoring the original color and texture of incomplete cultural heritage by projecting the image onto the surface of the object [9,10]. (4) Plane patterns restoration, restoring incomplete cultural heritage to complete state by placing the plane patterns of the incomplete areas behind its corresponding areas. However, all the methods mentioned above have insurmountable shortcomings. For physical restoration, the restoration is carried out on the body of cultural heritage by applying physical material to recover the missing part of the object, which is sometimes non-removable. For example, gypsum, as a kind of common restoration material for ancient incomplete ceramics, is easy to be retained by the surface of porous ceramics, or even permeate into the pore of ceramics [11]. And the welding [12] and gluing [13] for the restoration of ancient metal artifacts are largely irreversible and will do more impact on cultural heritage. Strictly speaking, any physical restoration is an intervention to cultural heritage. For digital restoration, the restored cultural heritage is in a completely virtual form, which not only reduces people's visiting experience but also goes against the original intention of the exhibition of cultural heritage. For digital projection restoration, even it has no physical contact with cultural heritage, the lights emitted by the projection still do harm to the objects, especially for organic materials [14,15]. Plane patterns restoration is, to some extent, a non-intervention method, but the plane patterns behind the cultural heritage are always lower than the surface of the cultural heritage since most cultural heritage has a certain thickness, so the restoration effect is not good.
Virtual imaging technology, also known as phantom imaging technology, is not a new technology. It has been widely used in museum exhibitions to show people a completely virtual cultural heritage and enhance the visiting experience. However, in the past, people ignored the enormous potential of this technology in virtual restoration. In this paper, we attempted to use the virtual imaging technology in the virtual restoration of incomplete cultural heritage. The results show that the virtual imaging restoration has better performances than traditional restoration methods from the perspective of keeping authenticity, non-interference, reversibility and discernibility of the cultural heritage.

Basic principle of virtual imaging restoration
Virtual imaging restoration is established based on the principle of a kind of visual illusion phenomenon that human common sense can sometimes cause erroneous spatial perception. This principle can further be explained by the example in Fig. 1. As shown in the gure, an incomplete bronze mirror is placed behind a transparent sheet that is inclined 45 degrees. Since the sheet is transparent, the incomplete bronze mirror behind the transparent sheet can be clearly observed. Just as sometimes people can see themselves from a piece of transparent glass, the transparent sheet also has a certain degree of re ection. A mirror-reversed digital image of the missing part of the bronze mirror is placed below the transparent sheet. As the transparent sheet is with an inclination of 45 degrees and has a certain re ectivity, the virtual image will be horizontally re ected into observers' eyes. With the common sense that "light travels in a straight line," the brain tends to think that the imaging lights come directly from the transparent space in front of it, that is to say, the brain will believe that the image of the missing part of the bronze mirror is located in the transparent space behind the transparent sheet. Because the distance of the virtual image is adjustable, by altering the size and spatial position of the image, the virtual image of the missing part and the bronze mirror can be joined together in the observers' eyes, thereby achieving the virtual restoration in situ.

Virtual imaging restoration device
As shown in Fig. 2, a device for the virtual imaging restoration of incomplete cultural heritage was established based on the above principle. The structure mainly consists of the imaging plate, the platform, the black block, the projection screen, and the base. And the imaging plate consists of a transparent sheet with an inclination of 45 degrees, two transparent supporting plates and a bottom plate (as shown in Fig. 3 and Fig. 2). The transparent supporting plates and the bottom plate provide supporting force so that the transparent sheet can maintain an angle of 45 degrees to re ect the image lights horizontally.
The properties of the transparent sheet play a key role in the imaging effect. Firstly, it should maintain as thin as possible (≤ 1 mm), to reduce the double-image phenomenon caused by light refraction (That is because both the inside surface and the outside surface of transparent sheet can re ect lights, and the texture of transparent sheet is different from that of air, when the lights enter the sheet, certain refraction will occur, making the re ected lights of two surfaces do not entirely coincide, resulting in the double image phenomenon. The thicker the transparent sheet is, the double-image caused by refraction will be more visible). Secondly, the transparent sheet should possess high transmissivity to ensure people can see a clear and undistorted image through it. At the same time, the transparent sheet should also have reasonable re ectivity to ensure su cient lights be re ected in observers' eyes so that people can see a clear virtual image. Finally, the transparent sheet needs to be rigid and unbending to reduce the interference of deformation caused by its self-weight. Therefore, a special kind of rigid thin plexiglass sheet that has high transmissivity and reasonable re ectivity in the visible light range was selected for the device.
The surface of the black block is diffuse and with pure black color to minimize the amount of extra re ected lights which may enter eyes through the transparent sheet. An opening is arranged on one side of the black block, and a black tablet computer is placed inside the opening as a projection screen. And the four sides around the opening are also black.

Virtual Restoring Of The Bronze Mirror In Situ
To examine the practicability of the virtual imaging technology in restoration, the virtual imaging restoration device was applied in the restoration of an incomplete bronze mirror.
Firstly, according to the symmetrical structure of the bronze mirror, the digital image of the missing part was obtained by computer, and the digital image was mirror-reversed and copied into the projection screen. Then the incomplete bronze mirror was placed on the platform behind the imaging plate, and the mirror-reversed digital image was projected to the imaging plate to obtain a virtual image of the missing part. Finally, the distance of the projection screen and the size of the digital image in the projection screen were adjusted to ultimately connect the virtual missing part with the incomplete bronze mirror. The bronze mirror before and after virtual restoration is shown in Fig. 4. When the projection screen doesn't work, the bronze mirror presents in the original incomplete state. When the screen projects, the bronze mirror is restored to a complete state with the incomplete mirror joined together with the virtual missing part, of which the color and texture are quite realistic. Therefore, by altering the running state of the projection screen, people can see both the complete and incomplete state of the bronze mirror as they wish.

Refer to conservation theory
Compared with traditional restoration methods, the virtual imaging restoration is a new one that has not been discussed yet. Actually, not all innovative methods can be used in restoration. Any method intends to be used in restoration must be carefully considered with the following fundamental theories of conservation.

Authenticity
Authenticity is the core concept in the modern theories of cultural heritage. The methods, whether physical or virtual, in the restoration of incomplete cultural heritage, should be based on convincing evidence. All algorithms of digital restoration methods are based on the principle of authenticity, which calculate the appearance of missing parts and carry out a completely virtual cultural heritage. But is completely virtual cultural heritage what we want? Compared with the completely virtual cultural heritage, people would rather see the real incomplete one. Actually, the completely virtual restoration deviates from the authenticity that people want the cultural heritage to present. In contrast, the virtual imaging restoration gives people the real state of cultural heritage to a greater extent. What people see in front of their eyes is the real incomplete cultural heritage that joined with the virtual missing part.
Is that virtual restoration better than physical restoration? It is undeniable that physical restoration often reinforces cultural heritage, while virtual restoration only makes cultural heritage look complete. But is physical restoration still necessary when the existing incomplete cultural heritage is stable and not need to strengthen it? In fact, physical restoration is exposed to more complicated authenticity problems, that is because the authentic it theory requires not only the authenticity of the appearance but also the authenticity of materials and processes [16], which is very hard to achieve. For example, the authenticity of the restoration on ancient metals is di cult to achieve, for we can hardly completely reproduce their processes. That is a challenge brought by the multidimensional information of physical restoration. The physical materials carry various information such as morphology, material and process, which undoubtedly increases the di culty of restoration. Obviously, the more complex the information they carried, the more content deviating from the authenticity may be introduced for restoration objects. On the contrary, refer to the virtual imaging restoration, we only need to take into account the authenticity of the morphology, which is easier to achieve under the technical conditions nowadays. Therefore, the virtual imaging restoration can restore cultural heritage in a more cost-effective way with less authenticity offset, while physical restoration may introduce more inauthenticity. From this point of view, the virtual imaging restoration may be a better choice for incomplete cultural heritage.

Reversibility and minimum intervention
Reversibility is a widely used but relatively vague concept. In simple terms, reversibility refers to recover the restored object to the previous form for any future need [17]. However, for physical restoration, absolute reversibility is almost impossible since the treatments implemented on the body of the objects will produce irreversible effects. As people have realized that true reversibility can never be achieved, some alternative concepts developed, such as repeatability [18] and removability [19]. Both of the two concepts accept the reality that the additional materials have irreversible effects on the restoration objects. They only require that the treatments do not cause too much hindrance to restoration in the future. In a sense, repeatability and removability are both the products of the compromise between reality and the ideal that people wish to achieve complete reversibility.
Minimum intervention is also a universal concept. It recognizes that the physical restoration will bring about irrevocable adverse effects so that when people have to intervene in cultural heritage for certain restoration purposes, they should try to minimize the intervention to the restoration objects [20]. In fact, the minimum intervention is the product of people's pursuit of authenticity. People hope that the restoration will not impair the authenticity of heritages and be non-intervention, which is obviously impossible, therefore born the minimum intervention.
Irreversible intervention to cultural heritage caused by physical restoration may result in future regret, but the virtual imaging restoration can satisfy the rigorous pursuit of people to perfectly restoration. The virtual missing part in virtual imaging restoration has no physical contact with the restoration object, strictly speaking, the lights that form the virtual image do not even reach the restoration object. Therefore, this kind of method is entirely reversible and non-intervention, which also makes the real reversibility and non-intervention have the value of re-discussing.

Discernibility
By following the Venice Charter in 1964, people have paid more and more attention to the concept of discernibility. They believe that "replacements of missing parts must integrate harmoniously with the whole, but at the same time must be distinguishable from the original so that restoration does not falsify the artistic or historic evidence [21]". However, there are some discrepancies in people's understanding of the discernibility in real restorations. Some people want it to look the same from a far distance and different from a near place, such as the very common tratteggio retouch [22].
Others want to make it indistinguishable to the naked eye, but from special angles or by the special treatment, it is distinguishable. Such as dyeing the outer surface of ancient bronzes to make it indistinguishable but doesn't dye the inner surface to distinguish the restored area [23], or adding the uorescent agent to restoration material to make it distinguishable under the ultraviolet [24]. However, in considering the purpose of the exhibition, the restoration traces should be recognizable, because the main group of visitors is the ordinary people, rather than the professionals who have access to view cultural heritage through ultraviolet lights or from the inside of cultural heritage. We cannot sacri ce ordinary peoples' rights to knowing the original state of cultural heritage. In contrast, the virtual imaging restoration can present both the complete and incomplete state of the cultural heritage by controlling the appearance and disappearance of virtual images, allowing the restoration area to be easily discerned by viewers.

Scope of application
The virtual imaging restoration can be well applied to the restoring of some cultural heritage. For instance, it can replace the commonly used plane patterns restoration for incomplete cultural heritage, of which the restoration areas are easy to discern but a little abrupt (Fig. 6). By contrast, the virtual imaging restoration ensures that the restored patterns are realistic and easy to discern by controlling the projecting of virtual images. It is especially suitable for cultural heritage that is usually viewed from a xed angle such as painting, fresco, textiles and paper (that is because the ideal viewing angle range of the virtual imaging restoration is limited, only from the front can get a better viewing effect, from other angles it will lead to obvious dislocation).
Since the virtual imaging restoration is entirely supported by the virtual image, there is no physical support provided. Therefore, it is only suitable for objects that are su cient to maintain their stability.
In addition, this method is exible in showing the state before and after restoration, so it is also very suitable for pretesting the restoration plans and providing a exible reference for actual restoration work, which could reduce the possible operating errors.

Conclusion
Restoring incomplete cultural heritage to an intact state will give people a more understandable way to comprehend the historical information of cultural heritage. The virtual imaging technology provides a virtual restoration method, which can restore the incomplete cultural heritage in situ by joining the virtual image and the incomplete cultural heritage. The virtual imaging restoration maintains the authenticity of the incomplete cultural heritage to a greater extent and is entirely reversible, non-intervention and easily discernible. For the restoration of cultural heritage that is usually viewed from a xed angle and does not require physical supplementary to maintain their stability, the virtual imaging technology may a better solution. Moreover, the virtual imaging restoration device can also be a useful tool to provide a reference for the actual restoration work.

Declarations
Availability of data and materials All data, models, and code generated or used during the study appear in the submitted article.
Competing interests Figure 1 The schematic diagram of the virtual imaging restoration of bronze mirror.

Figure 2
The schematic structure of the virtual imaging restoration device. the imaging plate; the transparent sheet; the transparent supporting plates; the bottom plate; the platform; the black block; the projection screen; the base.  The schematic structure of the imaging plate. the transparent sheet; the transparent supporting plates; the bottom plate. Images of the bronze mirror before (a) and after (b) virtual restoration by using the virtual imaging restoration device.

Figure 5
Patterns restoration used in Gold ornaments of Liao Dynasty, Inner Mongolia Museum, China.