Cultural heritage characteristics and damage analysis based on multidimensional data fusion and HBIM–taking the former residence of HSBC bank in Xiamen, China as an example

Abstract

In the context of digitalizing cultural heritage, HBIM is garnering increasing attention from heritage conservationists for its advantages. In conservation practices for cultural heritage, expressing regional characteristics and documenting in HBIM has become increasingly crucial, and these needs are inextricably linked to the need for models with a high level of detail (LOD). This paper presents a combination of Terrestrial Laser Scanning surveys, UAV photogrammetry, and handheld laser scanning to scan cultural heritage. The scan data obtained from multiple digital surveys serves as the foundation for modelling HBIM. For the regional characteristics, the HBIM model was built with detailed geometric shapes, configurations, materials, and colors of each element, as well as complex patterns and damage using the parametric models and mesh models. The information of the building and its components stored in the HBIM model. Consequently, obtained a highly detailed HBIM that not only captures the intricate details of the ideal state and real-world damage and degradation with minimal error but also serves as a repository for a multitude of information types. It has been concluded that a high LOD is essential to representing the regional geometric feature of a building and recording the damage of cultural heritage.

Introduction

In recent years, with the evolution of digital technology and digital twinning, the digitization of cultural heritage has emerged as a crucial technique in the preservation of digital cultural heritage [1]. The use of digital technology for cultural heritage protection involves both urban and architectural scales. However, there is less analysis of architectural heritage from the component scale, while the damage and restoration of architectural heritage begins from the perspective of architectural elements. The recording of cultural heritage also needs to be based on architectural elements. For the digital analysis of cultural heritage components, it remains an unsolved problem. The reason is that high-accuracy models are needed to support the display of features and analysis of damage. High-accuracy models not only require a lot of manpower and time to modelling, but also require a lot of annotations and supplementary information. Complete data information is the key to enriching cultural heritage. This requires professional researchers and other disciplines to work together for a long time to achieve. However, Traditional architectural heritage analysis relies on two-dimensional drawings and documents to record architectural heritage. In recent years, there has been an emerging trend of adding information to three-dimensional models to achieve a combination of 3D visualization and 2D information to display cultural heritage. Merely relying on 2D drawings and documents is insufficient to fully comprehend the wealth of information embedded within architectural heritage. A 3D visualization model needs to be constructed, which combines 2D drawings and documents, to unravel the structural intricacies and other associated details of the cultural heritage [2]. Building upon the concept of Building Information Modelling (BIM) for the cultural heritage domain, Murphy proposed Historic Building Information Modelling (HBIM), an approach to generate comprehensive engineering drawings and 3D models derived from laser scanning and photogrammetry techniques [3]. Utilizing HBIM facilitates the integration of a three-dimensional visual model with diverse digital datasets into a single file, thereby maintaining the connectivity between various elements in the 3D model and their corresponding data. The HBIM practice in previous studies only expressed structural or geometric features, but seldom analyzed the structural and tectonic relationships of building components in cultural heritage. Brumana, R. (2013) and Argiolas, R. (2019) both created structural models of accurate vaults, but they did not specifically distinguish between structural layers and decorative layers, such as the distinction between masonry layers and stucco layers [4, 5]. Especially in the case of Chinese cultural heritage, these buildings often have many complex component connections, different types of components with different tectonic methods in different heritage, or special regional structures. Expressing only the appearance features is not enough, as it will miss a lot of information and data, and cannot achieve comprehensive protection of cultural heritage.

The standard operational procedure of HBIM begins with a digital survey, to establish a foundation for HBIM modelling, we use digital survey technologies, specifically 3D scanning, which enabled us to gather information on architectural components and spatial relationships that characterize cultural heritage. The digital survey offers the advantage of collecting data with greater accuracy and reducing operating time without the need to physically interact with the measured objects, thereby minimizing potential physical damage to the heritage. Which yields a set of points carrying defined coordinates, called “point clouds”, along with geometric and colorimetric data [6]. This process, described as “scan to BIM”, involves several methods that have been developed and are presented in numerous works in the literature. Some studies provide examples of using scanning data and point clouds to generate HBIM models through techniques such as Terrestrial Laser Scanning (TLS) and photogrammetry [7,8,9]. Other studies, such as those by Karachaliou, E. (2019) and Martínez-Carricondo, P. (2020), demonstrate the use of Unmanned Aerial Vehicle (UAV) Photogrammetric Survey to generate 3D point clouds as a basis for reconstructing HBIM models. The application of 3D handheld laser scanning (HLS) in the field of cultural heritage aims to obtain a high-precision 3D model, which often has complex surface patterns. Patrucco, G. uses HLS to record digital models of movable cultural relics [10], while Adami, A. uses it to obtain 3D models of cultural heritage and architectural elements [11]. And Allegra, D. collects scan data of architectural elements, and builds 3D models of scan elements [12].

It can be seen that UAV Photogrammetric Survey is suitable for acquiring cultural heritage data at urban scales, TLS is suitable for acquiring cultural heritage data at architectural scales, and HLS is suitable for acquiring cultural heritage data at component scales. Each scanning method has its advantages and disadvantages. If point cloud data from multiple methods are fused and registered, more complete and accurate scanning data can be obtained. Additionally, there are also cases where TLS, photogrammetry, and UAV [13,14,15,16] survey are combined to scan cultural heritage to obtain more complete and detailed 3D scan data, and then an HBIM model is established. Some scholars have found that the registration of TLS and UAV-obtained point clouds can compensate for the defects of the two types of point clouds, resulting in more complete and accurate scan data. He, Y. used TLS and UAV to obtain point clouds data of ancient Chinese buildings, and used the registered point cloud model as a support for 3D printing of 3D model [17]. Lin, G. used the registered point cloud as a support for the modelling of large-scale traditional Chinese buildings [18].

The digitization of cultural heritage involves not just the generation of 3D models, regardless of their level of detail, but also the production of documentation for cultural heritage management. This documentation, which is the focus of digitization of cultural heritage, encompasses architectural objects and their constituent elements, along with attributes such as dimensions, materials, temporal information, damage, deterioration, and relationships [19]. Currently, this part cannot automatically generate engineering drawings and special documents, and requires a large amount of manual work to establish documents. HBIM is a tool for building 3D models and providing visualization models, and it can also serve as a resource library for creating and managing information and data on cultural heritage, which is different from other 3D modelling software. The majority of data in HBIM is derived from 3D scan data, while a minor proportion is obtained through traditional on-site surveys and photographic surveys. The information regarding cultural heritage provided by 3D scan data exceeds other data types, encompassing geometry shape, size measurements, location, materials, colors, and deterioration information. The acquired scan data is subsequently processed, enhancing data organization and infusing intelligence using BIM software platforms.

A considerable number of scholars have investigated the utilization of HBIM as a tool to collect, store, retrieve, and exhibit information from the realm of cultural heritage. These HBIM models were developed for documentation, conservation, and management [20]. Additionally, they were designed to accurately record information on the deterioration of cultural heritage assets, and the status of restoration efforts, and to monitor the deterioration of individual components [21]. Survey data were converted into identifiable parameters, and key fields and hyperlinks were incorporated into HBIM. In fact, due to the limitations of BIM software, it has not been designed for the modelling and management of cultural heritage assets. To circumvent these limitations, some researchers have developed external platforms that integrate BIM with external databases, enabling more comprehensive data collection and facilitated data management [22, 23]. These information data come from on-site surveys, document surveys, and digital surveys.

Although previous studies have proposed various methods to incorporate scan data into HBIM to enhance information about a structure, due to the limitations of BIM software, the amount of information data about cultural heritage that can be stored in HBIM is still limited. Especially when it comes to representing building damage, it is typically documented using pictures and text within 3D models or component models, or by mapping 2D images onto 3D models [24]. The establishment of a 3D model of damaged components serves multiple purposes. On one hand, in contrast to simpler methods of acquiring information, such as photographs, videos, or 3D point clouds, this model offers a more intuitive visualization of the component's appearance. Additionally, it can be leveraged for physical and environmental analysis, facilitating the inference of damage causes. On the other hand, a 3D model of damaged components not only preserves and documents the styles of buildings and components across different periods but also supports various applications, including exhibitions in online museums.

The management and establishment of HBIM differ according to the distinctive regional characteristics of cultural heritage. Following the same workflow as other digital scan surveys, similarly, Chinese cultural heritage surveys employ the TLS and UAV Photogrammetric Survey approach to obtain 3D scan data and point clouds [25, 26], and then build HBIM model based on the 3D scan data. The modelling of HBIM model for Chinese cultural heritage requires a more manual construction approach. This is due to the significant differences between Chinese wooden architectural heritage structures and modern architectural heritage structures made of wood-masonry compared to European architectural heritage structures. These structures possess complex coupling and connection relations among their components. While 3D laser scanning can capture the building's surface and individual components, digital measurement technology fails to accurately capture these coupling relations. Therefore, many components of the 3D model still require manual modelling, and some components may require a combination of manual surveying techniques.

In the HBIM research on China's cultural heritage, most of the research focuses on wooden architectural heritage, including the classification methods of wooden components, the parameterization methods of components, the practice of scan-to-BIM, and the practice of establishing digital archives and thematic drawings [27,28,29,30,31]. Unlike wooden structures, there have been many cultural heritages that combine wood and masonry since the beginning of modern China. Their structural relationship is not complex, but the construction hierarchy of individual components may be more complex. Similarly, some scholars have researched modern architectural heritage [32,33,34], but the discussion of the architectural structure and component construction hierarchy in HBIM is still vacant, in fact, this part is the advantage of BIM. There are many reasons behind this. Sometimes, destructive testing may be necessary, making it difficult to determine the structure or layers of construction material. And when using NURBS or Mesh models for modeling, they can only express the surface appearance of components, akin to representing a "shell". This approach does not capture the internal structural relationships within the component model. In China, there is relatively few research on the use of HBIM to create modern architectural heritage, especially in the southern Fujian. The LOD level of the practical HBIM model is low, the expression of architectural structures is not precise enough, and there is little data information in HBIM.

This study aims to address a research question, in the HBIM environment, generate highly detailed multi-dimensional models that support various stages of cultural heritage archive records, serve as information repositories for cultural heritage. Based on digital surveys using a combination of UAV, TLS, and HLS, comprehensive point cloud data is obtained from multiple scales perspectives, and high LOD level HBIM are modeled to support the use of building components as the basic unit for information recording. Finally, detailed HBIM with various types of data such as 3D parametric models, mesh models, 2D drawings, size data, construction relationships, material textures, numbers, documents, images, videos, and URLs are obtained. The HBIM records comprehensive information on the historical and current conditions of cultural heritage, deterioration and damage, restoration potential, and disease monitoring. This article identifies and deepens the complete scan-to-bim workflow for modern cultural heritage in southern Fujian, China, and establishes a LOD4 HBIM of the former residence of HSBC Bank. The focus is on recording historical construction information of component elements in the former residence of HSBC Bank, current deterioration and damage information, historical restoration results of components, monitoring of repaired components, and monitoring information on component diseases. This work translates the geometric outline and information data of cultural heritage into digital models based on Chinese specifications for classification and grading. The workflow of this article is shown in Fig. 1.

Fig. 1

The main step of HBIM modelling and documentation workflow

Materials and methods

Case study

In southern China's Xiamen Province, Kulangsu Island was added to the World Heritage List in 2017, making it China's 52nd World Heritage Site. The former residence of HSBC Bank, located at No.57 Kwu Xin Road on Kulangsu Island, is a key heritage conservation unit in China and is listed as a core element of Kulangsu Island's cultural heritage in the "Application for World Heritage Status." This building sits on the cliff-top of Bijia Mountain. The cliff below the building features a stone carving from the story of “a historical record about the rebuilding of the Sanhe Palace on Kulangsu Island”. The combination of the building and the cliff creates a unique blend of Chinese and Western culture.

The architectural style of this building is Colonial veranda architecture, with a construction area of 374 square meters. Its plan deviates from traditional Colonial veranda architecture as it features a non-geometrically symmetrical T-shaped layout (Fig. 2). Japanese scholar Terunobu Fujimori believes that this particular design stemmed from the Victorian era's penchant for diverse design concepts, which were further influenced by the "picturesque" trend towards architectural change. This single-story structure boasts a brick-and-wood structure and an intricate combination of multi-faceted slope roofs and flat roofs. The southeast and northwest exterior verandas serve not only as a place for leisure but also as a passageway for pedestrians. The verandas are crafted from unique, curved red bricks, which contrast with the traditional arches found in this style. Instead, they sport petal-shaped column heads made of concrete to enhance their aesthetic value. In between the columns, the railings are in the shape of vases, made of red bricks, it is a unique style of southern Fujian province. The building's foundation rests on site-derived granite that has been slightly flattened through manual trimming, followed by reinforcement with steel rods to establish a strong foundation.

Fig. 2

The former residence of the HSBC bank survey photos: a Photos of low-altitude aerial photography; b Photos of building facades

The building was originally designed by the British merchant Holiday, wise & co. as a villa in 1876. The building was designed by the British, using local building materials in Xiamen, and constructed by Xiamen craftsmen. In 1855, the building was transferred to Tait & Co. In April 1893, the building was transferred to the Xiamen branch of HSBC and became the governor's mansion. After the liberation of Xiamen, it was used as a dormitory for workers at the Kulangsu Shipyard. After several renovations, the building's external veranda was sealed off and used as a bedroom, and the interior of the building also suffered various types of renovations (Fig. 3). In March 2016, the former residence of the HSBC Bank began a restoration and protection design, restoring the building to its historical appearance.

Fig. 3

The damage suffered of the former residence of the HSBC bank: a The external veranda was sealed off artificially; b Additional construction of the building; c The deterioration and damage of walls; d The artificial transformation of the interior space of the building

The main focus of this restoration project is to demolish the walls of the enclosed outer corridor and restore it to its original open-air style. Due to previous renovations, the red brick columns and capitals were damaged, which will be restored during this project. Exterior additions and interior modifications that were not originally part of the building will be removed, returning the structure to its original state (Fig. 4). However, there are still some shortcomings in this restoration. Firstly, the red-brick columns are made up of specially curved red bricks, some of which have been found to have cracks and other damage. As a specific restoration plan has not been proposed, metal iron hoops are currently being used to support the columns and prevent further damage and collapse (Fig. 5). In addition, the red-brick bottle-shaped railing components were damaged during the enclosed outer corridor renovations, resulting in a lack of reference for the size and style of the components that can be restored (Fig. 6). Currently, red-brick lattices are being used as a replacement restoration component. Finally, a follow-up survey found that the foundation of the southeast outer corridor was unstable, necessitating an additional restoration and modification.

Fig. 4

The restoration project of the former residence of the HSBC bank: a Restoration of the railing at external veranda; b Restoration of the column capital at external veranda; c Restoration of floor of the building; d Restoration of the interior space of the building

Fig. 5

The metal iron hoops are being used to support the columns

Fig. 6

The red-brick lattice railing is used to replace the red-brick bottle-shaped railing

From the above, the scientific value of the former residence of HSBC Bank is:

• The location of the building is a cliff, which is unique and regional, requiring the use of multiple digital survey methods to obtain complete 3D scan data.

• This building is a case of regional architecture that combines European classical architecture structures with Chinese building materials and regional components. The architectural style of the building includes the fusion of traditional Southern Fujian architecture, colonial architecture from Southeast Asia, European classical architecture, and the unique style of Kulangsu.

• The historical information, current status information, and restoration information of the building need to be integrated into digital archives and displayed in a three-dimensional visualization manner, which is suitable for recording, management, and display using HBIM technology.

• The renovation of the building has lasted for a long time, supporting the exploration of multi-time-scale recording methods for the current state of damage and renovation process of architectural heritage.

Data survey and point registration

Digital scanning allows the rapid acquisition of complete 3D scan data of cultural heritage and has the advantage of accuracy, which is not possible with traditional surveys. For this reason, it is also commonly used for architectural, engineering and archaeological information acquisition. It is usually the first step in the HBIM building process, after which the scanned data are converted into point clouds or mesh models as a support for HBIM model building. After inspecting the former residence of HSBC Bank, considering its unique topography, its location on the top of a mountain, and the cliffs to its east and south, using a single type of digital survey method will result in many missing 3D scan data, and the incomplete and imprecise data will affect the level of detail of the final HBIM. This article has studied the method for converting laser and photogrammetric data into HBIM models, with a focus on cultural heritage located in complex environments that make it difficult to scan complete data. Using a digital surveying technique that combines TLS, UAV Photogrammetric Survey, and HLS, the article scans and records the shape, decoration, and building components, as well as current damage of the former residence of HSBC Bank. The article then employed the scanned 3D point clouds as a reference to establish an HBIM model.

TLS survey

The TLS surveying is carried out by the Trimble X7 laser scanner (Fig. 7), with a ranging range of 0.6 m–80 m and a measurement accuracy of 2 mm. It is equipped with a 500000 high-speed laser scanning system and three 10 million pixel lenses. Use TLS to scan the exterior facade, indoor rooms, and surrounding environment of the former HSBC Mansion. The internal parameters are set to: scan time is 2 Minutes, the number of points scanned each time is 11.8 MPts, and the point spacing of @ 10 m is 11 mm. Turn off HDR, and the white balance is automatic. Every time a site is scanned, point cloud data is automatically concatenated in Trimble Perspective software to view the scanning and concatenation status in real-time. In order to ensure that the scanned point cloud can be merged with the previous scan, the distance between two adjacent sites should not exceed 35 m, and there should be no obvious obstacles between the sites. Automatic registration ensures that all stations are in the same registration group. A total of 70 sites were ultimately scanned, with each site ranging in size from 100 to 200 MB and capturing 15 images. After scanning, a color panoramic image was created, where each point cloud was attached with RBG colors. Finally, a true color point cloud was created to restore the true color of the architectural heritage. These processes take approximately 4 h in total. After the scanning and splicing of each site are completed, the point cloud data is transmitted to Trimble RealWorks software to establish measurement sites cloud and obtain a 3D point cloud model.

Fig. 7

a Trimble X7 laser scanner scanning the building; b The distribution of TLS survey sites

UAV photogrammetric survey

After TLS survey, we obtained the TLS 3D point clouds of the mansion. Based on the scanning results of TLS, the roof section of the building had a significant point cloud missing, including the roof, ridges, chimneys, and roof tiles. In addition, due to the cliff on the east and south sides of the building, it was not possible to set up TLS stations, resulting in incomplete point cloud data for the exterior walls on the east side and the outer columns on the southeast side. Considering the regional uniqueness of the cliff and limitations of TLS, it was difficult to obtain complete data using TLS scanning, so UAV Photogrammetric Survey was used to supplement the missing parts (Fig. 8).

Fig. 8

a UAV scan the former residence of the HSBC bank; b the location of “Photogrammetry 3D” and manual flight

The oblique photography of UAV was carried out by DJI Phantom 4 RTK, which is a quadcopter drone with a maximum horizontal flight speed of 58 km/h and a maximum ascending speed of 6 ms/h. The drone weighs 1391 g and is equipped with a 1-inch 2000-megapixel CMOS sensor. The ground sample distance (GSD) can reach 2.74 cm at a flight height of 100 m. The "Photogrammetry 3D (Well shaped Flight)" function in DJI Phantom 4 RTK was used to shoot the roof, ridges, building facades, and surrounding environment of the building. The flight height was 100 m, the GSD was 2.74 cm/pixel, the maximum flight speed was 7.9 m/s, and the camera mode was timed shooting. The camera settings were as follows: picture ratio of 3:2, white balance mode set to daytime, exposure mode set to average metering, gimbal angle set to −45 degrees, shutter speed set to 1/1000. The overlap rate was set to 80% side overlap and 80% longitudinal overlap. The shooting area covered an area of 13,587.98 square meters. A total of 247 photos were taken with a resolution of 5472*3648 pixels per photo. In addition to using planned flight, we also used manual flight to let the drone fly close to the roof of the building to obtain more detailed information for obtaining a more detailed point cloud model with more dense points. A total of 132 additional photos were manually taken by controlling the drone. The 379 photos taken were imported into the DJI Terra software. Due to the RTK mode being turned on during the shoot, the photos already recorded geographic information coordinates (GPS). After importing the photos into the software, the positions were automatically arranged in order. Finally, a three-dimensional point cloud model was generated and exported. The oblique photography of UAV exported point clouds for 4 stations, which were color point clouds in *.las format, totaling 1,007,326,360 color points.

HLS survey

Considering the visual representation of deterioration and damage in cultural heritage, using high-precision TLS scanning can achieve detailed records of damage, but the accuracy is far inferior to that of HLS scanning. The point clouds generated by HLS are more densely spaced, resulting in a larger number of generated meshes, which provides more accurate representation of the damage. The HLS surveying is carried out by the IReal 3D hand-held laser scanner (Fig. 9). This article selects two different types of regional representative components for experiment and comparison. The first experimental object is the red-brick lattice, which is a component of the balustrade in architecture and is often used in other buildings in southern Fujian. It is a 3D model with deep geometric holes. The parameters of the HLS are set to Colorful Object Scanning Mode, Color Mode: Multicolor Model, Fill Light: Open, Splicing Modes: Feature, Scenes: Object, Object Size: Standard, Resolution: 0.7. Obtained from scanning point clouds of red-brick lattice, wrap the point clouds into a mesh model in IReal 3D, then mapping obtain a mesh model with materials, finally save as the *.asc format point cloud and the *.stl format mesh model. The same operation was used for the typical relief on KulangSu Island, which is a flat type of decorative component commonly used on building facades and walls. The point cloud in *.asc format and the mesh model in *.stl format were also obtained.

Fig. 9

a The red-brick lattice is scanned by HLS; b The typical relief is scanned by HLS

Manual survey

In Chinese cultural heritage, many components exhibit a coupled relationship, but the specific assembly method cannot be discerned through 3D scanning. Therefore, manual photography or physical examination of the component assembly is required to ascertain the distinct shape and style of each component [35]. To survey and document detailed components, a camera is employed to take photos and make recordings. This article uses the Nikon D750 camera, equipped with TAMRON 17–35 mm, NIKON 24–85 mm, and TAMRON 180–300 mm lenses, with photo pixels of 6016 * 4016. To measure the net height of the room and the dimensions of components, secondary measurements can be made using instruments like laser rangefinders and measuring tapes. These measurements can then be compared with point cloud data to ensure data accuracy. Additionally, a camera can be used to document the current condition of the building, including any damage, as well as construction records during the restoration process.

Point clouds registration

Through the digital survey described above, we obtained 3D scan data of the building, which can be converted into point clouds data as the basis for creating an HBIM. However, in reality, the TLS and UAV point clouds obtained through research are not only mutually isolated and lack connection, but also have certain limitations and missing points in their respective point clouds, and are not complete point cloud data.

Causes and areas of missing point clouds in this project include:

• Due to the limited scanning height of the TLS scanner and the obstruction of the building's veranda, it is difficult to scan the entire roof of the building, resulting in a serious lack of point cloud data for the roof.

• Due to the cliffs on the east and south sides of the building, it is difficult to set up TLS sites, resulting in partial missing point cloud data from the east to the south of the building.

• The content of the UAV point cloud only includes the exterior facades and roofs of buildings, and there are missing points in the interior of buildings and areas obscured by structures, such as the point cloud data missing inside the columns of the building's veranda.

• Using UAV for planning flights, because the angle of the camera's pan head is set to a fixed value, the point clouds of the roof cornice and eaves below will be missing.

In the face of these shortcomings, combining TLS point clouds with UAV point clouds can actually make up for each other's deficiencies and form complete 3D point cloud data (Fig. 10). By using TLS survey to obtain building point clouds in the *rwp format and UAV Photogrammetric Survey to obtain roof point clouds in the *las format. The point cloud files in different formats are isolated from each other, and the data and locations cannot be made interoperable enough to be accessed in the same file, so there is a need to align the point clouds in both formats. The fused point clouds as a rigid reference for modelling HBIM: as a preliminary evaluation of data, as a visual reference for HBIM models, as a dimensional basis for component families, as an alignment basis for the overall building, and as a geometric and spatial basis. There are currently many algorithms for point cloud registration, the point cloud alignment in this paper was performed in the commercial software Trimble RealWorks.

Fig. 10

The workflow of point clouds registration

In the "Register" interface in Trimble RealWorks, select one point cloud as the "Reference Cloud" and another as the "Moving Cloud". Select the special points of the two kinds of point clouds, and coarsely align the two kinds of point clouds to get the aligned point cloud model, which is based on the FPHP algorithm [36].

$$FPFH\left(p\right)=SPF\left(p\right)+\frac{1}{k}\sum_{i=1}^{k}\frac{1}{{\omega }_{k}}\bullet SPF({p}_{k})$$

After that, we can use the "Refine" function to further refine the alignment between the two point clouds, which is based on the ICP algorithm [37]. During the process, Trimble RealWorks will automatically calculate the refinement error and overlap between the two point clouds after alignment. The alignment results are shown in Fig. 11.

Fig. 11

The result of point clouds registration

Data classification

LOD level of the HBIM

The HBIM model of the former residence of HSBC Bank was created in Autodesk Revit^® software. Because of the significance of the building, to express the characteristics and the damage of South Fujian cultural heritage, the HBIM model is defined as LOD4. According the Definition of LOD in BIM for Heritage [38] and the Chinese standard GB/T 51301–2018 (Table 1), the LOD4 model in this article utilizes custom-developed components to represent the main features and service details of the building. These custom-developed components express the details of the building, which are jointly composed of precise geometric contours, structural relationships of components, and attached relevant information.

Table 1 The difference in the definition of LOD between UK’s and Chinese

This article argues that the 3D visual expression of decorative details and local damage is quite necessary. The expression and preservation of decorative details facilitate better protection of architectural heritage. Especially, the decorative components of modern buildings in southern Fujian were not constructed strictly according to existing shapes and rules. Instead, they were composed by piecing together various existing elements, which may have originated from local areas in southern Fujian or other regions. Furthermore, the 3D visual representation of local damage not only allows us to observe the detailed damage of components from a 3D perspective, but also combines with the 3D model of the building for a comprehensive expression, compared to the traditional method of indicating damage on 2D drawings. On the one hand, because the generic components of Revit^® cannot be applied to cultural heritage, especially to the cultural heritage in southern Fujian, the components in this building include both the generic components of southern Fujian's modern architecture and the unique component types of this building. Therefore, this article creates custom component families, the component families have high level of geometric detail, to express the details of the building components [39]. This method enhances the geometric expression and enhances the visualization information value of the model to show the regional specificity of the project. On the other hand, HBIM models that support damage restoration must be accurate enough to distinguish between structural components and decorative components inside the building. The number and style of components should be consistent with the current status of the architectural heritage, and the size measurements and geometric contours of components need to be accurately expressed, while the surface texture and detailed decoration of components can be expressed with more or less details. High-precision component models provide the most suitable carrier for recording geometric and non-geometric information, and also allow for deeper BIM-based analysis, support for renovation, visual display, and 3D structural simulation.

Component classification in HBIM

HBIM is faced with the establishment and management of cultural heritage models, which is different from construction management faced by BIM. Many components included in HBIM cannot be created using the native family in Revit^®, but fortunately, Revit^® provides many types of family templates that allow users to freely create models. Therefore, after surveying and analyzing the components of cultural heritage, using different family templates to establish families is also a necessary step in establishing HBIM.

Some examples present the classification of components, such as traditional Chinese architecture [27,28,29,30], and modern architectural components [32], while there are few case studies on HBIM for the modern cultural heritage of the southeast Fujian region, including modern wood-masonry and concrete cultural heritage. Therefore, based on the typological study of cultural heritage in the southeast Fujian region [40,41,42,43], the component classification of the building is conducted. After on-site investigation and analysis of 3D scan data, the number and classification of the building's components are shown in Table 2.

Table 2 Classification of components of the former residence of HSBC Bank

Modelling

Before modelling HBIM model, we need a concise 3D point cloud model as a support for HBIM modelling. This model contains the main structure of the cultural heritage, and can include a few numbers of environmental elements. A concise model can provide us with faster inspection speed and reduce the occlusion caused by redundant elements. Then, we should eliminate all extra points that do not need to the 3D point cloud data, it is necessary to perform a manual cleanup process on the point clouds. This process results in a more detailed and accurate 3D point cloud model.

The HBIM process involves a reverse engineering solution, in which parametric objects representing architectural elements are mapped onto laser scan or photogrammetric survey data. After the geomatics survey is conducted using photogrammetry techniques or TLS, the point cloud or its mesh (generated by the point cloud) is modelled into BIM objects [44, 45]. Point clouds to mesh is used as a means of obtaining scan data and quickly building accurate 3D visualization models, often of irregularly shaped buildings or with a large number of complex decorations and textures.

Parametric modelling

Due to the significant differences between the generic components in Chinese cultural heritage and the preset families in Revit^®, it is necessary to manually create custom-developed families. The advantage of using parameterization to create component objects is that they can change their geometric shape based on the parameterized dimensions, and view the component dimensions in real-time in the status panel. The establishment of these families resulted in an “ideal model” [46], which is built with a set of “sampling sizes” [47]. A. Barontini advises that considering consecutive inspections, graphical accuracy, and the change in the shape of each element due to damage and degradation, which will increase the computational burden [21].

Most family categories, including columns, beams, doors, windows, and other components, are created using Revit^® family templates. The geometric data and material information of the families are based on information from 3D scan data. According to the family library classification mentioned earlier, select the corresponding family template in Revit^® and draw reference planes based on point cloud data to serve as the basis for parametric modelling. Use functions such as extrusion, rotation, and fusion to create the solid geometry of the component, and use associated reference planes and internal family parameters to establish parametric components (Fig. 12). Families for walls, ceilings, floors, roofs, and stairs are created by modifying parameters such as structure, material, and construction in the native family. For components with coupled relationships in Chinese architectural heritage, data obtained from site surveys and point cloud data are used as a reference for component establishment, and parametric modelling within Revit^® is employed to create "families".

Fig. 12

Parametric modelling of the wooden shutter window

Mesh model generation

In this article, the generation of mesh models is illustrated with two types of components, such as the red-brick lattice of the railings in the project and the typical relief in KulangSu, to explore the 3D visualization of components with complex surfaces, as well as the 3D visualization of components with severe damage in buildings on KulangSu. The two practical methods are as follows.

Use the IREAL 2E handheld laser scanner to scan the component, obtain the scanning data in the IReal 3D software, and convert the scanning data into point cloud data. At this stage, the point spacing of the point cloud data can be adjusted. The point spacing selected in this article is @1 mm. Clear the redundant and unnecessary point cloud, then generate the mesh model from the point cloud and map the material to the mesh model. The holes and incomplete parts in the model will be automatically filled thanks to the automatic calculation in IReal 3D. The final generated point cloud is in *.asc format, with a point cloud number of 144,709, and the mesh model is in *.stl format, with a current mesh number of 325,918.

In addition, the mesh surface of the model is simplified in Meshlab, with a target number of face of 11,250. This step is to reduce the file size of the component model, so that it will not slow down the model when loaded into Revit^®. Appropriately simplifying the number of faces of the model will not reduce the features and details of the model. This method is more suitable for applications in 3D presentations.

Loading the final model into Revit^® with *.stl format, adjust its orientation and position, and then load it as an "embedded family" into the parapet family. Finally, load the model into the project (Fig. 13).

Fig. 13

The workflow of point clouds to mesh of red-brick lattice

Similarly, using IREAL 2E to scan the relief on the wall in the field, there is a penetrating crack in the middle of the component compared to other components. According to the scan data, it is converted to a mesh model in *.stl format in IReal 3D, and finally can be loaded into Revit^®. The current point is 1,626,892, and the current mesh is 4,647,146 (Fig. 14). The numbers of point clouds demonstrate that HLS, compared to TLS, captures significantly more points per individual component, enabling a more precise representation of architectural details, especially when dealing with decorative components that are more intricate in detail. Additionally, it enables the creation of a 3D model that damage of component.

Fig. 14

The result of point clouds to mesh of classical relief in Kulangsu

HBIM modelling and 3D visualization display of damage component

The registered point clouds are loaded into Revit^® and fused as a rigid reference for establishing HBIM, as a preliminary evaluation of data, as a visual reference for HBIM models, as a dimensional basis for component families, as an alignment basis for the overall building, and as a geometric and spatial basis. After that, load the established families into the project, in the same way, using the spatial positioning and appearance information provided by the point cloud as a reference, and give the families the correct parameter information and material information. Use these families to modelling a complete HBIM model (Fig. 15).

Fig. 15

Using the families modelling the former residence of HSBC Bank

Due to the large memory footprint of mesh models, loading them into Revit can slow down inspection efficiency. Therefore, this article references the method of Li, S. and uses mesh models used to represent complex components in buildings as "shadow families," which are turned on when they are needed for inspection [48]. Similarly, this method can be used for 3D visualization of damage to components in the building.

3D documentation data

Survey data is added to the artifacts in the completed HBIM model. The process of digital heritage archiving is not only limited to 2D drawings and 3D models, but also documents, pictures and images, which are used to enrich the contents of digital heritage archives. Through HBIM model, the information is stored, queried, modified and managed in 2D drawing and 3D model. As an information source, the HBIM model can continuously increase and update data information over time, and centralize related data information to avoid data isolation and loss. In past research, some scholars have been studying the information record based on HBIM, using HBIM as an information resource database [46, 49]. Moreover, Li argues that HBIM modelling as the “index frames”, which means that every element of cultural heritage can be used as a database for storing information [47].

To overcome these limits, it is necessary to find a suitable method to record data in the BIM environment. Thanks to the BIM parameters, it is possible to add images, texts and URL in per components, even more, use the plugin can achieve continuous recording of recovery projects, and monitoring of damage and deterioration. Use parameters means creating a classification method that applies to the architectural components of modern architectural heritage in southern Fujian. There should be many similar or different key fields. In previous studies, there are some examples that use Revit parameters in combination with 3D models to express damage information of components. Mol et al. added data, photos, test information, and other information to components in the Revit model, and also established a model of damaged components that changed over time [50]. Youn et al. used parameters in Revit, combined with 3D models of components, to express the damage condition of cultural heritage components [51]. And other previous studies have expressed the labeling of damaged components at a 2D level and then mapped them to a 3D model [22, 24].

Based on the semantics of Revit^®, HBIM serves as an indexing framework model for the preservation and restoration of architectural heritage. Each family in HBIM, after being numbered, has each parameter as a carrier of architectural heritage information records. The main recording method is to use the parameter function in Revit^®, and the content can be text, numbers, mathematical formulas, pictures, images, hyperlinks, etc.

In this study, all families in HBIM should be controlled by parameters and recorded information. Therefore, geometric rules and combination relationships between components within each family are specified from the modelling stage, and mathematical formulas and numerical constraints are used to control them. The information attached to each family in this article is divided into three categories: historical component information, current damage information, and restoration information [52] (Fig. 16).

Fig. 16

Family parameters of the red-brick column

The historical component information includes attribute information, geometric information, and material information. The attribute parameters of the family are the component number and component age, which are implemented using the "shared parameter" function, and the parameter type is set to "text". The geometric information and material information within the family represent the idealized geometric form of the component, using family parameters. Set the parameters for each part in the family template, then add labels to define the parameters, and use mathematical formulas to associate the parameters. Finally, adjust the corresponding parameters, and the remaining associated parameters will also change accordingly, resulting in changes in the geometric form of the family. Most importantly, the values of these parameters can be viewed and managed.

The current status of damage and restoration information is implemented using "shared parameters", and the parameter type is set to "text". Manually entered text information is used as a comment for the family. Their content is strictly defined and supports modification. They can ultimately be exported as Excel for statistics and analysis. This method can integrate information data such as family dimensions and column styles in the "family properties" page, and combine it with 2D drawings and 3D models to thoroughly record and archive the geometric outline, component dimensions, spatial positioning, and construction materials of the components. For the 3D visualization of the damage condition of the components, the "phased" function and "visibility/graphics" function in the view are used to create a new view, and the colors of the component materials are modified to express the different degrees of damage of the components [31, 53].

The restoration process is recorded using a self-developed plug-in called the "Cultural Heritage Restoration Manager." During the data assignment stage, this plugin allows us to select the Revit® project where the *.rfa families are located. The plug-in utilizes HBIM families as a recording medium to document the restoration process of components. It also records the pre-restoration and restoration process of components in a combination of graphics and text. The specific steps are as follows (Fig. 17):

• Step 1: In the "Enter Information" section, record the appearance and evaluation of the previous component, classify the degree and type of damage to the component, and add photos taken on-site.

• Step 2: In the "Information Modification" section, record the time of the restoration, use the text box to record the materials and practices used during the restoration process, and combine the on-site photos taken during the restoration process for recording. Finally, upload the saved data to the record library.

• Step 3: In the "Information Summary" section, review the information entered in the database, compare the status of the components before and after the restoration, and compare the status of the components during each stage of the restoration.

Fig. 17

Using a plug-in to record damaged and restored information

Results

From point clouds to HBIM model

The use of digital surveying technology to investigate cultural heritage is the beginning of the generation of HBIM. This process is supported by the creation of 3D models and component building. TLS survey and UAV photogrammetric survey approach point clouds with different degrees and parts of imperfection. These imperfections can be resolved through point cloud registration. After point cloud registration, the unnecessary parts of the complete point clouds should be eliminated, such as the environment, plants, and other buildings. The remaining part should be a concise and complete cultural heritage 3D point cloud model, which can access geometric appearance characteristics, structural characteristics of buildings, material and color information, but it cannot include the coupling relationship between components and the construction of components. These problems are solved by on-site manual investigation.

The component library has a strict classification, and all components undergo rigorous structural analysis before being established in the Revit environment based on accurate dimensions provided by 3D scan data. These components are custom-created families, controlled by parameters, which not only allow for adjusting and modifying the geometric outline and construction hierarchy of the components using parameters, but also support real-time access to the parameters of each component within the component. A total of 34 types of ethnic groups were established for the HBIM of the former residence of HSBC Bank, all of the families have high level of geometric detail. Then, using these components and with point cloud as a reference, the 3D model of the HBIM was constructed (Fig. 18).

Fig. 18

The HBIM model of the former residence of the HSBC bank: a East facade; b West facade; c North facade; d South facade

Besides generate projects using parameterized families, this HBIM integrates point clouds and mesh models obtained from TLS, UAV, and HLS. These scan data serve as the backbone of the HBIM. In addition, point clouds can also serve as a complementary expression of the HBIM (Fig. 19). In this article, the surrounding environment of the building obtained from point clouds is loaded into the HBIM, and is expressed together with the 3D model. Therefore, the HBIM model of the former residence of the HSBC bank is generated using custom components, with model details that conform to LOD4. And its components support the use of other modern architectural heritage in southern Fujian, complementing the component library types of modern architectural heritage in southern Fujian.

Fig. 19

The result of importing point clouds into HBIM, as a support for modelling or a supplement to HBIM

From point clouds to Mesh model

Import the mesh model (generate by point clouds) obtained from digital scanning into Revit^® as a "family". This method of creating components is based on a faithful reconstruction of architectural heritage from point clouds, which can restore the most authentic state of architectural heritage components, with less error compared to manual modelling.

The two of point cloud to mesh practices in this article use the "Cloud to Mesh Distance" command in Cloud Compare software to calculate the distance between the point cloud and the mesh grid, which is used to evaluate the accuracy of the reconstructed 3D model [54, 55]. The evaluation method uses ICP algorithm registration. From the results, it can be seen that the error distribution range of the red brick lattice is between 0.4–0.6 mm, while the error of the relief is 0 mm (Fig. 20). The reason for this difference is that on the one hand, the mesh model of the relief is automatically generated, while the mesh model of the red-brick lattice is semi-automatically generated, and there are always errors in the manually generated part. On the other hand, HLS is more suitable for planar-type components, compared with the red-brick lattice, the number of holes on the surface of the relief is fewer, and the depth of the holes is shallower. The point cloud obtained after HLS scanning has fewer missing points, these holes can lead to scanning failure and significant errors.

Fig. 20

a C2M result of the red-brick lattice; b C2M result of the damaged relief

3D documentation data of HBIM

This article's documentation is based on various parameters and secondary development plug-ins in Revit^®, and is completely conducted in the Revit^® environment. The method proposed in this article overcomes the limitations of Revit^® on information data management in HBIM. After categorizing the information obtained from cultural heritage surveys, information such as numerical values, text, images, videos, and hyperlinks are recorded through parameters and plug-ins, using the "family" in HBIM as the carrier. Finally, HBIM becomes an accessible, manageable, and modifiable information integration database, and is used as an indexing framework model. This article stores information entirely in the Revit® environment, using a combination of family parameters, 2D drawing annotations, and 3D model annotations to record and archive the heritage survey information and repair and protection process. The content of the 3D documentation data of HBIM is as follows:

• Supporting the documentation of damage and deterioration information of cultural heritage, as well as the recording of surveying and historical data.

• Supporting the recording of cultural heritage data obtained through on-site investigation.

• Supporting the recording of restoration process and deterioration monitoring.

• UAV Photogrammetric Survey provide cultural heritage images from a low-altitude perspective, including orthophotos and oblique photography images.

• Point clouds obtained through digital surveys are loaded into HBIM after registration and cleanup, not only serving as support for model construction, but also allowing for 3D visualization and serving as information supplements for HBIM models.

• 3D mesh model generated by point clouds is loaded into HBIM as a building component, serving as both an expression of the service details of the HBIM model and a 3D visualization of the damage and deterioration components.

• The components are created using parameterization, and the parameter information of the components is displayed, as well as the parameter information of each component.

• URLs such as images, videos, and recordings are used as data storage. The advantage of URLs is that they can connect not only local data files, but also web addresses, online cloud disks, pictures, and videos on the network.

• Combining 2D drawings with 3D models to accurately represent the geometric outline, detailed features, dimensional measurements, and component relationships of cultural heritage.

• Using various tools within the Revit® to query, manage, and modify data information.

Discussion

HBIM for the regional expression of cultural heritage

For the geographical characteristics of cultural heritage, this article uses the method of manifold digital survey (Sect. "Data survey and point registration") to cope with the special terrain of southeast Fujian, thereby obtaining accurate and complete scanning data. The terrain addressed in this article is cliffs. Similarly, this method applies to coastal, large-scale cultural heritage, as well as densely populated historical districts with high building density. After scan and registration, the previously isolated point cloud data can be integrated into a complete 3D point cloud model, which can serve as a support for the establishment of parameterized components, a foundation for the generation of mesh models, support for 3D model establishment, and a supplement to the 3D representation of HBIM. In order to highlight the regional characteristics of the cultural heritage of southeast Fujian in the model, including component appearance, component structure, building appearance, building structure, and environment, after classifying and grading the scan data (Sect. "Modelling"), a 3D model of cultural heritage was established using a parametric approach. The final HBIM has high service detail and accurately represents fabric types. This high-detail model accurately restores the current status and regional expression of architectural heritage, and also allows for the accurate visualization of damage and deterioration of cultural heritage (Sect. "From point clouds to HBIM model").

Multidisciplinary participation and continuous recording of 3D documentation

The archiving process is executed entirely within the Revit software, making it simple and easy to be accepted. Thanks to the support of various file formats in Revit, various types of information, including images, text, video, point clouds, URLs, CAD drawings, and PDF documents can be stored in HBIM. Moreover, Revit can also export models in *.ifc format to support 3D GIS applications and models in *.fbx format to support rendering and virtual display. Through this method, this article provides many information data recording methods for the digital protection of other cultural heritage, integrating traditional multiple recording resources and recording them in HBIM, achieving a combination of two-dimensional drawings with three-dimensional models, and recording the repair process of HBIM over time. It also monitors and records the damage and deterioration of components (Sect. "3D documentation data"). Therefore, non-professional BIM personnel can also classify and assign data to components. The HBIM model can support historical archive records of cultural heritage, data recording of current surveys, repair processes, and monitoring information. On the other hand, the restoration of cultural heritage involves the joint efforts of designers from multiple disciplines. Due to the various information recording formats in HBIM, it supports multidisciplinary collaboration and communication with construction teams.

Future perspectives

The final part of the research was to utilize the extensibility of the Revit software to practice monitoring the damage and deterioration of more types of architectural heritage, conducting a more detailed analysis of regional architectural defects, and develop a plugin in future work support the full lifecycle monitoring of cultural heritage. In fact, HBIM also has the potential for engineering quantity calculation and cost estimation. How to calculate the project cost, for the restoration of architectural heritage, and the reconstruction of historical neighborhoods, it is also a worthwhile aspect to explore. The visualization display of HBIM was explored, combined with VR and AR technologies, to expand the virtual display of building components, achieving. In addition, the combination of 3D GIS and HBIM displays allows for a more macroscopic view of architectural heritage, such as the precise three-dimensional display of Kulangsu Island International Historical Community down to building components. All of the above research is intended to be implemented in Revit, with the intention of using plugins to achieve these goals.

Conclusions

In conclusion, this article proposes a workflow that supports the establishment of high LOD and high detail service HBIM using various digital survey data. Finally, using building units as the carrier, various types of information on cultural heritage are recorded, making it possible to record, analyze, manage, view, share, and exchange data on cultural heritage in the HBIM environment. This method has made it possible to obtain a HBIM with parametric adjustment of components and rich documentation information, which aims to express the complex construction hierarchy and complex patterns in cultural heritage that are integrated into each individual element of the building, namely the building components. This modeling approach and information integration method greatly increase the amount of information in HBIM, allowing for complete expression and recording of the complexity of cultural heritage and architectural components in China, achieving a way of digital recording that combines 3D models with 2D drawings.

The modeling method proposed in this article combines manual establishment and automatic generation. The method proposed in this article overcomes the limitations of HBIM software storage and promotes the information recording process of HBIM or digital twin technology. Taking modern cultural heritage in China as a case study, various digital surveys provide multi-scale level cultural heritage data information. Based on historical research, current survey, and restoration planning, various time-dimension cultural heritage information can be obtained. The focus is on studying the classification methods and different recording methods for multi-stage, multi-time period, and multi-type information. The degradation and restoration of architectural heritage are analyzed from different time dimensions, promoting the process of information recording, analysis, and decision-making for cultural heritage. Modeling cultural heritage from the scale of building components is applicable to various types of architectural heritage, especially modern architecture in China and cultural heritage in East Asia. The generation of high-precision 3D models for complex structural architectural heritage has also become possible, allowing for the recording and analysis of various elements in cultural heritage, which may be a more comprehensive the whole building life-cycle recording method for cultural heritage protection.