Skip to main content

Integration of photogrammetry and laser scanning for enhancing scan-to-HBIM modeling of Al Ula heritage site

Abstract

The study highlights the significance of combining imaging and laser scanning techniques to enrich point clouds used for reliable documentation and Heritage Building Information Modeling (HBIM). The fusion-based approach was used to document Al Ula Heritage site in Saudi Arabia, a UNESCO World Heritage Site with well-preserved tombs from the first century BC. The HBIM parametric modeling process requires a detailed survey to collect all geometric data. Although terrestrial laser scanners (TLS) are an efficient tool for 3D recording of heritage scenes in a short period of time, the data resolution is insufficient for identifying and evaluating the spatial distribution of surface weathering forms. Furthermore, combining scans collected at different times may pose difficulties in recording the texture. These issues have an impact on the subsequent 3D modeling phase as well as the efficiency of interpreting and tracing surface features in the Scan-to-BIM process. The proposed workflow using imagery data to enhance both the geometry and coloring of laser point clouds. In addition to texture mapping, the high-resolution imagery is used to densify the laser data using dense image matching, allowing for a clear reading of the surface features, and serving as a useful tool to identify the type, extent, and severity of façade damage. TLS and imagery data were collected separately, with images taken at the best time and location for realistically recording of surface details. Finally, the captured images are orthorectified by TLS geometric information and used for HBIM texturing to provide realistic decay mapping. The results showed that the combination of TLS and photogrammetry allowed for the efficient collection of 3D data, which improved tracing and digitizing HBIM with complete mapping information. The research findings will be greatly useful in the management and planning of historic building conservation and restoration projects.

Introduction

Building information modeling (BIM) platform can greatly aid in the management planning, conservation, and restoration activities of the historical buildings [1,2,3,4,5]. BIM describes a structure's geometry as parametric objects with the attribute data required by various sectors, including engineering, architecture, archaeology, and materials. Unfortunately, the parametric objects in the current BIM library are unable to respond to the structural irregularities found in historic buildings [6, 7]. Such irregular and free form elements necessitate detailed and accurate surveys, which can be identified and replicated on the BIM platform [8]. Terrestrial laser scanning (TLS) and photogrammetry were presented as potential methods for producing point clouds as a geometric database for parametric modeling, a methodology known as Scan-to-BIM [9,10,11].

Laser scanner device directly records the object geometry, resulting of dense, reliable, and colored 3D point clouds. Despite TLS's potential, these systems can't be utilized solely for detailed and realistic recording of the historical structures [12, 13]. The majority of TLS can acquire digital images via an attached built-in camera to color their point clouds. However, because the optimal camera position is not always compatible with scanner position, color data may be of insufficient quality to visualize surface features and conditions [14]. Another issue is the difficulty in locating the outlines of surface features such as pathologies and crack extent in the deterioration region. Despite the fact that the scanner's model has an enormous amount of 3d points representing surface features, the mixed pixel effect, which occurs near surface edges and cracks, has the potential to corrupt the coordinates of their corresponding points [15,16,17,18]. Unfortunately, it is not possible to evaluate these errors' effects independently [19]. Due to previous issues, dimensional measurement, interpretation, and tracing in the Scan-to-BIM process may be inaccurate and unreliable for representing surface features.

On the other hand, photographs that were taken at ideal position and times offer better realistic representation of color and façade characteristics. Currently, photogrammetry is a cost-effective and efficient method of representing real-world structures [20]. Image processing can also be used in modeling and assessment of structural cracks and damage [21, 22]. The main challenge in image-based modeling is the main constraints on image quality, camera network, shadow, texture-less elements, and image scaling, which can affect photogrammetric processing and the final modeling results [23, 24].

Recent publications have demonstrated the feasibility of integrating multi-source data to digitize complex heritage structures [25, 26], as well as improving the input data to enhance object modeling in HBIM. For instance, Sztwiertnia et al. [27] used photos to determine the precise contour of the complicated shapes and elements in the point cloud before they were converted to 3D elements. Lopez et al. [28] used images combined with point clouds to improve displaying the border and shape of complex elements and edges during the geometry modeling in BIM platform. Banfi et al. [29] used orthophotos produced using photogrammetry and photo planes to enable better material analysis and help the archaeologist in understand the buildings components. The orthophotos were three-dimensionally mapped on the BIM models and used in a new immersive environment. Alshawabkeh et al. [30] supplemented TLS data in the inaccessible region with images obtained from an unmanned aerial vehicle, as a result, the overall geometry of the building is correctly rebuilt, which improves HBIM digitizing.

Cracks and material decay monitoring is a common and widespread issue in the built heritage. However, the implementing of BIM models in heritage that reflect a structure in its conditions is still a challenge [31]. As-built modeling with high levels of detail must detect and recognize finer and more complex elements in the scan to BIM process [32]. Lanzara et al. [33] and Conti et al. [34] used orthophotos as visual support for the HBIM digitization of the surface deterioration and cracks. In many studies, color legends have been used to map surface material and decay in a HBIM [35, 36]. Others attached the photos and orthophotos with the BIM objects as additional data for heritage preservation [37, 38]. Using BIM software's standard texture to represent surface material and decay, resulting in inaccurate mapping and information sharing [39,40,41]. The goal of this work was to create a method for integrating the results of imagery and laser scanning to enhance the scan to BIM process. Data fusion was approached during the recording of Lihyan, the son of Kuza tomb in Al Ula Heritage site, a UNESCO World Heritage Site with well-preserved tombs dating back to the first century BC.

In summary, this paper addressed the following contributions:

  1. 1.

    Fusion-based approach using the results of TLS and photogrammetry surveys to enrich scan to BIM process by supporting digitizing the geometry and cracking pattern.

  2. 2.

    Realistic textured HBIM workflow using true orthophoto created by combining TLS and photogrammetry aids in recognizing and tracking surface material decay and reconstructing realistic renderings of built heritage.

  3. 3.

    Developing a BIM library for the Nabataean architectural elements that includes high-detail parametric objects.

The paper is structured with the following sections: section Al Ula heritage site discusses the history of the Al Ula Heritage site, the Hegra city, and their tombs. Data collection and preprocessing are addressed in section Acquisition laser and photogrammetric data. Section Data fusion presents TLS and imagery data fusion. Section HBIM describes the BIM implementation. The results and conclusion are discussed in sections Discussion and Conclusion.

Al Ula heritage site

AlUla City, located approximately 300 km north of Madinah, has a remarkable natural and human heritage. AlUla, 52-hectare of rock, sand and oasis, has been a natural center of commercial activity for thousands of years and is the site of important trade routes that have existed since at least the 1st millennium BC. It is located on an ancient incense rout that reaches southern Arabia to Egypt and beyond. The sandstone mountains that surround Al Ula have had a significant impact on human history, providing a favorable environment for civilizations to flourish. The Dadanites, Lihyanites, and Nabataeans all built cities in this region over many centuries. The most well-known of Al Ula settlements is Hegra archaeological site, which is registered on UNESCO'S World Heritage Site list in 2008. Hegra, known as Madā’in Ṣāliḥ, was the second major city of the ancient Nabataean kingdom after Petra (Jordan), the Nabataeans settled there in the middle of the first century BC. The Nabataeans are best known for their impressive rock-cut tombs whose decorated façades give information on their artistic and hand-crafted skills. Nabataeans ruled the place until 106 A.D. when Roman Emperor Trajan conquers Hegra as the southern limit of its flourishing Empire [42]. Hegra city is best known for nearly 100 beautiful and remarkably well‐preserved tombs, see Fig. 1. They were built to house the remains of families or groups, the size and level of decoration reflecting their status. Simpler pit graves can be found higher up the mountains, likely containing the remains of lower-class people. The number and range of preserved tomb inscription at Hegra is unique. Many tombs have inscriptions that record who commissioned them and when, as well as the craftsperson who carved them. A striking and impressive building that rises over 72 feet is the Tomb of Liyhan, son of Kuza depicted in Fig. 2. This tomb, also known as Qasr al-Farid, which means "the lonely castle,". Visitors can observe that artists abandoned work before completion because the lowest half is unfinished.

Fig. 1
figure 1

Preserved tombs in Al Ula, Jabal al Khuraymat necropolis

Fig. 2
figure 2

Tomb of Lihyan, son of Kuza (Qasr al-Farid)

Acquisition laser and photogrammetric data

The data in our investigation collected from Lihyan, son of Kuza, tomb which is located in Hegra archaeological site. The facade and the interior of the tomb were scanned with a Leica RTC360 scanner. The scanner has a 360° × 300° field of view and can provide up to two million colored points per second over distances between of 0.5–130 m. The system automatically pre-registers the point clouds from the various scans, increasing field productivity and scan control. In order to color the produced point cloud, the system uses a calibrated 360 × 300 spherical image captured with a 36 MP camera. The TLS positions at the site were carefully designed so that point clouds from two adjacent locations overlapped sufficiently. Twenty-nine different scanner viewpoints, including four interiors, were chosen to provide 3D coverage of the tomb façade and the mountain surrounding it, the scanning workplan and examples of exterior scans are depicted in Fig. 3. The model has over 403 million points, and the overall of Ground Sampling Distance (GSD) of the façade model is around 6 mm. Figure 4 depicts the GSD of one of the collected scans, which is 4.9 mm.

Fig. 3
figure 3

Scanning work plan of the tomb and examples of collected scans

Fig. 4
figure 4

The Ground Sampling Distance (GSD) of one of the collected scans

After collected the TLS data, a post-processing step is used to eliminate noise data. The interior and exterior scans were then aligned into a single coordinate system using Autodesk ReCap software to create the final scene depicted in Fig. 5. The co-registration is carried out without the use of any Ground Control Point (GCP), instead relying on extracted tie points. According to the registration data report, more than 90% of overlapping points are within 6 mm of the corresponding project features. Figure 6 depicts the final model with color information collected using TLS camera, whereas Fig. 7 depicts the tomb interior scans.

Fig. 5
figure 5

Merging the scans into a single coordinate system

Fig. 6
figure 6

The final TLS colored point cloud of the tomb

Fig. 7
figure 7

The tomb interior scans

Color data captured with the system-built camera may not be of sufficient quality for texturing and feature interpretation. The device view points and locations that are identical for scanning data may not be optimal for the camera position to collect high resolution and quality images for texture mapping. Multiple scans and viewpoints of the large and complex structure can also result in different lighting conditions, resulting in a non-homogeneous appearance and color jumping in the final model. Figure 8 shows images taken by the laser system's installed camera. The images' different acquisition times result in significant differences in brightness and color. This will clearly interrupt the appearance of the textured 3D point clouds. Modern laser scanners can produce massive point clouds that represent the surface, but surface features and pathologies such as cracks, decay, and surface damage remain difficult to interpret and identify. The mixed pixel effect that creates in the spatial discontinuities of the surfaces will result in inaccurate coordinates of the data points, as depicted in Fig. 9, which may pose a challenge for accurate dimensional measurement. As a result of the preceding challenges, and in order to improve the geometric and radiometric information of laser data, additional digital images were collected for photogrammetric processing using a Canon EOS 5D Mark IV camera with a resolution of 6720 × 4480. The images were almost taken at the same time to ensure that the lighting conditions required for laser data coloring and true orthophoto were met. The selection of the camera geometry configuration for data acquisition is a critical step in the image-based modeling process. In our projects, the camera captured 173 closely spaced images in a circle with more than 80% overlap, as depicted in Fig. 10. The photos were taken with fixed focal length of 35 mm. Agisoft Metashape software was used for the image processing to produce 3D models with arbitrary coordinate system. The processing time for the all photographs was 9 h. TLS is used to establish Ground Control Points (GCP) in order to orient and scale photogrammetric data. Some point pairs' coordinates were found between the two models. Figure 11a shows the GCP locations and error estimation, with a 1.5 cm average error. The software calculates the interior and external camera orientation parameters while aligning photos. These parameters were used in texture mapping and true orthophoto processing. The final photogrammetric model is depicted in Fig. 11b with 9.9 million points and overall GSD of 1.4 cm. As a result of these findings, the utility of photogrammetric point cloud data is limited due to its lower accuracy when compared to TLS. Accuracy limitations due to image matching issues caused by factors such as a lack of image texture and other foreshortening factors mentioned in the introduction section.

Fig. 8
figure 8

Images taken with the scanner's built-in camera used for coloring point clouds

Fig. 9
figure 9

Mixed pixel effects on spatial discontinuities result in incorrect TLS data point positions and coordinates

Figure10
figure 10

Camera positions for photogrammetric data acquisition

Fig. 11
figure 11

a GCP estimation errors in color legend b GSD of photogrammetric point cloud

Data fusion

In order to enhance data quality and quantity for HBIM modeling, data fusion can be approached in three levels: At the first level, photorealistic impressions of laser-generated surfaces can be produced effectively using high quality radiometric imagery data. At the second level, a flexible realistic texturing approach for the HBIM model is proposed, utilizing true orthophoto generated from the TLS digital surface model and selected images taken from an independent camera. In the third level, the TLS dataset can be supplemented and densified in weathering surface areas with additional close-up images to interpret and improve BIM digitizing of cracks and linear surface features.

Texture mapping

The basic concept is to combine independent imagery with orientation parameters obtained from the previous photogrammetric process with TLS geometric data. Our own C +  + code was used to map the texture (RGB values) from images to the 3D point clouds using the collinearity equations, Eq. 1 and 2. The mathematical relationships are based on the fact that a photograph's perspective center (PC), the image point, and the associated object point are all located along a straight line, as depicted in Fig. 12. Within the equations (xa, ya) are the imagery coordinates of the object, while (XA, YA, ZA) are the corresponding coordinates in object space. The angular parameters (ω, φ, κ) are used to calculate the nine elements of the rotation matrix rij. The camera position coordinates are (Xo, Yo, Zo), the focal length is c, and the principal point coordinates are (xo, yo). Various photos have been arranged in the digital environment to texturize the survey data. The combination of the results enabled the creation of 3D data that is both metrically accurate and photorealistic, as illustrated in Fig. 13.

$$x_{a} = x_{a} - c\frac{{r_{11} \left( {X_{A} - X_{O} } \right) + r_{21} \left( {Y_{A} - Y_{O} } \right) + r_{31} \left( {Z_{A} - Z_{O} } \right)}}{{r_{13} \left( {X_{A} - X_{O} } \right) + r_{23} \left( {Y_{A} - Y_{O} } \right) + r_{33} \left( {Z_{A} - Z_{O} } \right)}}$$
(1)
$$y_{a} = y_{a} - c\frac{{r_{12} \left( {X_{A} - X_{O} } \right) + r_{22} \left( {Y_{A} - Y_{O} } \right) + r_{32} \left( {Z_{A} - Z_{O} } \right)}}{{r_{13} \left( {X_{A} - X_{O} } \right) + r_{23} \left( {Y_{A} - Y_{O} } \right) + r_{33} \left( {Z_{A} - Z_{O} } \right)}}$$
(2)
Fig. 12
figure 12

Camera registration parameters

Fig. 13
figure 13

Texture mapping of TLS data using independent images; the ground and surrounding areas are still textured from TLS camera

True orthophoto

The images are also used to create true orthophotos for HBIM texturing. Aside from realistic rendering, true orthophoto information in HBIM is useful for positioning objects, measuring distances, calculating areas, and quantifying surface decay. The quality of true orthophotos is determined by several factors, including image resolution, camera calibration and digital surface model (DSM) accuracy. The method removes the perspective distortions from the images using the DSM. A technique for automatic DSM generation is also offered by digital photogrammetry, but its accuracy is constrained by issues such as image matching and the lack of image texture. The digital surface model (DSM) from laser scanning was used as an alternative in our approach for accurate and reliable results. The color information is taken from the independent and oriented images obtained from the preceding photogrammetric process. The method, depicted in Fig. 14, begins by sampling the façade's 3D points in a regular grid, each with its own attached point. To store the spectral orthophoto data, a new grid with equivalent pixel size is created. The associated orthophoto pixel is then given the relevant imagery data using collinearity equations. The true orthophoto is created using the pixels from the source image that correspond to the visible points, while invalid pixels belong to shadow areas (without DSM data) are identified and labeled in white. The algorithm was developed using the C +  + programming language. Figure 15 depicts true orthophotos of images taken by the scanner device camera and an independent camera with a resolution of 0.6 mm/pixel.

Fig. 14
figure 14

True orthophoto workflow

Fig. 15
figure 15

a Sampling the 3D TLS points of the façade in a regular grid (single scan). b True orthophoto taken with a TLS camera c True orthophoto taken with a separate camera, in the white are the occluded areas

Densify point clouds

Even in dense scanning mode, the mixed pixel effect that occurs near surface edges and cracks has the potential to corrupt scanner point cloud coordinates. Using close-up photographs, image-based crack modeling can be an effective method. A hybrid approach combining data from the laser scanner and digital imagery was developed to support the modeling quality of such details in HBIM platform. Our method begins with photogrammetry, which collects data from close-up photographs taken at nearly constant distances from the weathering areas of the façade. The camera captured images with more than 80% overlap for successful model processing, the camera's widest shooting range was obtained by using a fixed focal length of 18 mm. At first, Wu's [43] VisualSFM software is used to match the images and determine camera orientation parameters, as depicted in Fig. 16a. The camera parameters were then exported to the IFP SURE software, which is based on the dense image matching algorithm developed by Rothermel and Wenzel [44]. The software determines 3D information for almost every pixel using a modified version of the Semi Global Matching (SGM) algorithm, Fig. 16b shows the disparity image using SURE software. Because the generated 3D point clouds have an arbitrary coordinate system, the scale information can be obtained from textured laser model. The registration phase was carried out using the cloud to cloud (C2C) method rather than the TLS control point because the unified point cloud will be exported to a BIM platform for digitizing the building elements and cracks features. The point cloud datasets were first registered in Cloud Compare software using some corresponding tie points to solve the pose and scale information of the imager point cloud. The software proceeded through the ICP registration process with a predetermined number of iterations. Following the spatial registration and scaling of the point cloud datasets, the registration analysis performed using cloud to cloud distance computation. This process determines the nearest neighbor's distance between the two data sets. As depicted in Fig. 17, it can be clearly seen that most of the data has a small deviation value, the large deviations in green areas represent weathering surfaces and cracks that have the potential to corrupt the coordinates of scanner point clouds due to mixed pixel effect or missing data. Densifying the damage areas, as illustrated in Fig. 18, will aid in improving the geometry and visual quality of the damage areas in the laser scanner data, as well as drawing the exact contour of the cracks. The results show 1–2 mm GSD is possible.

Fig. 16
figure 16

a Camera orientation using Visual SFM software b Disparity image using SURE software

Fig. 17
figure 17

C2C Registration results, 3d surface deviations analysis using distance computations values in color mode

Fig. 18
figure 18

Densify TLS data using imagery point cloud

HBIM

After combining the textured TLS and imagery point clouds, the data and the true orthophotos were uploaded and processed in the BIM platform, as shown in Fig. 19. The BIM modeling tool was the Autodesk Revit Architecture Software. Four stages were identified in the BIM modeling of the tomb. The first stage focuses on the modeling of regular components and uniform objects, like columns, walls, and plain façades. The second stage focuses on developing new families of repetitive Nabataean architectural elements with shared characteristics (parameters), like column capitals, cornices, pediments, and door frames. In the third stage, the true orthophoto was warped to the corresponding surfaces to create a textured HBIM model. The densify point cloud in the weathering regions is used in the final stage to model the crack patterns in HBIM.

Fig. 19
figure 19

Import the integrated point clouds into the BIM platform

The tomb point clouds were divided into longitudinal and cross sections in order to locate various library objects like columns, doors, walls, and other accessibility elements. The point cloud's characteristics were determined by closed polygons, which were then used to model the regular architectural components. It was critical to include width, length, and base dimensions when modeling columns and walls in order to produce the correct profile and present a parametric function that could modify all components. Direct tape measurements were used to determine the thickness of the walls and doors. Many of the tomb components have one-of-a-kind designs that did not exist in the BIM platform library. Thus, new families of Nabataean architectural components have been developed like those in Fig. 20. The façades of the Nabataean tombs share various architectural features and styles. The two sets of steps facing each other that crowned the facade are examples, as are the entablature supported by pilasters and supplemented by quarter columns on the inner side, cavetto cornice, and capitals. The new library was connected to the database of the different elements, allowing users to modify parameters in order to change the object shape. This library offers a workable solution for modeling the repetitive Nabataean. Figure 21a depicts the tomb BIM model, which includes all architectural elements. The texture and surface decay of cultural heritage objects is also important. Most HBIM applications address texture by utilizing the program's default texture or by developing a custom texture from the object image, resulting in an unreal visual effect. Our suggested approach emphasized data fusion in order to generate precise true orthophotos that can be used for realistic HBIM texturing. As shown in Fig. 21b, the warping over the HBIM geometry in the Revit software is reliable and accurate because the true orthophoto generated had the exact dimensions of corresponding BIM surfaces. This is accomplished in the software by applying a new decal type to the façade surface; the decal acts as a placeholder for the produced true orthophoto. The width and height of the produced true orthophoto should be the same as the physical size of the decal on the optional bar. The true orthophoto file can now be accessed and directly placed on the decal. Finally, as illustrated in Fig. 22, densifying point clouds in weathering areas improves visual quality and enables more accurate tracing of the exact contour outline of cracks in the BIM platform. The crack is then parametrically modeled, along with the lengths, widths, areas, causes, and descriptions that can be added and modified in the BIM properties window. These parameters can aid in monitoring the behavior of structural cracks over time, which is an important step in determining the structure's health.

Fig. 20
figure 20

Create new families for repetitive Nabataean architectural elements

Fig. 21
figure 21

a The final BIM models of the tomb. b Textured HBIM with true orthophoto

Fig. 22
figure 22

Crack pattern modeling in BIM platform

Discussion

The uniqueness and complexity of existing building components are the main challenges in HBIM. Modeling of architectural pathologies and damages that affect buildings is also critical for monitoring and assessing the state of conservation and existing architectural heritage. BIM library parametric objects and materials do not accurately represent irregular heritage structures, resulting in inaccurate information sharing and mapping. As a result, it is necessary to begin with a proper survey in order to correctly define the architectural geometric and radiometric characteristics. The workflow presented here enables access to TLS and imagery data in order to create a full integrated survey in a single and shared database, thereby improving the digitization of building geometry, decay, and damages modeling in the BIM process. The approach is flexible because imagery can be obtained independently of TLS data. The data we used in our investigation collected from tomb of Lihyan, the son of Kuza which is located at the Al Ula heritage site. Following is a summary of our proposed methods' main contributions to recent HBIM research.

  1. 1.

    Most 3D sensors include an integrated camera that records the RGB color values of the collected point clouds in order to provide model-recorded texture. Low resolution textured models can occur as a result of lighting and shadow conditions in both outdoor and indoor applications. Additionally, data acquisition from a long distance to reduce the number of required scans and the field time, or due to accessibility issues with the scan positions, will result also in a low-resolution textured model even in the presence of a high-resolution camera. Quality textured 3d model is important for accurate interpretation and digitizing of the geometry in BIM platform. The scientific community has developed several methods to enhance detection and tracing of detailed parametric features during the scan to BIM process using non-metric images [27, 28]. These images lack scale information, which may result in incorrect object shape positioning in the corresponding laser data. While, our proposed method projects the texture values directly on the point clouds using high quality digital images collected at the optimal position and time for texturing.

  2. 2.

    The ability to attach additional characteristics that define and highlight the damage causes, extent, and typology adds value to conservation activities and monitoring. In BIM platform, detecting and tracing surface damage in laser scanner data is a challenge. [29, 33, 34] used photogrammetric 2D orthophoto as a basis for detailed archaeological analysis and identification of damages in BIM surfaces. However. Accuracy limitations in photogrammetry-generated DSMs for large objects should be considered due to image matching issues caused by factors such as a lack of image texture and other foreshortening factors mention in the introduction sections. Furthermore, mapping 2D photos in HBIM surfaces will be restricted to vectorizing the various pathologies only on flat surfaces; surfaces with curvilinear directions must be discretized into a series of small faces in order to perform mapping correctly. Using close imagery and the dens image matching algorithm, the proposed method in this work densifies the point clouds in the damaged regions. This method, as illustrated in Figs. 18 and 22, improves the geometry and radiometric properties of damaged areas, avoids the mixed effect problem in laser data, which causes measurement errors in discontinuous surfaces, and draws the exact contour of the caracks for BIM tracing and modeling of crack pattern paths in curvilinear directions.

  3. 3.

    Texture, in addition to accurate geometry, can depict the state of the structure and material deterioration. Because managing a new structure is the primary goal of current BIM platforms, texture has become less important. In heritage BIM modeling, texturing is frequently handled using either the software standard textures [39, 40] or custom textures using object photo [45]. Others added the images to the BIM platform as supplemental data to aid in conservation of heritage [37]. On the other hand, our proposed method for texturing HBIM uses reliable true-orthophoto produced by combining TLS and photogrammetry. Because the true orthophoto and corresponding HBIM surfaces are of the same dimensions, the warping over the HBIM geometry will be precise and consistent, as shown in Fig. 23. In fact, taking the independent images as orthogonal to the surface as possible will reduce occlusion issues and allow for better interpretation of surface characteristics. Stakeholders may also benefit from the topography and terrain BIM modeling [46]. Figure 24 depicts modeling of over 100,000 TLS points representing the contours of the surrounding terrain in Revit.

Fig. 23
figure 23

As textured as HBIM using true orthophoto

Fig. 24
figure 24

Terrain modeling and HBIM integration

Conclusion

The paper describes an approach that combines TLS with image-based modeling techniques to enrich scan to BIM process for better interpretation and plotting of heritage complex objects. The data integration enables highly realistic 3D models that add graphic and geometric data to the related BIM database. Crack detection and assessment is a common problem in the built environment; this paper presents a reliable study intended to evaluating the feasibility of integrating TLS and imagery data to improve modeling cracks in scan to BIM process and accurately mapping their length, orientation, and width. In addition, an efficient HBIM texturing methodology using true orthophotos produced using TLS point clouds and close-range imagery was proposed. With the help of the HBIM texture data, we can recognize and track surface material decay and reconstruct realistic rendering of the built heritage. The fusion-based approach provided was used to model the Nabataeans tomb of Lihyan, son of Kuza in Hegra archaeological site. A library of high-detail parametric BIM objects was created to supplement the Nabataeans' architectural objects. Future research should investigate how the workflow can be used to detect surface pathologies and cracks automatically using feature detection algorithms and deep learning.

Availability of data and materials

All data generated or analysed during this study are included in this published article.

Abbreviations

BIM:

Building information modeling

HBIM:

Heritage building information modeling

TLS:

Terrestrial laser scanner

UNESCO:

United nations educational, scientific, and cultural organization

MP:

Mega pixels

RGB:

Red, green, blue

PC:

Perspective center

C2C:

Cloud to cloud

References

  1. Solla M, Gonçalves LMS, Gonçalves G, Francisco C, Puente I, Providência P, Gaspar F, Rodrigues H. A building information modeling approach to integrate geomatic data for the documentation and preservation of cultural heritage. Remote Sens. 2020;12(24):1–24. https://doi.org/10.3390/rs12244028.

    Article  Google Scholar 

  2. Costantino D, Pepe M, Restuccia AG. Scan-to-HBIM for conservation and preservation of cultural heritage building: the case study of san Nicola in Montedoro Church (Italy). Appl Geomatics. 2021. https://doi.org/10.1007/s12518-021-00359-2.

    Article  Google Scholar 

  3. Baik A. The use of interactive virtual Bim to boost virtual tourism in heritage sites Historic Jeddah. ISPRS Int J Geo-Inform. 2021. https://doi.org/10.3390/ijgi10090577.

    Article  Google Scholar 

  4. Di Stefano F, Gorreja A, Malinverni ES, Mariotti C. knowledge modeling for heritage conservation process: from survey to HBIM implementation. Int Arch Photogramm Remote Sens Spat Inf Sci ISPRS Arch. 2020;44(4/1):19–26.

    Article  Google Scholar 

  5. Bruno N, Roncella R. HBIM for conservation: a new proposal for information modeling. Remote Sens. 2019. https://doi.org/10.3390/rs11151751.

    Article  Google Scholar 

  6. Sampaio AZ, Pinto AM, Gomes AM, Sanchez-lite A. Generation of an HBIM library regarding a palace of the 19th century in Lisbon. Appl Sci. 2021. https://doi.org/10.3390/app11157020.

    Article  Google Scholar 

  7. D’amico A, Currà E. From TSL survey to HBIM, issues on survey and information modeling implementation for the built heritage the case study of the temple Di Bacco Ravello. Proc Int Conf Educ Res Comput Aided Archit Des Eur. 2017;2:39–48. https://doi.org/10.52842/conf.ecaade.2017.2.039.

    Article  Google Scholar 

  8. López FJ, Lerones PM, Llamas J, Gómez-García-Bermejo J, Zalama E. A review of heritage building information modeling (H-BIM). Multimodal Technol Interact. 2018. https://doi.org/10.3390/mti2020021.

    Article  Google Scholar 

  9. Murphy M, Mcgovern E, Pavia S. Historic building information modelling (HBIM). Struct Surv. 2009;27(4):311–27. https://doi.org/10.1108/02630800910985108.

    Article  Google Scholar 

  10. Andriasyan M, Moyano J, Nieto-Julián JE, Antón D. From point cloud data to building information modelling: an automatic parametric workflow for heritage. Remote Sens. 2020. https://doi.org/10.3390/rs12071094.

    Article  Google Scholar 

  11. Rocha G, Mateus L, Fernández J, Ferreira V. A scan-to-BIM methodology applied to heritage buildings. Heritage. 2020;3(1):47–67. https://doi.org/10.3390/heritage3010004.

    Article  Google Scholar 

  12. Alshawabkeh Y. Color and laser data as a complementary approach for heritage documentation. Remote Sens. 2020;12(20):1–24. https://doi.org/10.3390/rs12203465.

    Article  Google Scholar 

  13. Gines JLC, Cervera CB. Toward hybrid modeling and automatic planimetry for graphic documentation of the archaeological heritage: the Cortina family pantheon in the cemetery of Valencia. Int J Archit Herit. 2019. https://doi.org/10.1080/15583058.2019.1597214.

    Article  Google Scholar 

  14. Dostal C, Yamafune K. Photogrammetric texture mapping: a method for increasing the fidelity of 3D models of cultural heritage materials. J Archaeol Sci Reports. 2017;2018(18):430–6. https://doi.org/10.1016/j.jasrep.2018.01.024.

    Article  Google Scholar 

  15. Chaudhry S, Salido-Monzú D, Wieser A. A modeling approach for predicting the resolution capability in terrestrial laser scanning. Remote Sens. 2021;13(4):1–23. https://doi.org/10.3390/rs13040615.

    Article  Google Scholar 

  16. Tang P, Akinci B, Huber D. Quantification of edge loss of laser scanned data at spatial discontinuities. Autom Constr. 2009;18(8):1070–83. https://doi.org/10.1016/j.autcon.2009.07.001.

    Article  Google Scholar 

  17. Stałowska P, Suchocki C, Rutkowska M. Crack detection in building walls based on geometric and radiometric point cloud information. Autom Constr. 2021;2022:134. https://doi.org/10.1016/j.autcon.2021.104065.

    Article  Google Scholar 

  18. Wang Q, Sohn H, Cheng JCP. Development of a mixed pixel filter for improved dimension estimation using AMCW laser scanner. ISPRS J Photogramm Remote Sens. 2016;119(September):246–58. https://doi.org/10.1016/j.isprsjprs.2016.06.004.

    Article  Google Scholar 

  19. Mechelke K, Kersten TP, Lindstaedt M. Comparative investigation into the accuarcy behaviour of the new generation of terrestrial laser scanning systems. Opt 3-D Meas Tech. 2014;2007(I January):319–27.

    Google Scholar 

  20. Mikita T, Balková M, Bajer A, Cibulka M, Patočka Z. Comparison of different remote sensing methods for 3d modeling of small rock outcrops. Sensors (Switzerland). 2020. https://doi.org/10.3390/s20061663.

    Article  Google Scholar 

  21. Valença J, Puente I, Júlio E, González-Jorge H, Arias-Sánchez P. Assessment of cracks on concrete bridges using image processing supported by laser scanning survey. Constr Build Mater. 2017;146:668–78. https://doi.org/10.1016/j.conbuildmat.2017.04.096.

    Article  Google Scholar 

  22. Alshawabkeh Y. Linear feature extraction from point cloud using color information. Herit Sci. 2020;8(1):1–13. https://doi.org/10.1186/s40494-020-00371-6.

    Article  CAS  Google Scholar 

  23. Arza-García M, Gil-Docampo M, Ortiz-Sanz J. A hybrid photogrammetry approach for archaeological sites: block alignment issues in a case study (the Roman camp of a Cidadela). A hybrid Photogramm approach Archaeol sites Block alignment issues a case study the Rom camp A Cid. 2019;38:195–203.

    Google Scholar 

  24. Schonberger JL, Frahm JM. Structure-from-motion revisited. In Proc IEEE Comput Soc Conf Comput Vision Pattern Recognit. 2016;2016:4104–13. https://doi.org/10.1109/CVPR.2016.445.

    Article  Google Scholar 

  25. Murtiyoso A, Grussenmeyer P, Suwardhi D, Awalludin R. Multi-scale and multi-sensor 3D documentation of heritage complexes in urban areas †. ISPRS Int J Geo-Inform. 2018. https://doi.org/10.3390/ijgi7120483.

    Article  Google Scholar 

  26. Reinoso-Gordo JF, Gámiz-Gordo A, Barrero-Ortega P. Digital graphic documentation and architectural heritage: deformations in a 16th-century ceiling of the Pinelo Palace in Seville (Spain). ISPRS Int J Geo-Inform. 2021. https://doi.org/10.3390/ijgi10020085.

    Article  Google Scholar 

  27. Sztwiertnia D, Ochałek A, Tama A, Lewińska P. HBIM (Heritage building information modell) of the Wang stave church in Karpacz-case study. Int J Archit Herit. 2021;15(5):713–27. https://doi.org/10.1080/15583058.2019.1645238.

    Article  Google Scholar 

  28. López FJ, Lerones PM, Llamas J, Gómez-García-Bermejo J, Zalama E. A framework for using point cloud data of heritage buildings toward geometry modeling in a BIM context: a case study on Santa Maria La Real De Mave Church. Int J Archit Herit. 2017;11(7):965–86. https://doi.org/10.1080/15583058.2017.1325541.

    Article  Google Scholar 

  29. Banfi F, Brumana R, Stanga C. Extended reality and informative models for the architectural heritage: from scan-to-Bim process to virtual and augmented reality. Virtual Archaeol Rev. 2019;10(21):14–30. https://doi.org/10.4995/var.2019.11923.

    Article  Google Scholar 

  30. Alshawabkeh Y, Baik A, Miky Y. Integration of laser scanner and photogrammetry for heritage BIM enhancement. ISPRS Int J Geo-Inform. 2021. https://doi.org/10.3390/ijgi10050316.

    Article  Google Scholar 

  31. Chiabrando F, Lo Turco M, Rinaudo F. Modeling the decay in an HBIM starting from 3d point clouds a followed approach for cultural heritage knowledge. Int Arch Photogramm Remote Sens Spat Inf Sci ISPRS Arch. 2017;42(2W5):605–12.

    Article  Google Scholar 

  32. Bruno S, Musicco A, Fatiguso F, Dell’Osso GR. The role of 4D historic building information modelling and management in the analysis of constructive evolution and decay condition within the refurbishment process. Int J Archit Herit. 2021;15(9):1250–66. https://doi.org/10.1080/15583058.2019.1668494.

    Article  Google Scholar 

  33. Lanzara E, Scandurra S, Musella C, Palomba D, di Luggo A, Asprone D. Documentation of structural damage and material decay phenomena in H-BIM systems. Int Arch Photogramm Remote Sens Spat Inf Sci. 2021;XLVI-M–1–2:375–82.

    Article  Google Scholar 

  34. Conti A, Fiorini L, Massaro R, Santoni C, Tucci G. HBIM for the preservation of a historic infrastructure: the Carlo III bridge of the Carolino aqueduct. Appl Geomatics. 2020. https://doi.org/10.1007/s12518-020-00335-2.

    Article  Google Scholar 

  35. Malinverni ES, Mariano F, Di Stefano F, Petetta L, Onori F. MOdelling in HBIM to document materials decay by a thematic mapping to manage the cultural heritage: the case of “Chiesa Della Pietà” in Fermo. ISPRS Ann Photogramm Remote Sens Spat Inf Sci. 2019;42(2/W11):777–84.

    Article  Google Scholar 

  36. Santagati C, Papacharalambous D, Sanfilippo G, Bakirtzis N, Laurini C, Hermon S. HBIM approach for the knowledge and documentation of the St john the theologian cathedral in Nicosia (Cyprus). J Archaeol Sci Reports. 2020;36(July 2020):102804. https://doi.org/10.1016/j.jasrep.2021.102804.

    Article  Google Scholar 

  37. Fregonese L, Taffurelli L, Adami A, Chiarini S, Cremonesi S, Helder J, Spezzoni A. Survey and modelling for the BIM of Basilica of San Marco in Venice. Int Arch Photogramm Remote Sens Spat Inf Sci ISPRS Arch. 2017;42(23):303–10.

    Article  Google Scholar 

  38. Brumana R, Condoleo P, Grimoldi A, Banfi F, Landi AG, Previtali M, Brumana R, Condoleo P, Grimoldi A, Banfi F, Landi AG, Previtali M. HR LOD Based HBIM to detect influences on geometry and shape by Stereotomic construction techniques of brick vaults. Appl Geomatics. 2018;10(4):529–43.

    Article  Google Scholar 

  39. Martín-Lerones P, Olmedo D, López-Vidal A, Gómez-García-bermejo J, Zalama E. BIM supported surveying and imaging combination for heritage conservation. Remote Sens. 2021;13(8):1–11. https://doi.org/10.3390/rs13081584.

    Article  Google Scholar 

  40. Lo Brutto M, Iuculano E, Lo Giudice P. Integrating topographic, photogrammetric and laser scanning techniques for a scan-to-BIM Process. Int Arch Photogramm Remote Sens Spat Inf Sci ISPRS Arch. 2021;43(2–2021):883–90.

    Article  Google Scholar 

  41. Banfi F, Mandelli A. Computer vision meets image processing and UAS photogrammetric data integration: from HBIM to the extended reality project of Arco Della pace in Milan and its decorative complexity. J Imaging. 2021. https://doi.org/10.3390/jimaging7070118.

    Article  Google Scholar 

  42. Cassola-Cochin V. The development of Nabataean Mada’in Sālih (Saudi Arabia) into an attractive sightseeing: from traditional rejection to official recognition. Jordan J Hist Archaeol. 2020;14(4):81–96. https://doi.org/10.35516/0437-014-004-004.

    Article  Google Scholar 

  43. Wu, C. VisualSFM: a visual structure from motion system. http://ccwu.me/vsfm/.

  44. Rothermel M, Wenzel K. SURE—photogrammetric surface reconstruction from imagery. Proc LC3D Work. 2012;2016:1–21.

    Google Scholar 

  45. Banfi F, Previtali M, Stanga C, Brumana R. A layered-web interface based on HBIM and 360° panoramas for historical, material and geometric analysis. ISPRS Ann Photogramm Remote Sens Spat Inf Sci. 2019;42(2/W9):73–80.

    Article  Google Scholar 

  46. Fritsch M, Clemen C, Kaden R. 3D landscape objects for building information models (BIM). ISPRS Ann Photogramm Remote Sens Spat Inf Sci. 2019;4(48):67–74.

    Article  Google Scholar 

Download references

Acknowledgements

The authors express their gratitude to the Royal Commission for AlUla for their support and assistance with this research.

Funding

Not applicable.

Author information

Authors and Affiliations

Authors

Contributions

Conceptualization, YA and AB; methodology, YA and AB; software, YA; formal analysis, YA and AB; investigation, YA and AB; data curation, YA; writing—original draft preparation, YA; All authors reviewed the manuscript.

Corresponding author

Correspondence to Yahya Alshawabkeh.

Ethics declarations

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Alshawabkeh, Y., Baik, A. Integration of photogrammetry and laser scanning for enhancing scan-to-HBIM modeling of Al Ula heritage site. Herit Sci 11, 147 (2023). https://doi.org/10.1186/s40494-023-00997-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s40494-023-00997-2

Keywords