 Research
 Open access
 Published:
A study on the detection of bulging disease in ancient city walls based on fitted initial outer planes from 3D point cloud data
Heritage Science volume 11, Article number: 10 (2023)
Abstract
Bulging is a common disease suffered by ancient city walls, especially clay brick walls. Bulging affects or even threatens the stability of outer structures of ancient city walls. Therefore, accurate identification of bulges is one of the urgent issues to be addressed in ancient city wall conservation. In this study, a novel method is proposed for detecting and identifying bulge information using point cloud data collected from the outer surface of an ancient city wall. The nonbulging points are identified in the sliced point cloud of a city wall, and then the initial plane of the city wall is fitted more accurately with the aid of spatial constraints. Finally, the bulging damage is identified and extracted by comparing the initial plane with the original point cloud. This method was applied to the project of ancient city wall conservation and was compared with other existing methods in effectiveness and accuracy. The results show that the proposed method can detect efficiently budging disease in ancient city walls for architectural heritage conservation.
Introduction
Ancient city walls are architectural heritage sites that possess great historical value and reflect early urban planning and architectural techniques. One of the most wellpreserved types of walls is the outer clad brickwork, which comprises rammed earth inside and bricks outside [1]. Bulging refers to a type of external deformation damage of ancient city walls in which protruding bricks on the surface of the city wall form a cavity [2], and the main manifestation of this damage is the outward displacement of the bricks on the outer surface and damage to the brick layer structure. In severe cases, the collapse of the outer clad brickwork can occur, resulting in irreversible wall damage. It is important to obtain the extent and degree of bulging in a timely manner when considering preventive conservation and repair of ancient city walls.
Currently, existing studies have focused on the identification of bulging diseases and the monitoring of changes. According to the difference in the dielectric constant between the bulging cavity and the wall, ground penetrating radar can detect the existence of bulging in city walls from the reflected wave image [3]. Changes in the bulging can be obtained from the spatial coordinates of the observation target on the outer surface of the wall as monitored by a total station [4, 5], and they can also be obtained by sensors embedded in the wall, such as optical fibres [6]. However, the installation of equipment inside the wall is required in these existing methods, e.g., [6], which could cause unnecessary damage to the original city wall. or the coordinate data cannot cover the entire outer surface of city wall with sufficient density in some methods, e.g., [4, 5],therefore, it is difficult to simultaneously obtain the accurate degree and range of bulging.
Terrestrial laser scanning (TLS) is a measurement technique that actively acquires highprecision threedimensional (3D) geometric information on the surface of an object [7, 8]. Over the last few years, point cloud data obtained from TLS have been widely used in damage detection and deformation analysis of various types of buildings [9,10,11]. Processing point cloud data by slicing the point cloud to obtain a profile point cloud and then performing feature extraction analysis has become a common approach. For example, in tunnel boring projects, after obtaining the tunnel profile (tunnel contour lines) based on the sliced point cloud data, the tunnel deformation can be analyzed by counting the deviations in the measurement points [12,13,14]. To fit sliced point cloud data, Elberink used Markov chain Monte Carlo sampling and the Fourier interpolation method to obtain continuous smooth railway track curves [15], and Xu used the Bspline fitting method to extract the outlines of arched components [16].
In this paper, 3D TLS is adopted for the detection of bulging disease in ancient city walls. The main strategy is to acquire point cloud data through scanning the outer surface of ancient city walls and then compare them with the initial plane of ancient city walls to determine the bulging range and degree. Theoretically, the initial plane of an ancient city wall can be calculated in a very straight way if it is perpendicular to the ground surface. However, most surfaces of ancient city walls are not perpendicular to the ground surface and form a convergence angle with the plumb plane. Therefore, it is the key issue how to determine the initial plane of the city wall surface before the wall became bulging.
Currently, there are two methods that are employed to solve this issue. One method is to determine the inclination angle of the initial plane according to historical records. For example, Cao determined the height and the length of a wall according to a description of the Forbidden City wall in the Code of Great Ming Dynasty. This historic text states that "the height of the wall is three zhangs"(“zhang” is a length measure unit in ancient China, 1 zhang = 3.33 m) and "the top shrinks inward by one chi, two cuns and five fens"(1 chi = 10 cuns = 100 fens = 0.33 m) [17]. These values were then used to calculate the inclination angle. Because the units used in historical records are not sufficiently accurate, the accuracy in the first measurement method is limited. Particularly, for the city wall with long history, the overall settlement, inclination, and other deformations make it difficult in determining the initial position of wall surface from historical records.
The other method is to measure and fit the spatial point coordinate data of the wall surface to obtain the initial plane. For example, 3D point cloud data are acquired by scanning the outer surface of an ancient city wall, and the feature plane method [18] or the random sampling consensus (RANSAC) algorithm [19] is then directly applied to fit the point cloud to obtain the initial plane. In this method, point cloud data are directly used to determine the initial plane without considering the different features of the point clouds in both the bulging area and the nonbulging, as a result, the initial plane calculated for the wall would be in poor agreement with the true initial plane. In addition, RANSAC method generally has an unsatisfactory fitting effect when the noise point exceeds 60% [20]. Therefore, the accuracy of the existing methods for extracting bulging disease information of city walls cannot meet the requirements of extracting bulging disease information for city wall conservation.
In this study, a new method is proposed to detect bulging disease in ancient city walls based on point clouds obtained from 3D TLS. In the new method, a 3D comparison between the fitting plane and the point cloud of the city wall is used to detect bulging areas. The key point is to determine the initial fitting plane of the city wall as a benchmark. An ideal solution is to use points in flat areas (nonbulging areas) only to fit the initial plane in order to reduce the effect from points in nonflat areas (bulging areas) such as bulging areas. This impacts the performance but also causes errors due to the noise points in nonflat areas.
In order to overcome the difficulties in the existing methods, the proposed method adopts a twolevels strategy to determine the plane from the huge size of 3D point clouds. At the first level, we cut the point cloud slices and resample the points to achieve rough noise elimination. At the next level, spatial constraints like the missing, spalling, and bulging points are introduced to further remove the noise and refine the final plane. The result of the initial plane will be derived only from point cloud data in nonbulging areas and avoid the effect of the noise data in possible building areas. Therefore, the fitted initial plane and extracted bulging areas will be more accurate.
Methodology
As described above, the existing second method does not consider the features of different area and utilizes all point cloud data in the process of determining the initial plane. It causes poor accuracy of initial plane, but also needs extra workload to meet the requirements of preprocessing including the removal of redundant point clouds, such as parapet walls, drainage outlets and ground.
To reduce or eliminate the effect of building areas (noise data) on the initial plane and improve the performance, one strategy of point cloud slicing is firstly introduced in this study. Slicing the point cloud of the city wall can sample the points at the specified height of the city wall, and fitting these points can obtain ordered discrete points representing the bulging change of the city wall at the corresponding height. Then, the discrete points in the nonbulging region are identified based on the curved spatial geometric features in the sliced data. Thirdly, plane fitting was performed on the points of the nonbulging parts to obtain the initial plane of the city wall surface, the wall bulging disease information is extracted by comparing the initial plane with the original point cloud of the wall, and an accuracy evaluation is performed. Finally, the disease information is visualized. The main workflow is shown in Fig. 1.
In general, the method includes four main parts: point cloud slicing and fitting, feature points identification, fitting of the city wall initial plane and bulging disease information extraction, and accuracy evaluation.
Point cloud data slicing
Usually, the coordinate system of the 3D point cloud data collected in the field is the instrument coordinate system, which cannot be directly input into the algorithm. For the convenience, an independent coordinate system is necessary to be established according to the structural characteristics of city walls, illustrated in Fig. 2, where the Xaxis is the intersection line between the base of the wall and the ground as determined through both endpoints of the wall using the right direction (facing the wall) as the positive direction. the Zaxis trends upwards vertically, the Yaxis is perpendicular to the XOZ plane and points towards the wall. The origin of the coordinates is selected at any point on the Xaxis. The 3D point cloud data are converted into an independent coordinate system to facilitate point cloud data processing and algorithm description.
When a plane passes through the point cloud data, all points in a certain neighbourhood of the plane location are extracted and projected to the plane, these projection points are called the slice of point cloud data, and the width of neighbourhood is called the slice width. The slice of the point cloud data can be viewed as the point set located on a certain plane in the point cloud data. Assuming that several planes with equal intervals that are parallel to the horizontal plane pass through the wall point cloud data, several corresponding slices are obtained (Fig. 2). Let the slice width be w, the slice interval be d_{1}, and the upper and lower limits of the slice height be h_{max} and h_{min}, respectively. Then, a total of L sections can be obtained. The Zcoordinates range of the Jth section containing discrete points is expressed as follows:
The 2D discrete points of the Jth section are resampled at an interval of d_{2} to obtain N_{J} discrete points, and the discrete points are sorted in descending order according to the Xaxis values. Each section is resampled and sorted, and a point set Q is formed as follows:
where K represents the Kth discrete point in the Jth section. The schematic diagram of point cloud slice is shown in Fig. 2.
Piecewise fitting of slice contours
If there is no bulging on the outer surface of the city wall, then the discrete points in the successively connected slices can be approximated as a straight line. otherwise, they can be represented as a curve, and the curved section can be considered a representation of the bulging area on the slice [20]. Therefore, the key to bulge extraction is to identify the curved part of the line that connects the discrete points. Due to the presence of weathering or damage to the exterior clad bricks of ancient city walls (Fig. 3), the discrete points in the connected slices can also have many small local depressions. Therefore, to obtain the part of the curve that represents the bulge in the sliced data, it is necessary to fit the discrete points in the slice to a continuous smooth curve and, at the same time, remove the small local depressions as noise points. The actual masonry spalling and missing of the wall is shown in Fig. 3.
For curve fitting of the slice discrete points, only an appropriate algorithm can fit the curve to reflect the curved characteristics. otherwise, too much detail could be retained or necessary detail could be removed [21]. Here, a piecewise fitting strategy is adopted. The discrete points are first segmented, and then a polynomial is combined with the cubic Hermite interpolation function for fitting. At the same time, the improved Kmeans clustering algorithm is used to eliminate the noise points to ensure that the curved characteristics of the curve can be expressed, and reliable fitting accuracy can be obtained.
The discrete points of the section J in the point set Q form a point set Q_{1}, which is expressed as follows:
The discrete curvature of each point in the point set is calculated, and Kmeans clustering is performed on the discrete points according to the curvature value. According to the clustering results, the noise points are filtered through a neighbourhood search. The specific method is as follows.
Calculation of the discrete curvature of a point set
The three consecutive points in the point set Q_{1} are q_{JK1}, q_{JK}, q_{JK+1}, and t_{1}, t_{2} represent the lengths of the vectors between q_{JK} and two nearby points, as shown in Fig. 4.
The curve parameter equation composed of three points is obtained as follows:
The length of two segments is:
If the following conditions are met, then the expression becomes:
Then a_{1}, a_{2}, a_{3}, b_{1}, b_{2}, b_{3} can be solved according to the following equation:
Discrete curvature \(\kappa \) at point q_{JK} is expressed [22] as follows:
Noise filtering
Noise filtering mainly removes the small local depressions in the slice caused by damage to the city wall. Considering that these small depressions have a small range and large curvature value at the bottommost point of the depression, this area can be searched by setting the curvature threshold of the discrete points, and at the same time, the width and depth range of the small depressions can be limited to find other points inside the depressions, thus facilitating their identification. Because the curvature of the discrete points at the bottom of these small depressions is similar, it can be obtained by the Kmeans clustering method. Kmeans clustering is a classic unsupervised clustering algorithm, and its main idea is to group similar samples together to form clusters according to the distance between samples [23]. This approach is performed on the point set according to the discrete curvature, and the cluster with the largest discrete curvature is obtained, that is, the set of the bottom points of all small depressions. Then, the two thresholds, the depth and width, of the small depression are set and defined by the difference S of the Yaxis between adjacent points and the range coefficient \(R^{\prime}\) to determine the adjacent area. The specific noise filtering algorithm is obtained as follows:

(1)
The discrete points contained in the cluster with the largest discrete curvature are extracted as seed points to form a seed point set O.

(2)
Centerd at the seed point, within the range \(R^{\prime}\), a search is performed on both sides of the Xaxis. If the absolute value of the difference between the Ycoordinate of the seed point and the neighbouring point on one side is less than S, the neighbouring point is considered a noise point, and the search continues until the edge point of one side (at distance \(R^{\prime}\)) is determined. If the absolute value of the difference between the Ycoordinate of the seed point and the neighbouring point on one side is greater than S, the iteration is stopped on that side, as shown in Fig. 5. The search on both sides is complete.

(3)
The seed point set O and obtained noise points are removed, and the remaining points forms the point set Q_{2}, which is expressed as follows
$$ Q_{2} = \left\{ {q_{JF} \left {q_{JF} = \left( {x_{JF} ,y_{JF} } \right),\;\;F = 1,2,3, \ldots N_{J} } \right.} \right\}. $$(9)
Piecewise curve fitting of point sets
To accurately represent the curved characteristics of the slice contours, it is necessary to adopt a reasonable fitting strategy and function model [24]. Piecewise fitting is a method that divides a segment of complex discrete points into several segments of simple discrete points for curve fitting. this strategy has high fitting accuracy but maintains the curved characteristics of the curve. In this study, a cubic polynomial function is used to achieve piecewise fitting of the sliced data after the noise is removed, but the fitting curve of each segment is not continuous at the breakpoints. For a segment of discrete points, the endpoint condition of the cubic Hermite interpolation can satisfy the firstorder continuity of the interpolation function at the endpoints [25, 26]. In this study, cubic Hermite interpolation is used to achieve connection between the polynomial fitting curves.
Let the function expression of the cubic polynomial be expressed as follows:
In Eq. (10), b is the number of curve fittings, and a_{0}, a_{1}, a_{2}, … a_{b} are the undetermined regression coefficients of polynomial fitting.
Let the function expression of cubic Hermite interpolation be expressed as follows:
In Eq. (11),\(\alpha_{1} ,\alpha_{2} ,\alpha_{3} ,\alpha_{4}\) are the undetermined regression coefficients of the interpolation function. If the coordinates of two endpoints are (x_{L},y_{L}), (x_{R},y_{R}) in the interpolation function, then the expression of constraint (12) exists and is unique [27]:
Piecewise fitting is performed on the discrete points in point set Q_{2}, and the fitting method is as follows:

(1)
The number of segments u for piecewise fitting of point set Q_{2} is determined, the points are evenly distributed to each segment, and the number of discrete points in each segment is recorded as n_{1},n_{2},n_{3},…,n_{u}.

(2)
Cubic polynomial fitting is performed on the first segment of discrete points (x_{i},y_{i}), i = 1,2,3,…n_{1}, and the obtained polynomial equation is f_{1}(x). Discretization of f_{1}(x) at x = x_{i}, i = 1,2,3,…n_{1} obtains discrete points (\({\text{x}}_{{\text{l}}_{1}}\), \({y}_{{\text{l}}_{1}}\)), l_{1} = 1,2,3,…n_{1}.

(3)
For (\({\text{x}}_{{\text{l}}_{1}}\), \({y}_{{\text{l}}_{1}}\)), l_{1} = n_{1} − 4, n_{1} − 3, n_{1}2,…n_{1}, weighting on the point (\({\text{x}}_{{\text{n}}_{1}\text{4}}\), \({\text{y}}_{{\text{n}}_{1}\text{4}}\)) is performed, and the number of times the point is weighted is generally 0.2n_{1}. Both (\({\text{x}}_{{\text{l}}_{1}}\), \({y}_{{\text{l}}_{1}}\)), l_{1} = n_{1} − 4, n_{1} − 3, n_{1} − 2,…n_{1} and (x_{i}, y_{i}), i = n_{1}, n_{1} + 1, n_{1} + 2,…2n are fitted with cubic polynomials, and the obtained polynomial equation is denoted as f_{2}(x). Discretization f_{2}(x) at x = x_{i}, x = n_{1} − 4, n_{1} − 3, n_{1} − 2,…2n_{1} obtains discrete points (\({\text{x}}_{{\text{l}}_{2}}\), \({y}_{{\text{l}}_{2}}\)), l_{2} = 1, 2, 3,…n_{2}.

(4)
For (\({\text{x}}_{{\text{l}}_{1}}\), \({y}_{{\text{l}}_{1}}\)), l_{1} = 1, 2, 3,…n_{1} and (\({\text{x}}_{{\text{l}}_{2}}\), \({y}_{{\text{l}}_{2}}\)), l_{2} = 1, 2, 3,…n_{2}, (\({\text{x}}_{{\text{l}}_{1}}\), \({y}_{{\text{l}}_{1}}\)), l_{1} = n_{1} − 4, n_{1} − 3, n_{1} − 2,…n_{1} and (\({\text{x}}_{{\text{l}}_{2}}\), \({y}_{{\text{l}}_{2}}\)), l_{2} = 1, 2, 3,…5 represent overlapping sections of two discrete data segments in the Xdirection. This overlapping part is represented by an interpolation curve that employs (\({\text{x}}_{{\text{n}}_{1}\text{4}}\), \({\text{y}}_{{\text{n}}_{1}\text{4}}\)) and (\({x}_{5}\), \({y}_{5}\)) as the two endpoints of the interpolation curve, and the equation of the interpolation curve is obtained through the constraint in Eq. (12).

(5)
Steps (3) to (4) are repeated to obtain a complete, continuous and derivable curve for the discrete points of one section.
At the Xcoordinate of each discrete point in point set Q_{2}, the fitted curve is discretized, and the corrected coordinates obtained from the discretization form the overall point set P, which is expressed as follows:
Extraction of nonbulging points
To extract the nonbulging points more accurately from the sliced data, it is necessary to consider the spatial geometric characteristics of either the bulging area or the nonbulging area on the slice contour line. If there is no bulging of the city wall, the slice contour line may be a straight line, and the points in the sliced data can be directly extracted as nonbulging points. When there is only one bulging point, there may be a local bending and three inflection points (the two ends and the apex of the curve) on the slice contour line. If there are multiple bulging points in the section, there may be multiple bendings in the slice contour line, and the corresponding inflection points may also increase. For simplicity, the discrete points at the inflection points (both ends of the bending) and in a certain neighbourhood range can be treated as nonbulging points when the slope of the line formed between adjacent discrete points is small, as shown in Fig. 6. Therefore, the identification of nonbulging points can be achieved by identifying the inflection points and its type in the sliced data and determining the bending of the contour lines at the inflection points within a certain range on both sides. The identification of the type of inflection point is mainly based on the direction of the Ycomponent of the tangent vectors of the discrete points on the left and right sides of the inflection point. For an inflection point at one endpoint of the bending, the direction of the Ycomponent of the tangent vector of the discrete point on the left side is towards the wall, and the opposite direction corresponds to the discrete point on the right side. After the inflection points at the two ends of the bending are identified, the nonbulging points is determined by the slope of the connecting line between the two adjacent points. This study sets the threshold of the line slope between two adjacent points as \(\delta\) and the specific algorithm for identifying nonbulging points is obtained as follows, the form of identifying nonbulging areas is shown in Fig. 6.
Inflection point identification
For the point p_{JF}(x_{JF},y_{JF}) in point set P, the slope of tangent at x = x_{JF} is \({\varepsilon}\) _{JF}. then, the Ycomponent of the unit tangent vector at this point is:
If point p_{JF} satisfies condition (15), then point p_{JF} can be considered the inflection point at the endpoint of a bending. This relationship is expressed as follows:
The slope between point p_{JF}(x_{JF}, y_{JF}) and point p_{JF1}(x_{JF−1}, y_{JF−1}) is expressed as follows:
In Eq. (15), T(x_{JF−1}) represents the Ycomponent of the tangent vector at the adjacent discrete point on the left side of point p_{JF}, and T(x_{JF+1}) represents the Ycomponent of the tangent vector at the adjacent discrete point on the right side of point p_{JF}. The positive direction moves toward the wall, and the negative direction moves in the opposite direction.
Although some inflection points after the above treatment are closer to the initial plane than neighborhood points, but they are at the intersection of two bulging areas and still on the bulging area. In order to reduce the influence of these points on the fitting result, when the absolute value of the slope of an inflection point and other inflection points is greater than threshold \({\delta}\), this point is no longer regarded as an inflection point.
Extraction of nonbulging points
Using the retained inflection points as the center, calculates the slope of the retained inflection points and discrete points on both sides. If the slope is less than \({\delta}\), the discrete point is retained, and the search continues for the next discrete point on the same side. If the slope is greater than \({\delta}\), the calculation on this side is complete. If two points \({\text{p}}_{{\text{JF}}_{1}}\), \({\text{p}}_{{\text{JF}}_{2}}\)(\({\text{x}}_{{\text{JF}}_{1}}\text{<}{\text{x}}_{{\text{JF}}_{2}}\)) on both sides of the inflection point p_{JF} satisfy Eq. (17) as follows:
Then the discrete points in the interval [\({\text{x}}_{{\text{JF}}_{1}},{\text{x}}_{{\text{JF}}_{2}}\)] are denoted as nonbulging points. Each inflection points and the discrete points in its neighbourhood are identified to complete the identification of the nonbulging points in the sliced data.
Steps 1–2 are repeated to extract the nonbulging points in each section. All nonbulging points form a 3D point set P*(expression (18)), and the Zcoordinates correspond to the Zcoordinates of the original point cloud associated with each point.
Plane fitting and bulging extraction
The robust eigenvalue method is used to fit the plane comprising the nonbulging points and is based on the eigenvalue method. The calculated unit normal vector of the fitting plane and the distance from the origin to the plane are used as parameters, all the points are divided into inner points and outliers, and the final fitted spatial plane equation is obtained by iteration [28]. This method is widely used to eliminate noise points during the fitting process [29,30,31], can reduce the extraction error of nonbulging points, and can remove the few discrete points that lie on the bulging area but are arranged in parallel with the Xaxis.
The specific calculation method for fitting initial planes is described as follows:

(1)
Plane fitting is performed for all points in point set P* through the eigenvalue method, and the obtained equation that approximates the city wall initial plane is Ax + By + Cz + D = 0. The orthogonal distance from each point along the normal line to the plane is expressed as follows:
$$ H_{j} = \frac{{\left {Ax_{j} + By_{j} + Cz_{j} + D} \right}}{{\sqrt {A^{2} + B^{2} + C^{2} } }}\left( {j = 1,2,3 \ldots ns} \right). $$(19) 
(2)
The calculation of the standard deviation (SD) of the orthogonal distance H_{j} is obtained as follows:
$$ \sigma = \sqrt {\frac{{\left( {H_{j}  \overline{H}} \right)^{T} \left( {H_{j}  \overline{H}} \right)}}{ns  1}}, $$(20)where:
$$ \overline{H} = \frac{1}{ns}\sum\limits_{j = 1}^{ns} {H_{j} }. $$(21) 
(3)
When the H_{j} of a certain point is less than \(2\sigma\), the point is retained. otherwise, the point is considered an abnormal point and eliminated.

(4)
All of the retained points are fitted using the eigenvalue method, the orthogonal distance from each point to the fitting plane is calculated, and Steps 1 to 2 are repeated. If the orthogonal distances from all retained points to the plane are within the threshold, iteration ends, and the final fitting plane equation is obtained.
After the initial plane is obtained, the proceeded operation is to detect possible bulging areas. As described in "Piecewise fitting of slice contours", contour lines can be obtained within the slices at different heights of the city wall. However, the contour lines can only represent the horizontal bulge variation at the corresponding height of the city wall. Moreover, there are slice intervals between the contour lines, the “bulges” obtained only by the contour lines is discrete and cannot accurately reflect the degree and extent of the bulge in the vertical direction. Moreover, unlike initial plane, bulging surfaces vary from one to another without predefined wellknown mathematical forms to express them; therefore, it is not able or difficult to use the same fitting method to determine all bulging areas only from the sampled discrete bulging points. Therefore, in order to obtain accurately a complete bulge situation, the initial plane of the wall is taken as the reference and then the comparative analysis is performed between the initial plane and the whole point cloud of the wall. During the process of comparison, those points which are above the initial plane will be marked as bulging. Simultaneously, the offset distance between each original wall point and the initial plane is calculated. The offset distance is defined as the orthogonal distance from each point of the wall to the initial plane along the normal line. The offset distance is also recorded as the relative height of each wall point to the initial plane.
Error evaluation
The accuracy of the bulging extraction method was evaluated from two aspects: the accuracy of curve fitting and the angle and distance deviation between fitting planes.
The accuracy of curve fitting is usually evaluated using posterior variance, small error probability, and the mean absolute percentage error (MAPE) [32, 33]. The accuracy analysis method for the curve fitting used in this paper is as follows:
Point set P is obtained by curve fitting and discretization as follows:
Point set‾P is obtained by correlating each point in set P to the point before fitting as follows:
According to the residual error \(e_{JF} = y_{JF}  \overline{y}_{JF}\) at any point in point set P, the SD of the fitted original sequence is as follows:
The SD of the absolute error is expressed as follows:
where
Then, the posterior error ratio is as follows:
The small error probability is expressed by:
The MAPE is calculated as follows:
Based on the above calculation results, the piecewise curve fitting grade and the fitting accuracy can be determined.
The algorithm robustness is verified by calculating the inclination of the fitting plane, which is obtained by processing the point clouds of each city wall segment in the same global coordinate system. The calculation of the plane inclination is shown in Fig. 7.
In the figure, \(\vec{A}\left( {x,y,z} \right)\) is the unit normal vector of the fitting plane, and \(\vec{B}\left( {0,0,1} \right)\) is the unit normal vector in the Zdirection. Then, the inclination angle \(\alpha\) formed by the fitting plane and the XOZ plane is obtained as follows:
In the same way, the inclination angle formed by the fitting plane and the YOZ plane can be obtained.
Case study and results
The Forbidden City is located in Beijing and was built in 1420. This ancient city wall type is outer clad brickwork, its height is 9.27 m, its bottom width is 8.55 m, and its top width is 6.63 m. Although the Forbidden City wall is one of the most intact ancient city walls, it has experienced bulging disease. In particular, the inner part of western section of the wall has suffered more serious bulging damage, and temporary reinforcement measures have been taken in some areas, as shown in Fig. 8.
To extract information about the bulging disease experienced by this section of the city wall, point cloud data were collected from the outer surface over a length of approximately 190 m using a 3D laser scanner. Data were acquired using a FARO Focus3D X130 scanner with a measurement rate of 976,000 points/SEC and ranging accuracy of ± 2 mm at 25 m. During the scanning process, a total of 13 stations were scanned, each station is evenly distributed in the section of the experimental city wall, about 15 m away from the city wall, and the resolution of the obtained point cloud is 5 mm. The point cloud is acquired by frontal scanning. The material of each wall brick is the same. The wall bricks are laid by mortar, but the mortar is not exposed to the outer surface of the wall, so there is no difference in reflectivity caused by the material of the wall.
We use Geomagic Studio software for point cloud registration. Considering that the city wall scaffolds and the large amount of data could lead to a decrease in the calculation speed, the city wall was divided into three segments of 29 m, 54 m and 59 m in length. The preprocessed point cloud is shown in Fig. 9.
City wall point cloud slicing
Using the third segment of the city wall as an example, the point cloud of the preprocessed city wall was sliced. The lower and upper limits of the slice height (h_{min} and h_{max}) are 0.2 m and 8.6 m, respectively, the slice width w is 5 mm, the slice interval d_{1} is 30 cm, and a total of 29 sections were obtained, as shown in Fig. 10a. The slices of each section were sampled with horizontal interval d_{2} of 50 cm, the discrete points in each section were sorted from small to large according to the Xaxis value, and the results after sampling are shown in Fig. 10b.
Fitting contour lines
Using the slice at a height of 5 m (the 17th section) as an example, the Zcoordinate change of the discrete points in the same section is within 1 mm, as shown in Fig. 11a. The discrete points contained in each section were clustered according to the discrete curvature, and the number of clustering centers was set to 4. The clusters with the largest curvature were removed from the point set, and the filtered results are shown in Fig. 11b.
According to the length of the city wall and the sampling interval, the slice was evenly segmented according to the number of discrete points. In this study, the slice was divided into nine segments. Among them, the first to the eigth segments each of the fitted curve, each contain 11 discrete points, and the remaining segment contained 10 discrete points. The results after piecewise fitting are shown in Fig. 12, i.e., a firstorder continuous and derivable smooth curve.
Identification of nonbulging points
The fitted curve was discretized again at the Xcoordinate of each discrete point to obtain a new set of points. After the slope of the tangent line and the Ycomponent of the tangent vector of each point in the new point set were calculated, the identification of bulging points and extraction of nonbulging points were performed. The direction of the Ycomponent of the tangent vector of each point is shown in Fig. 13.
The slope threshold \({\delta}\) was set to 0.004 and the extracted nonbulging points are shown in Fig. 14a. The identified nonbulging points in both the 7th section (at 2 m) and the 27th section (at 8 m) are shown in Fig. 14b, c.
Extraction and visualization of wall bulging points
The nonbulging points of each section were extracted as shown in Fig. 15a. All these points were fitted using the robust eigenvalue method, and the results of robust plane fitting were obtained through iterative operations and are shown in Fig. 15b.
The city wall initial point cloud was compared with the fitting plane, and the bulging points of this segment of the city wall are shown in Fig. 16c. Figure 16a, b, respectively show the bulging of first and second segments. The bulging can be clearly presented in the colour map after visualization. For the city wall studied in this paper, there are four bulges with a horizontal length greater than 10 m, the bulging height usually reaches 15–20 cm, and for the most severe area, the bulging height can reach 30 cm.
Slicing Parameter selection and its effect on results
It is a crucial step to select the appropriate parameters for the performance of the proposed method. The main parameters interval point cloud slicing, piecewise fitting and nonbulging point extraction. The requirements for determining the parameters and the correlations between parameters are discussed in this section.
Parameters of point cloud slicing
The main parameters are listed in the Table 1.
The selection of h_{max}, h_{min} should avoid the influence of the redundant point cloud data from nonwall surfaces such as the drainage outlet and the parapet wall. therefore, h_{max} should be large enough to avoid the interference of the ground point clouds, and h_{min} should be a certain distance away from the parapet wall.
The vertical interval d_{1} and the horizontal sampling interval d_{2} directly determine the number of discrete points after sampling the wall point cloud. If d_{1} and d_{2} are too small, a large amount of redundant data will degrade the performance of the algorithm. For example, when two slices are on the same layer of brick, the contours obtained by each slice are almost the same, so the extracted nonbulging points are repeated and too dense. At the same time, too large values of d_{1} and d_{2} will cause distortion of the bulging characteristics of the sampling results. In terms of ShannonNyquist sampling theorem [34, 35], the sampling frequency should be greater than twice the highest frequency in the signal. Therefore, the upper limit of d_{1} and d_{2} depends on the size of the bulging area in the actual experimental wall segment. According to the preliminary survey, the smallest bulging area has a horizontal length of about 3 m and a vertical length of about 1 m. Figure 17 shows the two smallest bulging areas in the experimental wall section. Therefore, in this example, the value of d_{1} should not be greater than 0.5 m, and the value of d_{2} should not be greater than 1.5 m. Considering that the cladding bricks outside the wall are laid in the pattern of gardon wall bond and the length is about 0.45 m, the width is half of the length, and the thickness is about 0.1 m. In order to avoid multiple sampling on the same brick, the proper value range of d_{1} is from 0.1 m to 0.5 m, and the value range of d_{2} is from 0.225 m to 1.5 m. The actual situation of two bulging is shown in Fig. 17.
To select a proper slice width w, the density of the actual point cloud data should be considered. Simultaneously, it should also ensure that the sliced point clouds contain sufficient discrete points with even distribution. In this study, the resolution of the point cloud was 5 mm, for the reason of computing speed, w is the most appropriate point cloud resolution, so w is set to 5 mm.
Piecewise fitting parameters
The main parameters for piecewise fitting are listed in Table 2.
The number of cluster centers V could interfere to some extent with the filtering of points at the damage area of brick walls. When V is smaller, the number of discrete points in each cluster could increase, which may lead to severe local data missing. When V is larger, for the cluster with the greatest discrete curvature, the included discrete points on obvious damage area of walls might be incomplete. Besides a reasonable setting for V, the integrity of the identified original seed points after clustering should be maintained. In this study, when V is set to 4 or 5, the clustering result is better. When V is set to 3 or 6, the result is shown in Fig. 18.
The adjacent area range coefficient \(R^{\prime}\) and the Ycoordinate difference coefficient S can be used to identify one or multiple adjacent defect points on the basis of the cluster identification seed points, and the two need to be determined synchronously according to the actual brick damage length and depth of the city wall. \(R^{\prime}\) should ensure that a certain seed point is centered, usually between 1 and 3 times the resampling interval, so as to avoid false identification of defect points using Kmeans clustering when there is no obvious defect point in a slice of a layer. S is an empirical value, usually considering the depth of the brick defect, it is more appropriate to take 1 cm to 2 cm.
To determine u, the number of discrete points in each slice needs to be considered. Using the method in "Piecewise fitting of slice contours", there are at least five discrete points in each segment after segmentation. When there are too many discrete points in each segment, loworder polynomial fitting may not be able to obtain the proper bulging curve of the encasement brick. The cubic curve contains two inflection points, and in this study, the smallest bulging area has a horizontal length of about 3 m. Combined with the actual situation, the length of segments are set about 4.5 m or slightly larger, which is enough to ensure the accuracy and curve characteristics.
Parameters for nonbulging point extraction
Table 3 shows the required parameter for nonbulging point extraction.
Threshold \({\delta}\) plays a key role in the process of identifying nonbulging points. When \({\delta}\) is too large, the identified nonbulging points are inaccurate, and \({\delta}\) is too small, it could be difficult to completely identify the nonbulging points. \({\delta}\) is positively correlated with the bulge height of city wall. Generally, when the maximum bulge height of the wall is about 30 cm, \({\delta}\) can be set to 0.003, which can be enlarged or reduced in the same proportion with the maximum bulge height.
Accuracy evaluation and analysis
Precision evaluation
The accuracy of the curve fitting was verified. In total, there are 2962 discrete points used to fit each curve, and 29 fitted curves are obtained. The model accuracy evaluation criteria for curve fitting are shown in Table 4, and the classification criteria for fitting accuracy are shown in Table 5.
The calculated posterior error ratio C was 0.1886, the small error probability p was 0.9956, and the MAPE was 0.6876. In summary, the piecewise curve fitting method used to fit the contour line of the city wall has high accuracy.
Bulging extraction was performed on the three segments of the city wall, and the final fitting planes were obtained in the same global coordinate system, as shown in Fig. 19.
The inclinations of the three planes with both the XOY and YOZ planes are shown in Table 6 (unit: degree).
Comparison with existing methods
The existing two methods mentioned above were compared with the proposed method in this paper. The RANSAC algorithm [36, 37] was directly used for plane fitting on the point cloud data of the city wall, and the corresponding bulging disease information was extracted. In addition, the initial inclination angle of this segment of the wall was determined based on the historical recorded receding angle, and the initial plane of the wall was obtained and compared with the original point cloud to extract the bulging disease information. Taking the first segment of the city wall as an example, the visualization results of the three methods for extracting bulging diseases are shown in Fig. 20.
In order to verify the proposed method and compare the accuracy of extracted bulging areas in different methods, the point clouds of repaired wall are used as the actual city wall plane to identify the bulging areas in the city wall before repaired.
The overall bulging area and maximum bulging value of the wall obtained by using the repaired wall as actual bulging areas and three different methods are shown in Table 7. The actual photo and point cloud of the repaired wall are shown in Fig. 21.
Figure 22 illustrates the results of comparing bulging areas extracted with different methods.
Although the repaired wall plane cannot be absolutely considered as the initial plane, the repairment does not change the shape of the wall. Therefore, the repaired wall plane is highly close to the initial plane. As shown in Table 7, our method is far more accurate than RANSAC method and historical record method. Compared with RANSAC method, the accuracy in the maximum bulging value and bulging area is improved by 23% and 49% respectively; while compared with historical record method, the accuracy is improved by 32% and 68%, respectively.
At present, many studies have shown that bulge of city wall is mainly caused by rainwater seepage from the top of the city wall and freezing–thawing [2, 5, 19, 38, 39], therefore, the bulges generally develop downward from the top of the city wall, and the bulge development trend is obvious. According to the above visualization results, the bulges extracted using the two existing methods originates from the middle part of the city wall and extends around, which is different from the actual bulging pattern. However, the bulging areas obtained by the method in this paper are mainly concentrated on the top of the city wall, and all bulges spreads downwards from the top. This also indicates the new method can reflect the actual situation more accurate than the other two methods.
At the same time, a small number of points located in spalling or missing bricks show negative bulging values in the visualization. However, the visualization results of the existing methods all contains a large area of negative bulging values, and the maximum may even reach negative 15–20 cm, as shown in Fig. 20. This result means that there is a large area of wall bricks falling off, which does not conform to the actual preservation of the city wall. Therefore, the initial plane obtained by existing methods is inaccurate.
The quantitative comparative analysis above shows that the methods for directly obtains the initial wall plane can be largely affected by the discrete points of the bulging area, and the fitted initial wall plane is offset to the outside of the wall, which makes the extracted bulging areas and height inaccurate. Due to the poor measurement precision of the historical data, and the city wall deformations, such as settlement and inclination, there is a certain gap between the recorded receding angle of the city wall and the actual receding angle, so the repaired actual initial plane of the city wall has a large error. Overall, the bulging results extracted by the proposed method in this paper are close to the actual bulging pattern, and the extraction of the bulging value and area has been significantly and reliably enhanced.
Conclusions
This study provides a practical and feasible research idea for bulging extraction of regular planar walls. Because our method depends on the accuracy of the fitting plane, we design two levels strategy to eliminate the noise at the bulge of the city wall. At the first level, we cut the point cloud slices and resample the points to filter the rough noise. At the next level, on the basis of a method of fitting the contour line of the city wall, the spatial constraints like the missing, spalling, and bulging diseases of the city wall are introduced to filter the detailed smaller noise points. Because RANSAC method only obtains the fitting plane through onestep continuous iteration on the basis of random selection of points, it is not suitable for the case that the overall bulging range of the wall is large. Therefore, this method can't be applied to severe bulge disease detection. Compared to RANSAC method, our method can handle huge size data more efficiently and accurately, breaking through the degradation and limitation of RANSAC method that the error points of plane objects generally cannot exceed 60% of the total target. Therefore, the proposed method could be generalized to determine any geometry planes from the 3D point clouds.
In this paper, an actual city wall with bulging disease was taken as an example to verify the accuracy and reliability of the proposed method. The piecewise curve fitting satisfies the highprecision fitting requirement, avoids overfitting, and accurately reflects the bulging condition. The results obtained using the proposed method in this paper were also compared with the results obtained by direct fitting of the original point cloud data. The results showed that the proposed methods can avoid the effect from noise data and obtain more accurate and reliable results of building extraction than existing methods.
The method proposed in this study can be applied to the bulging extraction of walls with other shapes, such as brick walls, rammed earth walls, and concrete walls. However, the parameters used in the method need to be adjusted according to actual conditions. Realizing the adaptive selection of parameters could further improve the operational efficiency. At the same time, bulging at the bottom of city walls is usually rare, and the points at the bottom of a city wall can be weighted according to a certain rule in the process of initial plane fitting and obtain more accurate results.
However, it should be also noted that the proposed method in this study is focused on the ancient city wall with flat brick structure extending in a straight line. In general, most of the ancient city walls in China are buildings with regular flat shapes such as Xi 'an Ancient City Wall, Pingyao Ancient City Wall, and the ancient city wall of the Forbidden City in our case analysis [40]. The bulge of irregular walls will be taken into consideration in our further research.
Availability of data and materials
All data generated or analyzed during this study are included in this published article.
References
Liu XB, Ma X, Zhang BJ. Analytical investigations of traditional masonry mortars from ancient city walls built during ming and qing dynasties in China. Int J Architect Herit. 2015;10(5):663–73.
Yang C, Liu N. Disease analysis on the brick walls of modern and contempory architechures in Beijing, China. Cult Herit Sci Res. 2012;02:41–4.
Ercoli M, Brigante R, Radicioni F, Pauselli C, Mazzocca M, Centi G, et al. Inside the polygonal walls of Amelia (Central Italy): a multidisciplinary data integration, encompassing geodetic monitoring and geophysical prospections. J Appl Geophys. 2016;127:31–44.
Furini A, Paternò M, Pellegrinelli A, Russo P. Integrated measurement techniques for the monitoring of the ancient walls of Ferrara. Cham: Springer; 2015.
Sun B, Sun H, Song Q. Analysis of the influence of meteorological factors on the wall expansion of Ming dynasty’s rampart in Nanjing. Bull of Survey Map. 2020;09:136–9.
Qi J, Wang B, Wang X, Song Z, Cui Z. Application of opticalfiber sensing to concrete support and continuous wall strain monitoring. IOP Conf Ser Earth Environ Sci. 2019;349(1):012032.
Jia DF, Zhang WP, Wang YH, Liu YP. A new approach for cylindrical steel structure deformation monitoring by dense point clouds. Remote Sens. 2021;13(12):2263.
Wang Q, Kim MK. Applications of 3D point cloud data in the construction industry: a fifteenyear review from 2004 to 2018. Adv Eng Inform. 2019;39:306–19.
Shen YQ, Wang JQ, Puente I. A novel baselinebased method to detect local structural changes in masonry walls using dense terrestrial laser scanning point clouds. IEEE Sens J. 2020;20:6504–15.
SánchezRodríguez A, Riveiro B, Conde B, Soilán M. Detection of structural faults in piers of masonry arch bridges through automated processing of laser scanning data. Struct Control Health Monit. 2018;25(3): e2126.
Laefer DF, TruongHong L, Carr H, Singh M. Crack detection limits in unit based masonry with terrestrial laser scanning. NDT and E Int. 2014;62:66–76.
Qiu W, Cheng YJ. Highresolution DEM generation of railway tunnel surface using terrestrial laser scanning data for clearance inspection. J Comput Civ Eng. 2017;31(1):04016045.
Kang Z, Zhang L, Tuo L, Wang B, Chen J. Continuous extraction of subway tunnel cross sections based on terrestrial point clouds. Remote Sens. 2014;6(1):857–79.
Jia D, Zhang W, Liu Y. Systematic approach for tunnel deformation monitoring with terrestrial laser scanning. Remote Sens. 2021;13(17):3519.
Elberink S, Khoshelham K. Automatic extraction of railroad centerlines from mobile laser scanning data. Remote Sens. 2015;7(5):5565–83.
Xu X, Bureick J, Yang H, Neumann I. TLSbased composite structure deformation analysis validated with laser tracker. Compos Struct. 2018;202:60–5.
Cao X, Li D. A preliminary study on the status of the Forbidden City Walls and their protection. J Gugong Stud. 2013;01:342–53.
Zhang X. The application of 3D laser scanning in the safety detection of ancient buildings. Bull Survey Map. 2020;S1:155–8.
Song Q, Hu YG, Hou ML. Study on extraction and expression methods of bulging diseases in ancient city walls. Int Arch Photogram Remote Sens Spatial Informat Sci. 2019;42:77–82.
Lu J, Wang B, Zhou ZQ. Separate of multiobjects in image recognition by local feature. Acta Photon Sin. 2008;08:1708–12.
Cao Z, Chen D, Shi Y. A flexible architecture for extracting metro tunnel cross sections from terrestrial laser scanning point clouds. Remote Sens. 2019;11(3):297.
Tang P, Akinci B, Huber D. Characterization of three algorithms for detecting surface flatness defects from dense point clouds. Proc SPIE Int Soc Opt Eng. 2009;7239:72390.
Worring M, Smeulders AW. Digital curvature estimation. CVGIP Image Understand. 1993;58(3):366–82.
Arora P, Deepali VS. Analysis of Kmeans and Kmedoids algorithm for big data. Proc Comput Sci. 2016;78:507–12.
Lever J, Krzywinski M, Altman N. Points of significance: model selection and overfitting. Nat Methods. 2016;13(9):703–5.
Wang RH, Wang JX. The poisedness of interpolation problem for splines. Appl Numer Math. 2005;54:95–103.
Han X, Guo X. Cubic Hermite interpolation with minimal derivative oscillation. J Comput Appl Math. 2018;331:82–7.
Jaklič G, Žagar E. Curvature variation minimizing cubic Hermite interpolants. Appl Math Comput. 2011;218(7):3918–24.
Fernández O. Obtaining a best fitting plane through 3D georeferenced data. J Struct Geol. 2005;27(5):855–8.
Dey EK, Awrangjeb M, Stantic B. Outlier detection and robust plane fitting for building roof extraction from LiDAR data. Int J Remote Sens. 2020;41(16):6325–54.
Wang M, Tseng YH. Automatic segmentation of LiDAR data into coplanar point clusters using an octreebased splitandmerge algorithm. Photogramm Eng Remote Sens. 2010;76(4):407–20.
Chen H, Wang L, Zhao L, Chu W, Li H. Plane fitting of point clouds based on robust weighted total least squares method. Sci Survey Map. 2016;10:48–52.
Ning X, Liu S. Management forecasting and decisionmaking methods. 2009; 113–145.
Marčiukaitis M, Žutautaitė I, Martišauskas L, Jokšas B, Gecevičius G, Sfetsos A. Nonlinear regression model for wind turbine power curve. Renewable Energy. 2017;113:732–41.
Mineo C, Pierce SG, Summan R. Novel algorithms for 3D surface point cloud boundary detection and edge reconstruction. J Comput Design Eng. 2018;6(1):81–91.
Aryal A, Brooks BA, Reid ME, Bawden GW, Pawlak GR. Displacement fields from point cloud data: application of particle imaging velocimetry to landslide geodesy. J Geophys Res Earth Surface. 2012; 117(F1).
Nurunnabi A, Belton D, West G. Robust statistical approaches for local planar surface fitting in 3D laser scanning data. ISPRS J Photogramm Remote Sens. 2014;96:106–22.
Chen GQ, Li L, Li GM, Pei XJ. Failure modes classification and failure mechanism research of ancient city wall. Environ Earth Sci. 2017;76(23):1–15.
Zhou C, Li Y. Influence of rainfall infiltration on stability of Forbidden City wall. J Build Struct. 2020;41(S1):286–96.
Chang SD. Some observations on the morphology of Chinese walled cities. Ann Assoc Am Geogr. 1970;60(1):63–91.
Acknowledgements
We are grateful to the Palace Museum for sample data support. We would also like to thank Song Qing for her help in data collection.
Funding
This research was supported by the National Key Research and Development Program of China (2019YFC1520804), the National Natural Science Foundation of China (NSFC) (41930650) and the BUCEA Post Graduate Innovation Project (PG2022109).
Author information
Authors and Affiliations
Contributions
YH conceived the presented idea and put forward experimental suggestions. BF conducted and refined the analysis process and wrote the manuscript. All authors read and approved the final manuscript.
Corresponding authors
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
About this article
Cite this article
Hu, Y., Feng, B. & Hou, M. A study on the detection of bulging disease in ancient city walls based on fitted initial outer planes from 3D point cloud data. Herit Sci 11, 10 (2023). https://doi.org/10.1186/s40494022008566
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s40494022008566