A HBIM pipeline for the conservation of large-scale architectural heritage: the city Walls of Pisa

Abstract

In the architectural heritage field, a complete and in-depth knowledge of the assets is indispensable for any restoration and conservation strategy. In this context, the Historical Building Information Modelling (HBIM) technique is gaining much interest in supporting the diagnostic phase and the design and management of conservation activities. The HBIM provides opportunities to collect, organize and integrate information coming from different sources, inspections, and diagnosis techniques as well as to use standardized and effective tools for orienting cultural heritage asset management. This study addresses the challenges of developing HBIM for large-scale assets, that require adapting the conventional workflow to deliver results in a reasonable time. To this aim, a novel procedure involving a fit-for-purpose Inventory form and a scan-to-BIM approach is proposed. The data acquisition process is speeded up using multiple surveying techniques, and the modelling and information phases benefit from the interoperability among different tools that are already known by professionals in the field. As such, the main innovation lies in the ability to oversee the entire process through a single software, ensuring centralized and efficient control. This innovative process has been applied to investigate a significant portion of the city walls of Pisa, proving its ability to support the decision-making phase for planned conservation of large-scale architectural heritage. The emphasis is on the all-encompassing, interdisciplinary understanding of the assets across different scales. The suggested approach ensures a swift yet precise and reliable outcome in the diagnostic process and facilitating the critical temporal assessments and the review of information by any actor involved in the conservation.

Introduction

Planned preventive conservation involves safeguarding cultural heritage by carefully identifying critical conditions and systematically designing minimally invasive interventions [30, 39]. This approach proves more efficient both in terms of costs and results if compared to unplanned approaches that are often focused solely on remedial activities [72], which might not remove the root causes of disasters. In contrast, planned preventive conservation is a proactive management method that uses periodic monitoring, scheduled maintenance, and comprehensive condition assessment in order to prevent deterioration, damage, and eventually failure of the assets. In this context, a significant challenge is to devise straightforward techniques for establishing an effective management process for large-scale architectural heritage assets, such as ancient city walls, which demand substantial efforts to precisely document and represent their current state of conservation, as well as for designing minimum cost interventions.

Since the earliest times, city walls have been important for forming local identities and for protecting people and places against foreign invaders. In the present day, these assets possess great potential as cultural resources [14], serving as extensive and emblematic representations of civic pride and identity. A wide range of social, economic, and cultural forces fostered a process of re-interpretation and re-appropriation of city walls [12], such that the awareness of their values has opened to the application of the notion of Historic Urban Landscape [7]. Many ancient city walls and fortified systems have been effectively preserved with specific documentation and restoration activities [17, 75, 80]. Nonetheless, putting planned preventive conservation into practice for a considerable number of monumental structures dispersed over an extensive area is a difficult undertaking. As evidence of this, many failures have occurred across the world [3, 22], emphasizing the importance of ensuring systematic approaches to ensure the effective conservation, restoration, and maintenance of historical structures.

In this regard, since 2019, the Regional Government of Tuscany in Italy has promoted a systemic research regarding city walls and urban fortifications aiming at developing a fit-for-purpose methodology to address the conservation and inform risk governance of these historical assets in its territory [29]. The research is relevant for several reasons: firstly, more than 140 ancient walled systems have been counted only in this Region and a clear understanding of the state of conservation is missing; secondly, limited or poor economic resources are usually available to administrative bodies to promote planned preventive conservation of architectural heritage, which is often deemed as secondary with respect to other public needs; thirdly, the investigation requires the cooperation of multiple regional and local authorities who are in charge of their management, hence information, if available, is often fragmented; lastly, multiple sources and disciplines are involved in the process and need to effectively share knowledge within a digital archive that documents the city walls’ history, building technologies, construction phases, alterations and modifications, structural materials, past restoration and maintenance works. Therefore the management of such a great number of assets and data may be challenging as it demands for a lean approach to collect and manage data, while ensuring rapid access to consistent information.

To date, most research analyse city walls from the archaeological and socio-historical perspective [14, 25, 27, 36, 65], while the structural issues are still under-researched. The knowledge of these aspects is paramount as they affect the correctness of any intervention and conservation strategy. Few studies have focused on the rapid survey of city walls using LiDAR technology [23] or on the condition assessment by means of non-destructive or semi-destructive techniques [33, 44]. The analysis of mortars is of particular interest for researchers [26, 47, 62] to evaluate the characteristics with reference to construction phases, degradation processes, restoration measures and interventions. The results of documentation or, more rarely, testing campaigns are sometimes included in Geographic Information Systems (GIS) [8, 17, 18, 73]. Besides, several studies are exploring the application of Synthetic Aperture Radar (SAR) Interferometry to monitor ground settlements and movements of city walls [21, 61] and eventually set up early-warning systems.

Devising and implementing innovative tools for managing the conservation process of city walls has the potential to boost operational capabilities by promoting the exchange and dissemination of information among diverse professionals, including archaeologists, engineers, and various governmental entities spanning from regional to local levels. The adoption of Heritage Asset Management (HAM) principles could offer a significant contribution, as they advocate for decision-making that draws on multidisciplinary, knowledge-driven insights supported by thorough and up-to-date data [42]. Despite substantial advancements were introduced by Building Information Modelling (BIM) [59], a standardized procedure or framework for HAM is still absent. In the context of historical structures, Heritage BIM (HBIM) [28, 32] has been recognized as a valuable strategy for modelling architectural elements and managing vast amounts of information from diverse sources. The advantages of HBIM are numerous, encompassing the documentation of the present condition, the evaluation of interventions, the estimate of costs, and the supervision of activities [74]. Additionally, this approach allows for improving stakeholder’s cooperation, thus promoting informed decisions in terms of preservation, restoration, diagnosis and management since the earliest phases of the architectural design [15].

This research proposes and experiments a novel framework to analyse large-scale architectural heritage and to promote a knowledge-based decision making based on comprehensive up-to-date information. Specifically, the investigation of ancient city walls using HBIM techniques serves as the foundation for the development of a decision support system for the assets. As such, this contribution places particular importance on the inventory, survey, and modelling stages by employing an innovative scan-to-BIM approach. The case study herein examined is a substantial section of city walls of Pisa, a well-maintained defensive structure situated in Tuscany, Italy. These walls mainly consist of masonry curtain walls equipped with gates and towers, representing a noteworthy example of Medieval construction technique.

Related works

Over the last few decades, the advancement of digital data research has made it possible for academics and heritage specialists to accurately record the geometry and state of conservation of cultural assets through precise models. Although 2D media are still widely used by heritage professionals in the daily practice, 3D models have the benefit of offering an exhaustive view of the asset and its problems while maintaining a metrically accurate representation of the building geometry.

Overall, a great variety of methods have been devised for the reconstruction of architectural objects, enabling a precise collection of both geometrical and textural data, both inside and outside the asset, with accuracy down to a few centimetres [56]. This capability is made possible by the convergence of several factors, including advanced data capturing methods, the ample capacity of computer technology, and well-developed transmission and information networks. Terrestrial laser scanning (TLS), personal laser scanning (PLS), and digital photogrammetry, both terrestrial and aerial drone-based, are the most often utilized methods for capturing and processing data into sophisticated textured 3D models. Drones have greatly improved the capacity to survey segments of structures that were previously challenging to access. Another advancement is the integration of photogrammetry with laser scanning, pioneered by companies developing popular software for automated photogrammetry using Structure-from-Motion (SfM) and MultiView Stereo (MVS) surveys. Lately, the use of smartphones equipped not only with a high-quality camera but also featuring a Lidar sensor (such as the iPhone Pro 11–14) and a Real Time Kinematic (RTK) Global Navigation Satellite Systems (GNSS) device have been explored [70, 71]. Specifically designed for short distances, approximately up to 4–5 ms, and smaller objects, this system effortlessly generates a reasonably good quality 3D textured model, such as Pix4D viDoc. Even a common smartphone, with a native GNSS device, has the potential to emerge as an innovative and cost-effective 3D documentation tool [56].

In general, the selection of the most suitable approach is dependent on the specific application, object and the requirements of each case. Indeed, a combination of various modelling techniques is typically used concurrently, as no single method can guarantee a simultaneous achievement of high geometric precision, portability, complete automation, photo-realism, and low expenses. These aspects should be accompanied by flexibility and efficiency in the survey process [64]. This is even more relevant for large-scale cultural heritage monuments and sites considering that creating extensive, accurate and complete models can be time-consuming and costly [34]. On the contrary, these models require to put efforts in increasing the level of automation and balancing human critical interaction in the processing phase.

Once 3D models are generated, various types of information can be incorporated into them, resulting in enriched or ontological models [52]. This task involves initially documenting the heritage assets and retrospectively analysing multiple sources. Data-rich 3D models can be integrated into interfaces and platforms that also gather reports, analog/digital images, drawings, and archival documents [68]. This leads to the creation of digital libraries that facilitate the organization, access, and management of content. Some studies have also introduced a temporal dimension (both diachronic and synchronic) to the 3D architectural model [5, 16] to depict the building’s evolution throughout its life cycle. Limited research has explored how the utilization of 3D models and web-based digital platforms can orient restoration efforts [5], concurrently documenting project activities and progress.

Meanwhile, BIM has gained popularity within the civil engineering due to its capacity to generate and manage structured digital information and multidisciplinary design expertise. This process is supported by various tools that are consistently refined to attain higher quality, lower error rates, and cost reduction, thus ensuring a coordinated, coherent, and consistently updated working model. In this fast-evolving context, standardization is pivotal to ensure adequate technological levels, which affect the content quality and detail, digitization, interoperability, and collaboration [60]. The standardization of BIM practices in the European Union is the main objective of the CEN/TC 442 and its working groups that, since 2015, are responsible for developing and publishing standards and technical reports on the topic [11].

Different terminological attributes have been adopted by scholars to cluster the functionalities of BIM [74]. Specifically, 3D BIM refers to spatial models ensuring quantity take-off, 4D BIM also adds information on the construction scheduling, and 5D BIM further adds cost calculation [6]. By and large, the literature has categorised BIM introducing two main perspectives [45, 74]: a narrow perspective that focuses on the technical issues, intended as the creation of a digital model working as a repository or information management hub; and a broader perspective that accounts for all the interrelated functional, informational, and organizational/legal issues, in addition to technical ones.

The increasing complexity of modern infrastructure projects, characterized by extensive scale, often spanning kilometers and generating numerous interactions with the environment, has fostered the application of BIM in the field [31]. While the term BIM has been primarily applied in the building sector, the term I-BIM (Infrastructure Building Information Modeling) has been introduced to refer to infrastructure projects [2]. Notable projects such as the Gotthard Base Tunnel in the Swiss Alps [78], the Grubental Railway Bridge in central Germany [41], and the Stuttgart 21 metro project in Southern Germany have successfully used BIM to address challenges that would have been insurmountable with conventional 2D drawing methods [43]. In this context, numerous attempts have also been made to integrate GIS and BIM in both the architecture, engineering, and construction domain and the geospatial industry. This aims for a more holistic approach to the design, construction, and management of large-scale infrastructure [81]. Many applications in this regard involve the combined management of geospatial data and specific information about buildings and infrastructure [76]. This integration proves useful for planning and design, facilitating better 3D visualization of assets, assessing the impact of new infrastructure on the built environment, and monitoring and managing phases from infrastructure design to demolition in a centralized environment.

In the realm of historical building management, the collection and utilization of documentation, diagnostic investigations, and structural analyses are fundamental aspects. The potential of integrating the use of 3D models to support analyses is still to be fully explored. The study of a historical building typically generates a vast amount of information, ranging from construction phases and transformations over time to the materials, the state of conservation, and patterns of deterioration. The documentation and analysis of historical architecture require a system for data acquisition and organization that is flexible and capable of overcoming the typical fragmentation of information, especially when the study process unfolds over time. All this information can be collected and processed through the use of a collaborative process for the production and management of structured digital information aimed at creating a shared database. In other words, the application of the HBIM [32] principles and techniques can bring significant advantages in terms of rapid access to documentation, interoperability during analysis, multidimensionality in the design phase [79], ease in cost evaluation at every stage of the process, and improved management of maintenance and interventions, up to the management of various stages of the restoration site and subsequent planning of the building’s preventive conservation [55].

The development of reality-based HBIM procedures that rely on as-built/as-is models is driving the development of sophisticated surveying techniques in the field of architectural heritage. Data acquisition for the generation of 3D models can be carried out using traditional survey methods or point clouds. In the latter case, 3D models can be obtained either by directly triangulating the point-cloud data, which is affected more by the quality of the point cloud itself, or by using solid modelling techniques, which the user can handle both directly and parametrically depending on the time and precision of the desired results [4]. These models reflect the building’s real condition and may be semantically enhanced with a variety of data. Unlike the BIM process, which leverages the combination of elements chosen from a library usually already possessing information, HBIM implies the creation of a model of the construction, to which data is subsequently associated. The HBIM process allows for the structured cataloguing of information related to the asset, making it easily extractable and accessible at any time. It can be further enriched through thematic maps, photographs, and graphs, which can be directly processed on the model itself. A rich and organized cataloguing of collected data can be a crucial support in various stages of building restoration and management, proving particularly helpful in operational phases [20]. Such information may span diverse disciplinary fields, and their correlation can facilitate the generation of analyses. Additionally, there is the opportunity to utilize a dynamic model, one that is continually updatable and expandable, promoting collaboration among various professional roles [19].

In this multidisciplinary context, HBIM methodologies can be considered a promising and effective tool for the conservation of historic assets [54] serving as an efficient and effective approach for understanding, preserving, and restoring historical architecture, with the possibility of controlled management of all documentation within a unified environment. In [53], HBIM is employed for collecting and organizing data related to the degradation of existing wooden structures. In [66] for defining a decision-making approach for the best and most sustainable solution in wood conservation. [51] develop a new methodology in the HBIM environment to monitor the structural performance and human-centered environmental comfort of historic buildings. Finally, in [13], HBIM is applied for the control and monitoring of the structural safety of an ancient metal bridge. In [24], the digital representation of the built environment and the management of documentation are discussed, respectively, with the aim of better utilizing BIM projects through the use of portable tools and for the creation of as-built projects of structures and infrastructures. An useful review of the existing literature on the current state of HBIM implementation and research trends to date is presented in [48]. Even in the field of cultural heritage, the capabilities of the HBIM/GIS integration have been investigated [79]. Exploiting the capabilities of GIS, now a potent instrument in urban planning and resource management, serves various purposes in geographic and urban contexts, encompassing risk control, planning, analysis, and visualization [37]. Despite the growing applications of BIM technologies in HBIM workflows, the discipline still lacks clarity concerning information requirements from a tenant perspective [46]. Based on the outcomes of an analysis performed by [35] on 52 HBIM case studies pre-dating 2017, [46] highlight that in most cases that HBIM methodologies emphasize technical challenges, rather than seeking to understand the needs of presumed stakeholders. Furthermore, the lack of a standardized approach to HBIM implementation restricts the application of BIM technology and hinders the realization of associated benefits in the research and preservation of historical structures.

Finally, while numerous advancements have been made from a technical standpoint, significant challenges still remain, including the integration of information from diverse sources like material characteristics, historical modifications, and patterns of deterioration [38]. When combined with 3D representations, the organization of data have the potential to greatly enhance the efficacy of conservation practices. At present, numerous BIM platforms are employed by professionals for tasks such as modelling, visualizing, assembling, and managing architectural heritage knowledge. However, the seamless interaction between the available tools, referred to as interoperability, within a scan-to-BIM workflow and the absence of historical parametric object libraries are still subjects of ongoing discussion.

Materials and methods

Scan-to-BIM pipeline for large-scale architectural heritage

The main challenge in developing HBIM for large-scale architectural heritage is to maintain an adequately comprehensive level of detail to avoid constraining the process while ensuring its adaptability to various project needs. Besides, the necessity to attain a uniform level of knowledge for the whole asset adds an additional hurdle, which requires to invest efforts in the documentation phase along with the digital modelling.

The procedural pipeline herein presented is divided into different steps that allow for achieving of a comprehensive understanding of the assets through documentation and to efficiently develop a complete digital model (Fig. 1). The procedure is designed for efficiently acquiring and processing diverse types of data, which can be integrated into the informed 3D model. In this regard, the model can be interpreted as a container hosting and embedding multiple pieces of information, which together represent the content [49].

The primary benefit of HBIM resides in its capability to merge multiple datasets from various documentary sources, inspections, and diagnostic techniques into a unified model. As part of the proposal, a novel Inventory Form is herein presented aiming at providing a systematic organization for relevant data and at supporting the inspection and diagnosis tasks over time.In fact, in many historical buildings, information is often incomplete, inconsistent, obsolete or fragmented among different stakeholders [9] and across disciplines [45], therefore the documentation phase may also benefit from the availability of specific tools. Additionally, the form allows users to identify threats and consequently plan preventive conservation or activate immediate safety measures.

Additionally, the procedure focuses on how to generate as-built 3D models that preserve details concerning the external dimensions, material types, deformations, and crack patterns of large-scale structures. External dimensions and deformations are inherent geometrical properties of the model(s), while surface materials are implemented with texture mapping. We propose a workflow that allows for achieving a rapid, yet accurate and reliable results by speeding up data collection, the creation of 3D models and the association of any information. As such, it employs a scan-to-BIM strategy that reconstructs the three-dimensional scene from oriented images. This approach minimizes the need for labour-intensive and error-prone manual tasks. With reference to the type of assets under investigation, namely ancient city walls, the creation of accurate as-built models of all their elements, namely the architectural and structural ones, is feasible with a good and uniform level of detail.

Fig. 1

Scan-to-BIM pipeline for large-scale architectural assets

Documentation: a fit-for-purpose inventory form for city walls

City walls present peculiar features associated with their large extension and variety of forms and materials, as well as the great professionals involved in the knowledge and management process. The knowledge process aims at acquiring data on the assets from several disciplinary studies at different levels, both on a city-wide and detailed scale. The walled system can be analysed as a whole and documented in the light of previous studies, historical information, and archival material, but it can also be considered as a set of subsystems corresponding to the elements identified on the basis of morphological, geometric, architectural and historical criteria, namely walls, gates, bastions and towers. An improved knowledge of each element can be achieved through the acquisition of specific documents and the carrying out of field surveys (e.g., metric, photogrammetric or visual).

The survey and data acquisition encompass not only the geometry, but also the aspects such as historical context, construction techniques, and damage patterns [29, 58]. The analysis of the building technology makes it possible to catalogue both the construction techniques and the materials, each of which can be studied in depth with specific tests to characterize their physical, chemical and mechanical properties. It is also important to collect reports and documents on the construction activities, transformations, past interventions, partial destructions, and restoration activities that have affected the assets. At the same time, inspections and surveys can complement historical and archival research, thus offering direct insights to specialists.

Here, a fit-for-purpose ‘Inventory Form’ is presented (Fig. 2), to guide both on-site and offsite documentation of the city wall and its elements, and to gather data that serves as the initial basis for informing the 3D digital model. Every form is designed for one element/subsystem of the city walls, suitably coded based on the morphological and architectural classification (sect. 2.2 of the form). The form includes seven sections: (1) localization; (2) characterization; (3) accessibility for technical inspections; (4) documentation of the interventions; (5) material archive and masonry typology; (6) depiction of the surroundings and potential risks; (7) critical conditions to activate diagnosis or safety measurements. Three types of fields are available to accommodate information differently: (a) free field whose compilation is at the discretion of the compiler; (b) multimedia resources field, corresponding to a free field to be filled in with URL to external content to be collected in folders that can be shared with users; (c) restricted choice field, where the indication of the items to be inserted is associated with square symbols for single choice compilation, or with circle symbols for multiple-choice compilation. Table 1 presents a summary of the Inventory Form’s structure, including the type and purpose of data collected in each section.

Table 1 Structure of the Inventory Form

Fig. 2

Inventory form for ancient city walls

More specifically, Section 1 collects data regarding the localization of the element and its univocal identification. The localization is based on the administrative identification of the region, the municipality and, more specifically, the reference address. Possibly, reference geographical coordinates can be provided for geo-localization scopes. The univocal identification instead refers to the common denomination of the element sided by a code that designates it in a unique way.

Section 2 is divided into three sub-sections, mainly characterized by the restricted choice information field, but also containing the free field and the multimedia resources field. The first subsection is related to the historical characterization by collecting information on the prevailing construction typology of the element and the periodization of its historical transformations, modifications and alterations since the earliest construction phase up to the 19th century, when theories of restoration appear. The second subsection provides information on morphological and typological aspects, describing the type of element, curtain, tower or gate, its main dimensions, the battlement type, and the presence of terreplein. The third subsection collects the documentation relating to the state of conservation of the element and allows for adding links to URLs containing material on geometric surveys, in situ inspections and tests.

Section 3 collects information through restricted choice fields on how to use the asset, on accessibility to the neighbouring area, or on the visibility of its surfaces, highlighting the presence of adjacent buildings, inaccessible areas, or natural obstacles.

Section 4 collects information on past interventions. Through free fields, restricted fields and links to descriptive material, this section provides information on restoration and consolidation interventions that have affected the element from the second half of the 20th century up to the present day, and on the maintenance status of the asset, also reporting the original documentation available.

Section 5 is dedicated to the component materials of the asset and is compiled for each type of masonry in the element. Through both free and restricted choice fields, the masonry texture of the external surfaces is described, starting from the type of blocks and mortar joints to the morphology of the entire section, whether it is composed of several leaves or with an internal core.

Section 6 contains a description of the context of the element and the main hazards. Different types and intensities of hazards in the area are entered through restricted choice fields.

Finally, Section 7 identifies any critical conditions of the element, in view of the activation of the diagnostic process, provisional measures or consolidation and restoring interventions.

Development of a digital model

Important data sources for HBIM applications are the geometrical survey of the heritage assets other than documents regarding their significant features pertaining to history, construction techniques and materials, and actual conditions.

In this study, the geometrical survey process involves the utilization of photogrammetric methods in conjunction with topographical measurements. This combined approach is used to collect three-dimensional coordinates from numerous targets and control points located on the heritage structure (Fig. 1). Specifically, methods based on Structure from Motion (SfM) photogrammetry provide the opportunity for fast and less resource-intensive data collection [77], with respect to conventional photogrammetry. SfM methods reconstruct the three-dimensional geometry of the scene through automated and iterative procedures. These procedures ascertain the positioning, orientation and alignment of cameras based on an assortment of overlapping images, all without necessitating a pre-established network of reference points or targets.As such, these image-based methods may require extra computational effort in processing input photographs, but they have proved effective and economical in producing metrically correct 3D models.

In the acquisition phase, redundancy is a key requirement [77], which is why photographs are captured from various angles and under uniform lighting conditions. However, when dealing with city walls, practical challenges might emerge due to limited accessibility and the great extension of the object. This situation might require surveyors to resort to long-distance photography, which negatively affect the resolution of work. In fact, the distance between the camera and the object can lead to a reduction in the spatial resolution of the photographs, consequently resulting in point clouds with lower density.

Positional data (x, y, and z coordinates) is utilized to define control points that allow for proportionally adjusting the 3D reconstruction of the scene to an absolute coordinate system and for manually correcting the misalignment of any camera. These coordinates can be acquired through various range-based methods such as laser scanning, or more traditional survey instruments. Additionally, remote technologies like Global Positioning Systems (GPS) are incorporated in the photography process to enhance the alignment and positioning of cameras. It’s worth noting that these control points are also used in the subsequent texturing and information stages to set and spatially orient the projection planes for ortho-mosaics and annotation maps.

The 3D architectural model can be built starting from the dense point cloud following a refinement procedure that includes removing areas of low confidence and minimizing noise caused by reprojection errors. The resulting measured dense cloud is therefore processed to generate a textured mesh (i.e., a triangulated network) of the scene, which is by all means a 3D model [64] having good geometric accuracy, photorealism and complete details. Since city walls can be macroscopically described with planar geometries, even a lower number of points may be sufficient to triangulate the surface and generate a mesh that fits the actual geometry. Nonetheless, the accuracy of the textured model notably increases with the resolution and number of the photographs employed in the point cloud processing and with the number of measured points in each image. In this case, the number of sample points is higher and less noisy such that the requested manual adjustment is minimal, and the surface triangulation achieves better results.

This mesh-based 3D virtual representation is a powerful tool to facilitate the reading and interpretation of large architectural objects, such as ancient city walls, being less demanding in the processing and visualisation phase, thus ensuring portability and flexibility. Furthermore, the model can be used to create orthomosaics, which are high resolution 2D images obtained by projecting the reconstructed object over a reference surface. These orthomosaics can effectively support the diagnosis if used as a basis for annotating the wall’s properties and conditions. Due to the nearly planar geometry of the elements composing the circuit of city walls, the projection surface takes the form of a plane, which can be established by designating a minimum of three points onto the reconstructed mesh. These points may coincide with the ones targeted during the topographic survey.

The various materials and patterns of deterioration are commonly identified and marked on 2D images or representations of the asset. This is done by outlining polygons or areas directly on the 2D media. This operation is typically time-consuming as it involves manually drawing these geometries; however, recent research initiatives investigated the use of a human-centred Artificial Intelligence (AI) tools for the semantic segmentation of 2D orthographic images of masonry walls [57, 58] using the software TagLab [69].

Further operations can be performed to aid the information of a 3D model with the data collected during the documentation phase. The HBIM’s semantic structure reproduces the sections of the Inventory Form by setting customized properties.

Software modules

The scan-to-BIM procedure outlined in this paper is designed as a process involving well-established software tools. It starts with the creation of a 3D mesh geometric model of the structure to produce an HBIM.

The processing of photographs and positional data is performed with the SfM software Agisoft Metashape [1], which generates dense point clouds and textured tiled models, thereby improving the visual quality of the final model. This model can be saved in a three-dimensional format (like obj wavefront, fbx, 3Ds, etc.), ensuring that the texture remains directly linked to the geometry even when imported into different software. Afterwards, the final tiled textured model can be imported into the Rhinoceros software [50], which adeptly handles and edits complex geometries. The mesh model can be brought into the software Graphisoft Archicad [40] by using either the obj wavefront or fbx MotionBuilder format. These formats ensure the texture is associated with the model. Upon import, the processed model undergoes transformation into a.gsm library object.

The method proposed in this paper facilitates this step by using Grasshopper, an algorithmic modelling environment in Rhinoceros supporting a wide range of useful plug-ins. Specifically, the scan-to-BIM procedure employs the Grasshopper plug-in Archicad Live Connection developed by Graphisoft, which enables the connection between Rhinoceros and Archicad. Archicad Live Connection simplifies the generation of Archicad’s native BIM elements in Grasshopper using familiar nodes, hence automatizing common processes that are often complex to be handled.

The advantage of this procedure lies in the fact that Grasshopper allows control over the segmentation process, the assignment of properties to the elements of the model and the opportunity to choose the type of BIM component to associate with the model, not solely with the.gsm library object, as the traditional method provides. The general workflow of the procedure is illustrated in Fig. 3.

Fig. 3

Workflow of the proposed procedure

Case study: the city walls of Pisa

Description

The city walls of Pisa were constructed throughout the Middle Ages, commencing in the 11th century, with the aim of protecting the settlement area of the historical Maritime Republic. Overtime, several sectors of the system have been modified or strengthened in response to evolving weaponry and associated defensive strategies [10]. The expansion and consolidation of territorial control, coupled with ongoing conflicts with neighbouring cities, prompted the creation of advanced and intricate defense systems, which are now regarded as exemplars of innovative brilliance and expertise. Despite gradually losing their defensive purpose, the walls and been preserved, with just a few sectors of the whole perimeter being destroyed or demolished.

Today, the overall length of the circuit measures approximately 7 km, with an average height of the curtain walls of about 11 m, and a thickness of roughly 2 m. The wall section is multi-layered: two outer layers with an almost regular texture enclose the irregular masonry core (Fig. 4d). The construction method and craftsmanship trace back to local traditions of the area, and materials were quarried from nearby locations. Notable materials that have been detected are the sedimentary “Breccia” stone from Asciano, the San Giuliano Limestone, and a yellowish Livorno calcarenite (classified as “Panchina” Fm. Calcarenite) [57, 58, 63]. Figure 4 shows a set of emblematic features along the city walls of Pisa, encompassing material, geometry, and construction typologies.

Fig. 4

Emblematic features of the city walls: a Bastion San Giorgio (BAS.03) with traces of volumetric expansion and alterations in the wall’s texture; b curtain walls (CUR.01, CUR.02) and tower (TOW.02) showing the linear and regular configuration of the walled system; c curtain wall (CUR.47) and gate (GATE.15) that are partially buried under the street level; d remains of the gate “Porta Buoza” (GATE.16) attesting to the construction techniques of the wall’s section; e Tower Santa Maria (TOW.03) and adjacent curtain wall (CUR.05) from the external side of the walls; f external side of a curtain wall (CUR.12) displaying a full-height vertical crack. The identification codes are defined according to Table 2

The city wall circuit has been categorized based on the morphological characteristics of its elements, differentiating gates, curtain walls, bastions, and towers. Each element has been assigned an alphanumeric code corresponding to its typology and is numbered sequentially. Table 2 summarizes the adopted classification and displays the total number of elements in each morphological type. While, Fig. 5 shows the sectors analyzed in this investigation and corresponding to: three gates identified by the codes GATE.01, GATE.02, and GATE.03 (namely Porta Nuova, Porta del Leone, and Porta Santa Maria, respectively); five portions of curtain walls designated as CUR.01, CUR.02, CUR.03, CUR.04, and CUR.05; and three towers coded as TOW.01, TOW.02, and TOW.03 (Torre del Catallo, Torre del Leone, and Torre Santa Maria, respectively).

Table 2 Codification system for the city walls of Pisa

Fig. 5

General view and classification of the city walls of Pisa

Dataset

The investigated sectors of the city walls span in total a length of 405 m and have surface of 4392 sqm. Photographs were taken with a GPS-enabled iPhone 11 camera having a 1/2.55-inch sensor and a 12MP resolution. Accessibility constraints, often imposed by trees, roads, and barriers, dictated the distance from the wall, ranging from 7 m to 12 m. Photographs were acquired in longitudinal strips to ensure substantial overlap, averaging around 70% in the whole digital reconstruction. If the distance didn’t permit capturing the full wall height, two images were taken from bottom to top. Additionally, wall walk images were captured to ensure comprehensive detail and resolution for the upper portions. The data collection process spanned multiple days and involved various time periods to ensure uniform lighting conditions, thereby preventing excessive direct light and harsh shadows. Concurrently, positional data were gathered using a Total Station, which delivered accurate and high-quality coordinates for a set of control points situated along the city walls.

The acquisition process spanned multiple days and involved various time periods to ensure uniform lighting conditions, thereby preventing excessive direct light and harsh shadows. Concurrently, positional data were gathered using a Total Station, which delivered accurate and high-quality coordinates for a set of control points situated along the city walls.

Results and discussion

The documentation of the examined sectors of the city walls followed the codification system in Table 2, such that each and every coded element has a specific Inventory Form reporting all the important features and information. This phase was quite challenging due to the diverse institutions involved in the management of city walls, namely the Municipality owning the asset, the Office for Heritage Protection in charge of the preservation, the Opera della Primaziale Pisana for the access in the area within Piazza del Duomo, and eventually local cooperatives that organize touristic activities along the circuit.

Given the large dimensions of city walls, the combination of several surveying techniques has proven to be very effective, while no other individual method would deliver a comprehensive and suitably precise model as efficiently and quickly. Photographs and positional data underwent processing using the SfM software Agisoft Metashape [1]. The choice of using handheld, lightweight and low-technology devices to take photographs, namely the iPhone 11, was primarily driven by the need to employ low-cost equipment and reduce acquisition costs. Its 12-Megapixel resolution is more than sufficient for processing in Metashape, and the data format, specifically known as EXIF (Exchangeable Image Format) metadata for images, is a high standard with geolocalization likewise compatible Metashape.

The dense point cloud of the sector of the city walls resulted from a total of 486 photographs captured across an approximate length of 400 m. The number of Marker Points along the walls was 200, and the accuracy of the reconstruction and markers’ positioning was set to 5 mm. This value determined an average total error (in x, y, z coordinates) of 30 mm and a maximum total error of 44 mm. This set-up produced a good-quality scaled point cloud of the object under investigation (Fig. 6), which served as the basis for creating the 3D model.

Fig. 6

Positioning of photographs along the wall curtain (CUR.01), dense cloud of the tower (TOW.03) and Marker Points, perspective view and details of the city walls of Pisa processed with SfM photogrammetry

The 3D model was generated through the automatic triangulation of the dense point cloud within Agisoft Metashape. This process provides as ultimate result a 3D textured mesh that represents the walls’ sector (Fig. 5). Figure 7 illustrates the steps for the 3D reconstruction of a portion of the walls, showcasing the comparison between rectified actual photographs (a), the dense point cloud (b), the solid visualization of the model (c), and the tiled textured model (d). Both the mesh and texture retain a satisfactory level of detail, compatible with the large dimension of city walls, as the digital representations are able to reproduce the geometric irregularities and the surface finishing of the walls.

Fig. 7

3D reconstruction process and level of detail of a portion of the CUR.04: a rectified real photograph; b dense point cloud; c model in solid visualisation; d tiled model textured

Starting from the textured model, eight high-quality orthomosaics have been generated by projecting the wall’s surfaces over a plane, which was established through the definition of three Marker-Points in Agisoft Metashape. Each orthomosaic was employed as “annotation plane” for documenting the material properties and the deterioration patterns, such as decay, material degradation, cracks, and deformations, which have been also included in the Inventory Form in section 5 and 2.3, respectively. Every annotation plane encompasses closed 2D entities that are situated on the plane itself and helps to identify and outline distinct attributes of the studied sector. The annotation of material properties has been performed with the aid of AI-powered tools, as presented in [57, 58], while deterioration patterns have been segmented manually using Rhinoceros. The most recurrent deterioration types are efflorescence, salt crust, alveolization and erosion of calcarenite, moist spots, and vegetation either on the wall surface or on the upper walk (Fig. 8). Vertical cracks and out of plumbs are frequent, sometimes with signs of repointing, but often in critical state if we regard the wall slenderness.

Fig. 8

Sample descriptive form for the phenomena “Biological colonization - plants” and “Alveolization”

Fig. 9

Four annotation planes used for the curtain walls coded as CUR.04 and aligned to the 3D model. Each plane is then projected over the model as shown for the annotation plane regarding the diverse “materials”

After importing the 3D model in Rhinoceros as a continuous textured mesh, the geometry was sliced to clearly differentiate the elements based on the proposed classification system (Table 2). Besides, the annotated planes were imported in Rhinoceros and positioned accurately in the 3D space using the same set of Marker-Points as in Agisoft Metashape, namely retrieving the coordinates that were used for generating the orthomosaics (Fig. 9).

The Grasshopper canvas for generating informed 3d models is shown in Fig. 10 which refers to an emblematic element of the city walls. The input geometry corresponds to the 3D textured meshes generated in Metashape and imported in Rhinoceros. The mesh model is segmented by means of the Grasshopper Mesh Split component (canvas block “segmentation”), using the closed 2D entities predefined in the annotation planes (Fig. 9). This operation produces new sub-meshes which can be associated with different properties. At the same time, a file containing the three-dimensional entities is prepared within Graphisoft Archicad, involving the definition of specific properties for historical, material, structural, and morphological information. This set of attributes forms the semantic structure of the HBIM model, which is finalized using the Options/Property Manager.

Fig. 10

Grasshopper canvas for generating informed 3D models. The procedure is applied to a sample gate within the circuit of the city walls of Pisa

As shown in Fig. 10, the meshes produced during the segmentation step are thus linked to the Grasshopper-Archicad Live Connection component Morph-solid (in the case study the entity “morph” was chosen). Here the meshes are connected with the relative Morph Settings (canvas block “default settings”). Among these settings, it is possible to load all the new customized properties of the “HBIM identity data” and of the “HBIM-Design parameters related to masonry types”, specially defined in the Archicad file, and proceed with the assignment and compilation of the information related to the elements of the architectural element, directly in the Grasshopper environment.

The property settings are set up in Grasshopper following the organization of the Inventory Form reported in paragraph . These settings are associated to Archicad software settings and can be automatically identified and imported, although defined in the Grasshopper environment. Among the settings, the user can load all the new customized properties, which were defined in the Archicad file, and thus proceed with the assignment and compilation of the information related to the elements of the architectural heritage directly in the Grasshopper environment. In the case study, the Morph Setting component in the canvas allows for connecting Grasshopper to Archicad and to pass on the model from one platform to the other. As such, the pipeline is rooted in and proves the interoperability among Metashape, Rhinoceros-Grasshopper and Archicad allowing for high flexibility and efficiency in the creation and information of HBIM. This feature enables users to work in parallel and update data simultaneously on two software environments at once, thus avoiding duplication of work. The canvas can be replicated for each element of the circuit of the city walls.

At the end of this procedure, the Synchronize tool in Grasshopper can import the mesh and all related HBIM data into the Archicad file (Fig. 11). The historical artifact information is ultimately visible and manageable within the Archicad environment, facilitated through the configuration of Graphic Override Combinations to which Graphic Override Rules are associated (Fig. 12).

To ensure accessibility of various stakeholders to the information about the project in Archicad and the final HBIM, it is possible to export a specific sheet containing all customized information about the object defined in the property setting. This file can be read with an.ifc reader like Solibri [67], allowing every user to access the aforementioned HBIM information.

Fig. 11

A sample view of the final HBIM in Archicad. The selection in green is the mesh corresponding to the gate Porta del Leone, coded as GATE.02

Fig. 12

Graphic view of the curtain wall CUR.04 and its diverse materials, which are displayed as annotated textures in the HBIM

Conclusions

Ancient city walls are characterized by a peculiar configuration given by linear development, great extension, and recurrent geometries that are conducive to the systematization of knowledge into HBIM. This work proposes a novel procedural pipeline designed to ensure an efficient documentation and conservation of large-scale architectural heritage, and in particular city walls. The workflow facilitates the creation of as-built 3D models that retain external dimensions, deformations, damage patterns and materials. This results in a comprehensive and sufficiently accurate model, functioning as an HBIM for sharing and tracking data among heritage experts.

Diverse pieces of information can be collected in a structured way by means of the Inventory Form. At first, the form allows for the localization and characterization of the elements of the systems, assessing the consistency and historical-artistic value through typological and morphological features, as well as the current state of conservation and any modifications undergone over time. Then, it focuses on the accessibility, in view of extending control to any unsupervised areas, where the methods for inspection will need to be determined. Afterwards, it documents masonry types and materials and describes the context, with specific attention to possible exposure to hazards that may increase the level of attention to be given to a specific element. Finally, it clearly identifies the major critical conditions to activate a more in-depth diagnosis or lead to immediate safety interventions.

The proposed approach employs a scan-to-BIM technique, performing an automated 3D model reconstruction from oriented images. This reduces the need for labor-intensive and error-prone human procedures. The provided case study showcases the efficient interaction between the textured mesh generated by Agisoft Metashape and other pieces of information such as 2D annotated media and data gathered by means of the fit-for-purpose Inventory Form. These are further processed using software such as Grasshopper, McNeel Rhinoceros, and Graphisoft Archicad, namely specialized commercial software applications that are commonly used by professional architects or engineers. Although the tools are not novel in themselves, the innovation in the proposed software flow lies in the ability to oversee the entire process through a single software, Grasshopper, ensuring centralized and efficient control. The workflow excels in rapidly importing detailed mesh components into the Graphisoft Archicad BIM environment. Additionally, it allows for compiling HBIM information within the Grasshopper environment using a familiar node system. Overall, the scan-to-BIM pipeline is rapid, streamlined, and highly adaptable for professionals. Furthermore, it ensures prompt information exchange and interoperability between a variety of tools that are currently being used by professionals in this field.

The resulting HBIM can effectively support the diagnosis of ancient city walls, as well as diverse large-scale architectural heritage. In particular, the application of this pipeline can be extended to numerous large-scale architectures having linear structures, both in urban and suburban areas, such as ancient aqueducts, city embankments along rivers, and infrastructure spread throughout the territory, such as ancient bridges or historical pathways. It achieves this by facilitating the presentation of data over time, allowing for critical temporal evaluations, and permitting the access to information by any actor engaged in conservation efforts. In this manner, the proposed framework harmoniously aligns with the tenets of Heritage Asset Management (HAM), as it encourages shared decision-making based on comprehensive and up-to-date data.