Development of a virtual interactive system for Dahua Lou loom based on knowledge ontology-driven technology
Heritage Science volume 11, Article number: 178 (2023)
The Dahua Lou loom, pivotal to Nanjing Yunjin weaving, constitutes an integral part of global intangible cultural heritage. Its intricate weaving technique remains unmatched by modern machinery, marking it as a vital cultural artefact warranting protection. However, current virtual interactive systems grapple with adequately demonstrating its craftsmanship due to limitations like system iteration, multi-system integration, and data interoperability. To address these challenges, this study proposes a method that synergizes knowledge ontology, virtual reality technology, and data-driven design.Knowledge ontology enables enhanced management, reuse, and wide-ranging dissemination of domain knowledge, improving system interoperability. This methodology is utilized in constructing the loom model, animation demonstration, and in integrating it into the virtual interactive system. This multifaceted application of knowledge ontology significantly bolsters the system’s efficiency and optimizes its development, maintenance, and integration processes.This research provides crucial advancements in domain knowledge modeling, 3D visualization, and virtual reality interactive systems, playing a significant role in preserving cultural heritage. Despite certain limitations, it offers an immersive, intuitive, and enriched design experience. The study concludes with a critique of the developed system, discussions on the encountered challenges, and future research directions include improving and expanding the knowledge ontology based on the opinions and practical experiences of domain experts, enhancing system compatibility, and conducting broader evaluations.
The Nanjing Yunjin weaving is a valuable Intangible Cultural Heritage (ICH) of the Chinese nation and even the world. The Yunjin adornment weaving technique presented by the Dahua Lou loom shows complex patterns of “individual flower varying color” and “common warp broken weft”, which is one of the weaving techniques that modern machines have not yet been able to replace . The complex structure and superb craftsmanship of the Dahua Lou loom have attracted widespread attention from scholars of cultural heritage protection.
Currently, users can better understand and appreciate the craftsmanship and historical value of the Dahua Lou loom through virtual reality(VR) technology [2, 3], but traditional Virtual Interaction System (VIS) face problems such as iteration and multi-system integration . Among them, VR technology is not ideal in terms of intelligent and interactive expression of domain knowledge. Secondly, traditional virtual interactive systems require manual processing of a large amount of data and associated relationships, making system maintenance costs relatively high. Moreover, virtual systems may encounter problems such as low data interoperability and difficulties in information exchange when integrating with other systems, limiting the effectiveness of VR technology in conveying Yunjin culture to Yunjin enthusiasts and researchers. Lately, a novel concept, Galleries, Libraries, Archives, and Museums laboratories (GLAM Labs), has started playing a significant role in the field of cultural heritage preservation. GLAM Labs advocate the establishment of laboratory environments in libraries, archives, museums, and similar venues worldwide to explore the use of emerging technologies, such as VR and ontology technologies, for the digitization and reuse of cultural heritage.GLAM Labs emphasize collaboration with users, researchers, and communities, realizing more profound and diversified cultural heritage utilization through shared data and tools, as well as innovative projects and activities. In addition, GLAM Labs pay special attention to enhancing the accessibility and utilization of ICH, such as the Da Hua Lou loom that we are studying.Consequently, the objectives of this research are closely aligned with the mission of GLAM Labs. We will employ the methods and philosophies of GLAM Labs to optimize our Virtual Reality system, aiming to better protect and display ICH.
In recent years, the collaborative design of knowledge ontology (KO) and VR technology has provided new possibilities for the protection and display of cultural heritage . The application of KO in the field of cultural heritage protection mainly focuses on the management of complex resources and data, and effective information retrieval [6, 7]. KO development has cross-platform interoperability, and logical capabilities for concept integration and mapping , playing a crucial role in promoting collaboration and information exchange in the cultural field and advancing the development of cultural heritage protection. At present, VR systems developed based on KO are widely used in fields such as architecture , aerospace , and clothing design , providing immersive interactive environments for design, simulation, and training. However, research on integrating KO and VR technology for the interactive development and optimization of ICH is still relatively lacking. Therefore, this study aims to propose a collaborative method based on KO to address the problems in the current Dahua Lou loom virtual interactive system.
The main objectives of this study include
Constructing the Dahua Lou loom KO to promote the inheritance and digitalization process of Yunjin culture by optimizing knowledge management, reuse, and sharing in the field. As a general knowledge representation method, KO is easy to integrate with other knowledge systems to achieve data interoperability and the dissemination and sharing of domain knowledge.
Application of KO in loom modeling and animation demonstration: ensuring the accurate and consistent representation of complex relationships between entity parts, weaving steps, and other elements through knowledge mapping, thereby achieving accurate guidance and optimization of KO for loom modeling and animation demonstration.
Integrating KO with the loom virtual interactive system: based on KO data parsing and mapping, integrating the KO and the virtual loom interactive system into a Unity-based  platform to achieve real-time, interactive simulation and optimization, while improving the efficiency of system development, maintenance, iteration, and integration.The rest of this paper is organized as follows: “Related work” section describes three main related fields of this study: virtual interaction in museums, the construction of intangible cultural heritage knowledge ontology, and the methods and techniques for integrating knowledge ontology with virtual reality models. These fields provide the theoretical basis and technical foundation for this study. “Methodology” section describes in detail the method structure and process of this research, including the KO construction, 3D modeling, knowledge mapping, and virtual interaction processes used in this research. “Assessment and discussion” section discusses the results of the developed system and its effects in promoting the design and optimization of the Dahua Lou loom virtual interactive system, and analyzes and discusses some of the problems and limitations encountered in the research. “Conclusion” section summarizes the main findings and contributions of this research, and proposes suggestions and prospects for future research on the Dahua Lou loom development in the field of VR systems based on KO.
This chapter delves into the three major areas relevant to our research: virtual interactions in museums, the construction of knowledge ontologies for ICH, and methods and technologies for integrating knowledge ontologies and virtual reality models. Each section will focus on how these technologies and methods are applied to the protection and exhibition of Dahua Lou loom, aiming to explore their potential display forms in future museums.
Restoration and display of Dahua Lou loom through virtual interactions
Modern museum-goers increasingly seek an active involvement in acquiring knowledge and information rather than passively receiving it . As the use of virtual reality technology in museums becomes more prevalent, future museums are expected to evolve towards digitalization and cloud technology. ICH, like Dahua Lou loom, can be displayed in future museums through a new exhibition format that combines knowledgeability, entertainment, and interactivity, overcoming limitations related to space and maintenance personnel. For example, HTC Vive’s collaboration with the Louvre Museum in France enabled viewers to appreciate and interact with “Mona Lisa” in a virtual space, as if having a temporal dialogue with Mona Lisa, unlocking various mysteries and legends about her.Footnote 1 The immersive art exhibition “FLOWERS BY NAKED” in Japan combined elements of flower arrangement, innovative technology, light and shadow, music, and olfactory experiences, significantly enhancing the audience’s immersive experience. These cases demonstrate the enhancement and transformation of museum experiences through virtual reality technology and provide useful references for our research.
Significant progress has been made in research on the application of virtual reality and augmented reality technologies in museums. For example, Carrozzino M and others proposed a classification of VR devices for cultural heritage applications based on their characteristics in interaction and immersion . Hu QW and others used Unity 3D and network services to create a real-time virtual 3D scene of the Jinsha archaeological site museum composed of georeferenced sequential panoramic images and 3D models . L Errichiello and others studied the role of virtual reality in improving visitors’ experiences at destinations and cultural attractions . These studies reveal the extensive application and potential of virtual reality and augmented reality technologies in the field of cultural heritage and museums.
As for the integration of loom models with modern technology, some studies have been done in academia. For instance, Portales C and others conducted research on the design and implementation of SILKNOW's virtual loom, an interactive tool that records, preserves, and displays historical weaving and knitting techniques . Yuma Taru and others studied the cultural object unique to the Atayal weaving machine—the weaving box, providing valuable references for designing successful cross-cultural products and the intertwined experience of design and culture during the design process.
Although current studies have explored the possibility of integrating loom models with modern technology, little research has focused on ancient textile machines, especially Dahua Lou loom. Therefore, this research aims to apply virtual reality technology to the restoration and display of Dahua Lou loom. With the continuous development of virtual reality and computer technology, we may be able to combine traditional textile cultural heritage virtual experiences with museum exhibitions, providing audiences with a unique and appealing cultural experience.
Construction of ICH knowledge ontology
In order to accurately display the knowledge of the Dahua Lou loom in a virtual environment, this research constructs a thorough and specific knowledge model. It explicitly articulates various concepts and reveals their relationships to support the mapping of 3D models in a virtual space. Compared to product design knowledge models like Function-Behavior-Structure, our Dahua Lou loom knowledge model places more emphasis on systematic integration and knowledge guidance. Ontology, as a method of knowledge representation, can express structural concepts and relationships in knowledge, transcending traditional knowledge representation methods such as dictionaries and taxonomies. Therefore, we chose ontology knowledge tree to map the knowledge model to the 3D model of the Dahua Lou loom, realizing the 3D visualization of Dahua Lou loom knowledge.
In the field of ICH, many scholars have extended the CIDOC CRM model according to different application scenarios. For example, some scholars have built corresponding ICH knowledge bases based on domain ontology, taking drama elements , Indian classical dance , and bronze artifacts  as examples, providing semantic search, mobile program development, and information dissemination services. Based on a deep consideration of the resource characteristics of the traditional weaving machine field and existing ontology construction standards, we have drawn on the research results of the previous project team in constructing ontology in the field of cloud brocade to construct the KO of the Dahua Lou loom.
Thus, this study will further explore the interactive experience of the Dahua Lou loom in future museums by applying virtual reality technology based on the systematic knowledge ontology of the Dahua Lou loom.
Methods and techniques for the integration of ontology and virtual reality
Many studies in the field of ontology and virtual interaction focus on the application of ontology. Among them, some research uses ontology to describe entities and their relationships in virtual reality, to support the reasoning and search functions of VIS. Other researches use ontology to describe user behavior and interaction in virtual reality, to support the personalization and recommendation functions of VIS.
In this study, we choose to use MongoDB as the storage and interacti “specially Dahua Lou on tool for KO, to achieve efficient operation and update of KO. MongoDB is a flexible and scalable NoSQL database that has been widely used for the storage and interaction of KO. Due to its high performance, ease of use and scalability, MongoDB has become an effective tool for handling complex data, ensuring data security and integrating with other systems. For example, in the field of construction, researchers have found that embedding MongoDB into Building Information Modeling in the form of NoSQL can effectively manage and retrieve project experience information . In the field of biomedicine, MongoDB is used to explore effective methods for managing VCF-based genetic variations and OWL-based biological ontologies . In the field of intelligent transportation systems, researchers have combined MongoDB and distributed ledger technology Multichain to create a vehicle-centered information system, providing an effective method for balancing credibility and performance . In the issue of information system security, researchers have developed a method to introduce security mechanisms into MongoDB databases, which can automatically generate solutions for security issues including permission modification, new roles, and view combinations . In the field of health data management, MongoDB is used to store and process large-scale health data, and researchers have designed a MongoDB data storage pattern based on ontology, which has a significant improvement compared to the relational model .
In summary, MongoDB has demonstrated its advantages in handling complex, large-scale data and integrating with other applications in the fields of architecture, biomedicine, intelligent transportation, information security, and health data management. These researches provide a solid foundation for us to achieve efficient operation and update of the Dahua Lou loom KO in Unity.
Overall, the discussion in this chapter provides theoretical and technical foundations for this study, demonstrating the new trends and possibilities in future museum display formats. This study will explore the display formats and interactive experiences of the large flower tower loom in future museums based on these theories and technologies, and in combination with the characteristics of the large flower tower loom.
In this section, the process of integrating the KO of the Dahua Lou loom into the VIS is described in detail.
Construction of Dahua Lou loom knowledge ontology
Building an appropriate Dahua Lou loom KO can effectively express related concepts, reveal intrinsic relationships between knowledge, guide the construction of 3D models of the loom and the production of animation demonstrations, and lay the groundwork for the mapping of knowledge data in the VIS. This comprehensive and structured domain knowledge representation facilitates efficient acquisition, processing, and accumulation of Dahua Lou loom domain knowledge, enhancing the potential for knowledge reuse and sharing.
Knowledge sorting of Dahua Lou loom
The Dahua Lou loom is a textile device suitable for weaving large adornment fabrics, created by ancient Nanjing craftsmen. The Dahua Lou loom is one of the largest and most ingenious machines in the world’s hand-weaving industry. Its wooden frame is 5.6 m long, 4 m high, 1.4 m wide, and composed of 1,924 parts, including bamboo parts and a small amount of iron products. The structure of the Dahua Lou loom can be roughly divided into five major parts: the body, flower tower, opening mechanism, weft beating mechanism, and warp delivery and take-up mechanism. Each part contains a variety of parts, such as the parrot rack, duckbill, dog brain, pig foot, and sheep horn, etc. During the weaving process of the Dahua Lou loom, it usually requires two weavers to operate at the same time. They need to sing according to specific chants and coordinate operations according to this rhythm: threading, leading filaments, picking flowers, and throwing the shuttle, while singing and weaving. Two workers can only complete 7–8 cm of Yunjin in a day, hence the phrase “inch Jin, inch gold” is the best interpretation of this ancient traditional craft. This study attempts to build a KO in the field of the Dahua Lou loom, structurally representing elements such as the mechanical principles of the loom, parts assembly, and raw materials, revealing their relationships to form an interconnected and intercommunicative knowledge network to support understanding and simulation of the Dahua Lou loom VIS. By summarizing and analyzing literature, consulting expert opinions, and referring to existing resources (for example, “Chinese Classified Thesaurus”, national standards, and industry standards), the scope of the Dahua Lou loom field has been defined, and a professional terminology system has been established.To further refine the domain knowledge concepts, this study obtained five representative Dahua Lou loom weaving videos from the Nanjing Yunjin Research Institute and used Labelme  to annotate them. By annotating videos, the corresponding components of each weaving step can be identified. Finally, the collected multi-source heterogeneous data are processed structurally, and the domain knowledge of the Dahua Lou loom determined by this study is shown in Fig. 1. The data collection and integration process involves the combination of video resource processing and domain expert knowledge integration to create a comprehensive and accurate dataset representing the Dahua Lou loom domain knowledge. This dataset serves as the foundation for building a KO and developing a VIS, ensuring the system accurately reflects the complexity and precision of loom operation and weaving technology.
Defining the hierarchical structure and properties of the knowledge ontology
Through the above work, this study collated and integrated the core concepts of the Dahua Lou loom field, determining the six core categories that constitute the basis of the Dahua Lou loom KO: Dynasty, Modern Exhibition Places, Textiles, Raw Materials, Physical Components, and Weaving Steps. This study established a top-down hierarchical structure by grouping concepts into classes and subclasses to reflect their relationships and dependencies within the field. The six core classes represent the main elements and relationships in the field, providing a structured framework for organizing knowledge. Each core class is further divided into subclasses, which represent more specific concepts within the field. For example, the “Weaving Steps” class can be subdivided into various steps involved in the weaving process, such as “Threading”, “Picking Flowers”, and “Throwing Shuttles”. In addition to the hierarchical structure, the construction of the KO also needs to define object properties and data properties to represent relationships between concepts. Object properties are used to represent relationships between entities, while data properties represent relationships between entities and data values. Different classes contain corresponding properties and relationships. For example, the Weaving Steps class has relationships with Physical Components (usesPart), Textiles (producesProduct), and Raw Materials (requiresMaterial). The properties of each class include name, description, type, source, function, order, etc. These properties provide additional information about the entity, which is crucial for capturing the complex interrelationships in the KO. The main relationship properties in the KO are shown in Table 1. By constructing the structure of domain knowledge, a clear, hierarchical expression of Dahua Lou loom domain knowledge is provided. This structure captures the main elements and relationships within the field, providing strong support for the visualization of the KO and the integrated application of virtual interaction.
Implementation and management of knowledge ontology
To construct the conceptual architecture of Dahua Lou loom domain knowledge, this study used the ontology modeling tool Protégé to gradually establish a hierarchical structure of categories . Solid lines indicate the relationship between subclasses and instances, while dashed lines indicate attribute relationships. The ontology relationship diagram clearly displays the framework structure of the constructed ontology, as shown in Fig. 2. After completing KO modeling, it is stored in the form of Web Ontology Language (OWL) . OWL files are based on XML syntax and provide a rich array of elements and attributes to describe concepts, attributes, and relationships in the ontology.To realize the virtual visualization of Dahua Lou loom knowledge, this study used the dotNetRDFFootnote 2 ontology parsing library to parse the OWL file, converting it into computer-readable and executable C# code. The ontology parsing program acts as a bridge between the ontology file and the development of the VIS, parsing the Dahua Lou loom ontology text into a computer-executable program for implementing subsequent virtual interaction development. The entire ontology parsing process is divided into three steps: (1) loading the OWL ontology file using the dotNetRDF library, extracting information such as types, instances, and attributes from the ontology; (2) building C# classes and attributes corresponding to ontology concepts and attributes. For example, the “Dynasty” class corresponds to the C# class Dynasty, whose attributes include name, start date, and end date, etc..: (3) traversing instances in the ontology, converting them into C# objects, and adding the parsed C# objects to the ontology data structure. This process realizes the conversion from ontology to object model and provides knowledge reasoning, searching, categorization, and communication functions for subsequent virtual interaction development.To ensure the efficient operation and updating of KO in the Unity development environment, this study adopted MongoDB  as the storage and interaction measure. MongoDB provides a document-based data storage method suitable for storing entities, relationships, and attribute information. This study used the official MongoDB C# driver to connect to the MongoDB database, enhancing query and replication speed and reliability through index and replication technology, caching, and optimization algorithms. To increase the system's scalability, this study used a distributed MongoDB architecture to store the parsed OWL ontology. These measures not only facilitate the storage and access of parsing results but also greatly improve the performance, stability, and scalability of the VIS.
3D modeling and animation production of Dahua Lou loom guided by ontology
Under the guidance of KO, 3D modeling tool 3ds Max and animation tool Blender were used to realize the 3D modeling and animation demonstration of the Dahua Lou loom .
Mapping relationship between knowledge ontology and 3D modeling and animation
The close association between the Dahua Lou loom KO and 3D modeling and animation is expressed through the knowledge mapping between the loom KO and the 3D model [31, 32]. Please see Table 2 for details. Specifically, mapping relationships are established between concepts such as entity components, weaving steps, and attributes in the KO and the 3D model and animation elements to ensure the authenticity and reliability of models and animations. This mapping relationship enhances our understanding of the working principle of the Dahua Lou loom. In practice, these mapping relationships not only ensure the accuracy, consistency, and logic of 3D models and animations but also improve the quality of 3D modeling and animation demonstrations of the Dahua Lou loom.
Implementation of 3D modeling and animation
This study used 3ds Max  and Maya skeleton modeling technology  to model and animate the Dahua Lou loom. After lighting, texture, and particle system rendering, the model and animation files containing dynamic parameters and static information were exported and imported into the Unity development engine for material and script processing. During the modeling process, each component is an independent interactive object. The positioning and accuracy of each part need to be considered, modeled separately, without reassembling the parts. A 3D model of the Dahua Lou loom was constructed based on the entity components and weaving step information in the KO. In Blender, an animation demonstration was created to show various steps in the weaving process and the operation principle and internal structure of the loom . Ultimately, eight weaving animations of the Dahua Lou loom were produced, such as threading, wefting, flower collecting, and winding, as shown in Fig. 3. This method provides a new perspective and method for research and application in the loom field. In the future, the content of the KO can be further expanded to optimize and improve 3D modeling and animation demonstrations to meet more application needs.
Integration of ontology and virtual loom interaction system in unity
The system architecture for the integration of ontology and the Dahua Lou loom's VIS includes five key components: Unity project, parsed ontology data, User Interface (UI)  components, 3D models, and event listeners and handlers. Ontology data, as the source of system knowledge, undergoes a series of parsing and storage processes and is responsible for organizing the structure and behavior of UI components and 3D models. Accurate and context-related responses are realized by designing event listeners and handlers. Figure 4 provides an overview of the architectural integration of ontology and the VIS. In this architecture, the Unity project, as the central hub, hosts the virtual environment and 3D models, while connecting parsed ontology data, UI components, and event listeners and handlers . The figure shows the relationships and interactions between different components and presents the flow of information and control among them.
Implementation of interactive functionality
Parsed ontology data plays two key roles in the VIS: firstly, it populates the UI components such as info panels, labels, and interactive elements.secondly, it drives 3D model animations to accurately represent the virtual loom and its weaving steps.Utilizing the hierarchical structure of ontology knowledge facilitates browsing and searching knowledge content in the UI component filling and provides a reasonable user interaction interface in the form of menus and lists. Dynamic assignment and content updates for UI components can be achieved by scripting logic code. The parsed ontology data structure bridges the UI components and ontology knowledge data. To establish correspondence and interaction between UI components and knowledge elements, this study created a Canvas component in Unity3D, which controls the presentation of a group of UI elements, where all UI elements must be subclasses of Canvas. The Canvas structure is akin to a Jacquard loom instance, placing elements such as dynasty, fabric, and entity components on the same level within the canvas, and programming control functions for them.Additionally, in the VIS, parsed ontology data is used to drive and control virtual loom animation interactive behavior. Ontology data records detailed information about weaving steps and corresponding animations. By assigning accurate properties and behaviors to models and animations based on loom knowledge, a realistic, accurate representation of the Dahua Lou loom and its weaving process is achieved. A close connection exists between loom knowledge and animation interactive behavior, which can be implemented through the parsed ontology data structure and trigger and control loom animation interactive behavior, allowing users to explore and experience the loom operation process more deeply.
Event listening and response mechanism
The event listening and response mechanism in the VIS plays a key role in the interaction process between users and models . In close collaboration with ontology, the system achieves data referencing and updating, logical judgment and execution, and feedback generation, thereby improving the practicality of the system and user experience. Ontology, as the core of the system, contains structured information about the Dahua Lou loom and its weaving process. The event listening and response mechanism is responsible for handling user interaction behavior and providing users with real-time feedback based on the data provided by ontology. Firstly, the system adds event listeners to 3D models and UI components based on the data parsed from ontology. These listeners monitor user interaction behavior with objects in the virtual environment, such as clicking, dragging, and hovering. When interaction behavior is captured by the listeners, event handlers are triggered, and corresponding handling programs are called based on the received interaction behavior. In the event handling program, ontology data is referenced and logical judgment and operation are conducted based on the data. Specific operations and associations are shown in Table 3. For instance, when a user selects a model, the handler displays detailed data about the model based on the category and attribute information in the ontology. When the user adjusts model parameters, the handler updates the model state and recalculates the results based on the attributes, relationships, and action information in the ontology.
Assessment and discussion
The present study conducted a comprehensive evaluation of the KO and the design of virtual interactive experience for Dahua Lou loom, based on ISO9241-11 international standard [38, 39], with the aim to provide improvement guidance for subsequent system design and development.
Assessment of knowledge ontology
We conducted a thorough evaluation of the KO, including preprocessing, integrity, consistency, and applicability. The Owlready2 library was used for preprocessing, loading the ontology through get_ontology("Dahualou_loom.owl").load(), and preparing for subsequent evaluations.
The integrity evaluation model for Dahua Lou loom KO was designed in this study, which includes defining integrity constraint rules, checking ontology instance data, and handling abnormal data. These steps are performed using the Python library Owlready2. The specific integrity evaluation process is as follows: (1) For class integrity, the KO successfully covered 93.5% of the classes in the source data; (2) In the integrity assessment of object properties, the ontology covered 85.7% of the object properties; and (3) In the data property integrity assessment, the ontology covered 87.5% of the data properties. Overall, the integrity assessments of the classes, object properties, and data properties of the ontology all exceeded 80%, laying a solid foundation for subsequent research.
Protege and its built-in Hermit reasoner were used in this study for consistency checking. The checking process includes: importing the ontology, setting the reasoner, starting the reasoner, and debugging based on feedback. The specific consistency evaluation process is as follows: (1) Consistency of the class hierarchy: We checked whether the class hierarchy in the ontology is reasonable, such as confirming that all classes have superclasses to ensure that no isolated classes exist. The evaluation results show that all classes have superclasses; (2) property consistency: We verified whether the same property has the same definition and meaning in different classes, and whether the domain and value range of the property are appropriate to ensure the ontology's consistency and accuracy in describing various concepts. The evaluation results found that the domain coverage of data properties is complete, and the range coverage rate is 85.7%; and (3) relationship consistency: We checked whether the relationships in the ontology meet expectations, especially the domain and range situations of object properties, to ensure that the ontology can correctly represent relationships between entities. The results show that there is no missing situation in the domain and range of object properties. The evaluation results show that the ontology demonstrates good consistency in aspects such as class hierarchy, property definition, and relationship expression.
The ontology’s application credibility was evaluated in this study by a set of questions and standard answers set by experts. Each question is converted into a query statement and executed in the ontology. Then, application credibility is derived by analyzing matching concept pairs and comparing concept relationships. The evaluation process includes: parsing the question set, parsing the verification set, obtaining query results, and comparing results. This study can verify whether the ontology meets the actual application requirements from the perspective of user-concerned questions or knowledge content, thus evaluating the application credibility of the ontology. The query questions are transformed into statements recognizable by the ontology, mainly including querying all properties of a class, the attribute relationship between two concepts, and the hierarchical relationship. The result verification stage needs to compare the verification content and query results and calculate their similarity. These query statements can be mainly divided into three types: (1) Query all attributes of a certain class; (2) Query the attribute relationship between two concepts; and (3) Query the hierarchical relationship between two concepts. Specific query question examples and their corresponding query types are listed in Table 4. The result verification stage needs to compare the verification content and query results and calculate their similarity, which serves as the basis for evaluating application credibility. The specific calculation method is as follows: For query types (1) and (2), as the results are stored in the form of a character array, the credibility of the query results of each question can be determined by the proportion of successfully matched elements in the result set to the number of elements in the verification set. For query type (3), since the result is stored in the form of an enumerated type, the credibility of the question result can be 1 (the same result) or 0 (different results).
Through these steps, we can calculate the credibility of each question. Finally, the average credibility of all questions is used as the evaluation result of the ontology's application credibility. The final application credibility calculation formula is as follows:
where cred(application) represents the application credibility of the ontology, Q represents the question set, q represents a certain question in the question set, N(Q) represents the number of questions in the question set, and S(q) represents the similarity between the query result and the verification result of a certain question.
In this study, we referred to the probability interval commonly used in fuzzy decision and classification problems, set four credibility levels, and evaluated the application credibility of the ontology in a more intuitive and quantifiable way. The evaluation results are expressed in four levels: ‘Highly Credible’ (0.75–1), ‘Credible’ (0.50–0.75), ‘Less Credible’ (0.30–0.50), and ‘Incredible’ (0–0.30). In this study, the application credibility of the ontology reached 0.90, which belongs to the 'Highly Credible' level.
Assessment of the virtual interaction system
A deep analysis of the VIS was conducted in this study, with the evaluation system primarily including the dimensions of utility, efficiency, and satisfaction.
System performance evaluation
The system performance evaluation is a key part of the efficiency dimension of the feasibility assessment, focusing on objective indicators such as system response efficiency, animation accuracy, and UI functionality. Multiple tests were performed, and averages calculated to assess the system's response time during startup, loading, and user interaction. The data showed an average system startup time of 2.1 s, loading time of 3.7 s, and user interaction response time of 0.7 s, indicating stable technical indicators, acceptable system response efficiency, and important technical application value.Secondly, the animation's accuracy, fluidity, and visual effects during the simulated weaving process were evaluated. Qualitative assessments of the animations showed smooth, accurate, and aesthetically pleasing animations, improving the virtual weaving experience and quality.Lastly, the effectiveness of UI components in promoting user interaction was assessed by measuring the time and success rate of users completing specific tasks. Data showed a high user task success rate of 92%, indicating the design of UI components in this virtual interactive system meets user needs and provides a high level of system functionality.
User satisfaction evaluation
User satisfaction evaluation is a major part of the satisfaction dimension of the feasibility assessment. After ensuring the loom's virtual interactive system demo was error-free, 20 volunteers were invited for a user experience test to measure their satisfaction with the system. The volunteers, a balanced mix of genders and academic backgrounds, were assessed to ensure they had a similar knowledge base of the Dahua Lou loom, preventing gender and background knowledge from affecting the results. In a quiet, well-lit environment, volunteers completed a series of tasks related to the Dahua Lou loom, such as system environment exploration, loom component examination, and learning weaving steps. Post-test, they filled out a questionnaire regarding system usability, interactivity, and comprehensibility. This aimed to collect comprehensive feedback on the virtual loom experience system, providing direct data about system satisfaction. The average user satisfaction score was 4.2 out of 5, indicating a generally positive response.
After a comprehensive assessment of KO and VIS, the results were integrated and analyzed, and an in-depth discussion was conducted. The KO showed good performance in terms of completeness, consistency, and applicability, demonstrating a high level of practicality. However, it also exposed issues related to data coverage and handling complex relationships, which urgently need further optimization. As for VIS, the system performance and user satisfaction were evaluated as good, but the response efficiency and interface design need to be improved. User feedback emphasized the importance of intuitive design. In subsequent research, we will gain a deeper understanding of user needs and habits, and improve and update the interface and KO, to optimize user experience and enhance system performance. In summary, this study has achieved initial results in the construction of KO and VIS. Although there are flaws, through continuous iterative optimization, it is expected to better serve the research and inheritance of traditional crafts such as Dahua Lou loom.
This study proposed and implemented a collaborative method based on KO for interactive development of the Dahua Lou loom system in a VR environment. By integrating domain KO, VR technology, and data-driven technology, the efficient and accurate design of the VIS for the Dahua Lou loom was facilitated. The experimental results showed significant advancements in domain knowledge modeling, 3D modeling and animation technology development, and the comprehensive application of VR interaction systems. Furthermore, the positive feedback received from domain experts and users demonstrates that this method offers an intuitive, immersive, and informative design experience.During the design process, this method carefully considered the domain knowledge, constraints, and requirements of the loom, thus minimizing the chances of errors and sub-optimal designs. Its interactive nature allows designers to experiment with and optimize different configurations in real-time, stimulating creativity. Moreover, the collaborative aspect of this method enhances effective communication and knowledge sharing between domain experts, engineers, and software developers, leading to more informed decisions and more effective design outcomes.Despite the satisfactory results of this study, there are some limitations. First, the KO may not cover all knowledge in the Dahua Lou loom domain. Second, due to hardware and software limitations, the virtual reality-based interactive system may not be suitable for all potential users. Finally, the evaluation scope of this method is limited.To further improve and advance this study, subsequent research should focus on the following four aspects. First, improving and expanding the KO based on the opinions and practical experiences of domain experts. Second, enhancing system accessibility and compatibility to reduce hardware and software restrictions and improve system usability. Third, future research should involve larger-scale evaluations, including a wider range of volunteers, more diverse design scenarios, and more comprehensive evaluation dimensions. Lastly, further exploring the application of this collaborative method in other areas of the Dahua Lou loom (such as manufacturing, maintenance, and training), and considering extending this method to the design and development of other complex systems. This will provide technical support and guidance for workers and researchers in related fields and promote the protection and inheritance of cultural heritage.
Availability of data and materials
Data available on request from the authors.
Intangible cultural heritage
Virtual interaction system
- GLAM Labs:
Galleries, Libraries, Archives, and Museums
Web Ontology Language
Lu L, Liang X, Yuan GT, Jing LL, Wei C, Cheng C. A study on the construction of knowledge graph of yunjin video resources under productive conservation. Herit Sci. 2023. https://doi.org/10.1186/s40494-023-00932-5.
Ciurea C, Filip FG. Virtual exhibitions in cultural institutions: useful applications of informatics in a knowledge-based society. Stud Inform Control. 2019;28(1):55–63. https://doi.org/10.24846/v28i1y201906.
Montagud M, Orero P, Matamala A. Culture 4 all: accessibility-enabled cultural experiences through immersive VR360 content. Pers Ubiquit Comput. 2020;24(6):887–905. https://doi.org/10.1007/s00779-019-01357-3.
Jones D, Snider C, Matthews J, Yon J, Barrie J, Robinson K, Hicks B. Model-based information navigation for engineering documents. Comput Ind. 2020. https://doi.org/10.1016/j.compind.2020.103254.
Jasko S, Skrop A, Holczinger T, Chovan T, Abonyi J. Development of manufacturing execution systems in accordance with industry 4.0 requirements: a review of standard- and ontology-based methodologies and tools. Comput Ind. 2020. https://doi.org/10.1016/j.compind.2020.103300.
Kalita D, Deka D. Ontology for preserving the knowledge base of traditional dances (OTD). Electronic Libr. 2020;38(4):785–803. https://doi.org/10.1108/EL-11-2019-0258.
Candela G, Escobar P, Carrasco RC, Marco-Such M. A linked open data framework to enhance the discoverability and impact of culture heritage. J Inf Sci. 2019;45(6):756–66. https://doi.org/10.1177/0165551518812658.
Kim O, Jayaram U, Zhu LJ. A unified strategy to integrate information and methods across multiple training environments for assembly simulations. J Comput Inform Sci Eng. 2014;14(3): 031001. https://doi.org/10.1115/1.4027225.
Chokwitthaya C, Zhu YM, Lu WZ. Ontology for experimentation of human-building interactions using virtual reality. Adv Eng Inform. 2023. https://doi.org/10.1016/j.aei.2023;101903.
Fuchs M, Beckert F, Biedermann J. A collaborative knowledge-based method for the interactive development of cabin systems in virtual reality. Comput Ind. 2022. https://doi.org/10.1016/j.compind.2021.103590.
Dong M, Zeng XY, Koehl L, Zhang JJ. An interactive knowledge-based recommender system for fashion product design in the big data environment. Inf Sci. 2020;540:469–88. https://doi.org/10.1016/j.ins.2020.05.094.
Liu X, Jiang D, Tao B. Genetic algorithm-based trajectory optimization for digital twin robots. Front Bioeng Biotechnol. 2022. https://doi.org/10.3389/fbioe.2021.793782.
Lee H, Jung TH, Dieck MCT, Chung N. Experiencing immersive virtual reality in museums. Inform Manag. 2020. https://doi.org/10.1016/j.im.2019.103229.
Carrozzino M, Bergamasco M. Beyond virtual museums: experiencing immersive virtual reality in real museums. J Cult Herit. 2010;11(4):452–8. https://doi.org/10.1016/j.culher.2010.04.001.
Hu QW, Yu DB, Wang SH, Fu CW, Ai MY, Wang WD. Hybrid three-dimensional representation based on panoramic images and three-dimensional models for a virtual museum: data collection, model, and visualization. Inf Vis. 2017;16(2):126–38. https://doi.org/10.1177/1473871616655467.
Errichiello L, Micera R, Atzeni M, Del Chiappa G. Exploring the implications of wearable virtual reality technology for museum visitors’ experience: a cluster analysis. Int J Tour Res. 2019;21(5):590–605. https://doi.org/10.1002/jtr.2283.
Portales C, Perez M, Casanova-Salas P, Gimeno J. Virtual loom: a tool for the interactive 3D representation of historical fabrics. Multimed Tools Appl. 2021;80(9):13735–60. https://doi.org/10.1007/s11042-020-10294-w.
Lombardo V, Pizzo A, Damiano R. Safeguarding and accessing drama as intangible cultural heritage. ACM J Comput Cult Herit. 2016. https://doi.org/10.1145/2812814.
Jain N, Bansal V, Virmani D, Gupta V, Salas-Morera L, Garcia-Hernandez L. An enhanced deep convolutional neural network for classifying Indian classical dance forms. Appl Sci Basel. 2021. https://doi.org/10.3390/app11146253.
Isa WM, Zin NA, Rosdi F, et al. An ontological approach for creating a brassware craft knowledge base. IEEE Access. 2020;8:163434–46. https://doi.org/10.1109/ACCESS.2020.3022795.
Oti AH, Tah JHM, Abanda FH. Integration of lessons learned knowledge in building information modeling. J Constr Eng Manag. 2018. https://doi.org/10.1061/(ASCE)CO.1943-7862.0001537.
Liu J, Qu Z, Yang M, Sun JL, Su SH, Zhang L. Jointly integrating VCF-based variants and OWL-based biomedical ontologies in MongoDB. IEEE ACM Transact Comput Biol Bioinform. 2020;17(5):1504–15. https://doi.org/10.1109/TCBB.2019.2951137.
Distefano S, Di Giacomo A, Mazzara M. Trustworthiness for transportation ecosystems: the blockchain vehicle information system. IEEE Transact Intell Transp Syst. 2021;22(4):2013–22. https://doi.org/10.1109/TITS.2021.3054996.
Mate A, Peral J, Trujillo J, Blanco C, Garcia-Saiz D, Fernandez-Medina E. Improving security in NoSQL document databases through model-driven modernization. Knowl Information Syst. 2021;63(8):2209–30. https://doi.org/10.1007/s10115-021-01589-x.
Sen PS, Mukherjee N. An ontology-based approach to designing a NoSQL database for semi-structured and unstructured health data. Cluster Comput. 2023. https://doi.org/10.1007/s10586-023-03995-y.
Li XL, Mou LC, Lu XQ. Video parsing via spatiotemporally analysis with images. Multimed Tools Appl. 2016;75(19):11961–76. https://doi.org/10.1007/s11042-015-2735-x.
Cheng YJ, Chou SL. Using digital humanity approaches to visualize and evaluate the cultural heritage ontology. Electron Libr. 2022;40(1–2):83–98. https://doi.org/10.1108/EL-09-2021-0171.
Wu CK, Wu P, Wang J, Jiang R, Chen MC, Wang XY. Ontological knowledge base for concrete bridge rehabilitation project management. Autom Constr. 2021. https://doi.org/10.1016/j.autcon.2020.103428.
Sorokina M, Merseburger P, Rajan K, Yirik MA, Steinbeck C. COCONUT online: collection of open natural products database. J Cheminform. 2021. https://doi.org/10.1186/s13321-020-00478-9.
Durrant JD. Prot2Prot: a deep learning model for rapid, photorealistic macromolecular visualization. J Comput Aided Mol Des. 2022;36(9):677–86. https://doi.org/10.1007/s10822-022-00471-4.
Lee PC, Xie W, Lo TP, Long DB, Tang XF. A cloud model-based knowledge mapping method for historic building maintenance based on building information modelling and ontology. KSCE J Civil Eng. 2019;23(8):3285–96. https://doi.org/10.1007/s12205-019-2457-0.
Mantovani A, Piana F, Lombardo V. Ontology-driven representation of knowledge for geological maps. Comput Geosci. 2020. https://doi.org/10.1016/j.cageo.2020.104446.
Flotynsk J, Walczak K. Conceptual knowledge-based modeling of interactive 3D content. Vis Comput. 2015;31(10):1287–306. https://doi.org/10.1007/s00371-014-1011-9.
Liang H, Deng SJ, Chang J, et al. Semantic framework for interactive animation generation and its application in virtual shadow play performance. Virtual Real. 2018;22(2):149–65. https://doi.org/10.1007/s10055-018-0333-8.
Chang MML, Nee AYC, Ong SK. Interactive AR-assisted product disassembly sequence planning (ARDIS). Int J production Res. 2020;58(16):4916–31. https://doi.org/10.1080/00207543.2020.1730462.
Buyuksalih G, Kan TN, Ozkan GE, Meric M, Isin L, Kersten TP. Preserving the knowledge of the past through virtual visits: From 3D laser scanning to virtual reality visualisation at the Istanbul catalca Incegiz caves. PFG J Photogramm Remote Sens Geoinform Sci. 2020;88(2):133–46. https://doi.org/10.1007/s41064-020-00091-3.
Naik H, Bastien R, Navab N, Couzin ID. Animals in virtual environments. IEEE Transact Vis Comput Graph. 2020;26(5):2073–83. https://doi.org/10.1109/TVCG.2020.2973063.
Zamora-Musa R, Velez J, Paez-Logreira H. Evaluating learnability in a 3D heritage tour. Presence. 2017;26(4):366–77. https://doi.org/10.1162/PRES_a_00305.
Cao HK, Zhang ZG, Evans RD, et al. Barriers and enablers to the implementation of intelligent guidance systems for patients in Chinese tertiary transfer hospitals: usability evaluation. IEEE Transact Eng Manag. 2021. https://doi.org/10.1109/TEM.2021.3066564.
This research was supported by experimental data created in the Nanjing YunJing Institute. We also thank the ICH bearers of Nanjing YunJing for semantic annotation and data verification for this study. At the same time, we would like to express our deepest gratitude to the anonymous reviewers, whose comments improved this work tremendously.
This research was supported by the Jiangsu Provincial Social Science Foundation Project “Research on Knowledge Extraction and Organization of Nanjing Cloud Brocade Video Resources Based on Knowledge Meta” (Project No. 21TQC006). Jiangsu Provincial University Social Science Foundation Project “Research on the Implementation Path of the Living Heritage of Cloud Brocade Values in the DH Perspective” (Project No. 2021SJA0135).
The authors claim there is no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Lu, L., Li, M. Development of a virtual interactive system for Dahua Lou loom based on knowledge ontology-driven technology. Herit Sci 11, 178 (2023). https://doi.org/10.1186/s40494-023-01027-x
- Dahua Lou Loom
- Knowledge ontology
- Virtual reality
- Yunjin weaving
- Cultural heritage protection