Visitor’s experience evaluation of applied projection mapping technology at cultural heritage and tourism sites: the case of China Tangcheng

Abstract

Research on digital cultural heritage is concerned with the implementation of projection mapping (PJM) technologies, projection viewing, and interactive programs at cultural heritage sites. As PJM technology has come to play an increasingly important role in attracting visitors to museums, heritage pavilions and heritage sites, the topics of digital cultural heritage and digital cultural tourism have become widely discussed in professional and academic circles. However, questions have begun to emerge over the past decade about the relevance of the content presented using PJM at heritage sites and tourist attractions to the sites’ cultural value, and various researchers have attempted to evaluate the effectiveness of PJM on the visitor experience and generate proposals for improvement. Unfortunately, the usefulness of these attempts has been limited by several methodological shortcomings. Therefore, this study proposes an original system for evaluating visitor’s cultural experiences. By evaluating the effectiveness of PJM on visitor’s cultural experiences, a methodology and a set of guidelines for applying PJM that promotes cultural understanding were proposed, and further to achieve an integrated understanding of visitor’s tendency to recall PJM information. Furthermore, a trial run of the system was conducted by the authors in a study of a digital media campaign in October 2021 and the data derived from this investigation are presented in this article as a reference point for comparable cultural heritage and tourism sites.

Graphical Abstract

Introduction

With advances in technologies such as immersive and projection mapping (PJM) and virtual reality (VR) and the widespread use of such technologies in the design and implementation of cultural heritage sites, the concepts of digital cultural heritage have been widely discussed in academic and scientific circles [1, 2]. The significance of applying digital technologies to cultural heritage conservation has been discussed to various degrees since 1990, and PJM, VR, and mixed reality (MR) technologies became the centerpiece of digital culture interpretation around 2010 [3, 4]. Meanwhile, because of its advantages in terms of cost-effectiveness, reusability, and versatility, PJM technology has become the preferred approach for cultural heritage sites [5].

However, over the past decade, conflicts and controversies about the visitor experience and cultural representation at such sites have begun to emerge due to questions about the relevance of the content of PJM used at heritage sites and tourist attractions to the sites’ cultural value [6]. Following the success of events an increasing number of designers began to wonder “whether visitors genuinely perceive the culture at a PJM event as the designers intended” [7]. Accordingly, several professional and prospective developers have attempted to analyze the visitor experience with a view to evaluating the effectiveness of PJM on the cultural experience and to thus generate proposals for improvement and redesign [8]. However, the methods of feedback collection and questionnaire design used in such studies—both commercial and administrative—had shortcomings in terms of factors such as incompleteness and lack of specificity, as well as treating visitors to heritage sites as passive observers [9, 10].

Regarding research on the visitors’ experience in cultural heritage tourism sites, although most studies focused on topics unrelated to PJM, the analysis methods and theoretical underpinnings are nevertheless well worth referencing.The American Customer Satisfaction Index (ACSI) has developed a model to measure visitor satisfaction with history and culture [11]. The ACSI model has been used in numerous studies on evaluate satisfaction and loyalty in heritage tourism. With reference to the concept of “loyalty” (The tourist can evaluate their level of satisfaction at the destination once he has internalized the perceived value [12]. The level of satisfaction will determine if you recommend and revisit to the location), some academics have used structural equation modeling (SEM) analysis to examine visitor satisfaction and “loyalty” at the Citadel of the Catholic King in order to understand the connection between “perceived cultural quality” and “perceived heritage quality” [13]. Meanwhile, there also have research focused on the emotional, psychological and physical impact that the “smell” of heritage sites exerts on visitors, suggesting that the smell of heritage sites directly influences the ways in which visitors understand and engage with historical and cultural information [14]. This study argues that both “smell” and “projection” can be recognized as an external influence or stimulation on a particular sense of the human body to achieve a specific experience of culture. Therefore, it is essential to calibrate the relationship of these external factors to the influence of the visitors’ experience through a scientific methodology.

Furthermore, a significant amount of research indicates that as world heritage sites continue to improve their construction and tourism experience, plenty of visitors are no longer satisfied with the ordinary “strolling” experience, instead inclining to learning more about historical and cultural knowledge, enriching the cognition of national and world history, and knowledge-based tourism experience during their visit [15,16,17]. This allows cities with cultural heritage sites to become more competitive in the development of the tourism industry [18, 19]. Meanwhile, heritage site tourism is also developing a significant social and interactive dimension in some countries or regions. Researchers in Japan have thoroughly studied the “festival” events held at cultural heritage sites and have clearly demonstrated that it can foster relationships between locals and visitors, improve the visitors’ experience, and disseminate cultural information [20]. This study argues that there is a logical coherence between the practice of holding “festival events” and the application of PJM at heritage tourism sites, both approaches are based on a cultural presentation to generate interest to the heritage sites and to disseminate cultural understanding, and there are instances of regional cultural performances that incorporate PJM in Japanese “festival events” in existence.

Nevertheless, despite there is currently an increasing number of articles on digital cultural heritage, they mostly concentrate on technological developments. Studies tend to mention “the benefits of PJM for cultural heritage sites,” but they rarely concern themselves with the specific intellectual information that visitors may acquire through such technologies or any negative experiences they may have with them [21]. Since 2020, the theme of developing a system for evaluating cultural heritage sightseeing offerings has begun to attract interest [22]. The concept of “proven precision in value evaluation” was mentioned during the third conference on “Digital Cultural Heritage: FUTURE VISIONS” [23]. Later the same year, an evaluation system based on a cultural heritage site project in the old city of Zuoying, Taiwan, was designed [24]. However, such evaluation systems emphasize visitor’s adaptability and engagement with digital media technology and tend to collect little information related to the specific knowledge visitors may acquire through the use of such technologies or any negative experiences they may have with them [25].

In the period 2020 to 2021, the authors begin to address the issue of the information tendency conveyed in the heritage sites’ PJM cases [26]. However, research on the use of digital media technology in heritage sites has concentrated on technical applications, heritage conservation and restoration, only limited research on the cultural information and the information tendency conveyed by PJM. The author contends that regulated design guidelines and scientific analysis for PJM information can serve as a benchmark for innovation and advancement for all those practitioners in the industry. This can also help to address the issue that PJM content is not relevant to heritage history and culture. Therefore, after the case studies of PJM events in 45 heritage sites around the world,the author summarizes the information tendency conveyed by PJM events in heritage sites, classifies the types of PJM information displayed with high frequency in building-surface PJM and medium-sized PJM programs, and discussed the reasons for and responses to several special instances, providing a preliminary guideline for the design of PJM improvements. However, the guidelines still need to be discussed and improved from the perspective of the visitors’ experience because the results do not consider the feeling of visitors as information receivers. The results do not refer to the question of which information types of visitors prioritize to remember, and which information leads to negative experiences for visitors.

Therefore, this study aims to propose an original system for evaluating visitor’s cultural experiences, to get an in-depth perspective on the effectiveness of PJM for visitors’ experience. By referring to the data result from a trial run of the system, a methodology for applying PJM that promotes cultural understanding were proposed. Furthermore, this study also achieves an understanding of visitor’s tendency to recall PJM information through the data analysis, for providing a set of guidelines to improve the design of PJM in heritage and tourist sites.

Methods

Interview survey and analysis

To understand the potential influence of PJM on the visitor’s cultural experience and complete the design of evaluating system, seven designers who are currently engaged in digital media design careers and three developers of digital media technology-related companies were interviewed for this study [27].

The study first decided to take “attitudes of developers and designers towards visitors’ experience” as a starting point to gradually understand issues like “difficulties in the current design and development work,” “most concerned visitors’ feedback,” “expectations and problems,” and to summarize and analyze the points of agreement.

By referring to a professional reference book on interview surveys [28], this study set 10 basic interview survey questions, six of which were to be answered jointly by designers and developers, two specifically for designers and two specifically for developers, i.e., eight questions for each respondent. Meanwhile, only three more representative designers’ responses are analyzed in-depth here to make the analytic process more efficient. Furthermore, considering the language preference of the respondents in Chinese, Japanese and English, this study adjusted the interview process and the actual interview content and sequence of questions varied slightly from the written plan. Therefore, the assembled interview questions here only show the abbreviated version in English (Table 1).

Table 1 Interview survey questions

According to the analysis of Tables 2 and 3, we can summarize the following 6 key reference intelligence:

• The industry is gradually diverting the attention to visitors’ experience surveys, but commonly lack a methodical and scientific methodology.

• “Overall experience and PJM comments,” “Impressive cultural knowledge and scenes,” “Negative comments on various aspects,” and “Post event publicity” are the sorts of visitor feedback that designers/developers are most concerned about.

• The necessity for “historical rigor” also inevitably limits the creativity and efficiency of the designer while creating historically and culturally relevant PJM content. This is also the reason why some of the heritage sites PJM events rarely display cultural and historical information. Therefore, we need to explore the result of visitors’ tendency to recall cultural information from PJM with scientific methods, to help designers filter out “non-core” but “design-challenging” cultural information.

• The practitioners consider PJM to be a technological approach sufficient to improve the way of cultural dissemination and interpretation of heritage sites. Therefore, it was necessary for practitioners to have a thorough awareness of the actual effects of PJM on the visitors’ experiences, particularly the condition of knowledge experience, thus provide PJM content that is informative, innovative, and highly visitor relevant.

• Designers and developers rarely transform identity into “experiencers” to review the PJM contents they developed. Therefore, this study highlighting that to ensure the rationality of the questionnaire design, developers and designers should conduct “self-evaluation” before creating a questionnaire to understand the current information tendency of the PJM content.

• Designers are more concerned with the amount of understanding and propensity of visitors to recall the cultural content demonstrated in PJM, while developers are more concerned with the cultural experience and touring comfort of visitors through PJM events.

Table 2 Analysis of developer responses

Table 3 Analysis of designer responses

By understanding the questions currently encountered in their work, the following requirements were formulated as a reference for the system design:

• Developers and designers need to analyze and evaluate the information tendency conveyed by the PJMs they designed,

• How significant is the influencing level of PJM on the visitors’ experience at the cultural heritage site, (including evaluation of PJM, perception of cultural knowledge, negative feelings, willingness to publicize, etc.)

• Which cultural information has the greatest potential to attract visitors, as well as which information is vital but neglected,

• Whether visitors would be willing to introduce the event to others after the tour or to accompany friends and family for another visit.

Timeline analysis method

To provide practitioners with a more efficient way to conduct a “self-evaluation” of their impending PJM programs, this study decides to utilize the authors’ original timeline analysis method to identify the current defects in PJM design using time-scaled data.

The timeline analysis method was developed to deconstruct the PJM content and analyze overall tendencies in the information. This method provides a completely chronological arrangement of the various items of cultural information in a line analytical chart, which helped the developers conduct their self-evaluation in a more scientific and sequential manner.

Such method uses the scene transitions of PJM as time nodes to analyze the content of the cultural information conveyed by PJM in each period. The information type is distinguished by the original color classification system of the authors, which allows the researcher to determine the information tendency of the event based on the overall color propensity. Meanwhile, we decide to superimpose multiple color blocks on top of the primary color blocks to indicate the multiple types of information for the one period. The details of this method refer to Additional file 1: Figure S1, S2.

Design of the evaluation system

Based on the interview survey, this study argues that the questionnaire should be structured around four categories of feedback: “overall PJM experience,” “knowledge experience,” “negative experience,” and “subsequent comments”. Furthermore, to further explore the effects of PJM application on various aspects of the visitors’ experience, it is suggested that the structural equation model (SEM) be used to examine the link between the “overall PJM experience” group’s question data as a base factor and other group factors.

Subsequently, the details of the evaluation system were introduced. The evaluation system has a tripartite structure, with a “pre-investigation and inference hypothesis phase,” a “questionnaire design and data gathering phase,” and an “analysis and discussion phase,” also referred to as “Phase P,” “Phase Q,” and “Phase A” (Fig. 1) [15, 29].

Fig. 1

Framework and procedures of the evaluation system

“Phase P” refers to the pre-investigation phase. During this phase, the investigators were required to conduct an inspection of each site and building where the projection was to be applied [30]. Simultaneously, the investigator was required to conduct a self-evaluation to review the precise information conveyed by the PJM via the authors’ original timeline analysis method.

The timeline analysis method was developed to deconstruct the PJM content and analyze overall tendencies in the information.This method provides a completely chronological arrangement of the various items of cultural information in a line analytical chart, which helped the developers conduct their self-evaluation in a more scientific and sequential manner.

“Phase Q” refers to the formative investigation phase. In this phase, the investigator first needed to categorize the cultural information revealed in the PJM into four categories, “macro,” “micro,” “environment,” and “performance,” and to incorporate information such as question options into the “knowledge experience evaluation” section.

“Macro” refers to information regarding the strict logic of historical chronology and the macro perspective on the epoch, i.e., the historical background and development, particularly the site’s historical origins, significant historical events that took place there, and the process of its eventual restoration; “micro” refers to information regarding independent individual historical narratives and the demonstration of cultural and artistic elements, i.e., costumes, patterns, individual stories, myths, literature, and scenes of daily life in history; “environment” refers to the cultural information contained in the physical materials on site, such as the architectural structures, bridges, spectacular walls, and paintings; and “performance” refers to the information presented in the performance and by the performer. Sorting the information into four categories is a method for deriving data on visitor’s tendencies to recall cultural knowledge and achieve a profound understanding of it [31].

Regarding the rationale for classifying PJM information into the four categories, following extensive case study work, this study concluded that PJM is an approach to gather visitors and convey “generalized” cultural information in a quick period. In this regard, “macro” represents PJM content that summarizes the historical timeline and construction process of the site, while “micro” represents PJM content that summarizes traditional art, artifacts, and other detailed The survey of Phase 2 questionnaire receivedhistorical information. Both “macro” and “micro” are cultural information conveyed by PJM directly and serve to a certain extent replacing the role of signage text, helping visitors to extract cultural knowledge from the information types they are interested in. On the other hand, in “environment” and “performance”, PJM is used as a visual support tool to attract visitors’ attention to exquisite architectural details or dance performances that easily neglected, enabling visitors to extract the cultural information from the physical environment and performance. We can say that the four information categories correspond to the four directions in which visitors extract cultural information.

The next task was to design the questionnaire and this study categorized the common problems into four categories based on the information acquired from the interviews [32, 33]:

• Evaluation of the overall PJM experience (enjoyment/acceptability/sense of atmosphere/comprehension / adaptation)

• Evaluation of the knowledge experience (historical information/relics and architecture/folklore information/ornamental elements)

• Evaluation of negative experience (numbness/tiredness/unclear theme/physical discomfort)

• Evaluation of subsequent comments (enriched knowledge/intention to revisit/public promotion/interest in heritage)

Meanwhile, the study conducted an instant interview survey about one minute after the questionnaire in “Phase Q.” The investigators only needed to ask visitors about any negative feedback or feedback that would conflict with the hypothesized results and were able to extract valuable information by making the most of the period immediately after administering the questionnaire when the visitors still had some patience [34, 35].

Finally, “Phase A” refers to the data analysis phase. The investigators needed to enter the data they gathered into specialized analysis software (SPSS and NVivo) for analysis [36, 37].

On-site questionnaire and data collection

To test the feasibility of the evaluation system, the authors conducted a trial run of the system in a study of a digital media campaign in the Tangcheng national scenic area, Xiangyang City, Hubei Province, China, from 1 to 7 October 2021, of which they were co-investigators. For comfortable and efficient completion of the questionnaire, we decided to input the questions into the professional questionnaire design platform “Wenjuanxing” (a Chinese platform providing functions equivalent to those of Amazon Mechanical Turk), which outputted QR codes for tourists to scan face to face [38] (Fig. 2).

Fig. 2

Partial architecture of Xiangyang Tangsheng (Figure from official promotional video. http://www.zgtcly.com/)

As a site for PJM events, Tangcheng possesses an abundance of projectable vehicles (including large buildings, pagodas, and walls; small buildings, courtyards, and bridges; PJM platforms in the water), and is thus suited to serve as a comprehensive reference for other sites to explore the use of PJM on multiple vehicles [39]. Meanwhile, Tangcheng has been restored to the greatest extent possible to the style, size, materials and spatial layout of the Tang court buildings, and offers a wide range of PJM programs and a scientific touring process, each of the three pillar PJM events has a complete and independent program, ensuring that the PJM experience is not homogeneous and resulting in more generalizable data. Furthermore, the culture of the Tang dynasty has a significant status in the development and spread of Asian culture, as Tang dynasty costumes, architecture, and patterns continue to be widely inherited in Japan, Korea, the US, Australia, and Singapore.Therefore, this study argues that these PJM events, with Tang culture as the centerpiece, are suited to serve as a reference for a study of the use of PJM in cultural dissemination [40].

The Xiangyang Tangcheng cultural scenic spot, also known as “China Tangcheng,” located in Xiangyang city, Hubei province, China, is recognized as one of China’s 4A-level tourist attractions (on a scale that goes up to 5A). A prominent cultural tourism attraction and filmmaking base, the construction, encompassing over 3.7 million square meters [41].

Accordingly, the authors collaborated with the Tangcheng staff to implement the original evaluation system for a digital media event at Tangcheng in October and completed the data collection in cooperation with them. The relevant authorizations for the authors to publish the investigation data in research papers were obtained.

Multivariate analysis of the data

With the aim of verifying the effectiveness of PJM on the visitor’s experience through a hypothesis testing approach, this study will use multivariate analysis methods such as “Structural equation model (SEM)” and “Chi-square test” to further analyze the data [42]. Furthermore, cross-tabulations and “Pareto diagrams” will be used to discuss the visitor’s tendency of recall the PJM information.

Results

Timeline analysis of three major PJM events

Prior to the official investigation, we first disaggregated and analyzed the three main PJM events in Tangcheng through a timeline analysis. The results of the information tendency derived from the timeline analysis can be used as an important reference in questionnaire design and later data analysis.

Tangcheng produces three relevant PJM events: zhu que zhan guo wei (showing the military power of the nation in front of the Zhuque Gate, a building-surface PJM), xun meng hu yu lou (the fantastic Hu Yu citadel, a holographic in-water PJM), and si ji tang qiao (The Tang Dynasty Four Seasons Bridge, a bridge-surface PJM). Considering the higher frequency of mixed performances involving PJM and dance in Tangcheng, it was decided to distinguish the information on dance performances using dotted-line color blocks in the timeline analysis chart.

Building-surface PJM generally tends to be used to display macro-intelligence type information regarding historical development and architectural structures (Fig. 3) and is used especially frequently in military history and architectural reconstruction contexts. Meanwhile, despite providing historical information about the Tang dynasty, the performance lacks a complete story logic and continuity, so the historical and cultural information extracted is quite limited.

Fig. 3

Building-surface PJM timeline analysis chart

In-water PJM is used more often to convey micro-intelligence type information, such as traditional paintings, patterns, artifacts, mythological stories, and opera elements (Fig. 4). In-water PJM uses symbols from ancient Chinese mythology, including phoenixes and dragons, which represent prosperity and power, among others. It is notable that even though dragon and phoenix patterns have been used for a long time in China, their extensive use and spread to the world as a Chinese cultural symbol began during the Tang dynasty [43]. Meanwhile, in- water PJM also exhibits part of Buddhist culture and minority dances, both of which appear in conjunction with the warfare and diplomatic history of the Tang Dynasty [44].

Fig. 4

In-water PJM timeline analysis chart

Similarly, the bridge-surface PJM is a complete combination of dance performance and PJM (Fig. 5). In addition to the demonstration of minority costumes and dances, it also contains numerous fictitious scenes from Tang Dynasty poetry. These scenes serve as an implicit connection to the period in which poetry culture prevailed [45]. It can be argued that the bridge-surface PJM display numerous ornamental cultural elements and indirectly introduce the historical background and the process of compatibility between foreign and Chinese cultures through cultural elements.

Fig. 5

Bridge-surface PJM timeline analysis chart

From the timeline analysis, we understand that Tangcheng’s PJM programs are typical of the “PJM combined with dance performance” format, which is very common and well received in the field.

Through referring the information presented in the timeline analysis, we inferred that “Painting and patterns” and “Folklore, poetry, or operas” comprised a vast majority of information types. In particular, the primary source of both categories of knowledge is the PJM dance performance. Along the same line, the authors argue that the PJM stage performance in Tangcheng represents an excellent showcase of traditional costumes, music, and dance. However, it lacks a degree of historical origins or scenario design, which highly possible to result in visitors only receiving a good cultural aesthetic experience, but not a quality historical knowledge experience. This means the relevant information prioritises “decoration and aesthetics” compared to “cultural connotation and historically significant”.

Data analysis of the Phase 1 questionnaire

The entire questionnaire comprises 25 questions, which were divided into two phases for distribution (Additional file 1: Table S3). The Phase 1 questionnaire contained 20 questions—two for basic information and the others for visitor feedback. Additionally, the Phase 1 questionnaire was distributed at the Tangcheng heritage site between the timeframe of October 1st to October 7th, 2021. Among the collected 1,672 responses, the number of valid responses were 1,101.

With a view to determine the effectiveness of the Tangcheng PJM applications in enhancing the visitor’s experience, this study will adopt the SEM approach to ascertain the actual effectiveness of PJM applications in Tangcheng pertaining to several aspects of the visitor’s experience.

Based on the Phase 1 questionnaire, Q3 to Q8 belong to the “PJM experience evaluation” dimension. Concurrently, all six questions refer to the comprehensive evaluation of the visitor’s PJM experience. Accordingly, we opted for this dimension as the base factor in the SEM analysis to evaluate the likelihood of the PJM experience substantially influencing the “Knowledge experience (Q9 to Q13),” “Negative experience (Q14 to Q16)”, and “Subsequent evaluation (Q17 to Q20).” The initial finding of “factor loading coefficients” as the output is leveraged to confirm the presentability of the variables on the same factor. Moreover, the null hypothesis represents “the variables that cannot be presented on the same factor” (Table 4).

Table 4 Factor loading coefficients

According to the four dimensions of PJM evaluation, each measurement item passed the test with a significant p-value of less than 0.01, while the null hypothesis was rejected. Meanwhile, measurement items possess standard loading coefficients greater than 0.4, thus indicating that the variance is sufficiently explained to present each variable on the same factor.

Subsequently, the required data determining the actual effectiveness of PJM on the visitor’s experience, “model regression coefficients”, (Table 5) is emphasised. In particular, the null hypothesis of this analysis relates to “no correlation between the application of PJM and relevant dimensions of the experience evaluation, thus concluding that the application of PJM has no bearing on visitor experience.”

Table 5 Model regression coefficients

According to Table 3, based on the paired-item “PJM experience evaluation” to “Knowledge experience evaluation,” “Negative experience evaluation”, and “Subsequent evaluation”, the cumulative significance p-value is 0.000***. Under the condition of null hypothesis being rejected, this path is valid, and the impact coefficients are 0.991 (Knowledge experience evaluation), 0.994 (Negative experience evaluation), and 1.0 (Subsequent evaluation), thereby indicating an extremely strong correlation and influence between the factors. Furthermore, the validity of the data refers to ESI for this article (Additional file 1: Table.S4).

Evidently, the data results indicate a significant correlation and influence between visitor’s evaluations of their PJM experiences and other types of experiences. Accordingly, the findings can be interpreted to signify the substantial influence on visitor’s knowledge experiences, negative experiences, and subsequent comments through the integration of PJM.

Meanwhile, the authors postulate likely adverse consequences stemming from the effectiveness of PJM. Specifically, the age discrepancy of visitors could potentially influence the overall effectiveness of PJM [46]. Accordingly, this study aimed at leveraging the chi-square test to explore the relationship between visitor’s age change and cultural tourism experience.

However, this study incorporates only age information as the independent variable and supplementary question data as the dependent variable for the chi-square test.

The null hypothesis of the chi-square test implies that “the evaluation of PJM experience isn’t influenced by age change.” Based on the tabulated results (Additional file 1: Table.S5), the p-values for all questions exceeded 0.100, which failed to present significance on the level while the null hypothesis was accepted, except for Q13. In actuality, the significance p-value of Q13 was 0.028**, which highlighted significance at the level and rejected the null hypothesis. Moreover, the prevalence of a major difference between age and Q13 was deduced (PJM enhanced my comprehension of the Tang Dynasty culture and broadened my horizon on cultural diversity).

With a view to corroborate the data results of Q13, this study plotted a visualisation graph of Q13 for further analysis (Fig. 4).

Evidently, senior visitors are dissatisfied with the PJM content since the proportions of those who “agree” and “agree somewhat” noticeably decline with age (particularly above the age bracket of 60), while the proportion of “undecided” has increased dramatically (Fig. 6). During the interview survey, senior visitors complained that several aspects of ancient philosophy, craftsmanship, and technology were either vaguely referenced or completely disregarded. Likewise, a group of visitors between the age group of 45 to 50 claimed that the historical knowledge was presented in a format that was too “fragmented” or incomprehensible.

Fig. 6

Data for cross-tabulate of question 13

Combined with the timeline analysis (Figs. 4, 5, 7), this study argued that even though PJM displayed many cultural elements with historical characteristics (motifs, sculptures, costumes, etc.) during the stage performance, the lengthy stage performance caused some senior visitors to lose patience. Take the case of the building-surface PJM, the visitors are well immersed in the content describing the historical development until 3m40s, but the content of the stage performance following that correlates very poorly with the information on the historical development, which leads to a logical interruption and a limited understanding of the cultural information.

Fig. 7

Individual scenes of the three programs reference (Figures from official promotional video. http://www.zgtcly.com/)

Consequently, we can conclude that the likelihood of visitor’s overall evaluation of the PJM experience being affected by age is negligible, i.e., PJM upholds good acceptance and excellent pervasiveness within each age group. However, the prolonged stage duration generally shifts the PJM information towards “decoration and aesthetics,” which triggers senior visitors predisposed to respecting the cultural implications to respond negatively more frequently.

Data analysis of the Phase 2 questionnaire

The Phase 1 questionnaire provided valuable insight into the overall experience and feedback of the visitors, however, the authors identified shortcomings in the design of the “knowledge experience evaluation” in the Phase 1 questionnaire after the first survey. We consider that the current survey is merely a comprehensive survey of knowledge experience without sufficient information on visitors’ tendency to recall PJM information, impressive scenes, etc.

It is important to indicate that this study had the plan to develop multiple-choice questions about “knowledge experience evaluation” at the beginning of the evaluation system design. However, for many complex reasons (traffic, collaborators’ time, organizers’ schedules, etc.), this study has not completed the whole question design on time. In order not to waste the first three days of the most popular period of the event to revise the questionnaire, so instead, we decided to split it into two surveys. Therefore, the authors followed up with six supplementary questions relevant to cultural knowledge about 25 days after the event, which were delivered to participants through the scenic spot’s visitor service system and other Chinese social media channels. The Phase 2 questionnaire was intended to supplement the data in the “Knowledge experience evaluation” section and explore “the level of effectiveness of PJM cultural information in retaining visitor’s attention.”

The survey of Phase 2 questionnaire received 236 valid responses among the total 361 responses (Additional file 1: Table S6). Additionally, the results were distributed via the official website, visitor service platform, email, social media, and teleconferences.

Since the Phase 2 questionnaire were developed in a multiple-choice format, this study derived and classified the cultural information presented in the three PJM events into four categories: macro, micro, environment, and performance as a reference for designing the Phase 2 questionnaire (Table 6). Essentially, the distinction between the four types of information is intended as a reference for the options design of multiple-choice questions. Meanwhile, by referring to the definitions of the four types of information in this study and the final data results, it is helpful to researchers and practitioners to understand more accurately the visitors’ tendency to recall PJM information and conclude more referential and generalized conclusions.

Table 6 Information summarization and categories

Based on these categories; firstly, “macro” refers to information regarding the establishment of the Tang dynasty and significant events in revolutionary and military history, which are classified under historical background and development. Secondly, “micro” encompasses Tang dynasty texts, patterns, and portraits, in conjunction with specific information describing scenes from Tang poems. Thirdly, “environment” includes physical vehicles such as walls, bridges, and pagodas. Finally, “performance” comprises sword dances, court dances, and the entirety of costume components.

In the context of PJM, its key application as the primary vehicle to convey the “macro” and “micro” information should be prioritised. In relation to the “environment” and “performance” categories, PJM serves as a visual effect-supporting tool, where the building or dance is the primary vehicle for conveying information [47].

Initially, we analysed visitor’s tendencies to recall the PJM information with the help of a “Pareto diagram.” The Pareto diagram is a graphical representation of the “two-eight principle”, i.e., options with cumulative percentages between 0 to 80% are deemed “high-impact” capable of influencing the conclusion. Conversely, options with a cumulative percentage of 80% to 100% are considered to be “insignificant” [48]. This study classified the influence level of PJM information into “Central/General/Non-central” by leveraging the characteristics of the Pareto diagram and referring to specific autonomous definitions, i.e., “central” for cumulative percentages ranging from 0 to 0.6, “general” for values between 0.6 to 0.8, and “non-central” for 0.8 to 1.

Next, the data results of Q4 (Building-surface PJM), Q5 (In-water PJM), and Q6 (Bridge-surface PJM) were organised into a Pareto diagram to reflect the four information types (Macro/Micro/Environment/Performance) through corresponding colours in the diagram and to summarise the visitor’s priority of recalling PJM information, we will also be analyzed in conjunction with a timeline analysis diagrams simultaneously.

With reference to the results of building-surface PJM, the information in the “central” level is frequently dominated by “macro”. Likewise, the frequency of “performance” and “environment” in “general” is relatively similar, and all the “micro” information assembles at the “non-central” level and possess a cumulative percentage value higher than 0.8 (Fig. 8).

Fig. 8

Pareto diagram for building-surface PJM

Furthermore, in conjunction with the timeline analysis, it is noticeable that the “Macro” information relating to the historical context remained strong impressive to visitors even though only about 1/3 of the total duration is shown. Conversely, visitors were generally impressed by the PJM information on the longer stage performance.

Regarding the results of in-water PJM, the distribution of various information classifications is consistently average, with “micro” and “macro” garnering a similar degree of attention at the “central” level. It is noticeable that the information of the “Phoenix and Golden Dragon” was not displayed for a prolonged period but still served as a key memorable information for visitors as it is a most classic traditional Chinese motif and mythological image (Fig. 9).

Fig. 9

Pareto diagram for in-water PJM

Based on the results of bridge-surface PJM, the evolution of “micro” information into the dominant component is evident, while “macro” information surprisingly appears in large numbers at the “non-central” level because history-related information is indirectly represented in this “PJM combined with dance performance” program. Furthermore, it is obvious that the “performance” information that was shown for the longest period remains unimpressive despite being constantly chosen with a relatively steady frequency (Fig. 10).

Fig. 10

Pareto diagram for bridge-surface PJM

With respect to the total amount of “high-impact information” (cumulative percentage value below 0.8), the result is “macro > performance > micro > environment”, which indicates that the “macro” information constitutes the central data type in the PJM display and embodies the basis of PJM events. In actuality, the display of “micro” and “performance” information is greatly influenced by the design and display style. In other words, the effectiveness level is influenced by the designer’s individual preference despite being on par with information.

However, emerging trends in information variation revealed that “macro” and “micro” information are hardly perceived at the “general” level. Surprisingly, even “macro” had 6 amounts at the “non-central” level despite being considered as the “most central information”. In accordance with this result, this study argues that visitor’s tendency to recall the information presented was intricately linked to the theme of the event. For example, the theme of the building-surface PJM, which was “Presenting Grandeur and Military Strength in front of the Zhuque Gate.” In conjunction with the timeline analysis chart, a vast majority of visitors selected the information related to military history, even though only approximately a quarter of the PJM length was allocated to portraying events in military history on the building’s surface. Despite denoting a significant historical event, the “Governance during the Zhenguan Reign” was rarely chosen owing to its irrelevance to the history of the war. Consequently, the results may suggest a definite conclusion—when recalling information from the PJM, visitors automatically correlate visuals with the event’s theme and actively seek cultural elements in line with the adopted theme.

To summarise the results derived from the Phase 2 questionnaire in simplified form:

• The “macro” information conveyed in the building-surface PJM is likely to retain visitor’s attention effectively. Similarly, the “micro” information conveyed in medium-sized PJM will generate an analogous response.

• Visitors instinctively associate the information conveyed by the PJM with the theme and name of the event upon recalling past experiences.

• Despite overlooking information from “performance” as the central information, it indicates a very stable frequency of selection and effectiveness, thereby implying that PJM stage performance is relatively acceptable but fails to serve as an innovative expression for the public.

Cross-tabulation and summarization of the Phase 2 questionnaire data

In favour of further refining the conclusions on the effectiveness of PJM, the study will integrate the data on trip frequency (Additional file 1: Fig.S7; Table S8–S10) and employ cross-tabulation to identify “which information continuously impresses the visitor contingent on the growth in trip frequency, which information is gradually ignored or concerned, and which information consistently receives less attention” [49].

Like the format of the Pareto diagram analysis stage (Fig. 11), “central” refers to information that inevitably garnered the attention of visitors to the PJM, i.e., critical information in the PJM (selected nearly or more than 150 times and the integrated percentage is close to or higher than 0.75%). Also, “general” refers to information that was attended to sporadically or varied with the event theme and trip frequency, i.e., information that is relevant, but not central to PJM (selected between 150 and 50 times with the integrated percentage gradually ascending or descending between 3.5% and 0.75%). Likewise, “non-central” refers to information that presents minimal general relevance, i.e., material that isn’t significant to the PJM and can be either replaced or redesigned (selected less than or close to 50 times and the integrated percentage is less than or close to 3.5%) [50]. On account of the limited amount of sampling, this study decided to exclude the data from the participants who had visited three times or more.

Fig. 11

Aggregate of three cross-tabulations

In Fig. 9, the distribution of colours demonstrates that the level of attention exhibited by visitors pertinent to the information on “macro” is highly consistent regardless of the frequency of trips. However, it also indicates that the dominance of “macro” information is conditionally restricted as a section of the proportionately lesser relevant “macro” information to PJM themes was distributed at the “non-central” level. Simultaneously, the attention of “micro” and “performance” information is comparable, wherein “performance” is concentrated in “central” and “general (ascend)” levels, while the aspect of “micro” is focused in “central” and “general (descend)” levels.

With the aim of enhancing the comprehensibility of the results in this section, a table summarising the contents of Fig. 11 is provided as follows: (Table 7).

Table 7 Visitor’s tendency to recall PJM information

The information labelled “central” is inextricably linked to the theme of the event and the development of the historical context, indicating that logical historical storytelling information (macro information) should remain as the predominant element in the design of PJM. “Micro” and “performance” information are comparable in terms of influence, while “environment” information is relatively less prominent, i.e., “macro > performance = micro > environment.” Furthermore, the “environment” information is fundamentally concentrated in “general”, while the visitor’s attention to information concerning the “environment” expanded with rise in frequency of visit.

Finally, it is unfortunate that this study cannot integrate analysis of the data from the two phases survey, as this would undoubtedly lead to better results. However, we found that during the second survey, respondents were able to recall the most impressive PJM information more calmly, and some respondents also indicated that if they were asked to fill in multiple-choice questions immediately at the end of the event, they would probably do so perfunctorily because there were too many options. Therefore, we believe that the results of the Phase 2 survey are still a scientific representation of visitors’ tendency to recall information. Compared to the immediate questioning on the day of the event, this approach avoids the possibility of perfunctory responses, however, it also leads to diminished recall of some “general” information from fresh memory.

Discussion

A discussion on the practicality and limitations of the evaluation system

This study formulated an evaluation system that assesses the visitor’s experiences and directs PJM developers to carry out self- evaluation through timeline analysis. On a similar note, the system advocates classifying PJM information into four types (Macro/Micro/Environment/Performance), with the goal of attaining in-depth comprehension of visitor’s actual experiences and offering innovative scientific references for the creation and improvement of PJM content. Notably, the authors postulate three key directions in which the evaluation system can deliver results— “overall experience evaluation,” “the effectiveness of PJM on visitor experience,” and “the visitor’s tendency to recall PJM information.” Accordingly, we will summarise and discuss the findings of Tangcheng’s PJM evaluation.

This study determined the high degree of effectiveness of PJM applications on a multidimensional visitor’s experience in Tangcheng through SEM analysis in the Phase 1 questionnaire, distinctly concerning the influence on visitor’s subsequent evaluation. Based on the results from SEM, the findings highlight the means of effective PJM operations for disseminating cultural information and the proposed requisite to generate rigorous PJM content. Meanwhile, the chi-square test revealed that PJM significantly influences the tourism experience of visitors belonging to every age group, thus indicating the effectiveness of PJM in Tangcheng programs. From the results of Q13 in the chi-square test, it is evident that the high influence of PJM isn’t perceived as a positive result, since excessively drawn-out stage performances and “micro” material devoid of historical context will culminate in a lack of cultural depth in the PJM content. In turn, an overall detrimental impact on the spread of culture will be observed.

Based on visitor’s tendency to recall PJM information, this study conducted a Phase 2 questionnaire that showcased the results in terms of “cumulative percentage” and “influence of trip frequency on recall”. Consequently, the results of “visitor’s tendency to recall PJM information” in Tangcheng are as follows: macro > performance = micro > environment. Additionally, visitor’s memory of “macro” information is exponentially influenced by the theme of the event, i.e., “macro” information which isn’t pertinent to the event’s theme is conventionally disregarded.

Since Tangcheng was the first destination to test the evaluation system, the results can only serve as a reference for PJM events in cultural heritage sites analogous to Tangcheng. Therefore, the authors predict that this evaluation system will be implemented in various forms of relevant events to collect generalised results on the “influence of PJM on cultural tourism” and “visitor’s tendency to recall PJM information.”

Discuss the data results in conjunction with the timeline analysis chart

This study adopted a timeline analysis to deconstruct the information composition of the three PJM events in Tangcheng (Fig. 12). Specifically, the composition of the timeline shows that regardless of the form of the PJM vehicle, Tangcheng’s PJM typically comprises “PJM with stage performance,” i.e., costumes, dance, and ornamental components function as the primary components of the event. Accordingly, the authors assumed that the visitor’s tendency to recall PJM information of Tangcheng PJM should lean towards the aspect of “micro” and “performance”. Following rigorous data analysis, the results of such an assumption (macro > performance = micro > environment) indicates that “performance” was selected quite frequently owing to the excessive length of stage performance. However, “macro” prevails as the focus of visitor’s overall impressions or the potential need for information.

Fig. 12

Timeline analysis of all the Tangcheng’s PJM

According to the authors, such a phenomenon ought to be associated with the potential attributes of PJM information. While designing the Q5 (in-water PJM) and Q6 (bridge-surface PJM) options in the Phase 2 questionnaire, the authors argued that even without directly presenting “macro” information characterised by a historical context, some components of both PJMs were reminiscent of the knowledge relevant to history. For instance, the Dragon and Phoenix motifs induced visitors to associate with the “Kaiyuan Heyday Period”. Additionally, the fusion of the Buddha statue with the Tang totem correlates with the era when Buddhist culture spread to China. Accordingly, this study included specific options of Q5 and Q6 regarding the historical context to ascertain the likelihood of respondents instinctively associating ornamental components with the historical context. Furthermore, the frequency of results stemming from selecting “macro” information illustrates that visitors tend to subjectively inculcate potential attributes of PJM information.

Subsequently, this result indicates the provision of displaying relevant information in tandem with potential “macro” attributes if PJM requires enhanced information diversity while also mandating rich historical context information. By diligently leveraging the “potential attributes” of information, it is possible to enrich the visual effects of PJM while concurrently illustrating the abundance of historical and cultural information. However, the findings also demonstrated the inadequacy of the “potential information analysis” displayed by the present timeline analysis approach as it merely examined the “apparent information” of the PJM exhibition.

Although Tangcheng’s analysis results were insufficient to serve as a generalised guideline, the authors argued that such results could serve as a valuable reference for the PJM design of heritage sites that “combine PJM with stage performance.” All three projects in Tangcheng were undertaken through a combination of PJM and stage performance, wherein such a display format often received widespread recognition in global scenarios.

Conclusion

This study designed and applied a system for evaluating the visitor’s experience at PJM events in cultural heritage sites. The system is dedicated not only to helping researchers acquire feedback and suggestions from visitors but also to helping designers rethink and improve the content of PJM through a scientific method of analysis. Meanwhile, this study summarizes the central ideological procedure of such an evaluation system here: self-analysis (the timeline analysis method)—formulation of hypothesis (feedback anticipation)—category PJM information (macro, micro, performance, environment)—design of questions (according to the hypothesis theory)—interview survey (parts that conflict with the hypothesis theory)—analysis and presentation of conclusions. Consequently, such a process will minimize the generation of meaningless questionnaire items and assist developers in acquiring the desired information much more directly.

Subsequently, Tangcheng’s data results provided us a preliminary understanding of how PJM affects visitors’ impressions of their integrated tourism experience and cultural comprehension. Meanwhile, the results of Tangcheng (macro > performance = micro > environment) can still be a decent reference for the PJM displayed in cultural heritage sites as “combine PJM with stage performance” form, even though it is necessary to implement the more visitor’s experience test and acquire data from relevant events to explore a generalized results on “visitor’s tendency to recall PJM information.”

In conclusion, the recommendations from the authors for developing strategies for implementing future PJM events at the Tangcheng scenic area according to the research data are as follows:

• In the case of macro information regarding historical development, it is difficult to make an impression on visitors with information that is not directly associated with the theme of the PJM event or with relatively unfamiliar historical stories [51].

• Several visitors will continue to be attracted to environmental features like the “stupa” and “building structure,” which have a distinct religious and technological character [52].

• It is recommended to adjust the duration ratio of dance performances and the PJM program appropriately, particularly in the building-surface PJM event; enhancing the duration ratio of the PJM demonstration will achieve greater effectiveness [53].

• It is recommended to increase the content regarding the historical development of the “Heyday Period” in the building-surface PJM event and to demonstrate the historical story with a logical and integrated narrative structure as far as possible [54].

• In cases where the PJM demonstration platform refers to a distinctive cultural building, city wall, or bridge (e.g., the Zhuque Gate or the Four Seasons Bridge), it is possible to strengthen visitor’s impressions of the cultural building itself by inserting PJM content regarding the building techniques and craftsmanship [55].

• For display the traditional aesthetic and cultural connotations, it is possible to approach the PJM stage performance style rather like “stage drama” with a narrative [56].

Additionally, regarding the limitations of this study, the authors argue that: since Tangcheng, the subject of this study, is more inclined to be a cultural tourism tourist site, this leads to the results of the study being more biased towards heritage sites with cultural tourism functions PJM projects for reference. Meanwhile, the authors expect that in the future, more practitioners will be involved in improving the evaluation of the PJM visitors' experience at heritage sites. By evaluating the PJM content and visitors' experience in various types of cultural heritage sites/tourism areas, the industry will have more scientific guidelines, using contemporary technology as a medium to disseminate cultural information and help heritage sites to public and cultural dissemination.