Open Access

Mind the gap: rigour and relevance in collaborative heritage science research

Heritage Science20142:11

https://doi.org/10.1186/2050-7445-2-11

Received: 5 January 2014

Accepted: 8 April 2014

Published: 28 April 2014

Abstract

Background

This paper examines if there is a ‘rigour-relevance gap’ in collaborative heritage science research and what enables and impedes effective collaboration between academic researchers and users of research evidence in practice. A quantitative attitudes questionnaire was distributed amongst the heritage science community: 210 responses were received. Respondents answered in relation to one project they had worked on in the UK in the previous five years. They were asked about their personal goals in relation to the project and whether these were achieved, satisfaction levels in relation to project outcomes and impact, level of agreement with a series of attitude statements to ascertain what helped and hindered projects, personal characteristics and project characteristics. The questionnaire was analysed using a factor analytic, segmentation and profiling approach.

Results

Most respondents sought both rigour and relevance in collaborative research, were generally satisfied with the outcomes of projects and reported positive experiences of collaboration. However, respondents were less satisfied that the impact of projects would be realised. Practice-focussed goals were associated with lower levels of self-rated achievement than academic goals and a sizeable minority reported challenges in collaborative research. Furthermore, researchers and users differed in terms of their goals, experiences and level of satisfaction. Users had high expectations that research would translate into practice, but did not always feel this was realised.

Conclusion

Results from this project will inform how to improve heritage science research collaborations.

Keywords

RigourRelevanceCollaborative researchHeritage scienceResearcherPractitionerFactor analysisCluster analysisSegmentationProfiling

Introduction

The boundaries of what constitutes collaborative research and who counts as a collaborator are not well defined, but at a basic level there must be direct co-operation between two people [1]. Current United Kingdom (UK) policy encourages innovation and good use of resources by helping researchers, developers, and innovators bring together specific knowledge, skills, technical resources and financial capital [2]. Similarly, at an international level, EU Horizon 2020 promotes links between education, business, research and innovation [3]. In HEI research there is a growing emphasis on creating ‘impact’ upon economy, society, public policy, culture and quality of life [4]. In the heritage science research domain, the need for collaboration and impact has been stressed by the 8th European Commission Conference on Sustaining Europe’s Cultural Heritage, ‘Cultural Heritage Research meets Practice’, in its Ljubljana Declaration [5].

Collaboration between researchers and practitioners is integral to heritage science, which encompasses all technological and scientific work that can benefit the heritage sector, through improved management, enhanced understanding of cultural heritage or increased public engagement [6]. The vision of the UK National Heritage Science Strategy (2010) was that the understanding and preservation of material cultural heritage would be enhanced by collaborative science and technology research [7]. As such, heritage science can be viewed as an applied science, in which ‘impact’ is always an intention.

Although the benefits of collaborative heritage science research have been clearly articulated, bringing researchers and practitioners together to collaborate on research projects is not without challenges. Different partners may have very different ideas about important research questions, appropriate methods, who should benefit from research and what the benefits should be. Academic partners might be constrained by the need to publish, whereas practitioners may be inclined to only support research in which practical outcomes are guaranteed [8].

Studies of the relationship between academic research and practice identify several stages of research utilisation [9, 10]: transmission of results to end users, understanding and referencing of results by end users, efforts to adopt results, influence of results on practice/policy and then application (e.g. in the form of new policy or products). Studies looking at how far research climbs the utilisation ladder show that simply customising research to users’ needs does not necessarily increase utilisation. Research designed within the user’s context has a stronger relationship with utilisation: users must see research as pertinent, timely and credible. Such an acceptance of academic research by users may evolve through strong institutional and social linkages that are allowed to develop over time and when users invest resources in research [11, 12].

The benefits of collaboration can be broad and rich, contributing to many personal and organisational goals. In addition to application, users appreciate the opportunity for knowledge exchange [13]. A survey of arts and humanities researchers collaborating with non-HEIs showed that researchers found teaching and learning, the core business of HEIs, to be greatly enriched by collaboration, and also that collaboration produced new insights into research [14]. An evaluation of the UK Technology Strategy Board’s collaborative research funding programme found that access to resources and expertise, improved skills and knowledge and improved attitudes to collaboration were more frequently identified as benefits of collaboration than direct applications and financial impact [15].

However, collaborators may find that they have different working cultures and practices, and might work on different timescales. Collaboration may be rewarded or supported (e.g. through time for research) by some organisations but not others. Research from the National Academies [16] in the United States of America (USA) found that barriers to effective collaboration included a lack of understanding and trust arising from differences in internal processes, decision making and organisational interests.

Other research has looked at the impact of project design on the effectiveness of collaborative research. Team composition is important, requiring not only experience and expertise, but also an open attitude towards interdisciplinarity and collaboration [17]. The key role of the principal investigator in developing and leading collaborative networks has been highlighted in research [18] and the needs of early career researchers in other research [19]. There is also research looking at the impact of project size and budget [20] and level of interdisciplinarity [21] on project achievement.

Underpinning both the understanding of each other’s needs and goals and the formal and informal processes of working together might be the strength of relationships and the nature of communications. Language and communication issues have been much discussed in the context of interdisciplinary research [17, 22, 23]. One issue identified in the literature is the time needed for collaborators to understand each other’s approaches and develop shared languages or concepts. This can happen formally, such as through the development of shared conceptual models [24], or informally through regular contact and sharing of knowledge and ideas.

Many of the challenges of collaborative research detailed above have been raised in discussions of ‘rigour’ and ‘relevance’. In the fields of management science, design science and information systems there is a body of research and debate about whether applied research can be both rigorous and relevant. In essence, the ‘rigour relevance gap’ refers to challenges in knowledge production and knowledge transfer [25]. The collaborative process may be challenging [26], research may not be perceived to be usable by practitioners [27] and academic researchers may feel that robust evidence is not taken up by practitioners [28].

For some, the rigour-relevance gap is seen as unbridgeable, not only because of the competing goals of researchers and practitioners, but because of the separate logics and approaches to research problems of different institutions. This means that knowledge transfer cannot occur because the systems operating in different institutions cannot communicate with each other [26]. In contrast, others propose that deep partnerships and training in research methods can result in high quality scholarship that is also useful [29]. Some have suggested that rigour and relevance are not separate concepts but are both aspects of research quality [30, 31]. By seeking rigour, research may also become credible, consistent and transparent. By seeking to be relevant, or contributory, research may become original and generalisable.

Heritage science may share some of the challenges of collaborative research apparent in other fields, one of which may be a rigour relevance gap. However, this has not been yet been researched. Therefore, the study reported in this paper explores whether there is such a gap and, if so, what could help bridge the gap. The study focused on recent collaborative research in the UK, thus reducing possible biases due to international differences in funding and management. However, the study has relevance beyond the UK, and beyond the heritage science field.

A mixed methods [32] approach was employed to explore attitudes towards collaborative research. Project partners from a range of disciplines were first consulted on their experiences and attitudes. The results of this consultation were used in the design of a quantitative attitudes survey, which also included some qualitative questions. The qualitative and quantitative research methods complimented each other by allowing results to emerge from the heritage science community and related disciplines, while at the same time the prevalence and structure of attitudes in the community could be assessed in a statistically robust and reliable way.

Respondents were asked to think about one collaborative project they had worked on in the UK in the previous five years. They were then asked about their personal goals at the outset of the project and whether these were achieved, their level of satisfaction with project outcomes and impact, their level of agreement with a series of attitude statements about what helped and hindered their project, their background (specialism, experience, place of work and role) and their project (size and complexity).

Segmentation and profiling techniques were used to explore the relationships between these variables. Factor analysis has been used across a wide range of situations, from exploring the symptom structure of psychological disorders [33], to exploring personality types [34, 35], to investigating the value of heritage collection to users and visitors [36, 37]. The analysis is used to reduce and find structure in attitude data and can be used to create attitude measures. As a further technique, cluster analysis is often used in studying personality types [38], conducting local area analysis of government statistics [39] and segmenting markets [40]. Clustering can be used to isolate groups of people with similar attitudes. Clusters are profiled on other variables, such as demographics, in order to explore how attitudes affect behaviour or vice versa. Factor Analysis and Cluster Analysis were applied in this study to create measures of the enablers and impediments of effective collaboration, and then explore the reasons for varying experiences of collaborative research. The results were complemented by thematic analysis of respondents’ comments.

Analysis of the questionnaire was used to provide evidence in relation to three research questions. Evidence for a rigour relevance gap would include findings relating to knowledge transfer (translation of research into practice) and knowledge production (challenges in the collaborative process). Differences in the experiences and attitudes of academic researchers and users of research evidence in practice would also point to a rigour relevance gap. Finally, the analysis sought to identify enablers and impediments of effective collaboration.

Method

Definitions and inclusion criteria

The study was concerned with inter-institutional collaborative heritage science research between academic researchers (‘researchers’) and users of research evidence in practice (‘users’). Academic researchers could be based at a HEI or an independent research organisation (IROs). Users could be based at a museum, library, archive or gallery (MLAG), a heritage organisation, in industry, in a small to medium enterprises (SMEs), in self-employment, or at an HEI-based MLAG. Users of research evidence included practitioners such as conservators, curators, librarians, archivists, collection managers, policy makers and those seeking to exploit research findings in business and product development. For the purposes of the questionnaire study, collaborative research referred only to collaborations that involved both academic researchers and users. Interdisciplinary projects involving only academic researchers from different disciplines and no users were not of interest to this study.

Examples of collaborative activities included, but were not restricted to, formal funded research projects with multiple partners, sharing of expertise and resources between institutions, collaborative development of products, policy and new ways of working, co-supervision of research students and case studies.

When completing the Mind the Gap questionnaire, respondents were asked to think about one project they had completed (or which was near to completion) and which they had worked on in the UK in the previous five years.

Questionnaire development

The development of the questionnaire followed an established process [41, 42], in which qualitative and desk research are used to gather information about topics of interest and then converted into questionnaire format and piloted. Focus groups with partners took place in February 2013. Partners represented heritage science and a range of other disciplines such as linguistics, medicine, anthropology and history. The focus groups were used to collect information about experiences of collaboration and capture the language of collaborative research. The data were recorded and transcribed, and then organised into themes, cross-referenced with the literature and used in the design of the questionnaire. The questionnaire was piloted, for comprehension and ease of use, using guidelines from cognitive interviewing [43] on a small sample of heritage researchers and then edited before being launched.

A copy of the questionnaire can be found in Additional file 1.

Questionnaire content

Results from the preliminary research were used in two key sections of the questionnaire.

Goals

Respondents were asked to choose five goals, from a list of 24 goals, that they had at the outset of their project (regardless of whether these were also project goals), cf. Additional file 1, Section 2. The list included goals relating to improving practice, applications, academic knowledge, research evidence, access to and understanding of collections, professional development, networks, resources, collection management and research outputs. The aim of this section was to identify which goals were most frequently held and whether goals differed between groups of respondents. Respondents were also asked to rate whether each of their five goals was ‘achieved’, ‘partially achieved’ or ‘not achieved’.

Enablers and Impediments

Fifty attitude statements about what helps and hinders collaborative research were constructed in accordance with question design guidance [41, 42], cf. Additional file 1, Section 3. Respondents were asked to rate their agreement with each statement on a seven point scale from 1-strongly disagree to 7-strongly agree. This section provided the data for a factor analysis, with the aim of producing a set of reliable measures of the enablers and impediments of collaborative research.

The following data were also collected.

Satisfaction ratings

Questionnaire respondents were asked how satisfied they were with the outcomes (the results of the project and project aims) and impact (communicating research to users and translating results into practice) of their project, cf. Additional file 1, Section 2. Satisfaction was rated on a seven point scale from 1 (not at all satisfied) to 7 (completely satisfied).

Researcher/User

Respondents were gave a rating between 1 and 7 to indicate if they felt their role in the project was closer to an academic researcher (1) or an end user of research evidence in practice (7), cf. Additional file 1, Section 1. They were then classified as a Researcher (scored 1 or 2), Researcher-User (scored 3–5) or User (scored 6 or 7). Although many researchers are also users (e.g. academic researchers use research in teaching) and vice versa (e.g. curators research the significance of collections) respondents were asked to self-define their role.

Project and personal characteristics

Respondents were asked about their background (discipline, job title and experience) and their role in the project (seniority, subject specialism, place of work), cf. Additional file 1, Section 1. They were also asked about their project (number of people, partners and subjects).

Open-box questions

There were three open ended questions: ‘What else interested you about this project?’ and ‘What else do you think helped/hindered the project?’, cf. Additional file 1, Sections 2 and 3.

Distribution and sample characteristics

The questionnaire was distributed online via Survey Monkey (http://www.surveymonkey.com) for six weeks from April-June 2013. There was a good return rate of 210 eligible responses, resulting from approximately 550 personal invites to participate, largely made up of contacts of the Centre for Sustainable Heritage and the Arts and Humanities Research Council]/Engineering and Physical Sciences Research Council Science and Heritage Programme (AHRC/EPSRC SHP). Around half of respondents were based at HEIs and half at non-HEIs (e.g. MLAG, heritage organisations, industry/SMEs and self-employed). Around half of respondents said they represented a STEM (science, technology, engineering or mathematics) discipline during their project, and there were 50 responses from arts and humanities disciplines. Around half of respondents identified strongly as Researchers, and the remainder were split between Users and respondents with dual roles.

The respondent groupings illustrated in Figure 1 were used in the analysis of the questionnaire. In some cases the classifications presented in Figure 1 were simplified or edited for analysis purposes (e.g. excluding small categories and ‘no answers’).
Figure 1

Characteristics of the Mind the Gap questionnaire sample. Personal and project characteristics as self-reported by questionnaire respondents. Numbers in parentheses represent the number of respondents in each category of response.

Overview of analysis methods

A number of techniques from quantitative and qualitative attitudes research were used to analyse the questionnaire. All the data management and analysis was conducted in Statistical Package for the Social Sciences (SPSS, version 19). A copy of the dataset is available from the Centre for Sustainable Heritage.

Exploratory factor analysis

In the context of attitude questionnaires, factor analysis is a technique that reduces a large number of inter-related variables to a smaller number of latent dimensions. The goal is to achieve parsimony by using the smallest number of explanatory concepts to explain the maximum amount of common variance in a correlation matrix [44].

In this study, factor analysis was used to reduce fifty attitude statements about what helps and hinders collaborative research down to a small set of factors representing enablers of and impediments to collaborative research. Each factor was used as a dimensional measure of each enabler/impediment. Further analysis explored the relationship between the measures and other variables, such as satisfaction and achievement ratings, project characteristics and personal characteristics. For full details of the method and analysis see Additional file 2.

Cluster analysis

Cluster analysis refers to a range of algorithms that can be used to find groups of cases within a dataset, such as questionnaire respondents, that are more similar to each other than they are to cases within other groups [40]. In this study, the intention was to find clusters of respondents who differed in terms of their attitudes to collaborative research (using their factor scores) and then explore whether these clusters also varied according to their goals, satisfaction and achievement ratings, personal characteristics and project characteristics. The aim was to explore reasons for varying levels of project success and effectiveness in collaboration.

Inferential statistics

Inferential statistical tests are used to test hypotheses about data [45]. One-way Analysis of Variance tests (ANOVA) and t-tests were used to compare groups of respondents in terms of mean satisfaction ratings, achievement ratings and factor scores. Chi-squared was used to test whether groups of respondents differed in terms of their goals, and also to generate cluster profile descriptions. Pearson correlations were used to assess the strength of linear associations between different types of ratings (e.g. satisfaction ratings and factor scores).

The critical significance level was set at p ≤ .05, one-tailed (equivalent to p ≤ .10, two-tailed, shown in tables of results in Additional file 2). One-tailed significance is reported as, based on extensive literature, there was an expectation that specific differences between subgroups of participants would emerge. A Bonferroni correction was used for post hoc comparisons (p ≤ .05, two-tailed). Descriptive statistics and the results of tests can be found in Additional file 2.

Thematic analysis

Detailed coding of the text was used to identify overarching themes within respondent’s open-box comments. Initial coding was conducted independently by two researchers, who arrived at very similar thematic structures. A revised coding scheme was then applied to the data. Occurrence of themes and codes was then quantified using frequency analysis. The aim was both to identify the main themes in comments relevant to research questions and also to provide a full description of the content of comments. The full coding schemes are presented in Additional file 3 along with a selection of quotes.

Note on box-plots

Where appropriate, attitude ratings are illustrated as box-plots of quartiles in order to illustrate range and distribution. The whiskers of each box plot extend to the maximum and minimum scores that were observed. The coloured box represents ratings falling between the 25th and 75th percentiles (i.e. approximately 50% of respondents). The horizontal line through the coloured box indicates the median score (50% of people score above this line and 50% below). The diamond marker indicates the mean score. Ratings and factor scores tended to be skewed towards positive attitudes, hence the median and 75th percentile are sometimes in the same position on box-plot charts.

Results and discussion

Is there a rigour-relevance gap in heritage science?

Although most respondents reported a positive experience of collaborative research and aspired to both rigour and relevance in their projects, there were four key pieces of evidence to suggest that there is a rigour-relevance gap in heritage science research.

Project satisfaction

Questionnaire respondents were asked how satisfied they were with the outcomes and impact of their project (see Figure 2). Most respondents were satisfied with outcomes (64% of respondents gave a rating of 6 or more and 86% gave a rating of 5 or more). Many were also satisfied with impact (49% gave a rating of 5 or more). However, ratings were statistically significantly higher for project outcomes in comparison with impact. Respondents were satisfied that the aims of their project were achieved, but were less satisfied that the impact of the project would be realised.
Figure 2

Box-plots of outcome and impact satisfaction ratings. Chart shows box plots of quartiles, the medians and the means. Note that for the Outcome satisfaction ratings, 64% of respondents gave a rating of 6 or more and 84% gave a rating of 5 or more (hence the median and 75th percentile markers are in the same position). Outcome (mean = 5.50, SD = 1.27) and Impact (mean = 5.15, SD = 1.15) ratings were significantly different to each other (t (208) = 5.18, p < .001).

Personal goals

Respondents were asked to choose five goals, from a list of 24, that they were personally most interested in at the outset of their project. Frequency analysis was used to identify the number/percentage of respondents who chose each goal (Figure 3). Exchange of ideas and expertise between different fields, relevance to practice (conservation and understanding of cultural heritage), new knowledge and a high quality evidence base were most frequently chosen. This suggests that the heritage science field seeks both rigour and relevance in research and that both practice-focussed and academic goals are widely held. However, two of the goals that were least frequently chosen relate to management (‘Standards and Guidelines’, ‘Policy and Strategy’). Therefore, some types of practice-focussed goals (e.g. conservation of and understanding of cultural heritage) were more frequently held than others (management).
Figure 3

Personal goals. Percentage of respondents who chose each of 24 goals as one of their five main personal interests at the outset of their project.

The findings were corroborated by open-box comments in response to the question ‘what else interested you about this project?’. Of the 102 comments, the most frequent set of responses were about the collaborative process (34%), particularly interdisciplinarity and working with diverse people and institutions. This was closely followed by comments about research and practice (33%), particularly evidence based practice (including challenging, critiquing and evaluating current practice), providing a better evidence base, improving practice through new applications and processes, problem solving and ensuring relevance and a deeper understanding of practice. Aspects of both rigour and relevance occurred frequently in comments and rigour appeared to be closely linked to relevance. In addition, in line with the literature, although there were a number of comments about direct applications, the wider benefits of collaboration were more frequently mentioned. The full coding scheme is presented in Additional file 3.

Achievement of goals

For each of the five goals they chose, respondents were asked to rate whether they felt the goal had been achieved, partially achieved or not achieved. The most frequently chosen goals received some of the highest achievement ratings (Figure 4). This was with the exception of one of the key goals of heritage science research: ‘Better Care and Conservation of Cultural Heritage’. Management goals (such as ‘Improved Collection Management’ and ‘Standards and Guidelines’) also received low achievement ratings. Participants in collaborative heritage science research sought both rigour and relevance in their research, but more frequently felt that they achieved learning or academic goals than translation of research into practice.
Figure 4

Achievement of goals. Percentage of respondents who reported that they had ‘achieved’, ‘partially achieved’ or ‘not achieved’ their goals.

Attitudes towards collaborative research

The ratings for 50 attitude statements about collaborative research were entered into a factor analysis. An eight factor solution was assessed as the most reliable and yielded the most consistent factor structure. Full details of the factor analysis can be found in Additional file 2.

The eight factors represented distinct aspects of respondent’s experience of and approach to collaborative research. Each factor was given a name representing an underlying theme linking the statements in the factor. Table 1 gives the factor names and an overview of the factor structure. Most of the themes identified during the initial focus group research and literature review are represented in the factor structure, with two notable exceptions: ‘Language and Communication’ and ‘Resources’ did not emerge in the factor analysis but are discussed later in this paper.
Table 1

The eight factor solution

Factor name

Description

Keywords

Attitude statements*

Ease of collaboration

This factor contains statements relating to the quality of the collaborative relationship with partners. In particular, whether the partnership felt open, trusting and natural and whether there were shared goals and interests. The factor also appears to refer to the ease/difficulties of interdisciplinary working, including establishing a shared language.

Trust, naturally, easily, open, common ground, interests, common understanding, aligned, share, felt, found, partners, people, other disciplines and practices, terminologies and concepts.

Q15: I found that project partners were open to other disciplines and practices

Q56: I felt that partners trusted me to know what I was doing

Q53: I found that collaboration arose naturally between people

Q22: I found my interests easily aligned with those of project partners

Q60: Partners in the project were willing to share data and results

Q41: I found it difficult to interest partners in my ideas (-)

Q26: I found it difficult to establish common ground with project partners (-)

Q51: I found it difficult to establish a common understanding of terminologies and concepts (-)

Collaborative working style

Statements in this factor relate to a respondent’s preference for working in collaboration or as an individual, and whether they were interested in collaborating and felt it was needed.

Contact, collaboration, essential, required, only, wanted, preferred, alone, sole/co authorship

Q31: I wanted to have lots of contact with the project partners

Q32: Collaboration was essential to my work on the project

Q35: I only collaborated because my institution required it (-)

Q20: I preferred to work alone (-)

Q58: I preferred sole authorship to co-authorship (-)

Interest in bridging disciplines

This factor represents a theme that emerged consistently during the analysis, but is less reliable than other factors. The factor relates to an interest in the process of collaboration, interdisciplinarity and communications as opposed to the products of collaboration.

Interested, language, bridge, between, partners, different disciplines and practices.

Q55: I was interested in the different language styles of project partners (e.g. from different disciplines and practices)

Q24: I was interested in finding ways to bridge between different project partners (e.g. across disciplines and practices)

Institutional recognition

Statements in the factor relate to institutional recognition and reward of a project, related to an institution perceiving a project as relevant to them.

Project, recognised, institution, positive, relevant, rewarded

Q33: The project was recognised within my institution as a positive thing

Q57: My institution didn’t think the project was relevant to them (-)

Q46: My institution rewarded participation in this project

Internal procedures & working practices**

Statements in this factor relate to real or perceived impediments to research due to working practices (particularly academic research) and users’ understanding of academic research processes. Respondents who agreed with the first three statements and disagreed with Q49 may also have found that negotiation was a key aspect of their work on the project.

Intellectual property protection, got in the way, internal procedures, users, didn’t understand, time-consuming, understanding, how academic research works, negotiation

Q27: I found that intellectual property protection got in the way of the research

Q17: Internal procedures got in the way of the project (e.g. contractual agreements and ethics applications)

Q36: Users in the project didn’t understand how time-consuming collaborative research is

Q49: Users in the project had an understanding of how academic research works (-)

Q25: The project was all about negotiation

Practice-focussed research

Respondents who agree with the first two statements in this factor wanted their project to be inspired by practical problems and communicated in non-academic terms. Their preference was for practice-focussed applied research rather than research focused on generating new knowledge. The factor is not as statistically reliable as other factors and overlaps with questions on goals.

Inspired, practical problems, communicate, non-academic terms, generating new knowledge, practical solutions

Q12: I felt it was important to communicate the results of the research in non-academic terms

Q14: I wanted the project to be inspired by practical problems

Q30: I felt that the project should be more about generating new knowledge than practical solutions (-)

Research quality**

Statements in this factor relate to the quality of research in a project. Respondents who agreed with the first four statements tended to disagree, or agree to a lesser extent, with the fifth statement (that there was a commitment to high quality research). Research quality appears to encompass both academic rigour and relevance to users.

Results, research, credible, uncomfortable, approaches, academic research, did not deliver, users, needed, lacked the depth, commitment, high quality

Q50: I did not feel that the results of the research were credible

Q52: I was uncomfortable with some of the research approaches in the project

Q54: The academic research did not deliver what users needed

Q28: The project lacked the depth of purely academic research

Q11: There was a commitment to high quality research (-)

Understanding partners’ research approaches**

Statements in this factor related to partners’ understanding of each other’s methods, goals and culture (e.g. language and ways of working). If a respondent agreed with statements in this factor it might imply a lack of mutual understanding, competing goals or dissatisfaction with both academics’ and users’ approaches (reflected in both academic methods being perceived as too complex and users too focused on applied outcomes). The factor also suggests that opportunities for knowledge exchange could facilitate mutual understanding.

Methodologies, academic researchers, complex, users, focused, applied outcomes, language, impenetrable, opportunities to interact, knowledge exchange, project partners, stick, established, way of working

Q39: Methodologies introduced to the project by academic researchers were too complex

Q40: Users were too focused on the applied outcomes of the project

Q38: I found the language used by some project partners impenetrable

Q43: Researchers and users did not have enough opportunities to interact (e.g., for knowledge exchange)

 

Q34: Project partners preferred to stick to their established way of working

Notes: *Factor scores were calculated by taking the mean rating across the attitude statements in each factor. Ratings labelled with ‘-’ were reversed when calculating factor scores. **Factor scores were reversed for the purposes of reporting results so that positive scores indicated enablement and negative scores indicated impediments.

The eight factors were used as measures of the enablers/impediments of collaborative research. Figure 5 shows the distribution of factor scores in the questionnaire sample. On average, respondents had high scores (above the neutral 4 score) for all factors. There was widespread agreement that respondents experienced ‘Ease of Collaboration’, had a ‘Collaborative Working Style’, were ‘Interested in Bridging Disciplines’, that their project had ‘Institutional Recognition’ and that their preference was for ‘Practice-Focussed Research’. Positive scores on Factors 5, 7 and 8 indicated that most respondents did not have concerns about ‘Research Quality’, difficulties due to ‘Internal Procedures & Working Practices’ or difficulties ‘Understanding Partners’ Research Approaches’. However, there were a range of scores for each factor, indicating a range of agreement and disagreement that will be explored in further analysis. For example, around 25% of respondents had low scores (below the neutral 4 score) for ‘Institutional Recognition’ and ‘Internal Procedures and Working Practices’ respectively.
Figure 5

Box plots of factor scores. Chart shows quartiles, medians and means for each factor.

Do researchers and users differ in their experiences of collaborative research?

The analysis further explored the rigour-relevance gap in heritage science research by comparing the experiences of academic researchers and users of research evidence in practice.

Researchers and users: personal goals

‘Researchers’, ‘Researcher-Users’ and ‘Users’ were compared in terms of their personal goals. Some of the most frequently chosen goals were shared by Researchers and Users (see Figure 6): ‘Knowledge Exchange’, ‘Taking Part in Research that is Relevant to Practice’, ‘Insights and New Knowledge’ and ‘High Quality Research Evidence’. However, Users were significantly more likely to choose practice-focussed goals (e.g. ‘Better Care and Conservation of Cultural Heritage’) than Researchers, who were significantly more likely to choose career development, intellectual and publication goals (e.g. ‘Peer Reviewed Journal Articles’). Personal goals reflected institutional goals: individual’s sought impact for their respective institutions in different ways (e.g. improved practice vs. publications). Researcher-Users shared some goals with Researchers and some with Users. For full results of statistical tests see Additional file 2.
Figure 6

Goals of researchers, researcher-users and users. Some of the most frequently held goals in the sample were shared by Researchers, Researcher-Users and Users. The chart also shows goals that differed significantly between the three groups of respondents (see Additional file 2 for results of chi-square tests).

Researchers and users: satisfaction ratings

Users and Researcher-Users combined gave significantly lower ratings of outcome satisfaction than Researchers (Figure 7). The same trend was observed for impact satisfaction ratings. The results suggest that some Users, particularly those with hybrid roles, were less satisfied with the results of projects than purely academic researchers. This may be because they had higher expectations that research would improve practice. For full details, see Additional file 2.
Figure 7

Project satisfaction ratings for researchers, researcher-users and users. a) outcomes satisfaction ratings and b) project impact satisfaction ratings. The charts show mean ratings and standard errors.

Researchers and users: achievement ratings

A composite achievement rating was created for each respondent by taking their mean achievement rating across the five goals they had chosen. Users and Researcher-Users combined had significantly lower achievement ratings than Researchers - see Figure 8 and Additional file 2.
Figure 8

Composite achievement ratings for researchers, researcher-users and users. The chart shows the mean achievement rating across the five goals that each respondent chose and standard errors.

Researchers and users: attitudes

Users and Researcher-Users combined felt they had more institutional recognition, but also more concerns about research quality, than Researchers (Figure 9). For full details see Additional file 2.
Figure 9

Factor scores that differed significantly between researchers, researcher-users and users. Based on the results of one-way ANOVAs with post-hoc comparisons (see Additional file 2 for full results).

Comments suggested that institutional support and recognition of collaborative research may depend on how well research speaks to an institution’s goals and priorities, and that research may be not supported otherwise (e.g. through time given for research). Comments also pointed to differences in working practices and cultures. Users sometimes felt that academic research takes too much time. Researchers sometimes felt that expectations about practice-led goals need to be realistic and that users sometimes lack research experience.

Enablers and impediments of effective collaboration

Profiling respondents using cluster analysis helped to illustrate what aspects of collaborative research and what characteristics of participants and projects were most strongly associated with project satisfaction, self-rated achievement and the collaborative process.

Cluster analysis results

Questionnaire respondents were clustered into groups using their eight factor scores. The analysis yielded three reliable clusters (Figure 10). See Additional file 2 for a technical description of the analysis.
Figure 10

Factor scores and outcome satisfaction ratings for three clusters of respondents.

The analysis usefully separated respondents who had a very positive experience of collaborative research (Cluster 2, 52%), from respondents who found collaboration more challenging and experienced more impediments (Cluster 1, 28%) and respondents whose were less interested in interdisciplinarity and less practice-focussed than the other clusters (Cluster 3, 20%). In addition to varying according to factor scores, the three groups varied according to their outcome satisfaction ratings (Figure 10). The same pattern was also observed for impact satisfaction and achievement ratings (see Additional file 2).

Profiles for each cluster were generated using information about the background of respondents and characteristics of their project. Significance testing was used to identify the key differences between clusters, the results of which are summarised in Table 2 (see Additional file 2 for full data and analysis).
Table 2

Summary of the cluster profiles

 

Cluster 1

Cluster 2

Cluster 3

Size of cluster

n = 58; 28% of respondents

n = 110; 52% of respondents

n = 42; 20% of respondents

Satisfaction/achievement

Lowest

Highest

High (Average – Above Average)

Factor scores

Lowest scores for: ‘Ease of Collaboration’, ‘Internal Procedures & Working Practices’, ‘Research Quality’ and ‘Understanding Partners’ Research Approaches’

High scores on all factors

Lowest scores for: ‘Collaborative Working Style’, ‘Interest in Bridging Disciplines’, ‘Institutional Recognition’, ‘Practice-Focussed Research’

Goals

Most likely to choose collections management goals (‘Standards and Guidelines’, ‘Improved Management of Cultural Heritage’)

Least likely to choose the ‘Access to Resources’ goal

Most likely to choose the ‘Access to Resources’ goal

Most likely to choose ‘Peer Reviewed Journal Articles’ as a goal and least likely to choose ‘Improved Management of Cultural Heritage’

Least likely to choose ‘Better Understanding of Cultural Heritage’ as a goal

Role

Most likely to report being in a management role, and slightly more likely identify as a Researcher-User

Slightly more likely to identify as a Researcher-User or User

Most likely to identify as a ‘Researcher’, and least likely to be report being in a management role

Discipline and specialism

Most likely to report being a conservation scientist

Most likely to be identify with Arts & Humanities disciplines and/or conservation

Most likely to report having a STEM subject specialism

Project size

Largest projects

Small-medium sized projects

Smallest projects

Experience

Less experienced (but a range of experience)

Most experienced (but a range of experience)

Least experienced

Note: Significance testing was used to generate the cluster profiles and all significant findings are included in the table. Italicized text indicates non-significant trends used to interpret the profiles.

Enablers and impediments of project satisfaction and achievement

Findings presented earlier in this paper demonstrated that Users of research evidence in practice tended to have lower project satisfaction and achievement ratings than academic Researchers, probably because Users were more frequently focussed on improving practice. The cluster analysis helped to identify other attitudes and characteristics of projects that were related to self-rated project success.

Compared to other clusters, the least satisfied group (Cluster 1) scored lower on ‘Ease of Collaboration’, and more frequently reported concerns about ‘Research Quality, difficulties ‘Understanding Partners’ Research Approaches’ and impediments due to ‘Internal Procedures and Working Practices’. On average, they worked on the largest projects and were more likely to include management and strategic goals (particularly ‘Standards and Guidelines’) amongst their top five goals. Non-significant trends indicated that this cluster were slightly more likely to be managers and Researcher-Users.

Conversely, when satisfaction ratings were highest (Cluster 2) there were also very high ratings for the ‘Ease of Collaboration’ factor. This factor measures aspects of the dynamic of collaboration such as trust, openness, relationships, shared goals and communication. When the dynamic was strong there were fewer impediments due to ‘Internal Procedures and Working Practices’, fewer concerns about ‘Research Quality’ and a greater ‘Understanding of Partners’ Research Approaches’. Respondents with this profile tended to be working on smaller than average projects, and were least likely to include management goals amongst their top five project goals.

This pattern of results was corroborated by the finding that there were correlations between some factor scores and the satisfaction and achievement ratings: ‘Ease of Collaboration’ and ‘Research Quality’ were strongly positively correlated with satisfaction and achievement ratings (see Additional file 2).

Profiling of the clusters suggested that the dynamic of collaboration was key to project satisfaction and self-rated achievement, and intertwined with perceptions of research quality and the formal processes of collaboration. Thematic analysis of comments supported this finding. When respondents were asked ‘what else do you think helped your project?’ their comments most frequently related to ‘people’ (78% of 128 comments): shared interests, a collaborative nature, enthusiasm, the mix of the team in terms of experience and expertise, good communication and opportunities for meeting and knowledge exchange were all important. Similarly, when people were asked ‘what else do you think hindered your research?’, ‘people’ was also a frequent category of comment (41% of comments): the clash of working cultures, lack of opportunities to meet, a lack of experience or expertise within the team, a lack of interest and communication challenges were noted. The full coding scheme is presented in Additional file 3.

Project size

The cluster profiles also suggested that the size and complexity of projects was important in shaping the success of collaborative projects. Based on the analysis described below, it can be proposed that there is an indirect relationship between project design and the perceived success of projects. Satisfaction and achievement ratings were associated with the dynamic of collaboration (as measured through factor scores) which was in turn associated with project size and complexity.

Project size was measured in three ways: number of other people, number of other partner organisations and number of other subjects represented in a project. Respondents were split into quartile groups for each of these measures and then factor sores were compared between projects of different size. Results for four factors where significant findings were found, and where there were similar trends across the four factors, are illustrated in Figure 11 (full details are in Additional file 2).
Figure 11

Relationships between three measures of project size and four factor scores. Significant findings are marked with an asterisk (based on the results of one-way ANOVAs, see Additional file 2). Top: Number of other people involved; middle: number of other partner organisations; bottom: number of other subjects represented in the project.

As the number of people in a project increased, so did concerns about ‘Research Quality’, whereas ‘Understanding of Partners’ research Approaches’ decreased. This suggests that rigour-relevance issues were more prevalent in larger projects and that this may have been related to communication and mutual understanding.

As the number of partner organisations increased, so did reports of impediments due to ‘Internal Procedures and Working Practices’. This may have been due to increased bureaucracy and also increased need to manage cross institutional relationships – as evidenced by reduced ‘Ease of Collaboration’ and ‘Understanding Partners’ Research Approaches’ scores.

Interdisciplinarity

There were significant non-linear relationships between the number of subjects in a project and three factor scores. As the number of subjects increased, so did concerns about ‘Research Quality’ and difficulties ‘Understanding Partners Research Approaches’, whereas ‘Ease of Collaboration’ reduced. This trend was observed up to 4 or 5 other subjects, after which the trend levelled off or reversed (6 or more other subjects). This suggests that increased interdisciplinarity was associated with increased communication and rigour-relevance challenges, but only up to a certain point, after which it did not create more impediments. The most interdisciplinary projects did not necessarily involve the most people or partners. They may have been constructed from several smaller teams.

The finding that some of the most interdisciplinary projects were protected from some of the impediments associated with larger projects is supported by analysis of comments in response to the question ‘What else interested you about this project?’. There was high interest in bringing diverse groups of people together with mixed backgrounds and skills in order to progress practice-focussed research.

Significant findings were also observed for a fifth factor, ‘Practice-Focussed Research’. As this factor was less reliable than others and overlapped with the questionnaire section on ‘Goals’, the results are presented in Additional file 2 only.

Interest in collaborating

Self-ratings of project satisfaction and achievement of goals are one way of measuring the success of projects. However, the benefits of collaboration are not always linked to achievement of goals such as translating research into practice. The questionnaire also allowed a more in depth look at the collaborative process itself and what may enable or impede this.

For example, the cluster analysis identified a group of respondents (Cluster 3, 21%) who were satisfied with their project but who, in comparison to other clusters, reported a lesser preference for a collaborative working style, a lower interest in bridging disciplines, less understanding of partner’s research approaches and less institutional recognition. This cluster were the most likely of the three clusters to be academic researchers (particularly STEM researchers), focussed on academic goals and resources and less practice-focussed. They were also the least experienced cluster. This potentially suggests a skills or experience gap for a sizable minority of participants in collaborative heritage science research. Further analysis helped to identify areas where the collaborative process can be better supported.

Project roles

Different roles within a project may have distinct aims and aspirations. Three types of role were compared in terms of their personal goals, their factor scores and satisfaction and achievement ratings (see Additional file 2): Project Managers/Supervisors (principal investigators, task leaders, project managers, PhD/KTP supervisors and post-doctoral fellows with management responsibilities), Partners (partners, co-investigators and steering committee/advisory group members) and Junior/Student Researchers (post-doctoral researchers without management responsibilities, research associates/assistants and PhD/KTP students).

The goals of senior and junior roles reflected their stage of career: Junior/Student researchers were more likely to include personal and career development goals amongst their top five goals than senior roles, who were more focussed on project goals such as ‘Access to Resources’ (Figure 12).
Figure 12

Significant differences between project roles in terms of goals. Tested using chi-square (see Additional file 2).

Attitudes to collaboration reflected the types of research activities each role was engaged in. Project Managers/Supervisors reported more impediments due to working practices, but a greater preference for a collaborative working style than Junior/Student researchers (Figure 13). Junior/Student researchers also reported a preference for a collaborative working style, however their ratings were significantly lower than for senior roles.
Figure 13

Significant differences between project roles in terms of factor scores. Tested using one-way ANOVAs with post-hoc comparisons (see Additional file 2).

Comments suggested that the opportunity to work in non-HEIs and form contacts with practitioners is highly valued by Junior/Student researchers: a number of PhD/KTP students made positive comments about their experience of working across institutions. Other comments suggested that the energy, enthusiasm and commitment of project participants, including student and junior researchers, may be a key enabler of collaborative research. However, a small number of comments addressed the challenges of collaborative studentships and whether students can meet project goals and make the most out of the learning experience.

Experience and expertise

The differences between senior and junior researchers suggests a skills gap, which may be related to stage of career, the types of activities each is engaged in and level of experience.

Respondents were asked about their level of experience in terms of number of years working on collaborative research projects and number of collaborative research projects worked on. As the experience of respondents increased in terms of number of years, so did their preference for a ‘Collaborative Working Style’ and their ‘Interest in Bridging Disciplines’. A similar pattern of findings was observed for number of projects: the more projects a respondent had worked on the greater their preference for a ‘Collaborative Working Style’, the higher their ratings of ‘Research Quality’ and the fewer challenges they found in ‘Understanding Partners’ Research Approaches’. Whereas length of experience (number of years) was related to interest and motivation, breadth of experience (number of projects) appeared to be related to collaborative skills and quality. Significant findings are illustrated in Figure 14. Full results are provided in Additional file 2.
Figure 14

Significant associations between experience and factor scores. Tested using oneway ANOVAs with post-hoc comparisons (see Additional file 2). Top: Experience (Years). Bottom: Experience (Projects).

The analysis also pointed to a non-linear relationship between experience and collaborative skills. As the experience of respondents increased so did their interest in collaboration and their collaborative skills, but this was with the exception of some of the most experienced respondents (>20 years experience; >10 projects).

Language and communication

Language and communication appear in the literature as significant impediments to collaboration but did not emerge as separate issues in the factor analysis of the questionnaire. Instead, language and communication issues emerged across many areas of the results. For example, statements relating to language and communication were distributed across the factor solution, demonstrating that language and communication underpin many aspects of effective collaborative research – see Table 2.

Resources

Resources’ did not emerge as a theme in the factor analysis. Statements about resources did not load onto any factor or were distributed across the factor structure, suggesting that resource issues may be associated with a number of aspects of the experience of collaboration. However, comparisons of clusters showed that ‘Access to Resources’ was a goal that distinguished between Clusters 2 and 3, being needed least frequently by Cluster 2 who most frequently reported positive experiences of collaboration. Responses to the open box question ‘what else do you think hindered your project?’ revealed a link between resources and other aspects of effective collaboration: 59% of comments related to ‘resources’, particularly ‘time’ needed to develop projects and relationships.

Barriers to participation in collaborative heritage science research

The impediments and enablers to effective collaboration reported in this paper may be valid for respondents to the questionnaire, who were made up of recent participants in collaborative heritage science research. However, the paper does not address barriers to participation in collaborative research, as those who had chosen not to, were not aware of, or were unable to participate in recent projects were not surveyed.

As an example, there were only nine respondents from industry/SMEs. Sixty-nine respondents said they had an industry/SME partner in their project, but HEIs were much more likely to have collaborated with MLAGs and Heritage Organisations (90%) than with industry/SMEs (30%). Comments from industry/SME respondents and research partners pointed to barriers to engagement in collaborative research despite an interest from both groups in collaboration. Academic research might not be viewed as relevant to industry/SME goals and expectations, there may be little institutional support in industry/SMEs for dedicating time to collaborative research, there may be few opportunities to make connections between researchers and industry/SMEs and there are differences in working cultures between different types of institution. However, there was also mutual appreciation of access to expertise, knowledge exchange and potential research partners. The development of applications/technologies is an area where HEIs were interested in more collaboration with industry/SMEs. However, expectations about moving from basic research to application may need to be managed. A key barrier to collaboration mentioned by one MLAG respondent was that there are limited funding routes for non-HEIs wishing to work with HEIs.

Conclusions

Most questionnaire respondents reported a positive experience of collaborative research and aspired to both rigour and relevance in their projects: nevertheless, there were four key pieces of evidence that a rigour-relevance gap exists in heritage science research. Although most respondents were satisfied that the aims of their project were achieved (84% satisfied), they were less satisfied that the impact of the project would be realised (49% satisfied). Practice-focussed project goals, such as better care and conservation of cultural heritage and collection management, were associated with lower self-rated achievement than academic goals. In addition, some types of practice-focussed goals were not widely held (management goals). Finally, a sizeable minority of respondents reported that they had encountered challenges in collaboration or were shown to be less interested or skilled in collaboration than other respondents.

Further evidence for a rigour relevance gap came from a comparison of academic researchers and users of research evidence in practice. Users had higher expectations than academic researchers that research would improve practice, but gave lower achievement ratings for their goals than academic researchers, were less satisfied with the outcomes of collaborative research and gave lower ratings of research quality.

Although the questionnaire results point to a rigour-relevance gap in heritage science, the results also challenge definitions of rigour-relevance and the validity of boundaries between researchers and users. For example, the factor analysis showed that rigour and relevance are not opposing aims in heritage science research. Factors such as ‘Research Quality’ and ‘Understanding Partners’ Research Approaches’ contained statements about both the rigour (quality, depth, complexity) of research and its relevance (needs-led, user acceptance, credibility). Similarly, goal choice and open box comments suggested that respondents sought both rigour and relevance in their research, and that rigour is often seen as a route to relevance and vice versa (see open box comments about ‘research and practice’, Additional file 3).

One explanation for these observations is that many respondents had dual roles as users and researchers and worked across institutions. Both rigour and relevance were aspects of research quality for these hybrid researchers.

Although boundaries are blurred, there remains evidence for a gap between researchers and users in terms of their expectations and achievements in collaborative heritage science research. Further contributions the questionnaire results can make is to question what constitutes achievement in collaborative research and to highlight areas where the expectations and cultures of different institutions and individuals could become barriers to effective collaboration. In particular, a wider view of the benefits of collaboration, beyond translating research into practice, may be useful.

The findings are in line with research in other sectors which points to almost intractable differences between research and user communities stemming from differing expectations about research and differing institutional cultures. However, the questionnaire also provided information about ways to bridge the rigour relevance gap and increase the quality and effectiveness of collaboration. The results suggest a number of areas which could be addressed.

Goals and expectations

Designing research to accommodate different types of goals or creating an understanding of different goals may be a key stage in effective collaboration (e.g. understanding the role and expectations of HEIs and non-HEIs in research).

Translating research into practice

The wider benefits of collaborative research, such as knowledge exchange, community building, training, developing long term relationships and building the evidence base, needs to be recognised.

Team composition

Experience and expertise is useful in terms of bringing formal and informal collaborative skills to a team. Attitudes to and interest in collaboration may also be important.

Practitioner-Researchers

There are many researchers in heritage science who identify as both researchers and users and who work across both HEIs and non-HEIs. In other fields (e.g. nursing, social work, policing) such individuals are known as ‘practitioner-researchers’ [46, 47] and play a role in bridging the rigour- relevance gap, having both research skills and knowledge of non-HEIs’ needs and priorities.

Early career researchers

Development of collaborative skills and involvement of non-HEIs in supervision and career development should be promoted. The Knowledge Transfer Partnership scheme has been acknowledged as a successful way of partnering HEIs and non-HEIs, and developing the skills of early career researchers by embedding them in practice environments [4850].

The dynamic of collaboration

The dynamic of collaboration was key to satisfaction and self-rated achievement. Methods of facilitating projects to promote shared goals, mutual understanding, good communications and shared language may be useful.

Research design

Iterative, reflective and participatory approaches [5153], could help bridge the gap between research and practice in heritage science.

Time

Respondents commented that time is needed to build relationships and develop research. Longer projects, or longer research programmes, could promote good team dynamics and allow time to link research to practice.

Interdisciplinarity

There was a strong drive for interdisciplinarity amongst respondents, in order to progress practice-focussed research. Larger, more complex projects require good project management and leadership.

Questionnaire findings have highlighted a number of areas for action in terms of the future of collaborative heritage science research. A more in depth look at each of these areas in order to identify best practice as applied to the heritage science field may be needed. The findings from this research were reported to project partners in September 2013 in order to support the development of recommendations for the future of collaborative heritage science research [54].

Authors’ information

CD is an independent research consultant and was previously a postdoctoral researcher at the UCL Centre for Sustainable Heritage until November 2013. CD’s background is in research psychology and she has also worked as a researcher in the public and third sectors. CD became interested in sustainable heritage after conducting a research project for the Museums, Libraries and Archives Council concerning the social outcomes of a museum funding programme.

NB is head of Collections Care for The National Archives, UK. Her research focuses on the development of improved collection management strategies and the development of protocols to assess the value of heritage collections, and technical examination of cultural heritage collections.

KF is a lecturer at the UCL Centre for Sustainable Heritage and Course Director of the MRes Heritage Science at the Centre. Her research revolves around heritage management with an emphasis on in situ conservation, community-led and participatory approaches to site management and public engagement.

PL has a background in conservation and in her current role as Head of Collection Care Research at Tate she is focussed on developing, leading and supporting research related to the conservation and management of Tate's collections. She is an accredited member of ICON and a trustee of the National Heritage Science Forum.

AT is Professor of Modern History and Director of the Centre for Imperial and Global History at the University of Exeter, previously Dean of the Faculty of Arts and Pro-Vice Chancellor for Research at the University of Leeds. Professor Thompson is a Council member of the UK AHRC, and the Leadership Fellow for the UK AHRC Theme: "Care for the Future: Thinking Forward Through the Past" - an important strand of which is heritage research.

MS is Senior Lecturer at the UCL Centre for Sustainable Heritage and Course Director of the MRes Heritage Science at the Centre. His research focuses on the development of new scientific tools and methods of study of heritage materials, collections and their interactions with the environment, particularly development and use of damage functions and integrated modelling of heritage collections.

Abbreviations

AHRC: 

UK Arts and Humanities Research Council

ANOVA: 

Analysis of Variance

EPSRC: 

UK Engineering and Physical Sciences Research Council

HEI: 

Higher Education Institution

IRO: 

Indpendent Research Organisation

MLAG: 

Museum, Library, Archive or Gallery

N: 

sample size

SD: 

Standard Deviation

SME: 

Small to medium enterprise

SPSS: 

Statistical Package for the Social Sciences

SHP: 

UK Science and Heritage programme

p: 

probability (significance level)

UCL: 

University College London

UK: 

United Kingdom

USA: 

United States of America

PAF: 

Principal axis factoring.

Declarations

Acknowledgements

Carolien Coon transcribed data from the Mind the Gap launch meeting focus groups and recruited participants to take part in the Mind the Gap questionnaire. Debbie Williams (AHRC/EPSRC SHP Coordinator) provided contact lists, distributed the survey by email and provided background information about the AHRC/EPSRC Science and Heritage Programme.

Project partners contributed to the development of the questionnaire through participation in the launch event, provided background literature and responded to questionnaire findings: Ben Rampton (Kings College London), Boris Pretzel (Victoria & Albert Museum), Dean Sully (UCL), Geraldine Horan (UCL), Haidy Geismar (UCL), Jerry Podany (Getty), John Hughes (University of the West of Scotland), Julie Harvey (Natural History Museum), Katy Lithgow (National Trust), Alfred Kieser (Zeppelin University), Kostas Ntanos (The National Archives, UK), Michael Moss (University of Glasgow) and Roger Kneebone (Imperial College London).

The Mind the Gap project was funded by the AHRC/EPSRC Science and Heritage Programme.

Authors’ Affiliations

(1)
Centre for Sustainable Heritage, University College London
(2)
The National Archives, Kew
(3)
Tate Research, Tate
(4)
Department of History, University of Exeter, Amory Building

References

  1. Katz JS, Martin BR: What is collaboration?. Research Policy. 1997, 26: 1-18. 10.1016/S0048-7333(96)00917-1.View ArticleGoogle Scholar
  2. Department for Business Innovation and Skills (BIS): Innovation and Research Strategy for Growth. 2011, Norwich: The Stationery OfficeGoogle Scholar
  3. Communication from the Commission (COM): Europe 2020: A Strategy for Smart, Sustainable and Inclusive Growth. 2010, Brussels: European CommissionGoogle Scholar
  4. Research Excellence Framework (REF): REF 01.2011: Decisions on Assessing Research Impact. 2011, Bristol: REFGoogle Scholar
  5. Declaration by delegates at the 8th European Commission Conference on Sustaining Europe’s Cultural Heritage. 2008, Ljubljana, Slovenia: ‘Cultural Heritage Research Meets Practice’, http://www.si.edu/mci/downloads/CHRESP%202008/CHRESP_Declaration.pdf,
  6. National Heritage Science Forum. http://www.heritagescienceforum.org.uk,
  7. National Heritage Science Strategy Steering Group: Our Vision and Strategy for Heritage Science. 2010, National Heritage Science Strategy Steering Group, It is available online at: http://www.heritagescience.ac.uk/resources/publications/NHSSFinalGoogle Scholar
  8. Harvey D, Viles H: Workshop on Research Methodologies in Heritage Science: Facilitating Interdisciplinary Collaboration. 2008, Pitt Rivers Museum, Oxford: Report of a workshop sponsored by the AHRC/ESRC Science and Heritage ProgrammeGoogle Scholar
  9. Cherney A, McGee TR: Utilization of social science research: results of a pilot study among Australian sociologists and criminologists. J Sociol. 2011, 47: 144-162. 10.1177/1440783310386831.View ArticleGoogle Scholar
  10. Knott J, Wildavsky A: If dissemination is the solution, what is the problem? Knowledge, Creation, Diffusion. Utilization. 1980, 1: 537-578.Google Scholar
  11. Landry R, Amara N, Lamari M: Climbing the ladder of research utilization - evidence from social science research. Sci Comm. 2001, 22: 396-422. 10.1177/1075547001022004003.View ArticleGoogle Scholar
  12. Landry R, Lamari M, Amara N: The extent and determinants of the utilization of university research in government agencies. Publ Admin Rev. 2003, 63: 192-205. 10.1111/1540-6210.00279.View ArticleGoogle Scholar
  13. Wouters J: CHRESP Questionnaire. 2008, Ljubljana, Slovenia: Presentation at the 8th European Commission Conference on Sustaining Europe’s Cultural HeritageGoogle Scholar
  14. Hughes A, Kitson M, Probert J, Bullock A, Milner I: Hidden Connections: Knowledge Exchange Between the Arts and Humanities and the Private, Public and Third Sectors. 2011, Cambridge: Arts and Humanities Research Council, Swindon; Centre for Busines ResearchGoogle Scholar
  15. Public and Corporate Economic Consultants (PACEC): Evaluation of the Collaborative Research and Development Programmes: Final Report. 2011, Cambridge-London: PACEC for the Technology Strategy BoardGoogle Scholar
  16. The National Academies: Overcoming Barriers to Collaborative Research: Report of a Workshop. 1999, Washington DC: National Academy PressGoogle Scholar
  17. Podestá GP, Natenzon CE, Hidalgo C, Ruiz Toranzo F: Interdisciplinary production of knowledge with participation of stakeholders: a case study of a collaborative project on climate variability, human decisions and agricultural ecosystems in the Argentine Pampas. Environ Sci Pol. 2013, 26: 40-48.View ArticleGoogle Scholar
  18. Boehm DN, Hogan T: ‘A jack of all trades': the role of PIs in the establishment and management of collaborative networks in scientific knowledge commercialisation. J Tech Tran. 2014, 39: 1-16. 10.1007/s10961-012-9270-y.View ArticleGoogle Scholar
  19. Bazeley P: Defining ‘early career’ in research. Higher Education. 2003, 45: 257-279. 10.1023/A:1022698529612.View ArticleGoogle Scholar
  20. Spanos YE, Vonortas N: Scale and performance in publicly funded collaborative research and development. R&D Management. 2012, 42: 494-513.View ArticleGoogle Scholar
  21. Cummings JN, Kiesler S: Collaborative research across disciplinary and organizational boundaries. Soc Stud Sci. 2005, 35: 703-722. 10.1177/0306312705055535.View ArticleGoogle Scholar
  22. Thompson J: Building collective communication competence in interdisciplinary research teams. J Appl Comm Res. 2009, 37: 278-297. 10.1080/00909880903025911.View ArticleGoogle Scholar
  23. Monteiro RM, Keating E: Managing misunderstandings: The role of language in interdisciplinary scientific collaboration. Sci Comm. 2009, 31: 6-28. 10.1177/1075547008330922.View ArticleGoogle Scholar
  24. Heemskerk M, Wilson K, Pavao-Zuckerman M: Conceptual models as tools for communication across disciplines. Ecology and Society. 2003, 7: 8-Google Scholar
  25. Fincham R, Clark T: Introduction: can we bridge the rigour-relevance gap?. J Manag Stud. 2009, 46: 510-515. 10.1111/j.1467-6486.2009.00834.x.View ArticleGoogle Scholar
  26. Kaiser A, Leiner L: Why the rigour-relevance gap in management science is unbridgeable. J Manag Stud. 2009, 46: 516-533. 10.1111/j.1467-6486.2009.00831.x.View ArticleGoogle Scholar
  27. Van de Ven AH: Engaged Scholarship: A Guide for Organizational and Social Research. 2007, Oxford: Oxford University PressGoogle Scholar
  28. Furnham A: The Engaging Manager: The Joy of Management and Being Managed. 2012, New York: Plagrave MacmillanView ArticleGoogle Scholar
  29. Hodgkinson GP, Rousseau DM: Bridging the rigour-relevance gap in management: it’s already happening!. J Manag Stud. 2009, 2009 (46): 534-546.View ArticleGoogle Scholar
  30. Martensen A, Martensen P: Extending rigor and relevance: towards credible, contributory and communicable research. Proceedings of the 15th European Conference on Information Systems. 2007, 1325-1333.Google Scholar
  31. Harty C, Leiringer R: Social science research and construction: balancing rigour and relevance. Proceedings of CME 25: Construction Management and Economics 25th Anniversary Conference - 'Past, Present and Future'. Edited by: Hughes W. 2008, 1571-1583.Google Scholar
  32. Brannen J: Mixed Methods Research: A Discussion Paper. 2005, Southampton: ESRC National Centre for Research Methods, NCRM Methods Review Paper (NCRM005), http://eprints.ncrm.ac.uk/89/1/MethodsReviewPaperNCRM-005.pdf,Google Scholar
  33. Watson D, Clark LA, Weber K, Assenheimer JS, Strauss ME, McCormick RA: Testing a tripartite model: II. Exploring the symptom structure of anxiety and depression in student, adult, and patient samples. J Abnorm Psychol. 1995, 104: 15-25.View ArticleGoogle Scholar
  34. Eysenck HJ: The Structure of Human Personality. 1953, New York: WileyGoogle Scholar
  35. McCrae RR, Costa PT: Validation of the five-factor model of personality across instruments and observers. J Abnorm Psychol. 1987, 52: 81-90.Google Scholar
  36. Dillon C, Lindsay W, Taylor J, Fouseki K, Bell N, Strlič M: Collections demography: stakeholders views on the lifetime of collections. Postprints of the Munich Climate Conference - Climate for Collections: Standards and Uncertainties. Edited by: Ashley-Smith J, Burmester A, Eibl M. 2012, 45-58.Google Scholar
  37. Robb J, Dillon C, Rumsey M, Strlič M: Quantitative assessment of perceived value of geological collections by ‘experts’ for improved collections management. The Geological Curator. 2014, 9: 529-538.Google Scholar
  38. Cattell RB: The description of personality: basic traits resolved into clusters. J Abnorm Soc Psychol. 1943, 38: 476-506.View ArticleGoogle Scholar
  39. Vickers D, Rees P: Introducing the national classification of census output areas. Population Trends. 2006, 125: 15-24.Google Scholar
  40. Wedel M, Kamakura W: Market Segmentation - Conceptual and Methodological Foundations. 1998, Boston: Kluwer Academic PublishersGoogle Scholar
  41. Oppenheim AN: Questionnaire Design, Interviewing and Attitude Measurement. 1992, London: ContinuumGoogle Scholar
  42. Jenkins C, Dillman D: Towards a theory of self-administered questionnaire design. Survey Measurement and Process Quality. Edited by: Lyberg L, Biemer P, Collins M, De Leeuw ED, Dippo C, Schwarz N, Trewin D. 1997, New York: Wiley Interscience, 7-Google Scholar
  43. Campanelli P: Testing survey questions using cognitive interviewing. International Handbook of Survey Methodology. Edited by: de Leeuw ED, Hox JJ, Dillman DA. 2008, New Jersey: Erlbaum, 10-Google Scholar
  44. Tinsley HEA, Tinsley DJ: Uses of factor analysis in counselling psychology. J Counsel Psychol. 1987, 34: 414-424.View ArticleGoogle Scholar
  45. Upton G, Cook I: Oxford Dictionary of Statistics. 2008, Oxford: Oxford University PressView ArticleGoogle Scholar
  46. Kara H: Research and Evaluation for Busy Practitioners: A Time-Saving Guide. 2012, Bristol: The Policy PressGoogle Scholar
  47. Fox M, Martin P, Green G: Doing Practitioner Research. 2007, London: SageGoogle Scholar
  48. Howlett RJ: Knowledge transfer between UK universities and businesses. Smart Innovation, Systems and Technologies. 2010, 5: 1-14. 10.1007/978-3-642-14594-0_1.View ArticleGoogle Scholar
  49. Regeneris Consulting Ltd: Knowledge Transfer Partnerships Strategic Review. 2010, Altrincham: Regeneris Consulting Ltd for the Technology Strategy BoardGoogle Scholar
  50. Council for Industry and Higher Education (CIHE): Key Attributes for Successful Knowledge Transfer Partnerships. 2012, London: CIHE for the Technology Strategy Board and the Research CouncilsGoogle Scholar
  51. Lewin K: Action research and minority problems. J Soc Issues. 1946, 2: 34-46. 10.1111/j.1540-4560.1946.tb02295.x.View ArticleGoogle Scholar
  52. Wenger E: Communities of Practice: Learning, Meaning, and Identity. 1998, Cambridge: Cambridge University PressView ArticleGoogle Scholar
  53. Balazs CL, Morello-Frosch R: The three Rs: how community-based participatory research strengthens the rigor, relevance, and reach of science. Environ Justice. 2013, 6: 9-16. 10.1089/env.2012.0017.View ArticleGoogle Scholar
  54. Bell N, Strlič M, Fouseki K, Laurenson P, Thomspon AS, Dillon C: Mind the Gap: Rigour and Relevance in Heritage Science Research. 2014, AHRC/EPSRC SHP final project report available at http://www.nationalarchives.gov.uk/about/mind-the-gap.htmGoogle Scholar

Copyright

© Dillon et al.; licensee Chemistry Central Ltd. 2014

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.