Evaluation of metadata describing topographic maps in a National Library
Heritage Science volume 8, Article number: 113 (2020)
Nowadays, map users come from a diverse disciplines and use maps for different purposes, but they all are confronted with the same task: finding the maps they need. Searching systems, used by museums or archives collecting maps, rely on metadata. However, the scope and method of saving map data in metadata are not often connected with the way in which users search for maps and therefore they are often unable to find the maps necessary for their research. Moreover, the ways of describing maps in metadata are different in each institution, what hinders the interoperability and wide using of data. In this context, the key questions are: what scope of metadata can provide a proper description of maps and how to evaluate if metadata from concerned institution fulfill these needs and give user the chance to find necessary maps. Therefore, the aim of our research was to evaluate topographic maps metadata in the context of the possibility to evaluate their usability of specific users’ needs in the National Library of Poland (NLP). The area of interest are topographic maps because they are the most heavily used for research and other purposes. To assess metadata for the purposes of evaluating the usability of topographic maps, a two-stage procedure was used. It covers the evaluation of topographic maps metadata, followed by their assessment in terms of whether maps found on their basis might be useful in our research. The research was carried out on 35,092 topographic maps issued in the years 1608–2017, in scales ranging from 1:10,000 to 1:500,000 from the National Library of Poland. These maps mainly cover the current territories of Poland, Belarus, Germany, Ukraine, Czech Republic, Lithuania, and Russia. The conducted analysis of topographic maps in the National Library of Poland revealed that the collected data are very interesting and match the users’ needs.
Map users come from a diverse and broad set of educational and other disciplines, but they all are confronted with the task of finding the map they need. Currently, the objects searching systems used by libraries or archives rely on metadata. However, the scope and method of saving map data in metadata are not connected with the way in which users search for map resources and therefore they are often unable to find the maps necessary for their research. Moreover, the ways of describing maps in metadata are different in each institution, what hinders the interoperability and wide using of data.
From this perspective, the main goal of our research is to evaluate metadata of topographic maps, as widely used maps collected in libraries, in the context of the possibility to evaluate their usability for specific users' needs in the National Library of Poland. The area of interest are topographic maps because they are the most heavily used for research and other purposes.
The degree of maps usability for specific users’ needs was determined by Grygorenko  on the basis of up-to-dateness, accuracy of map, and visualization. Vuorela  emphasizes the importance of visualization and accuracy (spatial and temporal) in the description of maps in his research. When a map is transformed into a digital version, the scale, map projection and presentation methods (symbolization) are important, because they can help the users find the maps they need. The scale informs us about the mathematical relationship between real objects, phenomena, and what is presented on the map, as well as the level of detail of data presentation on the map . Map projection provides information on distortions and facilitates the transformation of data from analog to digital form . Finally, the cartographic methods of presentation (symbolization) offer us additional information about what and how is presented (graphically) on the map and how it has been classified . This type of data can be collected in metadata, but the lack of them or improper recording in metadata make impossible both finding and wide use of maps.
Considering the above, three research questions were identified:
First research question (1RQ): What scope of metadata can provide a proper description of topographic maps?
Second research question (2RQ): To what extent do the topographic maps from the selected library fulfill the rules for the occurrence of map-specific information in metadata which give the best chance of finding maps that fulfil the user needs?
Third research question (3RQ): To what extent do topographic maps metadata support the evaluation of map usability?
The research undertaken is based on evaluating the quality of metadata, which, in turn, is the basis for evaluating the usability of maps for users' needs. The results of the evaluation of library resources shows whether the user will have the chance to find the resources they need.
According to Youngblood  and Fish et al.  map users come from a diverse disciplines, it ranges from social scientists and researchers, through anthropologists to historians. Each of these sub-groups uses maps in a different way and for different purposes, depending on their main fields of academic interest, from analyzing cultural phenomena to studying battles or invasions. Frajer and Geletič  emphasized the significance of old maps for landscape management in the given time and place.
Usually, users do not look for a specific map; they do not know the title, author, or publisher, so most bibliographic descriptions are practically useless . Secondly, users do not necessarily think about using coordinates in a search , although they are directly related to the geographic extent of the map and the objects presented on it . Unfortunately, maps have been mostly described by cataloguers as books , despite the fact that the standards include fields and elements of metadata that enable the description of cartographic documents [12,13,14,15], they are usually optional and not supplemented by librarians [11, 16]. As a result, the metadata range used for books description was adopted to the description of the maps. This is also due to the fact that certain information typical of books is typically also located on each of the maps . Additionally, many studies [18,19,20,21,22] emphasize the need to use geographic data to increase the interoperability of data from various scientific disciplines, which is rather time-consuming. However, the economic factor requires cataloguers to make the descriptions of objects in libraries shorter , while the lack of specialized skills make it difficult to extract properly the geographical data needed to complete the metadata record by librarians . This approach results in the fact that the information collected cannot be fully used by map seekers.
In addition to searching for specific objects, metadata can be also used to evaluate the quality of collection metadata gathered in a given library e.g. in the context of the interoperability . The evaluation of map metadata is an important issue in particular for large collections. Consistent application of coherent map description principles serves the purpose of their exchange and integration .
In metadata evaluation, different criteria can be taken into account. The most important of them are: completeness, accessibility, accuracy, and conformance [25,26,27,28]. Completeness means the degree to which an object has been presented using all available metadata . Accuracy is the degree to which supplemented metadata describe a digital object . Accessibility is defined as the degree to which a user can access and use data and conformance means how metadata helps users complete the tasks they specify . All these criteria decide about the metadata quality. However, all of them demands standardized metadata and their values. Limitation the metadata values to a mandatory set of elements, while the information that is the most important for specific users is collected in optional or recommended elements , as well as the lack of common rules for recording metadata values  make difficult or even impossible not only finding a map, but also their metadata evaluation and wide use.
National Library of Poland map collection
The National Library of Poland (NLP), supervised by the Ministry of Culture and National Heritage, is the main library in Poland and one of the most important national cultural institutions. It operates pursuant to the Polish Act on Libraries  and the National Library Statute . Its mission is to protect national heritage preserved in the form of documents in all formats, including manuscripts and audio-visual materials, and to acquire, store, and permanently archive Polish intellectual works, the most important international works, and foreign publications related to Poland .
The National Library cartographic collection contains over 130,000 objects, which include maps, atlases and globes. Among them, there are 35,090 topographic maps issued in the years 1608–2017, in scales ranging from 1:10,000 to 1:500,000 . These maps mainly cover the area of Poland, Belarus, Germany, Ukraine, Czech Republic, Lithuania, and Russia. The collection includes maps from three most important former cartographic centers: Lviv, headed by E. Romer (Książnica-Atlas), Warsaw, represented mainly by the Military Geographic Institute (German, Russian, and Austrian maps from the period 1772–1918), and modern topographic studies developed by the Head Office of Geodesy and Cartography.
All topographic maps collected in the National Library of Poland are described in library catalog with metadata that contained the same information as any other object in the library catalogue (book, magazine), i.e. title, creator, publisher, language, subject of study, rights of use, but also information characteristic for maps, such as map scale or/and map format (size) (Fig. 1). This additional information are added as the result of the map collection team initiative, who considered that this kind of data are necessary for the proper characteristics of the maps and hence their subsequent finding in the database.
Based on the map collection team experience, the guidelines for cataloguing cartographic documents were developed and implemented. Currently, the following documents define the method and scope of map description by metadata in the National Library of Poland catalogue:
MARC 21 formats ,
Instrukcja katalogowania dokumentów kartograficznych [English: Instructions for cataloguing cartographic documents], R. Szura, M. Krynicka, the National Library of Poland, 2012 ,
Przepisy katalogowania. Mapy, [English: Cataloging regulations. Maps], the National Library of Poland, 2018 .
In the years 2007–2019, the above rules were implemented into maps metadata by the team of the 5 to 17 cartographers and geographers. They used two different software Millennium and from 2019 Sierra (Innovative) to collect metadata .
Unfortunately, not all maps have a unified bibliographic description. This is due to the fact that the rules of description that exist in the National Library are constantly evolving and the way of writing information about them in metadata has changed during many years of describing library objects. After the introduction of such changes, the records that were developed according to older, earlier principles and guidelines were not corrected. Due to this, previously developed facilities do not comply with the rules currently in force. Since the introduction of new rules (sometimes since earlier times, because instructions are also the result of the evolution of metadata replenishment), newly developed metadata have been supplemented according to new instructions. Unfortunately, the metadata of the National Library of Poland does not provide information about the version of the instruction on the basis of which specific metadata was created. Knowledge about the instructions according to which specific records were compiled depends on the experience of individual librarians and their knowledge of collections.
The research focused on the evaluation of archival maps metadata so as to use them to evaluate the usability of topographic maps collected in the library. We chose topographic maps because they include elements of the geographic map content, with particular emphasis on topographic objects (topography as well as objects and point of interest). Therefore, they constitute the basic source of information about elements of the environment and topography of the area in the past and are widely used in cartographic, geographic, historical, and environmental research. They are the basic maps used in almost all fields of economy. Moreover, the content items of topographic maps are the basis for preparing different types of maps, e.g. tourist maps. They are detailed maps, and in each country the preparation of such maps is regulated by specific, strict guidelines.
In the research, a topographic map collection (32,090 topographic maps) from the National Library of Poland was used. The research was conducted from March to September 2019.
To assess metadata for the purposes of usability of topographic maps, a two-stage procedure was used. It covered the evaluation of topographic maps metadata (I stage), and then their assessment in terms of whether maps found on their basis could be useful in user research (II stage).
The first part of the procedure (Fig. 2)—evaluation of topographic maps metadata—is based on the method of evaluation of the accessibility of archival cartographic documents in libraries defined by the authors . This methodology uses evaluation criteria of accessibility of resources and their features to assess quality of libraries metadata. The criteria are based on those metadata elements that contain information providing the basic characteristics of a spatial data set, while features identify the difficulty of obtaining those metadata values and are differentiated by the weights. In this procedure metadata prepared in MARC 21 standard  were analyzed in relation to the metadata scope pattern based on the ISO (International Organization for Standardization) 19,115 standard  in order to compare metadata from different libraries.
In the presented research project, the first part of the procedure was based on the original methodology, which has been specified here in details and expanded on the basis of current trends in assessing the correctness of metadata resulting from the analysis of subject literature . Moreover, present research focused on the evaluation of topographic maps from one library, unlike previous studies that compared data from several libraries. The second part of the procedure was the proposal of a completely new method of using metadata quality assessment for the evaluation of the usability of topographic maps in user’s research.
Evaluation of the topographic maps metadata
The basis for the topographic map metadata evaluation stage was the assumption that these maps are spatial data and that they should be described as such data in a library. Therefore, a model metadata profile was adopted, and the rules of describing topographic maps existing in the library were compared with the adopted pattern together with the metadata of the selected topographic map collection. The metadata was obtained from NLP in the form of the xls file, as this is the only form that can be exported from the National Library system. To compare received metadata with the pattern, they were imported to the Microsoft Excel, where Author’s scripts and formulas were defined to classify and analyse obtained data.
The model metadata profile
Spatial data are currently described with use of the ISO 19,100 series standards  defining the elements and the way in which they should be included in the description of the resource to obtain its correct characteristics. The basis for developing the model metadata profile was the scope pattern , MARC 21 (MAchine-Readable Cataloging) , as well as literature on map development and their usability for various purposes [1, 41,42,43].
In current research, the same as in the original procedure, the model metadata profile (metadata elements, their definitions, and the features of recording metadata values, treated as a metadata scope pattern), which provides the proper characteristic of topographic maps, has also been accepted. It was the basis for defining evaluation criteria. However, some changes were introduced to the current profile in comparison to the original one. First of all, two previous criteria have been removed—Access Rights 1 and Access Rights 2. They have been merged into one—Rights criterion. This is the result of analyzes of libraries with large collections, e.g. Digital Repository of Scientific Institutes , POLONA , National Library of Scotland  and metadata stored in them, as well as the literature on MARC 21 . It has been found that the Rights criterion exhausts information on licenses for objects in libraries. The Orientation criterion was also removed, because the collected geographic coordinates allow the map to be properly oriented, so it was duplicate information. At the same time, information about mandatory/optional metadata elements was added to the profile. Moreover, definitions, values, and examples were associated with existing dictionaries and data recording standards.
The evaluation of the library metadata profile
The metadata elements defined in the model metadata profile were used as evaluation criteria, while their values were used as features that were weighted in the same way as in the original procedure. The evaluation of the library metadata profile for topographic maps also consists in determining the way and the rules for recording metadata values in the library metadata profile, and then assigning weight (wk) to individual metadata elements depending on the adopted principles. These weights are also indicators of the degree to which each of the metadata elements is consistent with the corresponding element of the model metadata profile. The weights were assigned to the features based on the study by Kuzma and Moscicka , and they took the values 0.0, 0.5, 0.8 and 1.0, where 0.0 means that data are not available in metadata, 1.0 means that data can be taken directly from metadata, and 0.5 and 0.8 mean that data can be extracted from metadata with more (0.5) or less (0.8) workload.
In order to compare the topographic maps metadata profile and the model metadata profile, based on the rules/ways and their weights, the degree to which the metadata values provided according to library rules can fulfil the model metadata profile for the whole metadata profile (E1) was adopted using the formula presented in , which was generalized for one library and for any number of criteria, so that it took the form of formula (1):
E1—degree to which the metadata values provided according to library rules can fulfil the model metadata profile,
n—number of metadata elements,
wk—weight of the k-th metadata element.
To interpret the degree to which the metadata values provided according to library rules can fulfill the model metadata profile (E1) in the library, an analogical scale as in the original procedure was used. It means that the degree of compliance is rated as “very good” if at least half of the metadata elements are assigned the weight of 1.0 and the rest receive 0.8, “good” if all elements receive at least 0.8, “sufficient” if all elements receive at least 0.5, and “poor” if it is less than 0.5 for all metadata elements.
The evaluation of topographic map collection metadata
The evaluation of the topographic map collection metadata was carried out taking into account the completeness and accuracy of individual metadata values. However, this part of the methodology is different from the original one. It is the result of the fact that the original research concerned the comparison of several libraries, and the current research serves to assess the resource from only one library. Thus, it required the reformulation of formulas and clarification of the definition of measurements used.
Completeness is defined as a degree to which all the information necessary to provide a comprehensive representation of the described resource is included in the metadata instance . In the current research completeness was measured for each metadata element (Eck) as the ratio of the number of maps with existing values in metadata element (mck) to the number of maps in the whole collection (m) using the formula (2):
Eck—completeness of the k-th metadata element,
mck—number of maps with existing values in the k-th metadata element,
m—number of all analysed maps in the library (the size of map collection).
Completeness was also calculated for the topographic maps collection metadata (Ec), as a sum of the completeness for each metadata element, using the formula (3):
Ec—completeness of topographic maps collection metadata,
Eck—completeness of the k-th metadata element,
n—number of metadata elements.
The second parameter used to evaluate the topographic map collection metadata was accuracy. Neumaier et al.  defined accuracy as the degree with which the actual data are described by the available metadata values. The metadata instance should provide as correct information about the resource as possible . As with completeness, accuracy was also determined for each metadata element and for the whole collection. The accuracy (Eak) for each metadata element was calculated using the formula (4):
Eak—accuracy of the k-th metadata element,
wk—weight of the k-th metadata element,
mck—number of maps with existing values in the k-th metadata element,
m—number of all analysed maps in the library (the size of map collection).
Accuracy was also calculated for the topographic maps collection metadata (Ea), as a sum of the accuracy for each metadata element, using the formula (5):
Ea—accuracy of the topographic maps collection metadata,
Eak—accuracy of the k-th metadata element,
n—number of metadata elements.
The results of the research carried out with use of the above methodology are presented in the "Results" subsection Evaluation of topographic maps metadata.
The evaluation of the usability of topographic maps
The second stage of the procedure, related to the evaluation of the usability of topographic maps, consisted of three steps:
Selection of the metadata elements necessary to evaluate the usability of maps (selected evaluation criteria which are factors that determine usability);
Assessment of the degree to which the metadata values provided according to library rules can fulfil the set of metadata elements necessary to evaluate the usability of maps,
Assessment of the degree of usability of topographic map collection.
The second stage was related to the usability of maps. It was assumed that some metadata elements play a key role in the assessment of the extent to which maps may be useful for specific research. Therefore, assessment of these metadata elements and their values has been separately conducted.
One of the research goals was to analyze metadata in terms of their use to evaluate the usability of maps for research purposes. All cartographic materials must be evaluated before using them . In the evaluation process, it is important to determine their usability as a basic material for interdisciplinary research. The degree of usability was determined by Grygorenko  on the basis of the following factors:
Up-to-dateness, which can be determined based on the year of publication, the nature and timeliness of the source data, and based on the time of compilation. Information about the release year is collected by libraries and map catalogers find this type of data because they are typical of all collections gathered by libraries. Finding information on the sources and time of the study poses a much bigger problem, as the latter are not collected anywhere. Once this information used to be included in the map metric, but currently, libraries do not have this type of documentation;
Accuracy of the map, which consists of position accuracy, coordinate system (map projection and reference ellipsoid), cartographic grid, scale, detail, and faithfulness;
Visualization, i.e. the graphic presentation of content by which we understand the presence of a legend on the map, the existence of instructions for the development of topographic and thematic maps, the detail of the classification adopted, and the principles of generalization [1, 43]. Due to the fact that topographic maps are subject to evaluation and the content and manner of presentation on this type of maps is known, the methods of terrain presentation will be subject to analysis.
To evaluate the usability of topographic maps, the following measures were used:
The degree to which the metadata values provided according to library rules can fulfill the set of metadata elements to the evaluation of map usability was calculated using the following rules:
For each metadata element—by defining weight wi, which is compatible with the degree to which each selected metadata element fulfills the adopted rules,
For the whole set of metadata elements used to evaluate the usability of maps (E2), using the formula (6):
h—number of metadata elements selected to the evaluation of usability of maps,
i—metadata element which describe the usability of maps,
wi—weight for i-th metadata element.
The evaluation of the degree of usability of topographic map collection:
For each metadata element was calculated, using the formula (7):
mai—number of maps with accurate values in metadata element which describe the usability of maps,
m—number of all analysed maps in library.
For the set of selected metadata elements, using the formula (8):
The results of the research carried out on the basis of the above methodology are presented in "Results" subsection: The evaluation of the usability of topographic maps.
Evaluation of the topographic maps metadata
The range of elements of the proposed model metadata profile is presented in Table 1. It contains 13 metadata elements. Eight of them are mandatory, while five are optional. Seven elements (1–7) provide basic info about the object, whereas the other six (8–13) contain information specific to maps.
Additional file 1: Appendix S1 shows details of the model metadata profile. It presents the definitions of particular elements of metadata, the proposed values (standards, vocabularies, and sets of values), and examples of metadata elements. Seven of them contain values proposed by ISO standards, among them are four elements taken directly from ISO 19115–1 (topic category, geographic location, distribution format, and map scale). For three metadata elements—rights, mapping methods, source materials—sets of acceptable values have been proposed. They were developed based on numerous publications [1, 42, 43] and applicable ISO standards. The set of rights (Table 2) was developed based on copyrights used in Europeana . The set of methods (Table 3) was based on trade literature , and the set of sources (Table 4) was based on trade literature  and the existing thesaurus .
The model metadata profile (Additional file 1: Appendix S1) and sets (Tables 2, 3, 4) are aimed at facilitating the completion of metadata by catalogers, making it easier for users to search for individual maps and ensuring the interoperability of archival collections with other libraries and content aggregators .
The evaluation of the library metadata profile
The evaluation of the library metadata profile was carried out on an example of metadata profile for topographic maps description developed in the National Library of Poland (NLP). In order to calculate the degree to which the metadata values provided according to NLP rules can fulfill the model metadata profile, the rules for the description of metadata characteristic for topographic maps in the National Library of Poland were analyzed and described in Additional file 2: Appendix S2. The table contains the name of the metadata, its definition, and values that the metadata should take according to standards or dictionaries, as well as examples of these values.
Assignment of metadata elements of the analyzed library to the model metadata profile are presented in Table 5. It shows where (from which metadata element of the National Library of Poland) the metadata for the model metadata profile was obtained from.
Table 6 presents weight allocation to individual metadata elements (wk) and the degree to which the metadata values provided according to NLP rules can fulfill the model metadata profile for the whole metadata profile (E1).
According to Kuzma and Moscicka , the following scale was used to interpret the degree to which the metadata values provided according to library rules can fulfill the model metadata profile for the whole metadata profile (E1) in the library:
The E1 is 9.6 (out of 13 points possible to obtain). This means that the degree to which the metadata values provided according to library rules can fulfill the model metadata profile for the whole metadata profile is “sufficient”.
Table 6 shows that typical data for objects in libraries, such as type of content, date, date range, subject, rights, and language are easy to obtain (weight = 1.0), while information such as as geographic location, scale of map, reference system, mapping methods, map format, and source materials used to create the map, relevant from the point of view of cartographic documents, is more difficult to obtain (weight 0.8 and 0.5). This means that data typical of the map description is more difficult to obtain than basic information about each library object.
The evaluation of topographic map collection metadata
To get a full picture of obtaining data from the National Library of Poland, the authors determined the completeness for each metadata element (Eck) depending on the number of maps with existing values in metadata element (mck) was calculated based on the formula (2) and completeness for the whole metadata profile (Ec) based on the formula (3), while the total number of topographic maps in the collection was 35,090. The results are presented in Table 7.
It turns out that the Eck for individual elements is close to 100% for metadata, which is typical for all objects in libraries (metadata elements 1–6) and for the characteristic for maps (8–13), and the worst described items are sources that were the basis for preparing maps. The overall completeness of the metadata profile Ec is 10.43, while the maximum can be 13. It means quite a “good” result.
According to the methodology, the degree to which compatible values can fulfill the model metadata profile for each metadata element—Eak was calculated on the basis of formula (4), and for whole metadata profile (Ea) it was calculated based on the formula (5). It includes weights indicating the easiness of obtaining metadata. The results are presented in Table 8.
The results in Table 8 show that typical elements of all objects collected in libraries, except Subject and Distribution format, are the most compatible with the model metadata profile. Scale of map is the best complemented metadata element describing maps (0.79), while geographic location is described worse (0.66), and for other metadata characteristic for maps the degree to which compatible values can fulfill the model metadata profile (Eak) is below 0.5. The value of Ea for the whole collection is 8.50, while the maximum can be 13. It means not very good result, slightly exceeding half of the points to be scored (degree only “sufficient”).
The evaluation of the usability of topographic maps
The second stage of metadata evaluation at the National Library of Poland concerns the possibility of using metadata to evaluate the usability of maps. The criteria, which have been described in Methodology and data in subsection: The evaluation of the usability of topographic maps, can be assessed based on selected metadata elements. Based on the suggestions of Grygorenko  and Ratajski , we have developed a set of data necessary to evaluate usability. Due to the fact that the group of map users is becoming wider, not only cartographers use them, but also researchers, geographers, and historians , so the usefulness of a given map must be easy to interpret and objective. Therefore, we defined metadata elements corresponding to the up-to-dateness, the accuracy of map, and the visualization that allow evaluating the usability of maps. They are presented in Table 9.
For this purpose, the degree of usability of maps was calculated based on the formulas: (6), (7) and (8). Table 10 presents the results of the degree of usability of maps on the basis of which E2, Eui and Eu were calculated.
Based on the results presented in Table 10, it was noted that the metadata elements that are best described are those that are most relevant to map users, i.e. Date, for which wi is 1.00, scale of map—0.79, geographic location—0.66. Other elements have a lower the degree of usability of maps—below 0.5, and also the weight for all of them is 0.5. This means that it is difficult to obtain this metadata from the National Library of Poland.
The evaluation of the topographic maps metadata
In the conducted research, the model metadata profile shows the answer to the first research question: What scope of metadata can provide a proper description of topographic maps? It turns out that there is no profile that uniquely and fully defines the metadata for topographic maps . The proposed profile distinguishes 13 metadata elements that are relevant to the evaluation of topographic maps metadata. Eight of them are mandatory and should be completed by librarians, and they are typical for all objects in libraries, except for geographic location. The other 5 metadata elements relate to the characteristics for maps and these are the reference system, scale of map, mapping methods, map format, and source materials used to develop the map.
It turned out that metadata corresponding to elements of the metadata profile was collected in various metadata fields in the National Library of Poland. Below are some examples related to the circumstances of obtaining metadata from the National Library metadata profile.
The subject is collected in two different fields: topical subject (MARC 21, field 650) and index term-curriculum objective (MARC 21, field 658). The topical subject was chosen to analysis because the data, which are gathered in this field, are more detailed than index term-curriculum. The topical subject is compatible with the model metadata profile. However, the topical subject is fulfilled only for 18,285 topographic maps, and the index term-curriculum objective for 34,734 maps.
Geographic location and scale of map are collected in three various fields: subject added entry-geographic name (MARC 21, field 651), coded cartographic mathematical data (MARC 21, field 034), and cartographic mathematical data (MARC 21, field 255). The research was carried out the value of subject added entry-geographic name (MARC 21, field 651). It was noted that it contains the names of the cities, voivodships the map concerns. Since no thesauri were developed for this field, 15,685 unique (different) object names were found. Unfortunately, there are errors related to the detailed definition of a given place, e.g. Babimost (Lubuskie Voivodeship; district) and Babimost (Lubuskie Voivodeship, Zielonogórski District, Babimost commune; around). In addition, information is placed once "around" and "district", which mean the same kind of place, but already show the place name as different. Another example in the selected set there are 70 maps for Bydgoszcz. Among these 70 maps the name Bydgoszcz is saved in 13 different ways. Due to the fact that the same places were saved differently here, this metadata element was not used to analyze metadata.
The cartographic mathematical data (MARC 21, field 255) contains the coordinates of a given place, but only for 16,114 objects and these data are not as easy to obtain as in the case of coded cartographic mathematical data (MARC 21, field 034). This field contains geographic coordinates and scale data in accordance with MARC 21 guidelines. This field was used for analyzes.
The National Library of Poland collects metadata, which is largely supplemented in accordance with the model metadata profile. As a result of the analysis, it was found the ways and rules to obtain metadata from the National Library of Poland to the pattern metadata profile (Tables 5, 6). It turned out that 6 out of 13 elements can be directly obtained from the National Library of Poland—these elements are typical for all objects collected in libraries—type of content, date range, subject/keywords, rights, language, date. The next two metadata elements—geographic location and scale of maps—could be collected relatively easily and extracted to model metadata profile. The weight of assigned metadata was 0.8, and the other elements were obtained using specialist knowledge. Unfortunately, the Distribution format was not collected in specific metadata element. The evaluation of the degree to which the metadata provided according to library rules can fulfill the model metadata profile for the whole profile (E1) is 9.6 out of 13 points possible to obtain, which means that data for all elements were obtained by using at least a simple analysis.
The completeness and accuracy provide more details about the second research question: to what extent do topographic maps from the National Library of Poland fulfil the rules on occurrence in metadata these map-specific information which give the best chance of finding maps that fulfil the user needs?
Analysis of completeness (Ec) demonstrated that 90–100% of the maps were accompanied by metadata describing type of content, date range, rights, language, date, scale of map, mapping methods, map format, and reference system. The overall completeness of the profile was 10.43, which means that if all the data could be obtained directly, the library rating would be “good”.
Unfortunately, it turned out that all metadata elements cannot be obtained directly. Therefore, when calculating accuracy, we took into account the extent to which the library collects the data in accordance with the values of the model metadata profile (mck) and how the metadata from the National Library of Poland profile is obtained to the model metadata profile (wk). It turned out that for type of content, date range, rights, language, and date the accuracy (Eak) was 1.00, which means that all data could be obtained directly for these metadata and that data were provided for all maps. For the scale of map Eak was 0.79. This means that for almost 100% of maps the metadata was present and accurate, but obtaining it required more effort than the others, i.e. we had to use simple analysis to obtain this data (wk is 0.8). The Eak for geographic location was 0.66, because the geographic coordinates were completed in over 80% in the appropriate way. The next group of metadata elements were the mapping methods and map format, and the Eak was about 0.5, which means that for 95% (mapping methods), 97% (map format), and 92% (reference system) map metadata was consistent with the values proposed in the model metadata profile, although extracting them from the National Library of Poland profile required using specialized knowledge of catalogers. In addition, catalogers in the National Library of Poland use a thesaurus related to mapping methods to supplement data, but they can modify and change the values from this thesaurus. Therefore, typing errors occur. The last two metadata elements were supplemented based on the data on the map, without using a thesaurus, so the value of Eak for these two elements metadata was 0.11 (source materials used to develop map) and 0.00 (distribution format).
It turned out that the more data is obtained without thesauri, the more often mistakes are made in copying information from the analog document format and the more difficult it is to transfer data from one profile to another. After all, the accuracy (Ea) for the whole profile is “sufficient” (8.50).
Due to the fact that the obtained results were “good” or “sufficient”, we checked the extent, to which these data can be used to determine the usability of topographic maps.
The evaluation of the usability of topographic maps
Based on our research, it seems that the evaluation of the usability of topographic maps (Eui) is the best for the date, geographic location and scale of map. Those are the metadata elements that are most important for searching by users [51,52,53]. Also, these metadata elements can be obtained directly (date) or with use of simple analyzes (geographic location, scale of map). It turns out (Table 11) that in the National Library of Poland are maps that have metadata consistent with the accurate values for few elements of the model metadata profile:
28,854 maps (representing 82% of the entire collection) have accurate values for date, geographic location, and scale of map,
28,773 maps (representing 82% of the entire collection) have accurate values for date, geographic location, scale of map, and mapping methods,
28,206 maps (representing 80% of the entire collection) have accurate values for date, geographic location, scale of map, mapping methods, and map format,
6481 maps (representing 18% of the entire collection) have accurate values for date, geographic location, scale of map, mapping methods, map format, and source materials.
Limitations of the research
The advantage of the research is that it was conducted based on a significant amount of data (35,090 maps), which allows for reliable results about the quality of the resources in the NLP. Nevertheless, the research itself was quite burdensome, as the records obtained from the library were saved in the xls format, and thus it was not possible to export the data from the library system to other data formats. In addition, the values are recorded in different way in individual records, which makes it impossible to fully automate data analysis. Therefore, it was not possible to import data into the database and many tasks were carried out manually, which was very time-consuming due to the huge amount of data. This problem stems from two root causes. Firstly, the system used in the library is not adapted to work with external systems or popular databases, it only serves the internal needs of the library. It also results in the fact that the exchange of data between the national library and other institutions is complicated, and often even impossible. Secondly, there are no developed and implemented rules for recording individual values in each of the metadata elements, there are no thesauruses, established thematic entries, etc. As a consequence, different librarians write the same data differently. There is also no control system for recorded data, which results in errors such as simple mistakes or typos. Also, it has a big impact on the efficiency and effectiveness of searching for data in the library, and may prevent finding the needed maps altogether. All this made it impossible to carry out the research efficiently.
The discrepancies in the recorded values of the metadata elements also resulted from another problem. Library metadata recording rules have changed many times in the library over the years, but this information is not recorded in the metadata. There is no information which rules were used for particular object—the old or the current ones. It is not known, therefore, whether a record, which we deem incorrectly written, is the result of a mistake or the application of the old rules. All this caused difficulties and prolonged the research, as each such case had to be analyzed separately and manually. Elimination of such problems would significantly facilitate data analysis in line with the proposed methodology.
Librarians from the National Library of Poland are among the first in Poland to have recognized the need to describe maps using map-specific information, and to introduce geographic information into the metadata of other types of objects. They did their utmost to put their findings into practice. They have developed instructions to assist librarians in preparing these descriptions, and have described thousands of maps accordingly. It should be underlined that these activities were conducted at the initiative of individual employees and reflected their awareness of the needs in this area and it was not a top-down policy of the library. As a result, a very important step has been made towards full-fledged map descriptions. It is a pity, however, that these steps were not made in cooperation with specialists in the field of creating databases, computer cartography, etc., as then many errors and inaccuracies in the prepared descriptions could have been avoided. However, it must not be forgotten that a great deal of work has already been done.
The biggest problem at the NLP is the fact that it collects valuable information about maps without assistance of good cartographers, yet the National Library—as the most important institution to collect maps in Poland—can afford to hire qualified staff to fulfil its main objective. It may also share information about maps with other libraries. In order to improve the interoperability of collections found in the National Library of Poland, it is worth developing thesauri, as well as controlled vocabularies. They could improve search results and also reduce the number of typing errors in the metadata entered.
Further research should also concern the possibility of using such a large and well-described collection. It could be very interesting to implement an idea of a sample map  to old maps based on metadata. It is imperative to work out a specific functionality based on a geographic information system which should be implemented in the retrieval systems.
The data from the National Library of Poland are not perfect, but this library is still a leader in the development of library map metadata. The research on the methods of describing maps in other institutions shows that results from e.g. 7 university (academic) libraries  differ significantly from the achievements of the NLP. Firstly, the number of maps described in them is much lower, ranging from 20 to 1733, while the NLP described 35,090 maps. Moreover, the metadata of academic libraries rated much worse. The degree to which the metadata values provided according to each library rules can fulfil the model metadata profile is comparable—in academic libraries it ranges from 7.6 to 10.2 (in most libraries it is "sufficient", it is "poor" in one only), with NLP rating at 9.6 ("sufficient"). It means that the proposed metadata profiles are at similar level of compatibility with the model metadata profile in both academic and national libraries. The difference is visible when we evaluate accuracy, namely the degree with which the actual data are described by the available metadata values. The best result among academic libraries is scored by the library of the University of Wroclaw (4.5), which provides the most cartographic documents (1733) from academic libraries. On the other hand, an interesting situation has been observed in Jagiellonian University library—despite a very good assessment of the way of writing data in metadata (9.6), many objects lacked the relevant data (1.6). These results, however, are much worse than in NLP, where accuracy was assessed at 8.5. It is all the more valuable as the number of objects assessed in NLP was disproportionately greater.
The proposed method of assessing metadata may be a useful tool to check whether the resources collected in a library can be found by potential users and applied in their research. The conducted analysis of topographic maps in the National Library revealed that the collected data are very interesting and have rich metadata. It should be emphasized that, apart from well-developed metadata, all collections (not just maps) are assigned geographic descriptors. These descriptors contain information about the geographic name of the place, their various forms over the time, as well as geographic coordinates. However, the use of metadata and geographic descriptors is limited since search engines do not use the full potential of the collected data, and the spatial analyses are not implemented. Development of the functionality of the service enabling effective search of old maps in libraries will be the subject of our further research. The cooperation with experienced librarians and GIS (Geographic Information System) specialists will be necessary to create a cost-effective and useful service This will allow library resources to play an important role in providing access to valuable information sources in a modern way.
The awareness of the above needs, as well as the conclusions drawn from the results of our research, are of great importance, mainly for those working directly with the analysed data and dealing with map descriptions. They have direct interest in it and understand the need to develop even better metadata descriptions, as it determines effective access to maps. Unfortunately, not everyone is aware of that yet and the changes are slow, also due to the lack of adequate financing.
Availability of data and materials
All data generated during this study are included in this published article, and its supplementary information files: Additional file 1: Appendix S1. The model metadata profile, Additional file 2: Appendix S2. Metadata profile used for topographic maps description in the National Library of Poland (chosen elements).
Geographic Information System
Infrastructure for Spatial Information in the European Community
International Organization for Standardization
National Library of Poland
- U.S. Geological Survey:
United States Geological Survey
Grygorenko W. Redakcja i opracowanie map ogólnogeograficznych. Polish [English: Editing and preparation of general geographical maps]. Warsaw: PPWK; 1970.
Vuorela N, Alho P, Kalliola R. Systematic assessment of maps as source information in landscape-change research. Landscape Res. 2002;27(2):141–66. https://doi.org/10.1080/01426390220128631.
Dempsey C. Map Scale. In: Geography realm. 2014. https://www.geographyrealm.com/map-scale/. Accessed 27 Sep 2020.
Borneman E. Types of Map Projections. In: Geography realm. 2015. https://www.geographyrealm.com/types-map-projections/. Accessed 27 Sep 2020.
Korycka-Skorupa J, Nowacki T. Cartographic presentation—from simple to complex map. Miscellanea Geographica. 2018;23(1):16–22. https://doi.org/10.2478/mgrsd-2018-0023.
Youngblood D. Map use across the disciplines. J Map GeogrLibr. 2006;2(2):33–66.
Fish CS, Piekielek NB. Targeting disciplines for GIS outreach using bibliometric analysis. J Map GeogrLibr. 2016;12(3):258–80. https://doi.org/10.1080/15420353.2016.1221870.
Frajer J, Geletič J. Research of historical landscape by using old maps with focus to its positional accuracy. Dela. 2011;36:49–67.
Williams P. Where do I start? A cartographic cataloguing code. Cartogr J. 2005;42(3):227–30.
Tedd L, Beamer A. Map metadata: essential elements for search and storage. Program. 2009;43:18–35. https://doi.org/10.1108/00330330910934084.
Lai P, Gillies C. The impact of geographical information systems on the role of spatial data libraries. Int J GeogrInfSyst. 1991;5:241–51.
Kiser T, Smeltekop N. A method for creating scanned map metadata for geoportals, library catalogs, and digital repositories: reworking existing MARC records of paper maps to create new records for their scanned counterparts. J Map GeogrLibr. 2018;14:109–31.
Kowal KC, Martyn C. Descriptive metadata for digitization of maps in books: a British Library project. LibrResour Tech Serv. 2011;53:108–20.
Borbinha J, Pedrosa G, Gil J, Martins B, Freire N, Dobreva M, Wyttenbach A. Digital libraries and digitised maps: an early overview of the DIGMAP project. In: Goh DHL, Cao TH, Sølvberg IT, Rasmussen E, editors. Asian digital libraries Looking back 10 years and forging new frontiers. ICADL 2007. LNCS, vol. 4822. Berlin: Springer; 2007. p. 383–6. https://doi.org/10.1007/978-3-540-77094-7_48
Andrew PG. A/V cataloging at the crossroads: cartographic resources cataloging basics using RDA. 2014. In: Workshop in OLAC, National Conference, Kansas City, Missouri, October 2014. http://olac2014.weebly.com/uploads/2/5/1/9/25198051/olacmoug2014-andrew-maps-slides.pdf. Accessed 1 Nov 2020.
Perkins C. Quality in map librarianship and documentation in the GIS age. Cartogr J. 1994;31(2):93–9.
Kuzma M, Moscicka A. Evaluation of the accessibility of archival cartographic documents in digital libraries. Electron Libr. 2018;36:1062–81. https://doi.org/10.1108/EL-06-2017-0130.
Reese T. Bibliographic freedom and the future direction of map cataloging. J MapGeogrLibr. 2006;2(1):67–97. https://doi.org/10.1300/J230v02n01_04.
Morse T. Mapping relationships: examining bibliographic relationships in sheet maps from Tillett to RDA. CCQ. 2012;50:225–48. https://doi.org/10.1080/01639374.2012.657292.
Larsgaard M. FRBR and cartographic materials: Mapping out FRBR. In: Taylor A, editor. Understanding FRBR: What it is and how it will affect our retrieval tools. Westport: Libraries Unlimited; 2007. p. 111–5.
van Loenen B, Onsrud H. Geographic data for academic research: assessing access policies. CartogrGeogrInf Sc. 2004;31(1):3–17. https://doi.org/10.1559/152304004773112730.
Kilfoil, J. Projecting the map collection: academic map libraries and communicating the value of services on the World Wide Web. Masters Paper. 2002. https://doi.org/10.17615/h10f-2955.
Weimer K, Andrew P, Hughes T. (2008), Map, GIS and Cataloging/Metadata Librarian Core Competencies, American Library Association, 2020. www.ala.org/rt/sites/ala.org.rt/files/content/publicationsab/MAGERTCoreComp2008.pdf. Accessed 27 Sep 2020.
Janowicz K. The role of space and time for knowledge organization on the semantic web. Semant Web. 2010;1:25–32.
Kuzma M, Moscicka A. Metadata evaluation criteria in respect to archival maps description: a systematic literature review. Electron Libr. 2020;38:1–27. https://doi.org/10.1108/EL-07-2019-0161.
Ochoa X, Duval E. Automatic evaluation of metadata quality in digital repositories. Int J Digit Libr. 2009;10(2):67–91. https://doi.org/10.1007/s00799-009-0054-4.
Gavrilis D, Makri D, Papachristopoulos L, Angelis S, Kravvaritis K, Papatheodorou C, Constantopoulos P. Measuring quality in metadata repositories. Lect Notes ComputSci. 2015;9316:56–67. https://doi.org/10.1007/978-3-319-24592-8_5.
Neumaier S, Umbrich J, Polleres A. Automated quality assessment of metadata across open data portals. J Data InfQual. 2016;8(1):Article 2. https://doi.org/10.1145/2964909.
Moen WE, Stewart EL, McClure CR. The role of content analysis in evaluating metadata for the US Government information locator service (GILS): Results from an exploratory study. University of North Texas Libraries, UNT Digital Library, 1998. https://digital.library.unt.edu/ark:/67531/metadc36312/. Accessed 27 Sep 2020.
Ustawa o bibliotekach. Polish [English: Act on libraries], Dz.U.2019.0.1479 (Polish). 1997. https://www.lexlege.pl/ustawa-o-bibliotekach/. Accessed 27 Sep 2020.
The National Library of Poland. Statut Biblioteki Narodowej. Polish [English: National Library Statute]. 2016. https://www.bn.org.pl/o-nas/statut/statut-bn/. Accessed 27 Sept 2020.
The National Library of Poland. The National Library of Poland. https://www.bn.org.pl/o-nas/o-instytucji. Accessed 27 Sep 2020.
The National Library of Poland. Databases. https://data.bn.org.pl/databases. Accessed 10 Jun 2019.
Primo of the National Library of Poland. 2020. https://katalogi.bn.org.pl/. Accessed 27 Sep 2020.
Library of Congress, MARC Standards Office. MARC21 Standards. 2019. https://www.loc.gov/marc/. Accessed 27 Sep 2020.
Szura R, Krynicka M. Instrukcja katalogowania dokumentów kartograficznych. Polish [English: Instructions for cataloging cartographic documents]. Warsaw: National Library of Poland, 2012. https://www.bn.org.pl/download/document/1350029482.pdf. Accessed 27 Sept 2020.
The National Library of Poland. Przepisy katalogowania. Mapy. Polish [English: Cataloging regulations. Maps]. 2018. https://przepisy.bn.org.pl/przepisy-katalogowania/mapy#mapy. Accessed 27 Sept 2020.
Report of the National Library of Poland. 2019. https://www.bn.org.pl/o-nas/sprawozdanie-bn. Accessed 27 Sep 2020.
International Organization for Standardization. Geographic information—metadata—Part 1: Fundamentals, ISO 19115–1. 2014. www.iso.org/iso/home/store/catalogue_tc/catalogue_detailhtm?csnumber=53798. Accessed 27 Sep 2020.
Zwirowicz A. Testowanie zgodności wyrobów z normami serii ISO 19100 oraz oznaczanie znakiem zgodności PN. Polish [English: Testing of products for conformance to ISO 19100 series standards and labelling with Polish standard conformance mark]. Ann Geomatics. 2009;7(4):77–83.
Salishtchev KA. Kartografia ogólna. Polish [English: General Cartopraphy]. Warsaw: PWN; 1998.
Pasławski J. Wprowadzenie do kartografii i topografii. Polish [English: Introduction to cartography and topography]. Wrocław: WydawnictwoNowa Era; 2010.
Ratajski L. Metodyka kartografii społeczno-gospodarczej. Polish [English: Methodology of socio-economic cartography]. Warsaw: PPWK; 1973.
Digital Repository of Scientific Institutes. 2020. https://rcin.org.pl/dlibra?language=en. Accessed 27 Sep 2020.
The National Library of Poland. POLONA. 2020. https://polona.pl/. Accessed 27 Sep 2020.
The National Library of Scotland. 2020. https://www.nls.uk/. Accessed 27 Sep 2020.
Europeana. Europeana collections, Terms & Policies. 2020. https://www.europeana.eu/portal/en/rights.html. Accessed 27 Sep 2020.
U.S. Geological Survey. USGS Thesaurus. 2020. https://www2.usgs.gov/science/about/. Accessed 27 Sep 2020.
Platia N, Chatzidakis M, Doerr C, Charami L, Bekiari C, Melessanaki K, Hatzigiannakis K, Pouli P. ‘“POLYGNOSIS”’: the development of a thesaurus in an Educational Web Platform on optical and laser-based investigation methods for cultural heritage analysis and diagnosis. HeritSci. 2017;5:50. https://doi.org/10.1186/s40494-017-0163-0.
van Elzakker C. Use of maps on the Web. In: Kraak M, Brown A, editors. Web cartography: developments and prospects. London: Taylor & Francis; 2001. p. 21–36.
Brovelli M, Minghini M. Georeferencing old maps: a polynomial-based approach for Como historical cadastres. e-Perimetron. 2012;7:97–110.
Buckland M, Chen A, Gey F, Larson R. Geographic search: Catalogs, gazetteers, and maps. Coll Res Libr. 2007;68:376–87.
Lamont M, Marley C. Spatial data and the digital library. CartogrGeogr Inform. 1998;25(3):143–9. https://doi.org/10.1559/152304098782383142.
Ahonen-Rainio P, Kraak M-J. Deciding on fitness for use: evaluating the utility of sample maps as an element of geospatial metadata. CartogrGeogrInfSci. 2005;32:101–12. https://doi.org/10.1559/1523040053722114.
The authors acknowledge the cartographers of the National Library of Poland. Their comments and point of view have been invaluable in the creation of this research.
The research was funded by the Military University of Technology in Warsaw, Poland under the University Research Grant, grant number 793/2020, realized in year 2020.
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Kuźma, M., Mościcka, A. Evaluation of metadata describing topographic maps in a National Library. Herit Sci 8, 113 (2020). https://doi.org/10.1186/s40494-020-00455-3