Comparison of metrics and document coverage of Google Scholar, Web of Science, and Scopus: the case of 18 geography researchers at the University of Vienna

A DIGEST OF

Gorraiz, J., Gumpenberger, C., Glade, T. On the bibliometric coordinates of four different research fields in Geography. Scientometrics, in press 

Access to the Full Text

OBJECTIVES
This study is a bibliometric analysis of the highly complex research discipline Geography. In order to identify the most popular and most cited publication channels, to reveal publication strategies, and to analyse the discipline’s coverage within publications, the three main data sources for citation analyses, namely Web of Science, Scopus and Google Scholar, have been utilized. the main research questions of this study are: §  Which are the most usual and most cited publication channels? § Which data sources are most applicable for Geography bibliometrics? Is Google Scholar indeed a useful complementary data source? §  Are there any bibliometric differences detectable between natural science and human science oriented research fields?
METHODOLOGY
Sample
Documents published by 40 faculty members at the  Department for Geography and Regional Research at the University of Vienna specialized in four different subfields: Geoecology (12), Social and Economic Geography (10), Demography and Population Geography (9), and Economic Geography (9)
Design
§ This study is primarily based on publication data collected for four evaluation exercises performed at the Department for Geography and Regional Research at the University of Vienna.   §  In all exercises, the publication data were delivered directly by the candidates, whose identity has to remain anonymous.  All bibliometric indicators added to the list of publications were controlled or recalculated in order to enable a correct and comparable analysis. § The main data source for coverage and citation analyses was “Web of Science -Core Collection”. Furthermore, Google Scholar and Scopus has been utilized as another primary source within the Demography and Population Geography , and  Google Scholar, “Web of Science -Cited Reference Search” en Economic Geography. The analysis in GS was performed by using the GS Citations Profiles.  §  Thee candidates were invited to create their individual profiles and make them publicly available for at least a couple of weeks. In addition, the tool ‘Publish or Perish’ was used to check and amend these profiles. In the cases where individual profiles were not available, respective queries have been made in order to assemble a complete data set
Period analyzed:  1990-
RESULTS
1.   The results show significant differences in the publication and citation habits among the four specialties studied. While in natural science, publications in highly ranked international peer-reviewed journals is of highest importance, the publications within the social science domain refer often to reports, book-chapters and also monographs 2. In addition, very heterogeneous and individual publication strategies, even in the same research fields, are observed. 3. Monographs, journal articles and book chapters are the most cited document types. 4. Monographs (Books) are completely covered in GS (100%), whereas coverage is less complete in WoS Cited Reference Search (~75 %) and almost zero in WoS Core Collection including both Book Citation Indices. Similar trends are reported for the other publication types. However, the percentage of book chapters covered in GS is still low (around 55 %) but high in comparison to WoS (14.1%) and WoS Cited Reference Search (32.9%). 5. No considerable differences are observed when using WoS or Scopus.  6. The inclusion of the Cited Reference Search allows for a significant coverage and is more practicable in WoS than in Scopus. Coverage & Metrics comparison Web of Science, Scopus, Google Scholar 7.   The number of papers and citations in Google Scholar is substantially higher than both the Web of Science (Core Collection plus Cited Reference Search) and Scopus for every subdiscipline. Depending on the bibliometric indicator, Google Scholar doubles, triples, quadruples, or quintuples the results offered by Web of Science and Scopus, as can be observed in the graph. 8. Spearman correlations performed for number of citations, citations per cited publication and h-index in the three data sources (WoS, Scopus and Google Scholar) were very strong (varying from 0.8 to 0.95).

CONCLUSION

We only discuss the conclusions concerning Google Scholar

What is already known on this topic

Google Scholar has a much broader document and citation coverage than Web of Science and Scopus. This study supports the findings of previous studies.

The values of the main citation indicators might differ in absolute values in GS, WoS, WoS Cited Reference Search and Scopus, but are comparable in terms of relative values. It confirms what other studies dealing with similar comparisons found.

In addition, it also confirms something we've stated over the years: Google Scholar is an indispensable source of data if one wants to study any of the disciplines of the Social Sciences and Humanities.

What this study adds

This work analyses the coverage differences of various databases as regards several specialties in the discipline of Geography. The small size and the limitations of the sample (only 9 researchers are studied in each subdiscipline), limited to researchers from a department of one university (Vienna), and a country (Austria), make it difficult to generalize the results to the specialties studied. However, it has value as a case study.

This study is one of the first to take advantage of the Cited Reference Search feature in Web of Science to calculate bibliometric indicators, and compare them with data from other databases, in a similar way as some of our products do: Classic Scholars' Profiles: Bibliometrics & Scientometrics and Scholar Mirrors.

Furthermore, this works proves that both the Web of Science with the Book Citation Index, and Scopus with its "Titles Expansion Program" are still incapable of covering the majority of books and book chapters published locally in each country. Their book collections are for the most part limited to elite publishers in English-speaking countries, which their language, geographic, and publisher biases are still there.

Lastly, this study states that "the accuracy of the citations in GS was very high (~95 %)", but unfortunately they don't offer more details or explanations about this issue.