Microsoft Academic: is the Phoenix getting wings?

A DIGEST OF

Harzing, A-W & Alakangas, S. 

Microsoft Academic: is the Phoenix getting wings?

Scientometrics. (in press)

Access to the Full Text

OBJECTIVES
1. To compare publication and citation coverage of the new Microsoft Academic with Google Scholar, Scopus, and the Web of Science. 2. To investigate the extent to which the findings change if using the more liberal “estimated citation count” in Microsoft Academic rather than the more conservative “linked citation count”
METHOD
Sample
145 Associate Professors and Full Professors at the University of Melbourne, Australia in 37 disciplines grouped into 5 broad disciplinary areas: Life Sciences, Sciences, Engineering, Social Sciences, and Humanities
Design
For each researcher, 4 indicators were calculated from 4 different sources. - Indicators: Number of papers; Citations received; h-index; hla-index - Sources: Google Scholar; Microsoft Academic; Scopus; Web of Science Publish or Perish (PoP) was used to conduct searches for Google Scholar and Microsoft Academic.
Period analyzed
All time. Data collected in the first week of October 2016.
RESULTS
Microsoft Academic coverage has improved substantially (an average growth of nearly 10%). The biggest increase is found for several books or book chapters, as well as some publications in minor journals. In terms of data quality, namely several erroneous year allocations, and citations that were split between a version of the publication with the main title only and a version with both the main title and a sub-title – have not yet been resolved. On average Microsoft Academic citations, are very similar to Scopus and Web of Science citations and substantively lower only than Google Scholar citations. As to disciplines, Microsoft Academic has fewer citations than Scopus and, marginally, than Web of Science for the Life Sciences and Sciences. In the Social Sciences, however, Microsoft Academic has a clear advantage over both Scopus and Web of Science, providing 1.5 to 2 times as many citations for the sample. The difference is even starker for the Humanities, where Microsoft Academic has a coverage that is 1.7 to nearly 3 times as high. Google Scholar citations were higher than Microsoft Academic citations for all but one individual in the sample FIG 1. Average number of papers and citations for 145 academics across Google Scholar, Microsoft Academic, Scopus and Web of Science    FIG 2. Average citations for 145 academics across Google Scholar, Microsoft Academic, Scopus and Web of Science, grouped by five major disciplinary areas Taking Microsoft Academic estimated citation counts rather than linked citation counts as our basis for the comparison with Scopus, Web of Science, and Google Scholar does change the comparative picture quite dramatically. Looking at our overall sample of 145 academics, Microsoft Academic’s average estimated citation counts (3873) are much higher than both Scopus (2413) and Web of Science (2168) citation counts, and very similar to Google Scholar’s average counts (3982). For the Life Sciences Microsoft Academic estimated citation counts are in fact 12% higher than Google Scholar counts, whereas for the Sciences they are almost identical. FIG 3. Comparison of average Microsoft Academic estimated citation counts with Google Scholar citation counts, grouped by five major disciplinary areas

CONCLUSIONS

The study suggests that the new incarnation of Microsoft Academic presents with an excellent alternative for citation analysis. Moreover, the comparison of citation growth over the last 6 months also suggests that Microsoft Academic is still increasing its coverage.

FINAL REMARKS

To the best of our knowledge, this work represents one of the first empirical analyses concerned to tackle with the estimated citation counts, a procedure followed by the new generation of academic search engines.

This constitutes a paramount shift in citation analyses as it manages estimated citations instead of real citations. In this sense, we would like to emphasize the following results provided:

a) When using the more liberal estimated citation counts for Microsoft Academic its average citations counts were higher than both Scopus and the Web of Science for all disciplines.

b) For the Life Sciences, Microsoft Academic estimated citation counts are even higher than Google Scholar counts, whereas for the Sciences they are almost identical.

c) For Engineering, Microsoft Academic estimated citation counts are 14% lower than Google Scholar citations, whereas for the Social Sciences this is 23%. Only for the Humanities are they substantially (69%) lower than Google Scholar citations.

When comparing Academic Search with Google Scholar, we need to take into account that Google Scholar does not work – at least currently – with estimated citations. Therefore, the more fair comparison should be with the so-called Microsoft Academic’s conservative citation counts. However, the similar results found in some disciplines may be a signal of accuracy in the estimation processes, which in turn may change the way in which academic search engines will work in the future.

Faced with this scenario, we may ask a question in the wind… do we need to estimate when we can gather everything?