2017 [Go back]
Book chapters
Delgado López-Cózar, E. Orduña-Malea, E., Martín-Martín, A., Ayllón, J. M.(2017).
Google Scholar: The big data bibliographic tool. En: Cantu-Ortiz, FJ. (ed.). Research Analytics: Boosting University Productivity and Competitiveness through Scientometrics. Boca Raton: CRC Press (Taylor & Francis Group), 2017 p.59-80 goo.gl/vvPf7g
he launch of Google Scholar back in 2004 meant a revolution not only in the scientific information search market but also in research evaluation processes. Its dynamism, unparalleled coverage, and uncontrolled indexing make Google Scholar an unusual product, especially when compared to traditional bibliographic databases. Conceived primarily as a discovery tool for academic information, it presents a number of limitations as a bibliometric tool. The main objective of this chapter is to show how Google Scholar operates and how its core database may be used for bibliometric purposes. To do this, the general features of the search engine (in terms of document typologies, disciplines, and coverage) are analyzed. Lastly, several bibliometric tools based on Google Scholar data, both official (Google Scholar Metrics, Google Scholar Citations) and some developed by third parties (H Index Scholar, Publishers Scholar Metrics, Proceedings Scholar Metrics, Journal Scholar Metrics, Scholar Mirrors), as well as software to collect and process data from this source (Publish or Perish, Scholarometer), are introduced, aiming to illustrate the potential bibliometric uses of this source.
Orduna-Malea, E., Delgado López-Cózar, E. (2017). Performance behaviour patterns in author-level metrics: a disciplinary comparison of Google Scholar Citations, ResearchGate and ImpactStory. Frontiers in Research Metrics and Analytics, 2, 14. https://doi.org/10.3389/frma.2017.00014
The main goal of this work is to verify the existence of diverse behavior patterns in academic production and impact, both among members of the same scientific community (inter-author variability) and for a single author (intra-author variability), as well as to find out whether this fact affects the correlation among author-level metrics (AutLMs) in disciplinary studies. To do this, two samples are examined: a general sample (members of a discipline, in this case Bibliometrics; n = 315 authors), and a specific sample (only one author; n = 119 publications). Four AutLMs (Total Citations, Recent Citations, Reads, and Online mentions) were extracted from three platforms (Google Scholar Citations, ResearchGate, and ImpactStory). The analysis of the general sample reveals the existence of different performance patterns, in the sense that there are groups of authors who perform prominently in some platforms, but exhibit a low impact in the others. The case study shows that the high performance in certain metrics and platforms is due to the coverage of document typologies, which is different in each platform (for example, Reads in working papers). It is concluded that the identification of the behavior pattern of each author (both at the inter-author and intra-author levels) is necessary to increase the precision and usefulness of disciplinary analyses that use AutLMs, and thus avoid masking effects
Martín-Martín, A., Orduna-Malea, E., Delgado López-Cózar, E. A novel method for depicting academic disciplines through Google Scholar Citations: The case of Bibliometrics. Scientometrics, in press. https://doi.org/10.1007/s11192-017-2587-4
This article describes a procedure to generate a snapshot of the structure of a specific scientific community and their outputs based on the information available in Google Scholar Citations (GSC). We call this method multifaceted analysis of disciplines through academic profiles (MADAP). The international community of researchers working in Bibliometrics, Scientometrics, Informetrics, Webometrics, and Altmetrics was selected as a case study. The records of the top 1000 most cited documents by these authors according to GSC were manually processed to fill any missing information and deduplicate fields like the journal titles and book publishers. The results suggest that it is feasible to use GSC and the MADAP method to produce an accurate depiction of the community of researchers working in Bibliometrics (both specialists and occasional researchers) and their publication habits (main publication venues such as journals and book publishers). Additionally, the wide document coverage of Google Scholar (specially books and book chapters) enables more comprehensive analyses of the documents published in a specific discipline than were previously possible with other citation indexes, finally shedding light on what until now had been a blind spot in most citation analyses
Orduna-Malea, E.; Martín-Martín, A.; Delgado López-Cózar, E. (2017). Google Scholar as a source for scholarly evaluation: a bibliographic review of database errors. Revista Española de Documentación Científica, 40 (4): e185. http://dx.doi.org/10.3989/redc.2017.4.1500
Google Scholar (GS) is an academic search engine and discovery tool launched by Google (now Alphabet) in November 2004. The fact that GS provides the number of citations received by each article from all other indexed articles (regardless of their source) has led to its use in bibliometric analysis and academic assessment tasks, especially in social sciences and humanities. However, the existence of errors, sometimes of great magnitude, has provoked criticism from the academic community. The aim of this article is to carry out an exhaustive bibliographical review of all studies that provide either specific or incidental empirical evidence of the errors found in Google Scholar. The results indicate that the bibliographic corpus dedicated to errors in Google Scholar is still very limited (n= 49), excessively fragmented, and diffuse; the findings have not been based on any systematic methodology or on units that are comparable to each other, so they cannot be quantified, or their impact analysed, with any precision. Certain limitations of the search engine itself (time required for data cleaning, limit on citations per search result and hits per query) may be the cause of this absence of empirical studies.
Orduña-Malea, E., Ayllón, J. M., Martín-Martín, A., Delgado López-Cózar, E. (2017). The lost academic home: institutional affiliation links in Google Scholar Citations. Online Information Review, 41 (6), 762-781.
Purpose: Google Scholar Citations (GSC) provides an institutional affiliation link which groups together authors who belong to the same institution. The purpose of this paper is to ascertain whether this feature is able to identify and normalize all the institutions entered by the authors, and whether it is able to assign all researchers to their own institution correctly.
Design/methodology/approach: Systematic queries to GSC’s internal search box were performed under two different forms (institution name and institutional e-mail web domain) in September 2015. The whole Spanish academic system (82 institutions) was used as a test. Additionally, specific searches to companies (Google) and world-class universities were performed to identify and classify potential errors in the functioning of the feature.
Findings: Although the affiliation tool works well for most institutions, it is unable to detect all existing institutions in the database, and it is not always able to create a unique standardized entry for each institution. Additionally, it also fails to group all the authors who belong to the same institution. A wide variety of errors have been identified and classified.
Research limitations/implications: Even though the analyzed sample is good enough to empirically answer the research questions initially proposed, a more comprehensive study should be performed to calibrate the real volume of the errors.
Practical implications: The discovered affiliation link errors prevent institutions from being able to access the profiles of all their respective authors using the institutions lists offered by GSC. Additionally, it introduces a shortcoming in the navigation features of Google Scholar which may impair web user experience.
Social implications: Some institutions (mainly universities) are under-represented in the affiliation feature provided by GSC. This fact might jeopardize the visibility of institutions as well as the use of this feature in bibliometric or webometric analyses.
Originality/value: This work proves inconsistencies in the affiliation feature provided by GSC. A whole national university system is systematically analyzed and several queries have been used to reveal errors in its functioning. The completeness of the errors identified and the empirical data examined are the most exhaustive to date regarding this topic. Finally, some recommendations about how to correctly fill in the affiliation data (both for authors and institutions) and how to improve this feature are provided as well.
Delgado López-Cózar, E. (2017). La investigación en Biblioteconomía y su impacto en la profesión ¿Puede ser Google Scholar una fuente de información adecuada para medirlo? Mi biblioteca: La revista del mundo bibliotecario 48, 10-11
Martin-Martin, A., Orduna-Malea, E, Harzing, A.W., Delgado López-Cózar, E. (2017). Can we use Google Scholar to identify highly-cited documents?, Journal of Informetrics, 11(1), 152-163. http://dx.doi.org/10.1016/j.joi.2016.11.008.
The main objective of this paper is to empirically test whether the identification of highly-cited documents through Google Scholar is feasible and reliable. To this end, we carried out a longitudinal analysis (1950 to 2013), running a generic query (filtered only by year of publication) to minimise the effects of academic search engine optimisation. This gave us a final sample of 64,000 documents (1,000 per year). The strong correlation between a document’s citations and its position in the search results (r= -0.67) led us to conclude that Google Scholar is able to identify highly-cited papers effectively.
This, combined with Google Scholar’s unique coverage (no restrictions on document type and source), makes the academic search engine an invaluable tool for bibliometric research relating to the identification of the most influential scientific documents. We find evidence, however, that Google Scholar ranks those documents whose language (or geographical web domain) matches with the user’s interface language higher than could be expected based on citations. Nonetheless, this language effect and other factors related to the Google Scholar’s operation, i.e. the proper identification of versions and the date of publication, only have an incidental impact. They do not compromise the ability of Google Scholar to identify the highly-cited papers.
Delgado López-Cózar, E. ; Martín-Martín, A.; Orduna-Malea, E. (2017). Classic papers: déja vu, a step further in the bibliometric exploitation of Google Scholar. EC3 Working Papers, 24. Granada 28th of June, 2017. https://doi.org/10.13140/RG.2.2.35729.22880/1
After giving a brief overview of Eugene Garfield’s contributions to the issue of identifying and studying the most cited scientific articles, manifested in the creation of his Citation Classics, the main characteristics and features of Google Scholar’s new service -Classic Papers-, as well as its main strengths and weaknesses, are addressed. This product currently displays the most cited English-language original research articles by fields and published in 2006
Orduna-Malea, E.; Martín-Martín, A. & Delgado López-Cózar, E. (2017). Google Scholar and the gray literature: A reply to Bonato’s review. EC3 Working Papers, 23. 11 February 2017 .DOI:http://dx.doi.org/10.13140/RG.2.2.20703.87207 .
Recently, a review concluded that Google Scholar (GS) is not a suitable source of information “for identifying recent conference papers or other gray literature publications”. The goal of this letter is to demonstrate that GS can be an effective tool to search and find gray literature, as long as appropriate search strategies are used. To do this, we took as examples the same two case studies used by the original review, describing first how GS processes original’s search strategies, then proposing alternative search strategies, and finally generalizing each case study to compose a general search procedure aimed at finding gray literature in Google Scholar for two wide selected case studies: a) all contributions belonging to a congress (the ASCO Annual Meeting); and b) indexed guidelines as well as gray literature within medical institutions (National Institutes of Health) and governmental agencies (U.S. Department of Health & Human Services). The results confirm that original search strategies were undertrained offering misleading results and erroneous conclusions. Google Scholar lacks many of the advanced search features available in other bibliographic databases (such as Pubmed), however, it is one thing to have a friendly search experience, and quite another to find gray literature. We finally conclude that Google Scholar is a powerful tool for searching gray literature, as long as the users are familiar with all the possibilities it offers as a search engine. Poorly formulated searches will undoubtedly return misleading results.
Congress
Martín‐Martín, A., Orduna-Malea, E., Delgado López-Cózar, E. (2017). Tapping the potential of an academic search engine for
bibliometric analyses: the case of Google Scholar. CWTS Seminar, 15 September, 2017
Martín‐Martín, A., Orduna-Malea, E., Delgado López-Cózar, E. (2017). Scholar Mirrors: Integrating
evidence of impact from multiple sources into one platform to expedite
researcher evaluation. STI 2017
Conference: Science, Technology and Innovation indicators. “Open indicators:
innovation, participation and actor-based STI indicators”. Paris
This poster
describes the creation of “Scholar Mirrors”, a prototype web application that
aims to provide a quick but accurate representation of a scientific discipline
by integrating data from multiple online platforms. We chose the discipline of
Bibliometrics / Scientometrics as a case study. After carrying out a series of
keywords searches in Google Scholar Citations (GSC) and Google Scholar (GS),
813 relevant researchers were identified. Researchers were further classified
as core (those who work mainly on Scientometrics) or related (those who work in
other disciplines, with occasional incursions into Scientometrics). Additional
information about these researchers was collected from other platforms
(ResearcherID, ResearchGate, Mendeley, and Twitter). Up to 28 author-level
indicators were collected about each researcher, as well as data about up to
100 of the most cited documents displayed in their GSC profile. The
document-level data from all GSC profiles, as well as the data extracted from
the keyword searchers in GS, was aggregated to create a list of the top 1000
most cited documents in the discipline. This document collection was further
processed to generate a list of the most influential journals and publishers in
the discipline. The results are accessible from the “Scholar Mirrors” website,
which presents the results in four sections: authors, documents, journals, and
book publishers. Lastly, the poster presents the main features of the web
application, and the main limitations and future challenges of the product.
.
Martín‐Martín, A., Orduna-Malea, E., Delgado López-Cózar, E. (2017). Journal Scholar Metrics: building an Arts, Humanities
and Social Sciences journal ranking with Google Scholar data. STI 2017
Conference: Science, Technology and Innovation indicators. “Open indicators:
innovation, participation and actor-based STI indicators”. Paris
This paper
describes the creation of “Journal Scholar Metrics” (JSM), a prototype web
application that ranks journals in the areas of Arts, Humanities, and Social
Sciences (AH&SS) on the basis of the citations their articles have received
according to Google Scholar Metrics (GSM). To identify as many AH&SS
journals as possible, a master list of 66,454 journals covered by various
databases was developed. All AH&SS journals in that list were searched on
GSM. Additionally, a series of keyword searches were carried out to identify
journals covered by GSM which weren’t present in the master list. A total of
9,188 AH&SS journals with names written in Latin characters were found in
the 2015 edition of GSM (which displays data about articles published between
2010 and 2014). Besides the journal-level indicators provided by GSM (H5-index
and H5-median), several additional indicators were computed (H5-citations,
H5-index and H5-citations without journal self-citations, and journal
self-citation rate). Journals are displayed by subject categories and by
country of publication. Quartiles were computed for each category, and journals
in a category were further classified either as core (high affinity to the
category) or related (partial affinity). A detail page for each journal is also
available, displaying journal indicators, as well as a list of other databases
were the journal is indexed.Orduña-Malea, E., Martín-Martín, A., Ayllón Millán, J. M., & Delgado López-Cózar, E. (2017). The Google Scholar Revolution: A big data bibliometric tool. En: Google Scholar Day: "Changing current evaluation paradigms". Cibermetrics Lab (IPP-CSIC), Madrid, 20 February 2017. http://hdl.handle.net/10481/45073
Delgado López-Cózar, E. The Google Scholar Revolution: Opening the academic Pandora's box. The transformation of scientific communication. En: 2nd International Conference on Academic Communication Journals. Universitat Pompeu Fabra 17th February 2017. http://hdl.handle.net/10481/45072
Reports
Delgado López-Cózar, Emilio; Orduña-Malea, Enrique; (2017). Proceedings Scholar Metrics 2017: H Index of proceedings on Computer Science, Electrical & Electronic Engineering, and Communications according to Google Scholar Metrics (2012-2016). EC3 Reports, 22. Granada, 27th of December, 2017. doi.org/10.13140/RG.2.2.11387.41768
The objective of this report is to present a list of proceedings (conferences, workshops, symposia, meetings) in the areas of Computer Science, Electrical & Electronic Engineering, and Communications covered by Google Scholar Metrics and ranked according to their h-index. Google Scholar Metrics only displays publications that have published at least 100 papers and have received at least one citation in the last five years (currently 2012-2016). The queries were conducted between the 18th and 22th of December, 2017. A total of 1,918 proceedings have been identified
Delgado López-Cózar, E.; Martín-Martín, A.; Orduña-Malea, E.; (2017). Índice H de las revistas científicas españolas según Google Scholar Metrics (2012-2016). EC3 Reports, 20. Granada, 26th November 2017. doi.org/10.13140/RG.2.2.29279.56484
Rankings are presented by scientific fields and disciplines of Spanish scientific journals listed in Google Scholar Metrics (GSM). They are sorted according to the h-index. It pretends with this work to test the breadth of coverage that has Google Scholar Metrics of Spanish scientific journals. The two criteria used by Google Scholar Metrics to include journals in its product are: have 100 work published and own at least one cite. Bibliographic searches were conducted between 18 and 21 July 2016. 722 journals have been identified, of which 323 are social sciences, 201 Arts and Humanities, 134 of Health Sciences, 64 of the Natural Sciences and Engineering. They make a total of 699 unique journals (the same journal could be classified in one or more areas)
Journal Papers
Martín-Martín, A., Orduna-Malea, E., Ayllón, J.M. , Delgado López-Cózar, E. (2016). Back to the past: on the shoulders of an academic search engine giant. Scientometrics, 107(3), 1477-1487. DOI: http://dx.doi.org/10.1007/s11192-016-1917-2.
A study released by the Google Scholar team found an apparently increasing fraction of citations to old articles from studies published in the last 24 years (1990–2013). To demonstrate this finding we conducted a complementary study using a different data source (Journal Citation Reports), metric (aggregate cited half-life), time spam (2003–2013), and set of categories (53 Social Science subject categories and 167 Science subject categories). Although the results obtained confirm and reinforce the previous findings, the possible causes of this phenomenon keep unclear. We finally hypothesize that “first page results syndrome” in conjunction with the fact that Google Scholar favours the most cited documents are suggesting the growing trend of citing old documents is partly caused by Google Scholar.
Orduña-Malea, E., Martín-Martín, A., Delgado López-Cózar, E. (2016). The next bibliometrics: ALMetrics (Author Level Metrics) and the multiple faces of author impact.El profesional de la información, 25(3), 485-496. DOI: http://dx.doi.org/10.3145/epi.2016.may.18
The main goal of this article is to describe the purpose and content of a new branch of bibliometrics: ALMetrics (AuthorLevel Metrics). ALMetrics is focused on the quantitative analysis of an author’s performance by measuring the dimensions of their intellectual activity as shown through varied metric indicators. This article will list, define, and classify the differentmetrics that are offered in newer information portals that showcase the scientific activity of authors. These metrics are grouped into five sets: bibliometrics (publication and citation), usage, participation, rating, social connectivity, and composite indicators. This new bibliometric specialty is necessary because of new trends in scientific assessment, which have moved analysis away from old bibliometrics (based on journal analysis and Impact Factor) towards new bibliometrics that analyze both documents and authors via a mix of indicators. Most importantly, ALMetrics responds to the researchers’ desire for both knowledge and acknowledgement
Martín-Martín, A., Orduna-Malea, E., Ayllón, J.M. , Delgado López-Cózar, E. (2016). A two-sided academic landscape: snapshot of highly-cited documents in Google Scholar (1950-2013) Revista Española de Documentación Científica ,39(4), e149. DOI: http://dx.doi.org/10.3989/redc.2016.4.1405.
The main objective of this paper is to identify and define the core characteristics of the set of highly-cited documents in Google Scholar (document types, language, free availability, sources, and number of versions), on the hypothesis that the wide coverage of this search engine may provide a different portrait of these documents with respect to that offered by traditional bibliographic databases. To do this, a query per year was carried out from 1950 to 2013 identifying the top 1,000 documents retrieved from Google Scholar and obtaining a final sample of 64,000 documents, of which 40% provided a free link to full-text. The results obtained show that the average highly-cited document is a journal or book article (62% of the top 1% most cited documents of the sample), written in English (92.5% of all documents) and available online in PDF format (86.0% of all documents). Yet, the existence of errors should be noted, especially when detecting duplicates and linking citations properly. Nonetheless, the fact that the study focused on highly cited papers minimizes the effects of these limitations. Given the high presence of books and, to a lesser extent, of other document types (such as proceedings or reports), the present research concludes that the Google Scholar data offer an original and different vision of the most influential academic documents (measured from the perspective of their citation count), a
set composed not only of strictly scientific material (journal articles) but also of academic material in its broadest sense.
Orduña-Malea, E., Martín-Martín, A., Delgado López-Cózar, E. (2016). ResearchGate como fuente de evaluación científica: desvelando sus aplicaciones bibliométricas. El profesional de la información, 25(2), 303-310. DOI: http://dx.doi.org/10.3145/epi.2016.mar.18
A study released by the Google Scholar team found an apparently increasing fraction of citations to old articles from studies published in the last 24 years (1990–2013). To demonstrate this finding we conducted a complementary study using a different data source (Journal Citation Reports), metric (aggregate cited half-life), time spam (2003–2013), and set of categories (53 Social Science subject categories and 167 Science subject categories). Although the results obtained confirm and reinforce the previous findings, the possible causes of this phenomenon keep unclear. We finally hypothesize that “first page results syndrome” in conjunction with the fact that Google Scholar favours the most cited documents are suggesting the growing trend of citing old documents is partly caused by Google Scholar.
The main goal of this article is to describe the purpose and content of a new branch of bibliometrics: ALMetrics (AuthorLevel Metrics). ALMetrics is focused on the quantitative analysis of an author’s performance by measuring the dimensions of their intellectual activity as shown through varied metric indicators. This article will list, define, and classify the differentmetrics that are offered in newer information portals that showcase the scientific activity of authors. These metrics are grouped into five sets: bibliometrics (publication and citation), usage, participation, rating, social connectivity, and composite indicators. This new bibliometric specialty is necessary because of new trends in scientific assessment, which have moved analysis away from old bibliometrics (based on journal analysis and Impact Factor) towards new bibliometrics that analyze both documents and authors via a mix of indicators. Most importantly, ALMetrics responds to the researchers’ desire for both knowledge and acknowledgement
The main objective of this paper is to identify and define the core characteristics of the set of highly-cited documents in Google Scholar (document types, language, free availability, sources, and number of versions), on the hypothesis that the wide coverage of this search engine may provide a different portrait of these documents with respect to that offered by traditional bibliographic databases. To do this, a query per year was carried out from 1950 to 2013 identifying the top 1,000 documents retrieved from Google Scholar and obtaining a final sample of 64,000 documents, of which 40% provided a free link to full-text. The results obtained show that the average highly-cited document is a journal or book article (62% of the top 1% most cited documents of the sample), written in English (92.5% of all documents) and available online in PDF format (86.0% of all documents). Yet, the existence of errors should be noted, especially when detecting duplicates and linking citations properly. Nonetheless, the fact that the study focused on highly cited papers minimizes the effects of these limitations. Given the high presence of books and, to a lesser extent, of other document types (such as proceedings or reports), the present research concludes that the Google Scholar data offer an original and different vision of the most influential academic documents (measured from the perspective of their citation count), a
set composed not only of strictly scientific material (journal articles) but also of academic material in its broadest sense.
A study released by the Google Scholar team found an apparently increasing fraction of citations to old articles from studies published in the last 24 years (1990–2013). To demonstrate this finding we conducted a complementary study using a different data source (Journal Citation Reports), metric (aggregate cited half-life), time spam (2003–2013), and set of categories (53 Social Science subject categories and 167 Science subject categories). Although the results obtained confirm and reinforce the previous findings, the possible causes of this phenomenon keep unclear. We finally hypothesize that “first page results syndrome” in conjunction with the fact that Google Scholar favours the most cited documents are suggesting the growing trend of citing old documents is partly caused by Google Scholar.
Working papers
Martín-Martín, A.; Ayllón, J.M. Orduna-Malea, E.; Delgado López-Cózar, E. (2016). 2016 Google Scholar Metrics released: a matter of languages... and something else. EC3 Working Papers, 22. 21st of July 2016.
Martín-Martín, A.; Orduna-Malea, E.; Ayllón, J.M. & Delgado López-Cózar, E. (2016). The counting house: measuring those who count. Presence of Bibliometrics, Scientometrics, Informetrics, Webometrics and Altmetrics in the Google Scholar Citations, ResearcherID, ResearchGate, Mendeley & Twitter”. EC3 Working Papers, 21. 19th of January 2016.
Following in the footsteps of the model of scientific communication, which has recently gone through a metamorphosis (from the Gutenberg galaxy to the Web galaxy), a change in the model and methods of scientific evaluation is also taking place. A set of new scientific tools are now providing a variety of indicators which measure all actions and interactions among scientists in the digital space, making new aspects of scientific communication emerge. In this work we present a method for capturing the structure of an entire scientific community (the Bibliometrics, Scientometrics, Informetrics, Webometrics, and Altmetrics community) and the main agents that are part of it (scientists, documents, and sources) through the lens of Google Scholar Citations.
The 2016 edition of Google Scholar Metrics was released on July 15th 2016. There haven't been any structural changes respect to previous versions, which means that most of its limitations still persist. The biggest changes are the addition of five new language rankings (Russian, Korean, Polish, Ukrainian, and Indonesian) and elimination of two other language rankings (Italian and Dutch). In addition, for reasons still unknown, this new edition doesn't include as many working paper and discussion paper series as previous editions.
Following in the footsteps of the model of scientific communication, which has recently gone through a metamorphosis (from the Gutenberg galaxy to the Web galaxy), a change in the model and methods of scientific evaluation is also taking place. A set of new scientific tools are now providing a variety of indicators which measure all actions and interactions among scientists in the digital space, making new aspects of scientific communication emerge. In this work we present a method for capturing the structure of an entire scientific community (the Bibliometrics, Scientometrics, Informetrics, Webometrics, and Altmetrics community) and the main agents that are part of it (scientists, documents, and sources) through the lens of Google Scholar Citations.
Additionally, we compare these author portraits to the ones offered by other profile or social platforms currently used by academics (ResearcherID, ResearchGate, Mendeley, and Twitter), in order to test their degree of use, completeness, reliability, and the validity of the information they provide. A sample of 814 authors (researchers in Bibliometrics with a public profile created in Google Scholar Citations was subsequently searched in the other platforms, collecting the main indicators computed by each of them. The data collection was carried out on September, 2015. The Spearman correlation was applied to these indicators (a total of 31) , and a Principal Component Analysis was carried out in order to reveal the relationships among metrics and platforms as well as the possible existence of metric clusters.
Books and book chapters
Orduna-Malea, E., Martín-Martín, A., Ayllón, J.M. , Delgado López-Cózar, E. (2016). La revolución Google Scholar: Destapando la caja de Pandora académica. Unión de Editoriales Universitarias Españolas (UNE), Universidad de Granada . ISBN 978-84-338-5985-3
Este libro es fruto de la intensa y extensa labor investigadora desplegada por los autores durante casi una década en el seno de la Universidad de Granada y la Universidad Politécnica de Valencia donde trabajan como profesores e investigadores. Resultado de todo este que hacer ha sido la publicación de más de 50 artículos, informes y documentos de trabajo así como el diseño de varios productos de evaluación bibliométrica (H Index Scholar, Journal Scholar Metrics, Scholar Mirrors, Publishers Scholar Metrics, Proceedings Scholar Metrics, Classics Scholar’ Profiles, Indice H revistas científicas españolas según Google Scholar Metrics, La Biblioteconomía y Documentación española según Google Scholar Citations), que han explorado terrenos ignotos para la evaluación científica y abierto por primera vez nuevas vías para captar la producción e impacto de los científicos y sus obras.
Google Scholar ha supuesto una auténtica revolución en la forma de buscar y de encontrar información científica y de acceder a ella de manera fácil y rápida. Hoy día su simple caja de búsqueda, convertida ya en un estándar de facto en la búsqueda de información, se encarga de mostrar numerosos y pertinentes documentos escritos en cualquier lengua, producidos en cualquier país, bajo cualquier formato y tipología documental. Y todo ello ofrecido de forma gratuita. Pero lo que fue creado para buscar información se ha transformado, sin quererlo, en una valiosísima fuente de datos para la evaluación científica con múltiples usos bibliométricos, es decir, en una auténtica “caja de pandora académica”.
El objetivo de este libro es, ante todo, describir minuciosamente las características y prestaciones de Google Scholar. Se traza su origen y evolución, se desmenuza su funcionamiento general, se examina su tamaño, cobertura y crecimiento, se pormenorizan las prestaciones y servicios que proporciona como buscador y se apuntan sus fortalezas, debilidades y peligros. Se realiza un repaso exhaustivo de toda la literatura científica que ha indagado empíricamente sobre el comportamiento de Google Scholar hasta hoy.
Pero también, esta obra pretende explicar los principales productos derivados de Google Scholar, aquellos que conforman lo que denominamos la familia Google Scholar: Google Scholar Citations, nacido como un servicio a los autores para que generen un perfil donde se muestren los documentos publicados y recogidos en Google Scholar, así como el número de citas que cada uno de ellos ha recibido, generando una serie de indicadores bibliométricos. Y Google Scholar Metrics nacido como ranking de publicaciones científicas que permite a partir de los datos de citaciones de Google Scholar y por medio de índice h identificar las más influyentes revistas, congresos y repositorios. Por otra parte, se ha incluido un somero análisis de aquellos productos independientes a la compañía y que se han gestado para ofrecer distintas herramientas bibliométricas: Publish or Perish, Scholarometer, H Index Scholar, Journal Scholar Metrics, Publishers Scholar Metrics, Proceedings Scholar Metrics y Scholar Mirrors.
En definitiva estamos ante un libro que ofrece una revisión omnicomprensiva a la par que minuciosa sobre lo que es a día de hoy Google Scholar y sus derivaciones.
Posts
Martín-Martín, A., Orduña-Malea, E., Delgado López-Cózar, E. (2016). The role of ego in academic profile services: Comparing Google Scholar, ResearchGate, Mendeley, and ResearcherID. The LSE Impact blog, 4 March 2016.
Ayllón, J.M., Martín-Martín, A., Orduña-Malea, E., Delgado López-Cózar, E.(2016).Índice H de las revistas científicas españolas según Google Scholar Metrics (2011-2015). EC3 Reports, 17: 6th October 2016. http://hdl.handle.net/10481/42829
Reports
Martín-Martín, A., Ayllón, J.M., Orduña-Malea, E., Delgado López-Cózar, E. (2016). Proceedings Scholar Metrics: H Index of proceedings on Computer Science, Electrical & Electronic Engineering, and Communications according to Google Scholar Metrics (2011-2015). EC3 Reports, 19. Granada, 13 December 2016. DOI: http://dx.doi.org/10.13140/RG.2.2.16076.41605 .
The objective of this report is to present a list of proceedings (conferences, workshops, symposia, meetings) in the areas of Computer Science, Electrical & Electronic Engineering, and Communications covered by Google Scholar Metrics and ranked according to their h-index. Google Scholar Metrics only displays publications that have published at least 100 papers and have received at least one citation in the last five years (2010-2015). The searches were conducted between the 7th and 12th of December, 2016. A total of 1634 proceedings have been identified.
Rankings are presented by scientific fields and disciplines of Spanish scientific journals listed in Google Scholar Metrics (GSM). They are sorted according to the h-index. It pretends with this work to test the breadth of coverage that has Google Scholar Metrics of Spanish scientific journals. The two criteria used by Google Scholar Metrics to include journals in its product are: have 100 work published and own at least one cite. Bibliographic searches were conducted between 18 and 21 July 2016. 1299 journals have been identified, of which 645 are social sciences, 337 Arts and Humanities, 177 of Health Sciences, 140 of the Natural Sciences and Engineering.
Journal Papers
Martín‐Martín, A., Ayllón, J.M., Delgado López‐Cózar, E.; Orduna‐Malea, E. (2015). Nature's top 100 Re‐revisited. Journal of the Association for Information Science and Technology, 66 (12),2714 . DOI: http://dx.doi.org/10.1002/asi.23570.
Orduña-Malea, E., Delgado López-Cózar, E. (2015). The dark side of Open Access in Google and Google Scholar: the case of Latin-American repositories. Scientometrics, 102(1), 829-846. DOI: http://dx.doi.org/10.1007/s11192-014-1369-5.
Since repositories are a key tool in making scholarly knowledge open access (OA), determining their web presence and visibility on the Web (both are proxies of web impact) is essential, particularly in Google (search engine par excellence) and Google Scholar (a tool increasingly used by researchers to search for academic information). The few studies conducted so far have been limited to very specific geographic areas (USA), which makes it necessary to find out what is happening in other regions that are not part of mainstream academia, and where repositories play a decisive role in the visibility of scholarly production. The main objective of this study is to ascertain the web presence and visibility of Latin American repositories in Google and Google Scholar through the application of page count and web mention indicators respectively. For a sample of 137 repositories, the results indicate that the indexing ratio is low in Google, and virtually nonexistent in Google Scholar; they also indicate a complete lack of correspondence between the repository records and the data produced by these two search tools. These results are mainly attributable to limitations arising from the use of description schemas that are incompatible with Google Scholar (repository design) and the reliability of web mention indicators (search engines). We conclude that neither Google nor Google Scholar accurately represent the actual size of OA content published by Latin American repositories; this may indicate a non-indexed, hidden side to OA, which could be limiting the dissemination and consumption of OA scholarly literature.
Since repositories are a key tool in making scholarly knowledge open access (OA), determining their web presence and visibility on the Web (both are proxies of web impact) is essential, particularly in Google (search engine par excellence) and Google Scholar (a tool increasingly used by researchers to search for academic information). The few studies conducted so far have been limited to very specific geographic areas (USA), which makes it necessary to find out what is happening in other regions that are not part of mainstream academia, and where repositories play a decisive role in the visibility of scholarly production. The main objective of this study is to ascertain the web presence and visibility of Latin American repositories in Google and Google Scholar through the application of page count and web mention indicators respectively. For a sample of 137 repositories, the results indicate that the indexing ratio is low in Google, and virtually nonexistent in Google Scholar; they also indicate a complete lack of correspondence between the repository records and the data produced by these two search tools. These results are mainly attributable to limitations arising from the use of description schemas that are incompatible with Google Scholar (repository design) and the reliability of web mention indicators (search engines). We conclude that neither Google nor Google Scholar accurately represent the actual size of OA content published by Latin American repositories; this may indicate a non-indexed, hidden side to OA, which could be limiting the dissemination and consumption of OA scholarly literature.
Orduña-Malea, E., Ayllón, J.M., Martín-Martín, A., Delgado López-Cózar, E. (2015). Aplicaciones métricas de Google Scholar para la evaluación del impacto científico. Actas de las 4ª Jornadas de Intercambio y Reflexión acerca de la Investigación en Bibliotecología .
La aparición de Google Scholar en 2004 supuso una revolución tanto en los procesos de
búsqueda de información académica como en los de evaluación científica (especialmente en las áreas de las Ciencias sociales y humanas) gracias a su ingente base de datos, basada en la recopilación de cualquier documento académico online, a pesar de los errores e imprecisiones existentes en la correcta vinculación de citas. Este trabajo presenta la elaboración de diferentes productos métricos de información elaborados a partir de Google Scholar (H Index Scholar, Publishers Scholar Metrics, Journal Scholar Metrics, La Biblioteconomía Española según Google Scholar Citations) con el propósito de mostrar la potencialidad y utilidad de esta base de datos a la hora de ser utilizada por los diferentes actores involucrados en la creación, difusión y evaluación de la actividad científica.
Orduña-Malea, E., Ayllón, J.M., Martín-Martín, A., Delgado López-Cózar, E. (2015). Methods for estimating the size of Google Scholar. Scientometrics, 104(3), 931-949 . DOI: http://dx.doi.org/10.1007/s11192-015-1614-6 .
The emergence of academic search engines (mainly Google Scholar and Microsoft Academic Search) that aspire to index the entirety of current academic knowledge has revived and increased interest in the size of the academic web. The main objective of this paper is to propose various methods to estimate the current size (number of indexed documents) of Google Scholar (May 2014) and to determine its validity, precision and reliability. To do this, we present, apply and discuss three empirical methods: an external estimate based on empirical studies of Google Scholar coverage, and two internal estimate methods based on direct, empty and absurd queries, respectively. The results, despite providing disparate values, place the estimated size of Google Scholar at around 160–165 million documents. However, all the methods show considerable limitations and uncertainties due to inconsistencies in the Google Scholar search functionalities.
Digests
Martín-Martín, A., Orduña-Malea, E., Ayllón, J.M., Delgado López-Cózar, E. (2015). Reviving the past: the growth of citations to old documents. Granada: EC3 Google Scholar Digest Reviews (4): 9 January 2015.
Martín-Martín, A., Orduña-Malea, E., Ayllón, J.M., Delgado López-Cózar, E. (2015). Reviving the past: the growth of citations to old documents. Granada: EC3 Google Scholar Digest Reviews (4): 9 January 2015.
Haddaway, N.R., Collins, A.M., Coughlin, D., Kirk, S. (2015): The Role of Google Scholar in Evidence Reviews and Its Applicability to Grey Literature Searching. PLoS ONE 10(9): e0138237.
Harzing, A.W., Alakangas, S. (2016). Google Scholar, Scopus and the Web of Science: a longitudinal and cross-disciplinary comparison. Scientometrics, 106(2), 787-804 .
Sittig, D.F., McCoy, A.B., Wright, A., Lin, J. (2015). Developing an Open-Source Bibliometric Ranking Website Using Google Scholar Citation Profiles for Researchers in the Field of Biomedical Informatics. Studies in health technology and informatics, 216, 1004-1004.
Ortega, J.L. (2015). How is an academic social site populated? A demographic study of Google Scholar Citations population, Scientometrics, 104(1), 1-18.
Jamali, H.R., Nabavi, M. (2015). Open access and sources of full-text articles in Google Scholar in different subject fields. Scientometrics, 105(3), 1635-1651
Levay, P., Ainsworth, N., Kettle, R., Morgan, A. (2015). Identifying evidence for public health guidance: a comparison of citation searching with Web of Science and Google Scholar. Research Synthesis Methods, (Online Version)
The objective of this report is to present a list of proceedings (conferences, workshops, symposia, meetings) in the areas of Computer Science, Electrical & Electronic Engineering, and Communications covered by Google Scholar Metrics and ranked according to their hindex. Google Scholar Metrics only displays publications that have published at least 100 papers and have received at least one citation in the last five years (2010-2014). The searches were conducted between the 8 th and 10th of December, 2015. A total of 1501 proceedings have been identified.
Orduña-Malea, E., Ayllón, J.M., Martín-Martín, A., Delgado López-Cózar, E. (2015).Improvements in Google Scholar Citations are for the summer: creating an institutional affiliation linking feature. EC3 Reports, 14: 14 September 2015 [http://arxiv.org/abs/1509.04515] .
This report describes the feature introduced by Google to provide standardized access to institutional affiliations within Google Scholar Citations. First, this new tool is described, pointing out its main characteristics and functioning. Next, the coverage and precision of the tool are evaluated. Two special cases (Google Inc. and Spanish Universities) are briefly treated with the purpose of illustrating some aspects about the accuracy of the tool for the task of gathering authors within their appropriate institution. Finally, some inconsistencies, errors and malfunctioning are identified, categorized and described. The report finishes by providing some suggestions to improve the feature. The general conclusion is that the standardized institutional affiliation link provided by Google Scholar Citations, despite working pretty well for a large number of institutions (especially Anglo-Saxon universities) still has a number of shortcomings and pitfalls which need to be addressed in order to make this authority control tool fully useful worldwide, both for searching purposes and for metric tasks.
Ayllón, J.M., Martín-Martín, A., Orduña-Malea, E., Delgado López-Cózar, E.(2015).Índice H de las revistas científicas españolas según Google Scholar Metrics (2010-2014).EC3 Reports, 13: 16 July 2015 [http://hdl.handle.net/10481/36998].
Rankings are presented by scientific fields and disciplines of Spanish scientific journals listed in Google Scholar Metrics (GSM). They are sorted according to the h-index. It pretends with this work to test the breadth of coverage that has Google Scholar Metrics of Spanish scientific journals. The two criteria used by Google Scholar Metrics to include journals in its product are: have 100 work published and own at least one cite. Bibliographic searches were conducted between 8 and 10 July 2015. 1069 journals have been identified, of which 560 are social sciences, 248 Arts and Humanities, 142 of Health Sciences, 119 of the Natural Sciences and Engineering
2014 [Go back]
Journal Papers
Delgado López-Cózar, E., Orduña-Malea, E., Jiménez-Contreras, E., Ruiz-Pérez, R. (2014). H Index Scholar: el índice h de los profesores de las universidades públicas españolas en humanidades y ciencias sociales. El Profesional de la Información, 23(1), 87-94. DOI: http://dx.doi.org/10.3145/epi.2014.ene.11 .
The H-Index Scholar is a bibliometric index that measures the productivity and scientific impact of the academic production in humanities and social sciences by professors and researchers at public Spanish universities. The methodology consisted of counting their publications and citations received in Google Scholar. The main features and characteristics of the index are explained. Despite technical and methodological problems that Google Scholar might have as a source of information, the authors estimate that they do not affect substantially the calculated h and g indexes, probably being the error lower than 10%. The total population analyzed was 40,993 researchers, but data are displayed only for 13,518 researchers, the ones located in the first tertile of their respective areas.
The H-Index Scholar is a bibliometric index that measures the productivity and scientific impact of the academic production in humanities and social sciences by professors and researchers at public Spanish universities. The methodology consisted of counting their publications and citations received in Google Scholar. The main features and characteristics of the index are explained. Despite technical and methodological problems that Google Scholar might have as a source of information, the authors estimate that they do not affect substantially the calculated h and g indexes, probably being the error lower than 10%. The total population analyzed was 40,993 researchers, but data are displayed only for 13,518 researchers, the ones located in the first tertile of their respective areas.
Delgado López-Cózar, E., Robinson-García, N., Torres-Salinas, D. (2014). The Google Scholar Experiment: how to index false papers and manipulate bibliometric indicators. Journal of the American Society for Information Science and Technology, 65(3), 446–454. DOI: http://dx.doi.org/10.1002/asi.23056 .
Google Scholar has been well received by the research community. Its promises of free, universal, and easy access to scientific literature coupled with the perception that it covers the social sciences and the humanities better than other traditional multidisciplinary databases have contributed to the quick expansion of Google Scholar Citations and Google Scholar Metrics: 2 new bibliometric products that offer citation data at the individual level and at journal level. In this article, we show the results of an experiment undertaken to analyze Google Scholar's capacity to detect citation-counting manipulation. For this, we uploaded 6 documents to an institutional web domain that were authored by a fictitious researcher and referenced all the publications of the members of the EC3 research group at the University of Granada. The detection by Google Scholar of these papers caused an outburst in the number of citations included in the Google Scholar Citations profiles of the authors. We discuss the effects of such an outburst and how it could affect the future development of such products, at both the individual level and the journal level, especially if Google Scholar persists with its lack of transparency.
Google Scholar has been well received by the research community. Its promises of free, universal, and easy access to scientific literature coupled with the perception that it covers the social sciences and the humanities better than other traditional multidisciplinary databases have contributed to the quick expansion of Google Scholar Citations and Google Scholar Metrics: 2 new bibliometric products that offer citation data at the individual level and at journal level. In this article, we show the results of an experiment undertaken to analyze Google Scholar's capacity to detect citation-counting manipulation. For this, we uploaded 6 documents to an institutional web domain that were authored by a fictitious researcher and referenced all the publications of the members of the EC3 research group at the University of Granada. The detection by Google Scholar of these papers caused an outburst in the number of citations included in the Google Scholar Citations profiles of the authors. We discuss the effects of such an outburst and how it could affect the future development of such products, at both the individual level and the journal level, especially if Google Scholar persists with its lack of transparency.
Orduña-Malea, E., Delgado López-Cózar, E. (2014). Google Scholar Metrics evolution: an analysis according to languages. Scientometrics, 98(3), 2353–2367. DOI: http://dx.doi.org/10.1007/s11192-013-1164-8 .
November 2012 the Google Scholar Metrics (GSM) journal rankings were updated, making it possible to compare bibliometric indicators in the ten languages indexed—and their stability—with the April 2012 version. The h-index and h-5 median of 1,000 journals were analysed, comparing their averages, maximum and minimum values and the correlation coefficient within rankings. The bibliometric figures grew significantly. In just seven and a half months the h-index of the journals increased by 15 % and the median h-index by 17 %. This growth was observed for all the bibliometric indicators analysed and for practically every journal. However, we found significant differences in growth rates depending on the language in which the journal is published. Moreover, the journal rankings seem to be stable between April and November, reinforcing the credibility of the data held by Google Scholar and the reliability of the GSM journal rankings, despite the uncontrolled growth of Google Scholar. Based on the findings of this study we suggest, firstly, that Google should upgrade its rankings at least semi-annually and, secondly, that the results should be displayed in each ranking proportionally to the number of journals indexed by language.
November 2012 the Google Scholar Metrics (GSM) journal rankings were updated, making it possible to compare bibliometric indicators in the ten languages indexed—and their stability—with the April 2012 version. The h-index and h-5 median of 1,000 journals were analysed, comparing their averages, maximum and minimum values and the correlation coefficient within rankings. The bibliometric figures grew significantly. In just seven and a half months the h-index of the journals increased by 15 % and the median h-index by 17 %. This growth was observed for all the bibliometric indicators analysed and for practically every journal. However, we found significant differences in growth rates depending on the language in which the journal is published. Moreover, the journal rankings seem to be stable between April and November, reinforcing the credibility of the data held by Google Scholar and the reliability of the GSM journal rankings, despite the uncontrolled growth of Google Scholar. Based on the findings of this study we suggest, firstly, that Google should upgrade its rankings at least semi-annually and, secondly, that the results should be displayed in each ranking proportionally to the number of journals indexed by language.
Orduña-Malea, E., Martín-Martín, A., Ayllon, J.M., & Delgado Lopez-Cozar, E. Delgado López-Cózar, E. (2014). The silent fading of an academic search engine: the case of Microsoft Academic Search. Online Information Review, 38(7), 936-953. DOI: http://dx.doi.org/10.1108/OIR-07-2014-0169 .
Purpose – The purpose of this paper is to describe the obsolescence process of Microsoft Academic Search (MAS) as well as the effects of this decline in the coverage of disciplines and journals, and their influence in the representativeness of organizations.
Design/methodology/approach – The total number of records and those belonging to the most reputable journals (1,762) and organizations (346) according to the Field Rating indicator in each of the 15 fields and 204 sub-fields of MAS, have been collected and statistically analysed in March 2014, by means of an automated querying process via http, covering academic publications from 1700 to present.
Findings – MAS has no longer been updated since 2013, although this phenomenon began to be glimpsed in 2011, when its coverage plummeted. Throughout 2014, indexing of new records is still ongoing, but at a minimum rate, without following any apparent pattern.
Research limitations/implications – There are also retrospective records being indexed at present. In this sense, this research provides a picture of what MAS offered during March 2014 being queried directly via http.
Practical implications – The unnoticed obsolescence of MAS affects to the quality of the service offered to its users (both those who engage in scientific information seeking and also those who use it for quantitative purposes).
Social implications – The predominance of Google Scholar (GS) as monopoly in the academic search engines market as well as the prevalence of an open construction model (GS) vs a closed model (MAS).
Originality/value – A complete longitudinal analysis of disciplines, journals and organizations on MAS has been performed for the first time identifying an unnoticed obsolescence. Any public explanation or disclaimer note has been announced from the responsible company, something incomprehensible given its implications for the reliability and validity of bibliometric data provided on disciplines, journals, authors and congress as well as their fair representation on the academic search engine.
Purpose – The purpose of this paper is to describe the obsolescence process of Microsoft Academic Search (MAS) as well as the effects of this decline in the coverage of disciplines and journals, and their influence in the representativeness of organizations.
Design/methodology/approach – The total number of records and those belonging to the most reputable journals (1,762) and organizations (346) according to the Field Rating indicator in each of the 15 fields and 204 sub-fields of MAS, have been collected and statistically analysed in March 2014, by means of an automated querying process via http, covering academic publications from 1700 to present.
Findings – MAS has no longer been updated since 2013, although this phenomenon began to be glimpsed in 2011, when its coverage plummeted. Throughout 2014, indexing of new records is still ongoing, but at a minimum rate, without following any apparent pattern.
Research limitations/implications – There are also retrospective records being indexed at present. In this sense, this research provides a picture of what MAS offered during March 2014 being queried directly via http.
Practical implications – The unnoticed obsolescence of MAS affects to the quality of the service offered to its users (both those who engage in scientific information seeking and also those who use it for quantitative purposes).
Social implications – The predominance of Google Scholar (GS) as monopoly in the academic search engines market as well as the prevalence of an open construction model (GS) vs a closed model (MAS).
Originality/value – A complete longitudinal analysis of disciplines, journals and organizations on MAS has been performed for the first time identifying an unnoticed obsolescence. Any public explanation or disclaimer note has been announced from the responsible company, something incomprehensible given its implications for the reliability and validity of bibliometric data provided on disciplines, journals, authors and congress as well as their fair representation on the academic search engine.
Working papers
Martín-Martín, A., Orduña-Malea, E., Ayllón, J.M., Delgado López-Cózar, E. (2014). Does Google Scholar contain all highly cited documents (1950-2013)? Granada: EC3 Working Papers, 19: October 29, 2014.arXiv preprint arXiv:1410.8464.
The study of highly cited documents on Google Scholar (GS) has never been addressed to date in a comprehensive manner. The objective of this work is to identify the set of highly cited documents in Google Scholar and define their core characteristics: their languages, their file format, or how many of them can be accessed free of charge. We will also try to answer some additional questions that hopefully shed some light about the use of GS as a tool for assessing scientific impact through citations. The decalogue of research questions is shown below:
1. Which are the most cited documents in GS?
2. Which are the most cited document types in GS?
3. What languages are the most cited documents written in GS?
4. How many highly cited documents are freely accessible?
4.1 What file types are the most commonly used to store these highly cited documents?
4.2 Which are the main providers of these documents?
5. How many of the highly cited documents indexed by GS are also indexed by WoS?
6. Is there a correlation between the number of citations that these highly cited documents have received in GS and the number of citations they have received in WoS?
7. How many versions of these highly cited documents has GS detected?
8. Is there a correlation between the number of versions GS has detected for these documents, and the number citations they have received?
9. Is there a correlation between the number of versions GS has detected for these documents, and their position in the search engine result pages?
10. Is there some relation between the positions these documents occupy in the search engine result pages, and the number of citations they have received?
Orduña-Malea, E., Ayllón, J.M., Martín-Martín, A., Delgado López-Cózar, E. (2014). About the size of Google Scholar: playing the numbers. Granada: EC3 Working Papers, 18: 24 July 2014. arXiv preprint arXiv:1407.6239.
The emergence of academic search engines (Google Scholar and Microsoft Academic Search essentially) has revived and increased the interest in the size of the academic web, since their aspiration is to index the entirety of current academic knowledge. The search engine functionality and human search patterns lead us to believe, sometimes, that what you see in the search engine's results page is all that really exists. And, even when this is not true, we wonder which information is missing and why. The main objective of this working paper is to calculate the size of Google Scholar at present (May 2014). To do this, we present, apply and discuss up to 4 empirical methods: Khabsa & Giles's method, an estimate based on empirical data, and estimates based on direct queries and absurd queries. The results, despite providing disparate values, place the estimated size of Google Scholar in about 160 million documents. However, the fact that all methods show great inconsistencies, limitations and uncertainties, makes us wonder why Google does not simply provide this information to the scientific community if the company really knows this figure.
The emergence of academic search engines (Google Scholar and Microsoft Academic Search essentially) has revived and increased the interest in the size of the academic web, since their aspiration is to index the entirety of current academic knowledge. The search engine functionality and human search patterns lead us to believe, sometimes, that what you see in the search engine's results page is all that really exists. And, even when this is not true, we wonder which information is missing and why. The main objective of this working paper is to calculate the size of Google Scholar at present (May 2014). To do this, we present, apply and discuss up to 4 empirical methods: Khabsa & Giles's method, an estimate based on empirical data, and estimates based on direct queries and absurd queries. The results, despite providing disparate values, place the estimated size of Google Scholar in about 160 million documents. However, the fact that all methods show great inconsistencies, limitations and uncertainties, makes us wonder why Google does not simply provide this information to the scientific community if the company really knows this figure.
Martín-Martín, A., Ayllón, J.M., Orduña-Malea, E., Delgado López-Cózar, E. (2014). Google Scholar Metrics 2014: a low cost bibliometric tool. Granada: EC3 Working Papers, 17: 11 July 2014. arXiv preprint arXiv:1407.2827.
We analyse the main features of the third edition of Google Scholar Metrics (GSM), released in June 2014, focusing on its more important changes, strengths, and weaknesses. Additionally, we present some figures that outline the dimensions of this new edition, and we compare them to those of previous editions. Principal among these figures are the number of visualized publications, publication types, languages, and the maximum and minimum h5-index and h5-median values by language, subject area, and subcategory. This new edition is marked by continuity. There is nothing new other than the updating of the time frame (2009-2013) and the removal of some redundant subcategories (from 268 to 261) for English written publications. Google has just updated the data, which means that some of the errors discussed in previous studies still persist. To sum up, GSM is a minimalist information product with few features, closed (it cannot be customized by the user), and simple (navigating it only takes a few clicks). For these reasons, we consider it a 'low cost' bibliometric tool, and propose a list of features it should incorporate in order to stop being labeled as such. Notwithstanding the above, this product presents a stability in its bibliometric indicators that supports its ability to measure and track the impact of scientific publications.
Orduña-Malea, E., Ayllón, J.M., Martín-Martín, A., Delgado López-Cózar, E. (2014). Empirical Evidences in Citation-Based Search Engines: Is Microsoft Academic Search dead? Granada: EC3 Working Papers, 16: 28 April 2014. arXiv preprint arXiv:1404.7045.
The goal of this working paper is to summarize the main empirical evidences provided by the scientific community as regards the comparison between the two main citation based academic search engines: Google Scholar and Microsoft Academic Search, paying special attention to the following issues: coverage, correlations between journal rankings, and usage of these academic search engines. Additionally, selfelaborated data is offered, which are intended to provide current evidence about the popularity of these tools on the Web, by measuring the number of rich files PDF, PPT and DOC in which these tools are mentioned, the amount of external links that both products receive, and the search queries frequency from Google Trends. The poor results obtained by MAS led us to an unexpected and unnoticed discovery: Microsoft Academic Search is outdated since 2013. Therefore, the second part of the working paper aims at advancing some data demonstrating this lack of update. For this purpose we gathered the number of total records indexed by Microsoft Academic Search since 2000. The data shows an abrupt drop in the number of documents indexed from 2,346,228 in 2010 to 8,147 in 2013 and 802 in 2014. This decrease is offered according to 15 thematic areas as well. In view of these problems it seems logical not only that Microsoft Academic Searchwas poorly used to search for articles by academics and students, who mostly use Google or Google Scholar, but virtually ignored by bibliometricians.
Reports
Martín-Martín, A., Orduña-Malea, E., Ayllón, J.M., Delgado López-Cózar, E. (2014). Proceedings Scholar Metrics: H Index of proceedings on Computer Science, Electrical & Electronic Engineering, and Communications according to Google Scholar Metrics (2009-2013). EC3 Reports, 12. Granada, 23 December 2014.
The objective of this report is to present a list of proceedings (conferences, workshops, symposia, meetings) in the areas of Computer Science, Electrical & Electronic Engineering, and Communications covered by Google Scholar Metrics and ranked according to their h-index. Google Scholar Metrics only displays publications that have published at least 100 papers and have received at least one citation in the last five years (2009-2013). The searches were conducted between the 15th and 22nd of December, 2014. A total of 1208 proceedings have been identified.
Repiso, R., Delgado López-Cózar, E. (2014). H Index Communication Journals according to Google Scholar Metrics (2009-2013). EC3 Reports, 10: 24 July 2014. [http://hdl.handle.net/10481/32830].
The aim of this report is to present a ranking of Communication journals covered in Google Scholar Metrics (GSM) for the period 2009-2013. It corresponds to the H Index update made last year for the period 2008-2012 (Repiso & Delgado López-Cózar 2013). Google Scholar Metrics does not currently allow to group and sort all journals belonging to a scientific discipline. Therefore, in an attempt to overcome this limitation, we have used the diversity of search procedures allowed by GSM to identify the greatest number of scientific journals of Communication with H Index calculated by this bibliometric tool. The result is a ranking of 434 communication journals sorted by the same H Index. Journals are also grouped by quartiles.
Ayllón, J.M., Martín-Martín, A., Orduña-Malea, E., Ruiz-Pérez, R. Delgado López-Cózar, E.(2014). Índice H de las revistas científicas españolas según Google Scholar Metrics (2009-2013).EC3 Reports, 9: 28 July 2014.[http://hdl.handle.net/10481/32471].
Rankings are presented by scientific fields and disciplines of Spanish scientific journals listed in Google Scholar Metrics (GSM). They are sorted according to the h-index. It pretends with this work to test the breadth of coverage that has Google Scholar Metrics of Spanish scientific journals. The two criteria used by Google Scholar Metrics to include journals in its product are: have 100 work published and own at least one cite. Bibliographic searches were conducted between 27 June and 2 July 2014. 1003 journals have been identified, of which 520 are social sciences, 218 arts and Humanities, 146 of Health Sciences, 119 of the Natural Sciences and Engineering .
Martín-Martín, A., Orduña-Malea, E., Ayllón, J.M., Delgado López-Cózar, E. (2014). Proceedings Scholar Metrics: H Index of proceedings on Computer Science, Electrical & Electronic Engineering, and Communications according to Google Scholar Metrics (2009-2013). EC3 Reports, 12. Granada, 23 December 2014.
The objective of this report is to present a list of proceedings (conferences, workshops, symposia, meetings) in the areas of Computer Science, Electrical & Electronic Engineering, and Communications covered by Google Scholar Metrics and ranked according to their h-index. Google Scholar Metrics only displays publications that have published at least 100 papers and have received at least one citation in the last five years (2009-2013). The searches were conducted between the 15th and 22nd of December, 2014. A total of 1208 proceedings have been identified.
Repiso, R., Delgado López-Cózar, E. (2014). H Index Communication Journals according to Google Scholar Metrics (2009-2013). EC3 Reports, 10: 24 July 2014. [http://hdl.handle.net/10481/32830].
The aim of this report is to present a ranking of Communication journals covered in Google Scholar Metrics (GSM) for the period 2009-2013. It corresponds to the H Index update made last year for the period 2008-2012 (Repiso & Delgado López-Cózar 2013). Google Scholar Metrics does not currently allow to group and sort all journals belonging to a scientific discipline. Therefore, in an attempt to overcome this limitation, we have used the diversity of search procedures allowed by GSM to identify the greatest number of scientific journals of Communication with H Index calculated by this bibliometric tool. The result is a ranking of 434 communication journals sorted by the same H Index. Journals are also grouped by quartiles.
Rankings are presented by scientific fields and disciplines of Spanish scientific journals listed in Google Scholar Metrics (GSM). They are sorted according to the h-index. It pretends with this work to test the breadth of coverage that has Google Scholar Metrics of Spanish scientific journals. The two criteria used by Google Scholar Metrics to include journals in its product are: have 100 work published and own at least one cite. Bibliographic searches were conducted between 27 June and 2 July 2014. 1003 journals have been identified, of which 520 are social sciences, 218 arts and Humanities, 146 of Health Sciences, 119 of the Natural Sciences and Engineering .
Reina Leal, L.M., Repiso, R., Delgado López-Cózar, E. (2014). H Index of scientific Nursing journals according to Google Scholar Metrics (2008-2012). EC3 Reports, 8: 1 March 2014. [http://hdl.handle.net/10481/31349].
The aim of this report is to present a ranking of Nursing journals covered in Google Scholar Metrics (GSM), for the 2008-2012. This is an update of previous ranking, based on the h index for journals included by GSM (Reina Leal, Repiso & Delgado López-Cózar, 2013). GSM doesn’t yet allow group and sort journals of a scientific discipline. Therefore, to the nursing discipline, can only be located 41 journals from the ten listings offered by GSM. However, in an attempt to overcome this limitation, we have used the diversity of search procedures allowed by GSM to identify the greatest number of scientific journals of Nursing with h index calculated by this bibliometric tool. Bibliographic searches were conducted between 3rd and 7th February 2014. The result is a ranking of 366 nursing journals sorted by the same h index, and mean as discriminating value. Journals are also grouped by quartiles.
The aim of this report is to present a ranking of Nursing journals covered in Google Scholar Metrics (GSM), for the 2008-2012. This is an update of previous ranking, based on the h index for journals included by GSM (Reina Leal, Repiso & Delgado López-Cózar, 2013). GSM doesn’t yet allow group and sort journals of a scientific discipline. Therefore, to the nursing discipline, can only be located 41 journals from the ten listings offered by GSM. However, in an attempt to overcome this limitation, we have used the diversity of search procedures allowed by GSM to identify the greatest number of scientific journals of Nursing with h index calculated by this bibliometric tool. Bibliographic searches were conducted between 3rd and 7th February 2014. The result is a ranking of 366 nursing journals sorted by the same h index, and mean as discriminating value. Journals are also grouped by quartiles.
Posts
Delgado López-Cózar, E. (2014). Sobre el número de documentos que indiza Google Scholar, reportaje de Jia You en el servicio de noticias de Science. EC3noticias, 30 septiembre 2014
Delgado López-Cózar, E. (2014). Google da un lavado de cara a sus Google Scholar Profiles=Google Scholar Citations. EC3noticias, 23 agosto 2014.
Delgado López-Cózar, E. (2014). Google da un lavado de cara a sus Google Scholar Profiles=Google Scholar Citations. EC3noticias, 23 agosto 2014.
Orduña-Malea, E., Delgado López-Cózar, E. (2014). Low visibility of Latin American repositories in Google Scholar: technical incompatibility or lack of web strategy? LSE Impact of Social Sciences blog, 31 July 2014.
Delgado López-Cózar, E. (2014). Lanzamiento de GOOGLE SCHOLAR DIGEST. Research on Google Scholar: empirical evidences. EC3noticias, 30 abril 2014.
Delgado López-Cózar, E. (2014. La nueva bibliometría: nuevos horizontes, nuevas oportunidades, nuevos peligros. Vino viejo en odre nuevo. EC3noticias, 11 abril 2014.
Delgado López-Cózar, E. (2014). Google Scholar: presente y futuro, fortalezas y debilidades, reportaje de John Bohannon en Science . EC3noticias, 3 enero 2014.
Digests
Orduña-Malea, E., Martín-Martín, A., Ayllón, J.M. & Delgado López-Cózar, E. (2014). Are Latin-American repositories invisible on Google and Google Scholar? Granada: EC3 Google Scholar Digest Reviews (3): 20 June 2014. | |
| Martín-Martín, A., Orduña-Malea, E.,Ayllón, J.M., & Delgado López-Cózar, E. (2014). The World Bank’s policy reports in Google Scholar. Are they visible, cited, and downloaded? Granada: EC3 Google Scholar Digest Reviews (2): 9 June 2014. | |
| Orduña-Malea, E., Ayllón, J.M., Martín-Martín, A. & Delgado López-Cózar, E. (2014). How many academic documents are visible and freely available on the Web? Granada: EC3 Google Scholar Digest Reviews (1): 27 May 2014. |
Journal Papers
Cabezas-Clavijo, Á., Delgado López-Cózar, E. (2013). Google Scholar and the h-index in biomedicine: the popularization of bibliometric assessment. Medicina Intensiva, 37(5), 343–354. DOI: http://dx.doi.org/10.1016/j.medin.2013.01.008 .
The aim of this study is to review the features, benefits and limitations of the new scientific evaluation products derived from Google Scholar, such as Google Scholar Metrics and Google Scholar Citations, as well as the h-index, which is the standard bibliometric indicator adopted by these services. The study also outlines the potential of this new database as a source for studies in Biomedicine, and compares the h-index obtained by the most relevant journals and researchers in the field of intensive care medicine, based on data extracted from the Web of Science, Scopus and Google Scholar. Results show that although the average h-index values in Google Scholar are almost 30% higher than those obtained in Web of Science , and about 15% higher than those collected by Scopus, there are no substantial changes in the rankings generated from one data source or the other. Despite some technical problems, it is concluded that Google Scholar is a valid tool for researchers in Health Sciences, both for purposes of information retrieval and for the computation of bibliometric indicators.
Cabezas-Clavijo, Á., Delgado López-Cózar, E. (2013). Google Scholar and the h-index in biomedicine: the popularization of bibliometric assessment. Medicina Intensiva, 37(5), 343–354. DOI: http://dx.doi.org/10.1016/j.medin.2013.01.008 .
The aim of this study is to review the features, benefits and limitations of the new scientific evaluation products derived from Google Scholar, such as Google Scholar Metrics and Google Scholar Citations, as well as the h-index, which is the standard bibliometric indicator adopted by these services. The study also outlines the potential of this new database as a source for studies in Biomedicine, and compares the h-index obtained by the most relevant journals and researchers in the field of intensive care medicine, based on data extracted from the Web of Science, Scopus and Google Scholar. Results show that although the average h-index values in Google Scholar are almost 30% higher than those obtained in Web of Science , and about 15% higher than those collected by Scopus, there are no substantial changes in the rankings generated from one data source or the other. Despite some technical problems, it is concluded that Google Scholar is a valid tool for researchers in Health Sciences, both for purposes of information retrieval and for the computation of bibliometric indicators.
Delgado-López-Cózar, E., Cabezas-Clavijo, Á. (2013). Ranking journals: could Google Scholar Metrics be an alternative to Journal Citation Reports and Scimago Journal Rank?. Learned Publishing, 26(2), 101-114. DOI: http://dx.doi.org/10.1087/20130206 .
The launch of Google Scholar Metrics as a tool for assessing scientific journals may be serious competition for Thomson Reuters' Journal Citation Reports, and for the Scopus-powered Scimago Journal Rank. A review of these bibliometric journal evaluation products is performed. We compare their main characteristics from different approaches: coverage, indexing policies, search and visualization, bibliometric indicators, results analysis options, economic cost, and differences in their ranking of journals. Despite its shortcomings, Google Scholar Metrics is a helpful tool for authors and editors in identifying core journals. As an increasingly useful tool for ranking scientific journals, it may also challenge established journals products.
The launch of Google Scholar Metrics as a tool for assessing scientific journals may be serious competition for Thomson Reuters' Journal Citation Reports, and for the Scopus-powered Scimago Journal Rank. A review of these bibliometric journal evaluation products is performed. We compare their main characteristics from different approaches: coverage, indexing policies, search and visualization, bibliometric indicators, results analysis options, economic cost, and differences in their ranking of journals. Despite its shortcomings, Google Scholar Metrics is a helpful tool for authors and editors in identifying core journals. As an increasingly useful tool for ranking scientific journals, it may also challenge established journals products.
Delgado López-Cózar, E., Repiso, R. (2013). The Impact of Scientific Journals of Communication: Comparing Google Scholar Metrics, Web of Science and Scopus. Comunicar, 41, 45-52. DOI: http://dx.doi.org/10.3916/C41-2013-04 .
Google Scholar Metrics' launch in April 2012, a new bibliometric tool for the evaluation of scientific journals by means of citation counting, has ended with the duopoly exerted by the Web of Science and Scopus databases. This paper aims at comparing the coverage of these three databases and the similarity their journal rankings may have. We selected a sample of journals from the field of Communication Studies indexed in the three databases. Data was recollected on 1720 November, 2012. 277 journals were identified to which we calculated their hindex and ranked them according to such indicator. Then, we analyzed the correlation between the rankings generated. Google Scholar Metrics dobles the coverage of the other databases, reducing the bias toward English language both; web of Science and Scopus have. Google Scholar Metrics shows higher hindex values (an average 47% higher than Scopus and 40% higher than Web of Science), allowing to better rank journals. We conclude that Google Scholar Metrics is a tool capable of identifying the main journals in Communication Studies offering results as reliable and valid as the ones Web of Science and Scopus show.
Google Scholar Metrics' launch in April 2012, a new bibliometric tool for the evaluation of scientific journals by means of citation counting, has ended with the duopoly exerted by the Web of Science and Scopus databases. This paper aims at comparing the coverage of these three databases and the similarity their journal rankings may have. We selected a sample of journals from the field of Communication Studies indexed in the three databases. Data was recollected on 1720 November, 2012. 277 journals were identified to which we calculated their hindex and ranked them according to such indicator. Then, we analyzed the correlation between the rankings generated. Google Scholar Metrics dobles the coverage of the other databases, reducing the bias toward English language both; web of Science and Scopus have. Google Scholar Metrics shows higher hindex values (an average 47% higher than Scopus and 40% higher than Web of Science), allowing to better rank journals. We conclude that Google Scholar Metrics is a tool capable of identifying the main journals in Communication Studies offering results as reliable and valid as the ones Web of Science and Scopus show.
Working papers
Cabezas-Clavijo, Á., Delgado López-Cózar, E. (2013). Google Scholar Metrics 2013: nothing new under the sun. EC3 Working Papers, 12 : 25 de julio de 2013. arXiv preprint arXiv:1307.6941.
Main characteristics of Google Scholar Metrics new version (july 2013) are presented. We outline the novelties and the weaknesses detected after a first analysis. As main conclusion, we remark the lack of new functionalities with respect to last editions, as the only modification is the update of the timeframe (2008-2012). Hence, problems pointed out in our last reviews still remain active. Finally, it seems Google Scholar Metrics will be updated in a yearly basis.
Main characteristics of Google Scholar Metrics new version (july 2013) are presented. We outline the novelties and the weaknesses detected after a first analysis. As main conclusion, we remark the lack of new functionalities with respect to last editions, as the only modification is the update of the timeframe (2008-2012). Hence, problems pointed out in our last reviews still remain active. Finally, it seems Google Scholar Metrics will be updated in a yearly basis.
Delgado López-Cózar, E., Ramírez Sánchez, M. (2013). Índice H de las revistas españolas de Historia según Google Scholar Metrics (2007-2011). EC3 Working Papers 10: 7 de febrero de 2013. arXiv preprint arXiv:1307.6941.
Google Scholar Metrics (GSM), which was recently launched in April 2012, features new bibliometric systems for gauging scientific journals by counting the number of citations obtained in Google Scholar. This way, it opens new possibilities for measuring journal impacts in the field of Humanities. The present article intends to evaluate the scope of this tool through analysing GSM searches, from the 5th through 6th of December 2012, of History journals published in Spain. In sum, 69 journals were identified, accounting for only 24% of the History journals published in Spain. The ranges of H index values for this field are so small that the ranking can no longer be said to show a discriminating potential. In the light of this, we would like to propose a change in the way Google Scholar Metrics is designed so that it could also accommodate production and citation patterns in the particular field of History, and, in a broader scope, in the area of Humanities as well.
Google Scholar Metrics (GSM), which was recently launched in April 2012, features new bibliometric systems for gauging scientific journals by counting the number of citations obtained in Google Scholar. This way, it opens new possibilities for measuring journal impacts in the field of Humanities. The present article intends to evaluate the scope of this tool through analysing GSM searches, from the 5th through 6th of December 2012, of History journals published in Spain. In sum, 69 journals were identified, accounting for only 24% of the History journals published in Spain. The ranges of H index values for this field are so small that the ranking can no longer be said to show a discriminating potential. In the light of this, we would like to propose a change in the way Google Scholar Metrics is designed so that it could also accommodate production and citation patterns in the particular field of History, and, in a broader scope, in the area of Humanities as well.
Reports
Ayllón Millán, J.M., Ruiz Pérez, R., Delgado López-Cózar, E. (2013). Índice H de las revistas científicas españolas según Google Scholar Metrics (2008-2012). EC3 Reports, 7. Granada, 18 de noviembre de 2013. [http://hdl.handle.net/10481/29348].
Rankings are presented by scientific fields and disciplines of Spanish scientific journals listed in Google Scholar Metrics (GSM). They are sorted according to the h-index. It pretends with this work to test the breadth of coverage that has Google Scholar Metrics of Spanish scientific journals. The two criteria used by Google Scholar Metrics to include journals in its product are: have 100 work published and own at least one cite. Bibliographic searches were conducted between 4 and 14 November 2013. 1051 journals have been identified, of which 541 are social sciences, 242 arts and Humanities, 154 of Health Sciences, 114 of the Natural Sciences and Engineering.
Rankings are presented by scientific fields and disciplines of Spanish scientific journals listed in Google Scholar Metrics (GSM). They are sorted according to the h-index. It pretends with this work to test the breadth of coverage that has Google Scholar Metrics of Spanish scientific journals. The two criteria used by Google Scholar Metrics to include journals in its product are: have 100 work published and own at least one cite. Bibliographic searches were conducted between 4 and 14 November 2013. 1051 journals have been identified, of which 541 are social sciences, 242 arts and Humanities, 154 of Health Sciences, 114 of the Natural Sciences and Engineering.
Repiso, R., Delgado López-Cózar, E. (2013). H Index Communication Journals according to Google Scholar Metrics (2008-2012). ). EC3 Reports, 6 : 26 october 2013.arXiv preprint arXiv:1310.7378.
The aim of this report is to present a ranking of Communication journals covered in Google Scholar Metrics for the period 2008-2012. It corresponds to the H Index update made last year for the period 2007-2011 (Delgado López-Cózar and Repiso 2013). Google Scholar Metrics doesnt currently allow to group and sort all journals belonging to a scientific discipline. In the case of Communication, in the ten listings displayed by GSM we can only locate 46 journals. Therefore, in an attempt to overcome this limitation, we have used the diversity of search procedures allowed by GSM to identify the greatest number of scientific journals of Communication with H Index calculated by this bibliometric tool. The result is a ranking of 354 communication journals sorted by the same H Index, and mean as discriminating value. Journals are also grouped by quartiles.
The aim of this report is to present a ranking of Communication journals covered in Google Scholar Metrics for the period 2008-2012. It corresponds to the H Index update made last year for the period 2007-2011 (Delgado López-Cózar and Repiso 2013). Google Scholar Metrics doesnt currently allow to group and sort all journals belonging to a scientific discipline. In the case of Communication, in the ten listings displayed by GSM we can only locate 46 journals. Therefore, in an attempt to overcome this limitation, we have used the diversity of search procedures allowed by GSM to identify the greatest number of scientific journals of Communication with H Index calculated by this bibliometric tool. The result is a ranking of 354 communication journals sorted by the same H Index, and mean as discriminating value. Journals are also grouped by quartiles.
Reina Leal, L.M.; Repiso, R.; Delgado López-Cózar, E. (2013). H Index of scientific Nursing journals according to Google Scholar Metrics (2007-2011) EC3 Reports, 5: 1 july 2013. arXiv preprint arXiv:1307.4329.
The aim of this report is to present a ranking of Nursing journals covered in Google Scholar Metrics (GSM), a Google product launched in 2012 to assess the impact of scientific journals from citation counts this receive on Google Scholar. Google has chosen to include only those journals that have published at least 100 papers and have at least one citation in a period of five years (2007-2011). Journal rankings are sorted by languages (showing the 100 papers with the greatest impact). This tool allows to sort by subject areas and disciplines, but only in the case of journals in English. In this case, it only shows the 20 journals with the highest h index. This option is not available for journals in the other nine languages present in Google (Chinese, Portuguese, German, Spanish, French, Korean, Japanese, Dutch and Italian).
Google Scholar Metrics doesnt currently allow to group and sort all journals belonging to a scientific discipline. In the case of Nursing, in the ten listings displayed by GSM we can only locate 34 journals. Therefore, in an attempt to overcome this limitation, we have used the diversity of search procedures allowed by GSM to identify the greatest number of scientific journals of Nursing with h index calculated by this bibliometric tool. Bibliographic searches were conducted between 10th and 30th May 2013. The aim of this report is to present a ranking of Nursing journals covered in Google Scholar Metrics (GSM), a Google product launched in 2012 to assess the impact of scientific journals from citation counts this receive on Google Scholar. Google has chosen to include only those journals that have published at least 100 papers and have at least one citation in a period of five years (2007-2011). Journal rankings are sorted by languages (showing the 100 papers with the greatest impact). This tool allows to sort by subject areas and disciplines, but only in the case of journals in English. In this case, it only shows the 20 journals with the highest h index. This option is not available for journals in the other nine languages present in Google (Chinese, Portuguese, German, Spanish, French, Korean, Japanese, Dutch and Italian).
The result is a ranking of 337 nursing journals sorted by the same h index, and mean as discriminating value. Journals are also grouped by quartiles.
Delgado López-Cózar, E., Marcos Cartagena, D., Jiménez Contreras, E., Ruiz Pérez, R. Índice H de las revistas españolas de Ciencias Sociales y Jurídicas según Google Scholar (2002-2011). EC3 Informes, 4, 29 mayo 2013. [http://hdl.handle.net/10481/26049].
Rankings are presented by disciplines of Spanish scientific journals of social sciences and law according to Google Scholar (GS) for the period 2002-2011. They are sorted according to the h-index which is the Bibliometric indicator adopted by Google. Intends this work check the breadth in coverage that has Google Scholar of Spanish scientific journals of social sciences and law and compare the differences over the period 2001-2010. Bibliographic searches were conducted in the month of March 2013. H-index is calculated from 1038 journals, of which 251 are Law, 157 education, 137 economy, 109 psychology, 87 sociology, 56 political science, 54 geography, 46 anthropology, 42 sports science, 39 urban, 36 library and information science and 24 communication.
Rankings are presented by disciplines of Spanish scientific journals of social sciences and law according to Google Scholar (GS) for the period 2002-2011. They are sorted according to the h-index which is the Bibliometric indicator adopted by Google. Intends this work check the breadth in coverage that has Google Scholar of Spanish scientific journals of social sciences and law and compare the differences over the period 2001-2010. Bibliographic searches were conducted in the month of March 2013. H-index is calculated from 1038 journals, of which 251 are Law, 157 education, 137 economy, 109 psychology, 87 sociology, 56 political science, 54 geography, 46 anthropology, 42 sports science, 39 urban, 36 library and information science and 24 communication.
Delgado López-Cózar, E., Ayllón Millán, J.M., Ruiz-Pérez, R. (2013). Índice H de las revistas científicas españolas según Google Scholar Metrics (2007-2011). 2ª edición.EC3 Informes, 3: 9 de abril de 2013. [http://hdl.handle.net/10481/24141].
Rankings are presented by scientific fields and disciplines of Spanish scientific journals listed in Google Scholar Metrics (GSM). They are sorted according to the h-index. It pretends with this work to test the breadth of coverage that has Google Scholar Metrics of Spanish scientific journals. The two criteria used by Google Scholar Metrics to include journals in its product are: have 100 work published and own at least one cite. Bibliographic searches were conducted between 15 and 30 December 2012. 943 journals have been identified, of which 407 are social sciences, 212 arts and Humanities, 216 of health sciences, 108 of the natural sciences and engineering and the health sciences.
Congress
Delgado López-Cózar, E. (2013). The Google Scholar Family: ¿Is it an alternative for the evaluation of science? In: IV Seminario EC3: Altmetrics y Unidades de Bibliometría. Universidad de Granada, Facultad de Comunicación y Documentación, 14-15 marzo 2013. [http://hdl.handle.net/10481/24142].
Tras marcar las acusadas diferencias existentes entre los sistemas tradicionales de información y medición científica (Thomson Reuters con su Web of Science y JCR; Elsevier con Scopus y SJR) y GS (Google Scholar), se desvelan las razones que explican la siguiente paradoja: ¿Cómo siendo tan distintos estos dos mundos pueden ofrecer unos resultados bibliométricos tan idénticos? Se ilustra este fenómeno con los resultados de varios estudios empíricos de ranking de revistas con muestras de diverso tamaño y naturaleza realizadas por el grupo EC3. El porqué de esta similaridad entre productos tan diametralmente divergentes no es otra que su composición: GS, cuyo motor rastrea la red científica y académica sin descaso, incluye en su seno todo el mundo tradicional (cerrado y controlado) al cual añade el mundo abierto y libre que circula en el espacio académico. Se sostiene que hoy día GS es, como fuente de información científica, una seria alternativa a las bases de datos tradicionales y se defiende que los productos bibliométricos derivados del mismo, y en concreto, los rankings de revistas (Google Scholar Metrics) ofrecen resultados tan solventes, fiables y válidos como los ofrecidos por JCR (Journal Citation Reports) o SJR (Scimago Journal Rank). En definitiva, se apuesta por Google Scholar como fuente para la medición científica. Las razones: mide más, pues abarca muchos más documentos (su tamaño duplica o triplica según áreas a sus competidores), es capaz de trabajar con grandes números (sus indicadores bibliométricos alcanzar más tamaño). Mide mejor porque cubre todo tipo de documentos, porque carece de los sesgos geográficos y lingüísticos de que adolecen los sistemas tradicionales. Crece más y lo hace de manera más rápida y sus resultados son estables en el tiempo y ofrecen un margen de error más que tolerable, producto de la falta de control técnico de la información. Es capaz de ofrecer luz donde antes solo existían tinieblas (Humanidades y Ciencias Sociales) Los dos grandes peligros o amenazas para las nuevas herramientas, no son científico ni técnicos, como pueden creer muchos científicos, sino humanos: por un lado, el principal enemigo es su propio creador (Google) que despreciando la exigencia de transparencia propia del ethos científico hace depender a sus productos de su caprichosa y entera voluntad. Por otro, la facilidad con que se pueden manipular sus datos. Desde el momento en que GS automáticamente rastrea, indiza y vacía cualquier documento de apariencia científica que cuelga de un dominio académico por voluntad de un autor sin sufrir ningún control externo previo (el de los repositorios es solo un filtro técnico que no opera sobre los contenidos) abre la posibilidad a que cualquier persona sin escrúpulos pueda manipular a su entero gusto y beneficio documentos que repercuten directamente sobre la evaluación de su producción e impacto bibliométrico. Poner en manos de todos los científicos, que son humanos, las herramientas que permiten manipular la producción y las citas pueden tener consecuencias imprevisibles o pueden convertir a estas herramientas en inservibles.
Tras marcar las acusadas diferencias existentes entre los sistemas tradicionales de información y medición científica (Thomson Reuters con su Web of Science y JCR; Elsevier con Scopus y SJR) y GS (Google Scholar), se desvelan las razones que explican la siguiente paradoja: ¿Cómo siendo tan distintos estos dos mundos pueden ofrecer unos resultados bibliométricos tan idénticos? Se ilustra este fenómeno con los resultados de varios estudios empíricos de ranking de revistas con muestras de diverso tamaño y naturaleza realizadas por el grupo EC3. El porqué de esta similaridad entre productos tan diametralmente divergentes no es otra que su composición: GS, cuyo motor rastrea la red científica y académica sin descaso, incluye en su seno todo el mundo tradicional (cerrado y controlado) al cual añade el mundo abierto y libre que circula en el espacio académico. Se sostiene que hoy día GS es, como fuente de información científica, una seria alternativa a las bases de datos tradicionales y se defiende que los productos bibliométricos derivados del mismo, y en concreto, los rankings de revistas (Google Scholar Metrics) ofrecen resultados tan solventes, fiables y válidos como los ofrecidos por JCR (Journal Citation Reports) o SJR (Scimago Journal Rank). En definitiva, se apuesta por Google Scholar como fuente para la medición científica. Las razones: mide más, pues abarca muchos más documentos (su tamaño duplica o triplica según áreas a sus competidores), es capaz de trabajar con grandes números (sus indicadores bibliométricos alcanzar más tamaño). Mide mejor porque cubre todo tipo de documentos, porque carece de los sesgos geográficos y lingüísticos de que adolecen los sistemas tradicionales. Crece más y lo hace de manera más rápida y sus resultados son estables en el tiempo y ofrecen un margen de error más que tolerable, producto de la falta de control técnico de la información. Es capaz de ofrecer luz donde antes solo existían tinieblas (Humanidades y Ciencias Sociales) Los dos grandes peligros o amenazas para las nuevas herramientas, no son científico ni técnicos, como pueden creer muchos científicos, sino humanos: por un lado, el principal enemigo es su propio creador (Google) que despreciando la exigencia de transparencia propia del ethos científico hace depender a sus productos de su caprichosa y entera voluntad. Por otro, la facilidad con que se pueden manipular sus datos. Desde el momento en que GS automáticamente rastrea, indiza y vacía cualquier documento de apariencia científica que cuelga de un dominio académico por voluntad de un autor sin sufrir ningún control externo previo (el de los repositorios es solo un filtro técnico que no opera sobre los contenidos) abre la posibilidad a que cualquier persona sin escrúpulos pueda manipular a su entero gusto y beneficio documentos que repercuten directamente sobre la evaluación de su producción e impacto bibliométrico. Poner en manos de todos los científicos, que son humanos, las herramientas que permiten manipular la producción y las citas pueden tener consecuencias imprevisibles o pueden convertir a estas herramientas en inservibles.
2012 [Go back]
Journal Papers
Cabezas-Clavijo, Á., Torres-Salinas, D. (2012). Google scholar citations y la emergencia de nuevos actores en la evaluación de la investigación. Anuario ThinkEPI, 6, 147-153. [http://hdl.handle.net/10481/20229].
Google scholar citations, a system aimed at researchers, attempts to outline researchers’ bibliometric profile, providing citation indicators and the h-index. We review other tools designed to measure visibility and academic impact on the web, such as Microsoft academic search and the new initiatives grouped under the label altmetrics or alternative indicators. Finally, we discuss how the appearance of Google scholar citations and other products may affect the two major sources of bibliometric data, ISI Web of science and Scopus, and how this can influence the evaluation of scientific research.
Delgado-López-Cózar, E., Cabezas-Clavijo, Á. (2012). Google Scholar Metrics: an unreliable tool for assessing scientific journals. El profesional de la información, 2012, 21 (4), 419-427. [http://hdl.handle.net/10481/21540] .
We introduce Google Scholar Metrics (GSM), a new bibliometric product of Google that aims at providing the H-index for scientific journals and other information sources. We conduct a critical review of GSM showing its main characteristics and possibilities as a tool for scientific evaluation. We discuss its coverage along with the inclusion of repositories, bibliographic control, and its options for browsing and searching. We conclude that, despite Google Scholar’s value as a source for scien- tific assessment, GSM is an immature product with many shortcomings, and therefore we advise against its use for evaluation purposes. However, the improvement of these shortcomings would place GSM as a serious competitor to the other existing products for evaluating scientific journals.
Delgado López-Cózar, E., Robinson-García, N. (2012). Repositories in Google Scholar Metrics or what is this document type doing in a place as such?. Cybermetrics, 16(1), paper 4. [http://hdl.handle.net/10481/22019] . The present paper analyzes GS Metrics, Google's newest product aiming at ranking journals according to their H-Index. Specifically, we analyze GS Metrics' decision of considering journals and repositories as equal and therefore, including them in the product. In this sense, the authors position themselves against this decision and provide several arguments of different nature warning against the shortcomings this product has. The first one is of a conceptual nature and is related to the definition of journal and repository. Secondly, they refer at the methodological issues mixing repositories and journals can bring out. Then, they deepen on many other flaws GS Metrics presents. Finally, GS Metrics and its possible use as an evaluation tool are discussed and possible solutions to its shortcomings are provided.
Working Papers
Cabezas-Clavijo, Á., Delgado López-Cózar, E. (2012). Las revistas españolas de Ciencias Sociales y Jurídicas en Google Scholar Metrics, ¿están todas las que son? EC3 Working Papers, 2: 17 April 2012. [http://hdl.handle.net/10760/16892].
New paper of EC3 about Google Scholar Metrics. The coverage of Spanish relevant journals in Social Sciences and Law is assessed. We conclude that Google Scholar Metrics shouldn't be used to evaluate national journals in these areas because of its limitations.
Cabezas-Clavijo, Á., Delgado López-Cózar, E. (2012). Scholar Metrics: the impact of journals according to Google, just an amusement or a valid scientific tool? EC3 Working Papers, 1: 11 April 2012. [http://hdl.handle.net/10760/16836].
In this note we review the most significant features of Google Scholar Metrics, pointing out their strengths and weaknesses (always bearing in mind that Google products are progressively incorporating new functionalities), and discussing the possibilities of the adoption of these tools by bibliometricians and policy makers.
Cabezas-Clavijo, Á., Delgado López-Cózar, E. (2012). ¿Es posible usar Google Scholar para evaluar las revistas científicas nacionales en los ámbitos de Ciencias Sociales y Jurídicas? El caso de las revistas españolas. EC3 Working Papers, 3: 23 April 2012. [http://hdl.handle.net/10760/16888].
In this paper, we test the validity of Google Scholar as a source for evaluation of journals within national environments in the fields of Social Sciences and Law. We rank journals covered by spanish databases IN-RECS and IN-RECJ according to its H index and G index, concluding the usefulness of this source for the assesment of Social Sciences journals, but not for Law journals.
In this paper, we test the validity of Google Scholar as a source for evaluation of journals within national environments in the fields of Social Sciences and Law. We rank journals covered by spanish databases IN-RECS and IN-RECJ according to its H index and G index, concluding the usefulness of this source for the assesment of Social Sciences journals, but not for Law journals.
Cabezas-Clavijo, Á., Delgado López-Cózar, E. (2012). Las revistas españolas de Ciencias Sociales y Jurídicas en Google Scholar Metrics, ¿están todas las que son? EC3 Working Papers, 2: 17 April 2012. [http://hdl.handle.net/10760/16892].
New paper of EC3 about Google Scholar Metrics. The coverage of Spanish relevant journals in Social Sciences and Law is assessed. We conclude that Google Scholar Metrics shouldn't be used to evaluate national journals in these areas because of its limitations.
Cabezas-Clavijo, Á., Delgado López-Cózar, E. (2012). Scholar Metrics: the impact of journals according to Google, just an amusement or a valid scientific tool? EC3 Working Papers, 1: 11 April 2012. [http://hdl.handle.net/10760/16836].
In this note we review the most significant features of Google Scholar Metrics, pointing out their strengths and weaknesses (always bearing in mind that Google products are progressively incorporating new functionalities), and discussing the possibilities of the adoption of these tools by bibliometricians and policy makers.
Delgado López-Cózar, E. (2012). From April to November: the growth of Google Scholar Metrics EC3 Working Papers, 9: 22 November 2012. [http://hdl.handle.net/10481/22466].
Last November, the Google Scholar Metrics journal rankings were updated, offering for the first time the possibility to compare the differences between the editions from April and November 2012 regarding the bibliometric indicators in the 10 rankings offered, as well as their stability. The H index and h-5 median of 1000 of journals are analyzed, comparing their averages, maximum and minimum values and the correlation coefficient within rankings. The bibliometric figures have experienced an important growth. In just 7 ½ months the index h of journals has grown by 15% and the median h-index by 17%. This growth takes place for all the bibliometric indicators analyzed (maximum and minimum values of the indices h and h-5 median) and practically in every journal, regardless of the language in which they are published. However, we found significant differences in growth rates depending on the language in which the journal is edited and on the values of the h index of journals published by language. Moreover, the journal rankings seem to be stable between April and November, reinforcing the credibility of the data held by Google Scholar and the solvency of the Google Scholar Metrics journal rankings. From the present analysis we suggest the following actions. Firstly, we suggest that Google should upgrade its rankings at least semi-annually or even at real-time, becoming a fully dynamic information system. Secondly, we believe Google Scholar Metrics follows a wrong policy by displaying only 100 journals for each ranking, and what is worse, using the same threshold despite of the language. We suggest that the results should be displayed in each ranking proportionally to the number of journals indexed by language.
Delgado López-Cózar, E., Cabezas-Clavijo, Á. (2012). Google Scholar Metrics updated: Now it begins to get serious. EC3 Working Papers, 8: 16 November 2012. [http://hdl.handle.net/10481/22439].
In this paper we perform a first analysis of the new version of Google Scholar Metrics (November 2012). Significant improvements have been detected, such as the possibility of consulting the most prominent publications by areas and disciplines, although only for those in English language. The citation statistics have been updated to 15 November 2012. Also the original set of journals has been modified along with the journal impact indicators, which has resulted on the entry of new journals. Finally, another major issue is the change of policy regarding the processing of documents indexed in repositories. Despite the improvements made, Google Scholar Metrics remains an inadequate product for scientific journal evaluation due to its methodological shortcomings.
In this paper we perform a first analysis of the new version of Google Scholar Metrics (November 2012). Significant improvements have been detected, such as the possibility of consulting the most prominent publications by areas and disciplines, although only for those in English language. The citation statistics have been updated to 15 November 2012. Also the original set of journals has been modified along with the journal impact indicators, which has resulted on the entry of new journals. Finally, another major issue is the change of policy regarding the processing of documents indexed in repositories. Despite the improvements made, Google Scholar Metrics remains an inadequate product for scientific journal evaluation due to its methodological shortcomings.
Delgado López-Cózar, E., Orduña Malea, E., Marcos Cartagena, D., Jiménez Contreras, E., Ruiz Pérez, R. (2012). JOURNAL SCHOLAR: Una alternativa internacional, gratuita y de libre acceso para medir el impacto de las revistas de Arte, Humanidades y Ciencias Sociales. EC3 Working Papers, 5: 12 May 2012. [http://hdl.handle.net/10481/20375].
The EC3 Research Group of the University of Granada presents a prototype of the JOURNAL SCHOLAR, a bibliometric index which intends to measure the performance of scientific journals in the fields of Arts, Humanities and Social Sciences by counting publications and citations through Google Scholar. The indicators used are the H-index and the G-index. Journals are ranked in quartiles and the citation windows ranges from 5 to 10 years for international or regional rankings. Data was retrieved on 10-15 July, 2011. We computed the H-index and the G-index of 462 journals. We conclude comparing the coverage and results offered by SCOPUS and WoS. JOURNAL SCHOLAR quadraples the coverage and reduces English-language biases. As it identifies more citations, the H-indexes are higher (an average of 44% higher than in SCOPUS and 77% than Web of Science) allowing to discriminate better the journals' positioning in the ranking. In order to show this tool's functionalities we present the results for Information and Library Science, offering a ranking of de 462 journals edited in 59 countries. We conclude that it is feasible and viable to generate bibliometric tools using Google Scholar. Despite its lack of control (selecting sources and normalizing data) we obtain reasonable results as valid and reliable as those from SCOPUS and WoS. And with a lesser cost than with the traditional bibliometric tools - considering the great cost of these tools elaborated by Thomson Reuters and Elsevier.
Delgado López-Cózar, E., Robinson-García, N.,Torres-Salinas, D. (2012). Manipulating Google Scholar Citations and Google Scholar Metrics: simple, easy and tempting. EC3 Working Papers, 6: 29 May 2012. [http://hdl.handle.net/10481/20469].
The launch of Google Scholar Citations and Google Scholar Metrics may provoke a revolution in the research evaluation field as it places within every researcher’s reach tools that allow bibliometric measuring. In order to alert the research community over how easily one can manipulate the data and bibliometric indicators offered by Google’s products we present an experiment in which we manipulate the Google Citations’ profiles of a research group through the creation of false documents that cite their documents, and consequently, the journals in which they have published modifying their H-index. For this purpose we created six documents authored by a faked author and we uploaded them to a researcher’s personal website under the University of Granada’s domain. The result of the experiment meant an increase of 774 citations in 129 papers (six citations per paper) increasing the authors and journals' H-index. We analyse the malicious effect this type of practices can cause to Google Scholar Citations and Google Scholar Metrics. Finally, we conclude with several deliberations over the effects these malpractices may have and the lack of control tools these tools offer.
The launch of Google Scholar Citations and Google Scholar Metrics may provoke a revolution in the research evaluation field as it places within every researcher’s reach tools that allow bibliometric measuring. In order to alert the research community over how easily one can manipulate the data and bibliometric indicators offered by Google’s products we present an experiment in which we manipulate the Google Citations’ profiles of a research group through the creation of false documents that cite their documents, and consequently, the journals in which they have published modifying their H-index. For this purpose we created six documents authored by a faked author and we uploaded them to a researcher’s personal website under the University of Granada’s domain. The result of the experiment meant an increase of 774 citations in 129 papers (six citations per paper) increasing the authors and journals' H-index. We analyse the malicious effect this type of practices can cause to Google Scholar Citations and Google Scholar Metrics. Finally, we conclude with several deliberations over the effects these malpractices may have and the lack of control tools these tools offer.
Reports
Delgado López-Cózar, E., Repiso, R. (2012). Índice H de las revistas de Comunicación según Google Scholar Metrics (2007-2011). [http://hdl.handle.net/10481/22483].
We present the h index H Index Communication journals according to Google Scholar Metrics . The journals are ranked according to their index H and G. The searches were conducted in July 2011 and measured the impact of the journal from articles published between 2007 and 2011.
We present the h index H Index Communication journals according to Google Scholar Metrics . The journals are ranked according to their index H and G. The searches were conducted in July 2011 and measured the impact of the journal from articles published between 2007 and 2011.
Delgado López-Cózar, E., Marcos Cartagena, D., Cabezas Clavijo, A., Jiménez Contreras, E., Ruiz Pérez, R. (2012). Índice H de las revistas españolas de Ciencias Jurídicas según Google Scholar (2001-2010). [http://hdl.handle.net/10481/20234].
We present the impact factors of Spanish journals listed in IN-RECS (Impact Index of Spanish Law Journals) calculated from Google Scholar. The journals are ranked according to their index H and G. The searches were conducted in July 2011 and measured the impact of the journal from articles published between 2001 and 2010.
We present the impact factors of Spanish journals listed in IN-RECS (Impact Index of Spanish Law Journals) calculated from Google Scholar. The journals are ranked according to their index H and G. The searches were conducted in July 2011 and measured the impact of the journal from articles published between 2001 and 2010.
Delgado López-Cózar, E., Marcos Cartagena, D., Cabezas Clavijo, A., Jiménez Contreras, E., Ruiz Pérez, R. (2012). Índice H de las revistas españolas de Ciencias Sociales según Google Scholar (2001-2010). [http://hdl.handle.net/10481/20235].
We present the impact factors of Spanish journals listed in Social Sciences IN-RECS (Impact Index of Spanish Journals of Social Sciences) calculated from Google Scholar. The journals are ranked according to their index H and G. The searches were conducted in July 2011 and measured the impact of the journal from articles published between 2001 and 2010.
We present the impact factors of Spanish journals listed in Social Sciences IN-RECS (Impact Index of Spanish Journals of Social Sciences) calculated from Google Scholar. The journals are ranked according to their index H and G. The searches were conducted in July 2011 and measured the impact of the journal from articles published between 2001 and 2010.
2010 [Go back]
Journal Papers
Torres Salinas, D., Ruiz Pérez, R., Delgado López-Cozár, E. (2010). Google Scholar: ¿una herramienta para la evaluación de la ciencia? Anuario ThinkEPI, (1), 254-257.
Google Scholar (GS) is a search engine that specializes in scientific information and in the identification of the citations that academic papers receive, making it a strong competitor for other citations indexes like Scopus and Web of Science. For this reason, several studies have attempted to evaluate its capacityas a bibliometric tool. Due to this interest, we present an introduction to its use. We discuss the following disadvantages: not all documents indexed in GS can be considered academic; it is not a transparent product because it offers no information on the sources covered; with respect to scientific journals, there is uneven coverage of the disciplines; its interface is too basic; there is an absolute lack of standardization in data; GS does not solve the citation problem; and, like Web of Science and Scopus, there are huge gaps in its coverage.
Torres Salinas, D., Ruiz Pérez, R., Delgado López-Cozár, E. (2010). Google Scholar: ¿una herramienta para la evaluación de la ciencia? Anuario ThinkEPI, (1), 254-257.
Google Scholar (GS) is a search engine that specializes in scientific information and in the identification of the citations that academic papers receive, making it a strong competitor for other citations indexes like Scopus and Web of Science. For this reason, several studies have attempted to evaluate its capacityas a bibliometric tool. Due to this interest, we present an introduction to its use. We discuss the following disadvantages: not all documents indexed in GS can be considered academic; it is not a transparent product because it offers no information on the sources covered; with respect to scientific journals, there is uneven coverage of the disciplines; its interface is too basic; there is an absolute lack of standardization in data; GS does not solve the citation problem; and, like Web of Science and Scopus, there are huge gaps in its coverage.
2009 [Go back]
Journal Papers
Orduña-Malea, E., Serrano-Cobos, J., Lloret-Romero, N. (2009). Spanish public universities in Google Scholar: presence, evolution and coverage of their scientific output. El Profesional de la Informacion, 18(5), 493–500. DOI: http://dx.doi.org/10.3145/epi.2009.sep.02 .
The validity of Google Scholar as a service that reflects properly the scientific output of a university is analyzed, comparing its coverage of Spanish public universities with that of Scopus. The presence and evolution of scientific documentation on the websites of the Spanish public universities from January to July (both months inclusive) 2009 is also studied. The results show that, despite finding some interrelationship between Scholar and Scopus concerning the productivity of institutions, there are large differences in the total results that override the latter as a valid reflection of university productivity. Finally, Scholar shows positive, albeit modest, growth during the studied period for most university websites analyzed.
The validity of Google Scholar as a service that reflects properly the scientific output of a university is analyzed, comparing its coverage of Spanish public universities with that of Scopus. The presence and evolution of scientific documentation on the websites of the Spanish public universities from January to July (both months inclusive) 2009 is also studied. The results show that, despite finding some interrelationship between Scholar and Scopus concerning the productivity of institutions, there are large differences in the total results that override the latter as a valid reflection of university productivity. Finally, Scholar shows positive, albeit modest, growth during the studied period for most university websites analyzed.
Torres-Salinas, D., Ruiz-Pérez, R., Delgado López-Cózar, E. (2009). Google Scholar como herramienta para la evaluación científica. El profesional de la información, 18(5), 501-510. DOI: http://dx.doi.org/10.3145/epi.2009.sep.03 .
Google Scholar is a search engine that specializes in scientific information and in the identification of the citations that academic papers receive, making it a strong competitor for other citations indexes. For this reason, several studies have attempted to evaluate its capacity as a bibliometric tool. Due to this interest, we present an introduction to its use and the advantages and disadvantages versus Scopus and Web of Science. First, its way of collecting information and features of its interface are analyzed. The following section describes the results that Google Scholar generates. Thirdly, we analyze the coverage of information sources and the different document types to be found, showing how this coverage universe offers different citations versus other products. Finally, we specify the standardization problems of Google Scholar and offer a number of precautions that must be taken into account when using Google Scholar as an evaluation tool.
Google Scholar is a search engine that specializes in scientific information and in the identification of the citations that academic papers receive, making it a strong competitor for other citations indexes. For this reason, several studies have attempted to evaluate its capacity as a bibliometric tool. Due to this interest, we present an introduction to its use and the advantages and disadvantages versus Scopus and Web of Science. First, its way of collecting information and features of its interface are analyzed. The following section describes the results that Google Scholar generates. Thirdly, we analyze the coverage of information sources and the different document types to be found, showing how this coverage universe offers different citations versus other products. Finally, we specify the standardization problems of Google Scholar and offer a number of precautions that must be taken into account when using Google Scholar as an evaluation tool.
Congress
Delgado López-Cózar, E. (2009). Google Scholar: ¿Herramienta de evaluación científica? In: II Seminario sobre evaluación y comunicación de la ciencia. Universidad de Granada. Granada, 2 de abril, 2009.[ http://hdl.handle.net/10760/14088].
Delgado López-Cózar, E. (2009). Qué es y cómo utilizar Google Scholar. VI Foro sobre evaluación de la calidad de la educación superior y de la investigación. Fundación Española de Ciencia y Tecnología (FECYT) / Universidad de Vigo. Vigo, 8-11 septiembre , 2009.
2008 [Go back]
Congress
Delgado López-Cózar, E. (2008). Qué es y cómo utilizar Google Scholar. V Foro sobre evaluación de la calidad de la educación superior y de la investigación. Fundación Española de Ciencia y Tecnología (FECYT) / Universidad del País Vasco. San Sebastián, 2-5 septiembre, 2008. ISBN 978-84-691-3972-1.
No hay comentarios:
Publicar un comentario