A blog by Rob J Hyndman 

Twitter Gplus RSS

Google scholar metrics

Published on 2 April 2012

Google has pro­duced another great tool for researchers, this time pro­vid­ing some met­rics on jour­nal cita­tions. Google Scholar Met­rics allows you to search on jour­nals rather than arti­cles, and to see the aver­age or median h-​​index for each journal.

For exam­ple, a search on “fore­cast­ing” yields the fol­low­ing results:

The h-​​index is the largest num­ber h such that at least h arti­cles in that pub­li­ca­tion were cited at least h times each. The h5-​​index is the h-​​index cal­cu­lated using only arti­cles pub­lished in the last five com­plete cal­en­dar years (2007−2011 in this exam­ple). So there were 29 arti­cles pub­lished in the IJF between 2007 and 2011 that have each been cited at least 29 times.

The h5-​​median is the median num­ber of cita­tions of the arti­cles mak­ing up the h5-​​index. That is, for the IJF the 29 arti­cles have received a median 43 cita­tions. By click­ing on the num­ber in the h5-​​index col­umn, you can see which arti­cles have been included.

Cur­rently, any dis­cus­sion of jour­nal met­rics is dom­i­nated by the ISI 2-​​year impact fac­tor, equal to the aver­age num­ber of cita­tions received per paper pub­lished in that jour­nal dur­ing the two pre­ced­ing years. In my view the h5-​​index is a far bet­ter mea­sure than the 2-​​year IF. Here are some rea­sons why.

  1. I like the fact that a 5-​​year index has been used, rather than the 2-​​year impact fac­tor favoured by the ISI. Two years is much too short, and leads to a lot of year-​​to-​​year vari­a­tion, at least in the fields I’m inter­ested in. Five years pro­vides a smoother mea­sure that will not change so much from year to year, yet will still rep­re­sent recent qual­ity rather than what the jour­nal might have been like many years ago.
  2. The h5-​​index can­not be dom­i­nated by one star paper. If there is only one great paper in the jour­nal, and every­thing else is not cited at all, the h5-​​index will be 1. On the other hand, the ISI 2-​​year impact fac­tor will be greatly increased by that sin­gle paper due to the non-​​robustness of the mean.
  3. It is harder to game the h5-​​index than the ISI impact fac­tor. One of the games that some jour­nals play is to force authors want­ing their arti­cle pub­lished in the jour­nal to cite other arti­cles pub­lished in the jour­nal, thus arti­fi­cially increas­ing the num­ber of cita­tions. This has a direct and imme­di­ate impact on the ISI impact fac­tor as it is based on aver­age cita­tions per arti­cle. But it will be harder to use this game on the jour­nal h5​-index​.To see this, imag­ine that the Jour­nal of Fore­cast­ing wanted to improve their h5-​​index to 30 and so beat the IJF. They cur­rently only have 5 papers with 30 or more cita­tions, so they would need to get another 25 papers up to that level, 14 of which cur­rently have fewer than 16 cita­tions. So that’s more than 14 extra JF cita­tions for each of those 14 papers — even with bogus cita­tions that’s not going to hap­pen. (I am not sug­gest­ing that JF ever plays such games, only point­ing out that it will be much harder for jour­nals to game the h5-​​index than the ISI 2-​​year impact factor.)

In sum­mary, the h5-​​index is sim­ple to under­stand, hard to manip­u­late, and pro­vides a rea­son­able if crude mea­sure of the respect accorded to a jour­nal by schol­ars within its field. While jour­nal met­rics are no guar­an­tee of the qual­ity of a jour­nal, if they are going to be used we should use the best avail­able, and Google’s h5-​​index is a big improve­ment on the ISI impact factor.


Related Posts:

4 Comments  comments 
  • Matthias Flor

    Thanks for the info. I com­pletely agree with your three argu­ments on the supe­ri­or­ity of Google’s h5 index over ISI’s impact fac­tor.
    Search­ing for my field (evo­lu­tion­ary biol­ogy), I noticed some strange behav­ior: If I enter evo­lu­tion in the search field, only jour­nals that have that exact word in the title are included, jour­nals such at BMC Evo­lu­tion­ary Biol­ogy do not appear. For this to hap­pen, I have to search for evo­lu­tion OR evo­lu­tion­ary. Strangely though, for your field of fore­cast­ing, search­ing for for­cast­ing spits out 5 jour­nal with for­cast­ing or for­cast in the title whereas a search for for­cast only returns the sin­gle jour­nal of Dia­betes fore­cast. Search­ing for for­cast­ing OR for­cast only returns the 4 jour­nals with for­cast­ing in their title but not Dia­betes fore­cast. So, it appears one needs to be care­ful with one’s search terms when try­ing to com­pare jour­nals belong­ing to a spe­cific field.

  • Pingback: Google Scholar Metrics: Bibliometria para publicações | Bibliotecários Sem Fronteiras - Biblioteconomia Pop

  • Pingback: Malaysian Citation Centre (MCC) | Research News

  • Tom­Tom

    Still, con­sider a very selec­tive jour­nal that pub­lishes only 10 arti­cles per year. Sup­pose more­over that each of these arti­cles will be cited 1000 times within the next five years. At best this jour­nal will get an h5 index of 50 (5*10)–much worse than PLoS One, and yet a much bet­ter jour­nal!
    So H5 SHOULD be nor­mal­ized by the yearly num­ber of papers, no?