Can I use Google Scholar for systematic review?

  1. Sampson M, McGowan J: Errors in search strategies were identified by type and frequency. J Clin Epidemiol. 2006, 59: 1057.e1-1057.e9.

    Article  Google Scholar 

  2. Maggio LAM, Tannery NH, Kanter SL: Reproducibility of literature search reporting in medical education reviews. Acad Med Aug 2011. 2011, 86: 1049-1054.

    Google Scholar 

  3. Boeker M, Vach W, Motschall E: Semantically equivalent PubMed and Ovid-MEDLINE queries: different retrieval results because of database subset inclusion. J Clin Epidemiol. 2012, 65: 915-916. 10.1016/j.jclinepi.2012.01.015.

    Article  PubMed  Google Scholar 

  4. Vanopstal K, Buysschaert J, Laureys G, Vander Stichele R: Lost in PubMed. Factors influencing the success of medical information retrieval. Expert Syst Appl. 2013, 40: 4106-4114. 10.1016/j.eswa.2013.01.036.

    Article  Google Scholar 

  5. Haig A, Dozier M: BEME guide No. 3: systematic searching for evidence in medical education–part 2: constructing searches. Med Teach. 2003, 25: 463-10.1080/01421590310001608667.

    Article  PubMed  Google Scholar 

  6. Lefebrve C, Manheimer E, Glanville J: Searching for studies. Cochrane handbook for systematic reviews of interventions Version 5.1.0. Edited by: Higgins JP, Green S. 2008, 1-21. The Cochrane Collaboration 2011. http://www.cochrane-handbook.org

    Google Scholar 

  7. Nourbakhsh E, Nugent R, Wang H, Cevik C, Nugent K: Medical literature searches: a comparison of PubMed and Google scholar. Health Info Libr J. 2012, 29: 214-222. 10.1111/j.1471-1842.2012.00992.x.

    Article  PubMed  Google Scholar 

  8. Heinzl S: Medizinreport: studien im fokus. Medizinisch-wissenschaftliche recherche: Google scholar ist PubMed möglicherweise überlegen. Dtsch Arztebl. 2012, 109: A–2467-

    Google Scholar 

  9. Gehanno J-F, Rollin L, Darmoni S: Is the coverage of Google scholar enough to be used alone for systematic reviews. BMC Med Inform Decis Mak. 2013, 13: 7-10.1186/1472-6947-13-7.

    Article  PubMed  PubMed Central  Google Scholar 

  10. Ovaska T, Fiander M, Weightman A: BMC medical informatics and decision making – user comments – is the coverage of Google scholar enough to be used alone for systematic reviews. 2013, http://www.biomedcentral.com/1472-6947/13/7/comments,

    Google Scholar 

  11. Giustini D: Is Google scholar enough for SR searching? No. – The search principle: views are my own. 2013, http://blogs.ubc.ca/dean/2013/01/is-google-scholar-enough-for-sr-searching-no/,

    Google Scholar 

  12. Giustini D: More on using Google scholar for the systematic review – the search principle: views are my own. 2013, http://blogs.ubc.ca/dean/2013/01/more-on-using-google-scholar-for-the-systematic-review/,

    Google Scholar 

  13. Giustini D, Boulos MNK: Google scholar is not enough to be used alone for systematic reviews. Online J Public Health Informatics. 2013, 5: 214-

    Article  Google Scholar 

  14. Booth A: “Brimful of STARLITE”: toward standards for reporting literature searches: toward standards for reporting literature searches. J Med Libr Assoc. 2006, 94: 421-e205.

    PubMed  PubMed Central  Google Scholar 

  15. Sampson M, McGowan J, Cogo E, Grimshaw J, Moher D, Lefebvre C: An evidence-based practice guideline for the peer review of electronic search strategies. J Clin Epidemiol. 2009, 62: 944-952. 10.1016/j.jclinepi.2008.10.012.

    Article  PubMed  Google Scholar 

  16. Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JPA, Clarke M, Devereaux PJ, Kleijnen J, Moher D: The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med. 2009, 6: e1000100-10.1371/journal.pmed.1000100.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Simera I, Moher D, Hoey J, Schulz KF, Altman DG: A catalogue of reporting guidelines for health research. Eur J Clin Invest. 2010, 40: 35-53. 10.1111/j.1365-2362.2009.02234.x.

    CAS  Article  PubMed  Google Scholar 

  18. Cochrane Editorial Unit: Methodological expectations of Cochrane intervention reviews (MECIR). 2013, http://www.editorial-unit.cochrane.org/mecir,

    Google Scholar 

  19. Sampson M, Zhang L, Morrison A, Barrowman NJ, Clifford TJ, Platt RW, Klassen TP, Moher D: An alternative to the hand searching gold standard: validating methodological search filters using relative recall. BMC Med Res Methodol. 2006, 6: 33-10.1186/1471-2288-6-33.

    Article  PubMed  PubMed Central  Google Scholar 

  20. Waffenschmidt S, Janzen T, Hausner E, Kaiser T: Simple search techniques in PubMed are potentially suitable for evaluating the completeness of systematic reviews. J Clin Epidemiol. 2013, 66: 660-665. 10.1016/j.jclinepi.2012.11.011.

    Article  PubMed  Google Scholar 

  21. Arroll B, Elley CR, Fishman T, Goodyear-Smith FA, Kenealy T, Blashki G, Kerse N, MacGillivray S: Antidepressants versus placebo for depression in primary care. Cochrane Database Syst Rev. 2009, 3

  22. Sinclair D, Zani B, Donegan S, Olliaro P, Garner P: Artemisinin-based combination therapy for treating uncomplicated malaria. Cochrane Database Syst Rev. 2009, 3

  23. McQueen J, Howe TE, Allan L, Mains D, Hardy V: Brief interventions for heavy alcohol users admitted to general hospital wards. Cochrane Database of Syst Rev. 2011, 8

  24. Bar-On ES, Goldberg E, Hellmann S, Leibovici L: Combined DTP-HBV-HIB vaccine versus separately administered DTP-HBV and HIB vaccines for primary prevention of diphtheria, tetanus, pertussis, hepatitis B and haemophilus influenzae B (HIB). Cochrane Database of Syst Rev. 2012, 4

  25. Bohlius J, Schmidlin K, Brillant C, Schwarzer G, Trelle S, Seidenfeld J, Zwahlen M, Clarke MJ, Weingart O, Kluge S, Piper M, Napoli M, Rades D, Steensma D, Djulbegovic B, Fey MF, Ray-Coquard I, Moebus V, Thomas G, Untch M, Schumacher M, Egger M, Engert A: Erythropoietin or darbepoetin for patients with cancer - meta-analysis based on individual patient data. Cochrane Database of Syst Rev. 2009, 3

  26. Boehm K, Borrelli F, Ernst E, Habacher G, Hung SK, Milazzo S, Horneber M: Green tea (camellia sinensis) for the prevention of cancer. Cochrane Database of Syst Rev. 2009, 3

  27. Guimarães MM, El Dib RP, Smith AF, Matos D: Incentive spirometry for prevention of postoperative pulmonary complications in upper abdominal surgery. Cochrane Database of Syst Rev. 2009, 3

  28. Verbeek JH, Kateman E, Morata TC, Dreschler WA, Mischke C: Interventions to prevent occupational noise-induced hearing loss. Cochrane Database of Syst Rev. 2012, 10

  29. Yip P, Middleton P, Cyna AM, Carlyle AV: Non-pharmacological interventions for assisting the induction of anaesthesia in children. Cochrane Database of Syst Rev. 2009, 3

  30. Okebe JU, Yahav D, Shbita R, Paul M: Oral iron supplements for children in malaria-endemic areas. Cochrane Database of Syst Rev. 2011, 10

  31. Ipser JC, Stein DJ, Hawkridge S, Hoppe L: Pharmacotherapy for anxiety disorders in children and adolescents. Cochrane Database of Syst Rev. 2009, 3

  32. Sultan A, McQuay HJ, Moore RA, Derry S: Single dose oral Flurbiprofen for acute postoperative pain in adults. Cochrane Database of Syst Rev. 2009, 3

  33. Oduyebo OO, Anorlu RI, Ogunsola FT: The effects of antimicrobial therapy on bacterial vaginosis in non-pregnant women. Cochrane Database of Syst Rev. 2009, 3

  34. Mestre T, Ferreira J, Coelho MM, Rosa M, Sampaio C: Therapeutic interventions for symptomatic treatment in Huntington’s disease. Cochrane Database of Syst Rev. 2009, 3

  35. Walters WH: Google scholar search performance: comparative recall and precision. portal: Libr and the Acad. 2008, 9: 5-24. 10.1353/pla.0.0034.

    Article  Google Scholar 

  36. García-Pérez MA: Accuracy and completeness of publication and citation records in the Web of science, PsycINFO, and Google scholar: a case study for the computation of h indices in psychology. J Am Soc Inf Sci Technol. 2010, 61: 2070-2085. 10.1002/asi.21372.

    Article  Google Scholar 

  37. Falagas ME, Pitsouni EI, Malietzis GA, Pappas G: Comparison of PubMed, Scopus, Web of science, and Google scholar: strengths and weaknesses. FASEB J. 2008, 22: 338-342.

    CAS  Article  PubMed  Google Scholar 

  38. Kulkarni AV, Aziz B, Shams I, Busse JW: Comparisons of citations in Web of science, Scopus, and Google scholar for articles published in general medical journals. JAMA. 2009, 302: 1092-1096. 10.1001/jama.2009.1307.

    CAS  Article  PubMed  Google Scholar 

  39. Mayr P, Walter A-K: An exploratory study of Google scholar. Online Inf Rev. 2007, 31: 814-830. 10.1108/14684520710841784.

    Article  Google Scholar 

  40. Google Inc: About Google scholar. 2013, http://www.google.com/intl/en/scholar/about.html,

    Google Scholar 

  41. Higgins JP, Deeks JJ: Selecting studies and collecting data. Cochrane handbook for systematic reviews of interventions. Edited by: Higgins JP, Green S. 2011, : John Wiley & Sons, Ltd, 1-21. 510

    Google Scholar 

  42. Jasco P: As we may search – comparison of major features of the Web of science, Scopus, and Google scholar citation-based and citation-enhanced databases. Curr Sci. 2005, 89: 1537-1547.

    Google Scholar 

  43. The Cochrane Collaboration: The Cochrane Library. Independent high-quality evidence for health care decision making. 2013, http://www.thecochranelibrary.com/view/0/index.html,

    Google Scholar 

  44. Booth A: Unpacking your literature search toolbox: on search styles and tactics. Health Info Libr J. 2008, 25: 313-317. 10.1111/j.1471-1842.2008.00825.x.

    Article  PubMed  Google Scholar 

  45. Haase A, Follmann M, Skipka G, Kirchner H: Developing search strategies for clinical practice guidelines in SUMSearch and Google scholar and assessing their retrieval performance. BMC Med Res Methodol. 2007, 7: 28-10.1186/1471-2288-7-28.

    Article  PubMed  PubMed Central  Google Scholar 

  46. Shariff SZ, Bejaimal SA, Sontrop JM, Iansavichus AV, Haynes RB, Weir MA, Garg AX: Retrieving clinical evidence: a comparison of PubMed and Google scholar for quick clinical searches. J Med Internet Res. 2013, 15: e164-10.2196/jmir.2624.

    Article  PubMed  PubMed Central  Google Scholar 

  47. Sharma P: Reflections on medical literature searches: letters to the editor. Transfusion. 2013, 53: 1858-1861. 10.1111/trf.12247.

    Article  PubMed  Google Scholar 

  48. Shultz M: Comparing test searches in PubMed and Google scholar. J Med Libr Assoc. 2007, 95: 442-445. 10.3163/1536-5050.95.4.442.

    Article  PubMed  PubMed Central  Google Scholar 

  49. McGowan J, Sampson M: Systematic reviews need systematic searchers. J Med Libr Assoc. 2005, 93: 74-80.

    PubMed  PubMed Central  Google Scholar 

  50. Anders ME, Evans DP: Comparison of PubMed and Google scholar literature searches. Respir care. 2010, 55: 578-583.

    PubMed  Google Scholar 

  51. Sampson M, Tetzlaff J, Urquhart C: Precision of healthcare systematic review searches in a cross-sectional sample. Res Synthesis Methods. 2011, 2: 119-125. 10.1002/jrsm.42.

    Article  Google Scholar 

  52. Booth A: How much searching is enough? Comprehensive versus optimal retrieval for technology assessments. Int J Technol Assess Health Care. 2010, 26: 431-435. 10.1017/S0266462310000966.

    Article  PubMed  Google Scholar 

  53. Wallace BC, Trikalinos TA, Lau J, Brodley C, Schmid CH: Semi-automated screening of biomedical citations for systematic reviews. BMC Bioinformatics. 2010, 11: 1-11. 10.1186/1471-2105-11-1.

    Article  Google Scholar 

  54. Giustini D, Barsky E: A look at Google scholar, PubMed, and scirus: comparisons and recommendations. J Can Health Libr Assoc. 2005, 26: 85-89. 10.5596/c05-030.

    Article  Google Scholar 

  • The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2288/13/131/prepub


Page 2

Review on References included* References found Hits Precision [%] Recall [%]
Antidepressants versus placebo for depression in primary care [21] 14 11 17,400 0.06 78.6
Artemisinin-based combination therapy for treating uncomplicated malaria [22] 49 49 5,320 0.92 100
Brief interventions for heavy alcohol users admitted to general hospital wards [23] 14 14 18,000 0.08 100
Combined DTP-HBV-HIB vaccine versus separately administered DTP-HBV and HIB vaccines … [24] 20 20 4,320 0.46 100
Erythropoietin or Darbepoetin for patients with cancer - meta-analysis based on individual patient data [25] 48 41 36,500 0.11 85.4
Green tea (Camellia sinensis) for the prevention of cancer [26] 51 48 17,900 0.29 94.1
Incentive spirometry for prevention of postoperative pulmonary complications in upper abdominal surgery [27] 11 11 13,400 0.08 100
Interventions to prevent occupational noise-induced hearing loss [28] 24 24 19,500 0.12 100
Non-pharmacological interventions for assisting the induction of anaesthesia in children [29] 17 16 29,300 0.05 94.1
Oral iron supplements for children in malaria-endemic areas [30] 70 62 67,800 0.09 88.6
Pharmacotherapy for anxiety disorders in children and adolescents [31] 22 21 17,700 0.12 95.5
Single dose oral flurbiprofen for acute postoperative pain in adults [32] 11 11 7,950 0.14 100
The effects of antimicrobial therapy on bacterial vaginosis in non-pregnant women [33] 24 24 18,700 0.13 100
Therapeutic interventions for symptomatic treatment in Huntington’s disease [34] 21 16 17,400 0.10 76.2
Total 396 368 291,190 0.13 92.9

  1. We used the Cochrane Database of Systematic Reviews issue 3/ 2013 (Gehanno et al. used Issue 3/2009 [9]).
  2. *: “References included“ is the number of single references of included studies with one major reference per study as retrieved from the Cochrane reviews.