Don’t Use PubMed as a Journal Whitelist

pubmed

Proceed with caution.

PubMed is not MEDLINE. I recommend against using PubMed as a list of quality journals for the purposes of finding a journal to publish in, evaluating academic performance, awarding grants and degrees, and assessing job candidates. In addition, I recommend against relying on PubMed for literature reviews, unless the search results are restricted to MEDLINE journals only.

MEDLINE continues to have relatively strict inclusion criteria. But PubMed is not the same entity as MEDLINE.  PubMed is MEDLINE plus thousands of other journals, including many low-quality and predatory journals. The criteria for inclusion in PubMed (or PubMed Central, the full-text repository associated with PubMed) are minimal compared to the MEDLINE standards.

A journal’s inclusion in PubMed does not mean the journal has a stamp of approval from NIH. There is such a low barrier to inclusion that researchers are advised to be suspicious of any journal that boasts about its inclusion in PubMed, especially if the boasting is prominently displayed on the journal’s main web page. PubMed inclusion is not an achievement that merits boasts.

It is misleading that these potentially low-quality articles, many of which have not undergone rigorous peer review, are featured prominently in PubMed searches. The default search bar (at the left side of the screen after a PubMed search is conducted) features a filter for “Free full text”. Predatory journals that upload articles get highlighted in this way, while it is relatively difficult to find the filter for limiting searches to journals included in MEDLINE.

MEDLINE

If you seek a whitelist of scholarly journals in the biomedical sciences, use MEDLINE. MEDLINE is a curated, vetted list of about 5,600 journals that have been reviewed and approved by the National Library of Medicine. These journals are just a small fraction of the nearly 26,000 journals indexed in PubMed.

There are two relatively easy ways to confirm that an item indexed in PubMed was published in a MEDLINE-indexed journal.

One is to search the NLM Catalog for the journal and to check the line for “current Indexing Status” to see whether the journal is listed as “Currently indexed for MEDLINE” (or as “Not currently indexed for MEDLINE”).

The other is to search PubMed for abstracts and to limit the results to items published in MEDLINE journals.

Limit your search.

Limit your search.

To do this:

  1. Enter your search terms into the PubMed search box shown above.
  2. On the search results page, click on “Show additional filters” on the lower left.
  3. Select “Journal categories” in the pop-up box that appears.
  4. When the revised menu is presented on the left of the search screen, look for the MEDLINE option under the journal options header. Click on MEDLINE to limit your search results to MEDLINE-indexed journals.

Be aware that this limiter option may exclude some entries for MEDLINE articles. For example, in-process and “ahead of print” citations may be marked as not being included in MEDLINE even if they are preprints in MEDLINE journals. However, this option will help searchers avoid clogging their searches with articles from predatory journals.

Conclusion

PubMed is a database that indexes articles from many thousands of journals, including predatory journals. As the proportion of articles from predatory and other low-quality journals increases in PubMed, the database’s value decreases. However, researchers in the biomedical sciences can use MEDLINE as a whitelist and can use PubMed to access MEDLINE if they use the appropriate search limiters.

Hat tip: Dr. Kathryn H. Jacobsen

37 Responses to Don’t Use PubMed as a Journal Whitelist

  1. Reinhard says:

    Don’t use Medline as a whitelist either. As a plant scientist I noticed a long time ago that many plant specific journals were not indexed in Medline. This situation improved, however, when checking some of the Springer journals in the area of Plant Science one can still find journals not indexed in medline although they are well accepted in the plant research community.

  2. Richard Horobin says:

    Thanks for this one, it is a great practical tip … as long as Reinhard’s cautionary note is noted. In the absense of a real-time listing of junk/predatory journals I shall use this procedure when evaluating dubious journals in papers submitted to biomedical journals on whose editorial boards I serve. But what is needed, y’know, is for the reputable publishers to fund a real-time listing, yes? — Richard Horobin

  3. shirley ainsworth says:

    The ‘Not currently indexed for MEDLINE’ line is also applied to previous titles if there has been a title change for example. So be careful!

  4. shirley ainsworth says:

    On a related note, for records from Web of Science Core Collection (now of Clarivate Analytics), it is difficult to detect whether the journals belong to Emerging Sources Citation Index which are just being tracked and under consideration, or to the main database which has passed through a more rigorous selection.

  5. Dorothy Drew says:

    It should be noted it’s a medical database, so it can’t be expected to be a comprehensive list of plant science related journals. It’s also worth noting that in a search of PubMed alternate terms may be substituted. For example looking at nutrients in Kale, pubmed will add in the generic term “brassica” not the specific species name. Which may be ok but if you only want kale you get a lot of non-Kale articles in PubMed. Any systematic review that states a specific search expression was done in PubMed must be taken with a grain of salt when they compare results to other databases — were general terms used that ended up with substitutes. (not part of not relying on PubMed for journal authenticity but an aspect most researchers don’t realize happens)

  6. tekija says:

    This is generally sound advice, except for literature reviews. Especially for systematic reviews, the search should be as wide as possible, and quality checks are applied after rather than during search phase.

  7. Joe says:

    For Engineering/IT journals use Inspec (by The IET, UK).
    For Humanities/Social Sciences use Erih Plus (by Norwegian Government).
    For Education journals use ERIC (by US Government).
    For Medical journals use MedLine (by US Government).
    For Philosophy journals use Philosopher’s Index.
    For Muslim world journals use Index Islamicus.
    For Jew Journals use Index to Jewish Periodicals.

    And that’s all. Enjoy…

  8. MrBean says:

    Your exhortation does not make much sense, Jeffrey Beall.
    If you consider MEDLINE as the only mean of “Whitelist” journals, you seem restricting the scientific knowledge to Medicine only because Medline is by definition a database for Medicine literature, but not for mathematics, physics, chemistry, agriculture, geology, etc.
    What would be abnormal, by contrast, is to find a journal specialized in Chemistry, Physics, Veterinary or Civil Engineering indexed in MEDLINE!
    So, your caution to consider MEDLINE as the unique proxy of whitelist journals cannot be valid.

    • Thanks for your comment, and I regret the confusion. This is meant to apply to bio-medical sciences research, not all research.

      • NeuroGene says:

        Even for biomedical research this is a problematic proposition. For example, my papers in the American Journal of Human Genetics are not indexed on MEDLINE, and I don’t think anyone can claim that this is a predatory or a low level journal.

      • Except, please hold on a second, no one has said that journals not in MEDLINE are predatory journals. I repeat, I never said that, and I am not sure how you leaped to that conclusion.

      • NeuroGene says:

        I will explain how – you said not to use PubMed, use Medline, because when using Pubmed you can get papers from predatory and other low quality journals. But by applying your suggested filter, you exclude great journals. This is my point. Therefore, if someone accepts your suggestion and will look at my CV through your filter, I will lose two papers in one of the best genetic journals, as well as other papers in other good journals. So there must be another way, that will be more accurate.

      • Marco says:

        Jeffrey said not to use PubMed as a whitelist, but you can do so with Medline. That is, do not automatically assume that journals included in PubMed are high quality, whereas you can quite safely assume that those in Medline are. This does not mean that journals that are not included in Medline are bad, just that you must do more due diligence when you wish to submit to such a journal than just check whether it is included in PubMed.

  9. Reblogged this on TrapperPhD's Outpost and commented:
    I frequently met colleagues in pharma industries, which did not even knew that PubMed is a limited source for bio-medical publications. What a pity … and what a risk for the departments those colleagues are working in.

    This post by “Scholarly Open Access”, a quite proficient blog on scientific literature sources, points out the tiny little (but quite important) difference between free PubMed and its professional sibling Medline.

  10. NoSpam says:

    There are journals on your personal list that are also listed as “indexed for Medline” on PubMed. Which, then, should a cautious author follow?

      • NoSpam says:

        Oncotarget is the one that immediately came to mind. It’s been a few months since I did my research, but based on my recollection, some of Bentham’s journals are in Medline, although I am not sure if any are from the open access arm — they all report Bentham Science as a publisher on PubMed. Frontiers’ journals, including Bioscience. Probably others, based on the breadth of your list and the presence of big names in it.

      • Easy says:

        World Journal of Gastroenterology by Baishideng is also listed.
        They have many editors (more than 1000) and I know at least one who practices plagiarism.

      • Herr Doktor Bimler says:

        Baishideng … have many editors

        Baishideng are spamming indiscriminately to recruit “senior scholars” / peer reviewers; and since they ask the potential recipients of this honoured title to submit a capsule biography and a photograph, I suspect that they’re also upgrading the recipients to the title of “Editor” (without bothering to tell them).

  11. AmandaRW says:

    I think there is some confusion that needs to be addressed. You state; “MEDLINE is a curated, vetted list of about 5,600 journals that have been reviewed and approved by the National Library of Medicine. These journals are just a small fraction of the nearly 26,000 journals indexed in PubMed.” This is a little misleading. If you count by the number of articles (as opposed to the number of journals), Medline contains approximately 94% of Pubmed coverage. The disparity in the number of journals is because many of the Pubmed-not-Medline journals have only one or two articles, while the indexed for Medline journals might have many more articles. Take as an example the Journal of Hospital Librarianship, which is not indexed by Medline. There are 6 articles from the journal found in Pubmed, because authors have opted to make them available through PubMed Central. To find other articles from JHL, you’ll need to search CINAHL, or other library literature sources.

    • Thanks for your helpful comment. To clarify, the point is that, of those 6% of articles, a certain percentage of them will be from predatory journals. If you are doing a literature review, you will probably want a way to exclude those, and limiting to MEDLINE journals is a good way to do this.

      Also, the blog post is mostly about selecting a whitelist for journals, as in selecting a journal to publish in, so the number or proportion of articles is not relevant to that. A curated list of 5,600 journals is more selective than an uncurated list of 26,000.

      • Aviv Shachak says:

        For a literature review, I would care more about the quality of the article itself (particularly in a systematic review in which quality is one of the criteria) and not the quality of the publisher or the journal. Although I agree that the quality of articles in predatory journals is likely to be low that doesn’t mean one cannot find good papers in them at all- especially if good researchers fell into the trap set by the journal.

        The first recommendation of the San Francisco Declaration on Research Assessment (DORA), which I’m sure you’ve heard of is:
        “Do not use journal-based metrics, such as Journal Impact Factors, as a surrogate measure of the quality of individual research articles, to assess an individual scientist’s contributions, or in hiring, promotion or funding decisions.”
        I think this should also apply to the decision of whether or not to include an article in a systematic review.

  12. PubMed also contains for example 7 papers which have been published in the ornithological journal “The Ibis” ( https://www.ncbi.nlm.nih.gov/pubmed?term=%22The%20Ibis%22%5BJournal%5D ).

    The Ibis (or Ibis) is a highly respected journal within the field of ornithology (the study of birds). It is published since 1859. All 7 papers in PubMed report ornithological topics. The Ibis is the official journal of the British Ornithologists’ Union https://www.bou.org.uk/about-the-bou/ The Ibis is currently published by Wiley http://onlinelibrary.wiley.com/journal/10.1111/(ISSN)1474-919X

  13. Easy says:

    Dove press and Oxford journals.
    They are not included but some of their journals seem to fit the criteria.

  14. […] Jeffrey Beall recommends “against using PubMed as a list of quality journals for the purposes of finding a journal to publish in, evaluating academic performance, awarding grants and degrees, and assessing job candidates.” Here’s why. […]

  15. […] avete cliccato su statistica, vi sarete accorti che per PubMed anything goes. Jeffrey Beall spiega come usare il filtro […]

  16. TKesteman says:

    I also note that, for a given journal (e.g. Malaria Journal), recent publications (2016) are not listed in Medline while previous ones are. I don’t think that this reflects a change in the quality of the journal -as it remains a respectful journal of importance in this domain-, but just a lag in Medline registration, maybe.

    Just pay attention to this when conducting a systematic review or an assessment of someone’s CV.

  17. I did not find Macrothink Institute’s Journal Of Agricultural studies in your list. Can I trust thhis journal?

  18. Journal of development effectiveness, Indian journal of plant protection, International Journal of Entomology Research, Journal of Agriculture and Rural Development in the Tropics and Subtropics and Sustainable agriculture research Canada. Can I trust any of these journals? Regards,

    • Journal of Development Effectiveness = not a predatory journal
      Indian Journal of Plant Protection = outside the scope of my work (a subscription journal)
      International Journal of Entomology Research = one of the journals published by ESci Journals Publishing, a publisher on my list here. I recommend avoiding all the journals from this publisher.
      Journal of Agriculture and Rural Development in the Tropics and Subtropics = not a predatory journal
      Sustainable agriculture research Canada = I think you mean Sustainable Agriculture Research published by the so-called Canadian Center of Science and Education. This publisher is on my list here and I recommend you avoid all its journals.

      Please consult my lists:

      Publishers = https://scholarlyoa.com/publishers/
      Standalone journals = https://scholarlyoa.com/individual-journals/

  19. sa lim says:

    Can you help me, I have an article I published in the journal of biology or environment, but must have an impact factor, please who has experience advise me the names of these journals,

    • Keith says:

      Sorry, your question is a little unclear. Are you saying that you have an article you want to publish in a biological or environmental sciences journal, and want to know which ones have legitimate impact factors? Or are you saying that you have already published an article in a particular journal (it’s unclear which one, since “the journal of biology or environment” doesn’t appear to be an exact title) and want to know if this journal has a legitimate impact factor?

  20. kpmitton says:

    Reblogged this on Ken Mitton, PhD FARVO and commented:
    An important post from the Scholarly Open Access blog by Jeffrey Beall, University of Colorado. Why you cannot trust many publications in PubMed searches these days, but how to confirm trusted journal list by searching just Medline. Know the difference.

  21. Jeannette Kunz says:

    I reserve the right to make my own choice of where I search for primary and background literature. Medline is an index for medical and basic science research. Whether you personally like it or not can not be the only “value” indicator there is. Sorry, you just lost my respect. You are completely biased.

Leave a Reply -- All comments are subject to moderation, including removal.

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: