49 Responses to Ongoing Questions about PLOS ONE’s Peer Review

  1. Analyst says:

    Beall, what should we understand here? PLOS ONE as a potential, possible, or probable predatory scholarly journal ?

  2. Nils says:

    One wonders whether there is a sleep specialist named Doctor Stone who received a request to review a geophysics paper…

  3. […] Source: Ongoing Questions about PLOS ONE’s Peer Review | Scholarly Open Access […]

  4. Wim Crusio says:

    Jeffrey, just a few comments here, as I am familiar with PLOS ONE’s procedures. PLOS ONE’s editorial board definitely is closely involved in editing articles, but as editors, not reviewers. When a manuscript is received, PLOS ONE staff performs some obvious checks that can be done by somebody who’s not an expert. The manuscript is then assigned to a board member (called “academic editor”), who can accept or refuse to edit the manuscript. This is largely an automated process and far from flawless, but given that several editors are solicited simultaneously, usually a suitable editor is found pretty rapidly (if it’s not in your field, it’s just a mouse click to decline editing it).

    Selecting reviewers, however, is absolutely not automated but strictly done by hand by the academic editor. Like for any other journal, an editor can search the database for suitable reviewers using key words (in the present case, “sleep” is an obvious, but not sufficient, choice). A good editor will then have a look at any people found this way. (Personally, I rarely use this facility, as I only accept to edit articles on subjects that I am familiar with and generally have a pretty good idea which colleagues would be suitable reviewers.) In any case, it’s the responsibility of the academic editor to find suitable reviewers. In the present case, something clearly went wrong. Even if the academic editor had by mistake entered the key word “sleep” in the “reviewer name” field, just a quick glance to the key words describing the reviewers scientific expertise should have told him that Dr. Sleep was absolutely not qualified for this manuscript.

    I think the biggest problem with PLOS ONE is their enormous size. With 5000 editors, it is not easy to enforce uniform standards. If I were them, I would certainly look into Dr. Harry Zhang’s track record to see whether this was a one time sloppy mistake or a more regular occurrence.

    One more pedantic remark. Every journal that practices peer review will contact possible reviewers by email. I would not call that a “spam email”.

    • Thanks! The spam email that I was referring to was one I personally received, and it’s copied at the link at the end.

      • Wim Crusio says:

        But in your post you also call the review invitation that Dr. Sleep received a “spam email” multiple times… Mind you, I’m not contesting that that email was erroneous, because it was, just not spam.

      • Bonbon says:

        Yes. That invitation to review the article is definitely not “spam”. He’s an inappropriate reviewer for the material, but it’s just a standard invitation to review.

      • lisaamartin says:

        Agree with Wim – the request for Dr Sleep to review was erroneous, not spam. Also, the ‘spam’ email you linked to at the end of your article was not, actually, directly from PLOS One. If you look at the footer you can see that it was sent on PLOS One’s behalf as a ‘carefully selected partner’ of Scientific Direct, a.k.a. Thomson Reuters. Thomson Reuters are the spammers, it seems, not PLOS One, or at least you need to update your subscription settings.

  5. shirley ainsworth says:

    I apologize if I sent this twice, not quite sure if I posted it…

    Isn’t this a little strange that the ‘spam email’ purports to be from

    Scientific Direct
    1500 Spring Garden Street
    4th Fl
    Philadelphia, PA 19130
    USA ?

    When I googled Scientific Direct I came across a site selling targeted mailing lists

    1500 Spring Garden Street seems to be the address for WoS/Thomson Reuters

    Doesn’t sound right to me…

    • lisaamartin says:

      I think Scientific Direct is a sales/marketing branch of Thomson Reuters, which sells ads to companies (or “carefully selected partners”) who want to reach a scientific/academic audience. Whether Dr Beall signed up to receive these updates or not is almost a moot point: the email didn’t come from PLOS One directly, so it is not they who are the spammers here.

      • Dan Riley says:

        Thomson Reuters/Scientific Direct do seem to be spammers–I get junk from them I never signed up for. If PLOS One hired TR/SD for a marketing campaign, that’s certainly a mark against PLOS One, and I would argue that companies that hire spammers are responsible for the resulting spam.

        But there’s also nothing uniquely OA about that. I get spam from TR/SD for pay-access journals, and I get spam directly from Elsevier and IOP for restricted access journals.

      • Wim Crusio says:

        I actually do not consider this type of email spam. As far as I know, TR is a reputable company that honors unsubscribe requests. That is a far cry from the multiple OMICS-related emails that I get every day, despite multiple requests to remove me from their email lists.

        Up to a point, I think promotional emails are legitimate. For me, the conditions are 1/ able to unsubscribe, 2/ reasonably targeted (i.e. don’t send a geophysicist info on a journal on the neurobiology of sleep), and 3/ only on rare occasions.

        When we started Genes, Brain and Behavior 16 years ago, we actually did the same thing as PLOS ONE apparently did: Blackwell got a list of email addresses from TR, by providing them keywords and journal names, which they used to identify authors in their Web of Science who might be interested in our new journal. As far as I remember, only one email was sent out (exactly because we wanted to avoid spamming people). Without that, it would have been almost impossible to get the message out that this new journal had come into existence.

        Having said all this, I do find it curious that PLOS ONE, which now publishes a sizeable proportion of all articles indexed in MEDLINE, thinks it still needs this kind of promotion…

      • Dan Riley says:

        I’m something of an absolutist on the subject. TR gets to send me exactly one unsolicited email, asking if I want to opt-in to their carefully targeted emails that they genuinely believe will be of interest to me. If I don’t opt-in, and they send me another email, that’s unsolicited commercial email, aka spam. Reputable companies use opt-in; opt-out is spam.

        Yes, they are aren’t as bad as OMICS. That’s a very, very low bar.

  6. Sito says:

    I do not like “open acess” in general (I guess a more honest name would be “pay per publish”) and PLoS ONE in particular (open access + accept everything (technically correct) to get more money).

    Having said that, I do not think PONE can be classified as a “spammer” or predatory journal (maybe just mentioning it in this blog could give that feeling).

    • Samir Hachani says:

      Sito you seem to equate open access with open access with APC ( Authors Processing Charges) which is not the case for all publishers.

      • Sito says:

        I guess they are equal “de facto”. The most important OA journals are APC (PLoS, BMC, …). You have to go quite down in the “impact/importance” list to find a OA journal that is not APC.

      • BG says:

        To clarify, APCs are a consequence of Open Access and the current publishing industry (technology and culture). Articles published, for example, in arXiv are Open Access (freely available in full to readers anywhere with an internet connection). Because it is supported financially by charity, this is possible. For other, more traditionally “publisher” publishers, which are commercial entities, they cannot simply publish Open Access for free as they can’t make any money off of Open Access articles via subscription. So, they charge an article processing charge to recoup the costs involve in the publishing process. There is no reason that alternative models can’t exist, such as charitably funded models like arXiv, or alternative ways of recouping costs such as via advertising on pages where articles are accessed. But they generally don’t for reasons of current technology, economics and not insignificantly, culture (researchers want to have their work published in a journal run by a commercial publisher because this is traditionally what publication means, and is a sign of prestige with, in many cases, direct consequences on career advancement).

    • wkdawson says:

      I don’t think the purpose is to get more money. The purpose is to allow a forum for scientific discussion.

      For example, I seem to recall that there was a power struggle in anthropology for a long time over whether there was interbreeding between Neanderthals and the local humans. PLoS One was willing to publish some of the work interbreeding. Now, people seem to suggest, oh that’s obvious, yet that view was not welcome anywhere for a long time.

      I am sure that the matter probably is quite complicated and difficult to piece together 40k years later. Nevertheless, I think the views of a qualified anthropologist on such a subject should be heard. There are times that scientists seem to hold out on demanding “evidence” not so unlike creationists. What does an intelligent person do in such a case, … find some other place where people are willing to listen. In the end, it is probably a mixture of the two different views, but let good scientists do good science.

      • Sito says:

        I do not see how charging $2,500 to anyone willing to give an opinion fosters scientific discussion.

        I’d say it restricts scientific discussion to people/groups with a lot of money/resources.

      • wkdawson says:

        And, of course, no kahuna has an “opinion” that is full of it and freely published in the top journals at the reader’s expense; oh no….. definitely not.

        So what does this say?; that publication in OA actually means something if it is not so expensive? Not every PLoS journal is $2500, and I think PLoS One is $1500, where the institutes typically pay the expense from grants etc.

        By making work OA, the information is freely available to everyone. There is no excuse for not reading the papers; unlike now where researchers with limited budges — including even researchers in Japan — are forced to pay $40 for an article. A paper with 100 references means $4000.

        At any rate, you are entitled to your opinion, but that is all that it is.

      • Sito says:

        Like Wim, In 25 years researching I NEVER paid for an article. Articles not covered by my institute’s subscription were always provided by the authors on request, as preprints. There always was “de facto” open access.

        The current open access model (pay per publish) is just a business model disguised as a “cool” “modern” free-access-to-knowledge stuff. And the mere existence of predatory open access (subject of this blog) is a proof of that.

      • wkdawson says:

        Fine. In the US, there are interlibrary loans, and sometimes people will answer your emails. Not everyone has your luxury life, sir.

      • Wim Crusio says:

        I know of at least one person who was in between jobs and had to pay these $1500 out of pocket… And in my 40 years of doing research, I have never ever paid even a dime to obtain an article. My institutions usually subscribe to most of the journals I need and in the rare cases that they didn’t have a specific journal, I would get a reprint (in the pre-Internet days) or a PDF (nowadays) from the authors. And my employer most certainly does not pay for OA fees, meaning that I have to cover those from my research budget…

      • wkdawson says:

        That surely can happen in this day and age of adjunct professors and temporary postdocs. Presently though, the person would also have to pay for color pictures $900 a pop in the reader-pays journals. Admittedly, there are situations where it is easy to display data as B&W, but often color really does improve the production drastically, and there is no denying that with pictures of biomolecules — which are generally hard to make understandable even for experts.

        Consider yourself fortunate that you have had the luxury of being able to go to a full library to read whatever research paper you wished — and that you can download what you need. I have worked at places like that myself before. However, in many parts of the world, this is not so, and I have known places like that too.

        This reminds me of Aesop and his story of the pig and the lambs. The lambs ask the pig, “why are you crying”. The pig answers, “for you (the sheep), the farm just wants your wool. Me, he wants my bacon”.

      • Barbara Piper says:

        Sito asks about paying $2500 for an article, and notes that in many years s/he has never paid a cent to publish. Those comments are not an indictment of PLOS, however, but simply an illustration of the difficult financial models that open access publishing has to work with. I agree that some of the current models look like pay-to-publish any nonsense, but I am not sure how to balance the goals of OA with the problem of costs. wkdawson mentions anthropology, and I wonder if HAU is good example: rigorous peer review, very careful editing, open access on-line publication. HAU does not advertise the price it charges to publish an article, but the issues that have been published so far suggest that the quality is extremely high and that no one is looking to make any money off of it…

      • wkdawson says:

        I am not an anthropologist, and I am not qualified to take sides on the so-as-to-say “assimilation” vs replacement debate on Neanderthals or any other related genetic mixing arguments (that is not the only one). There is, nevertheless, sufficient evidence now to support the hypothesis that some level of integration of the Neanderthal genome occurred within our own genome. I mentioned that example simply because I observed parts of this debate over the years and noted the PLoS One was generous enough to publish the minor opinion (at the time).

        You may be correct that HAU would provide a more focused forum, but only an anthropologist can tell you that.

  7. wkdawson says:

    That’s the first time I have ever seen an advertisement from PLoS. I recall receiving solicitations from the reader-pay publishers too, though notably none recently.

  8. Jan Nyssen says:

    Sorry Jeffrey, your remarks seems not fair. Any editor of a journal can make a mistake in selecting a reviewer. This can happen, though it is of course not a sign of professionalism. Again, if a journal sends a reminder to a reviewer, this is not to be considered as Spam.
    In view of many people, including myself, PLOS One is doing a good job, as it also contributes to decrease the power of a few commercial publishers like Elsevier and Wiley who charge high rates for downloading papers. (We scientists write the papers for free, we do the reviews for free, we advertise our papers for free, hence there is no reason for charging high rates for paper downloads).
    PLOs One’s tries to assist us in getting out of this vicious circle; its IF is around 3, which is not too bad. This journal has nothing to do with the low quality publishers that are generally named ‘predatory publishers’.

    Best greetings,


    • liorsham says:

      If it is an error of an editor you are correct that it is not spam. But I think the point of the post is not the error. The point is that the nature of the error shows that it could be that invitations to review articles are being sent by some automatic system. That’s a completely different issue from the error. If indeed an automatic system identifies reviewers based on keyword matching and sends invitations to review articles, these invitations are very close to spam.

      • Wim Crusio says:

        As someone with first-hand experience with the PLOS ONE system (I have been an academic editor since very early on), I can tell you that reviewers are absolutely not selected by an automatic system. Each reviewer has to be selected by an editor. In this particular case, the editor goofed. Editors are not infallible, after all, they’re just humans like everybody else.

      • wkdawson says:

        Up to now at least, when PLoS journals have asked me to review, it was at least reasonably related to my area. It was a topic that I was not competing directly on, but I always assumed that was because the editors try to avoid sending manuscripts to competitors. It has been a problem in the history of academia and most journals at least say something to that effect, whether they follow that rule or not.

        At any rate, the editor in this case really did screw up. Usually, a potential reviewer would just write back to the editor of the journal and say “you made a mistake”, there would be an apology, and the matter would be finished right then and there. OA on the other hand, doesn’t seem to be cut this kind of slack here.

        Maybe it is good that OA is all being painted with the same brush here. After all, if one doesn’t even try to act right under the glaring gaze of heaven’s bright sun, how much more would there be trouble when one operates in the darkness of night.

  9. “its” has been misspelled as “it’s” twice. ;-)

  10. mkoulikov says:

    Publishing in PLOS ONE is easy; the journal is not very selective.

    For what it’s worth, at least in the field that I’m most familiar with (library/information science), PLOS ONE has regularly published articles from some of the most well-regarded scholars *in* the field. So, if someone like Vincent Lariviere is willing to publish with them, I am willing to give them a *lot* of a benefit of the doubt.

  11. ZhenBai says:

    PLOS ONE as a potential, possible, or probable predatory scholarly journal

  12. LynnStarrs says:

    I agree that PLOS is having trouble with uniform standards. I reviewed for them once, and both reviewers had serious questions about the article. They published it without a re-review. I will never review (or publish, which I have) with them again.

    • Harry Hab says:

      Same experience here.

      • Ioannis says:

        Not same experience. Both me and the other reviewers had some doubts (largely overlapping) about the scope of one manuscript (from quite recognisable Authors I might add). Did not get published.

  13. Ari says:

    Dear Jeffrey,
    I agree with some of the people that wrote above. I am an editor for PLoS One and take my duties in it very seriously. Its always hard to find reviewers, always. So I allow the system to search for me, but I never take it for granted, I always pubmed the suggestions to see if they are appropriate. But if one of the editors, 5000 to your count does not do that, its his/her fault, and does not make the journal a spammer.

  14. Akilah says:

    I am a frequent reviewer for PLOS one and all of the manuscripts sent to me thus far have been specific to my research area. The manuscripts that I have reviewed were all scientifically rigorous and would have, in my opinion, been accepted in top tier content specific journals. I think that it is unfair to label PLOS One predatory based upon only a few examples. I must admit that the article link to the retracted paper about “The Creator” was a huge gaffe and I hope that PLOS One is taking more steps to improve its reputation.

  15. Wayne says:

    I’ve published a couple papers in PLoS One and always found the reviews I received to be as rigorous as any I’ve faced in other journals.

  16. Mark Hahn says:

    I also have both reviewed for and published in PLoS ONE. The requests for review have been appropriate and from respected colleagues who serve as Associate Editors. I see many fine papers published there. It is true that with such a large editorial board and so many submissions there will be some mistakes made in inviting reviewers or in publishing faulty work. PLoS should work to improve that. But PLoS One is definitely not a predatory journal.

  17. Aviv Shachak says:

    Adding to what others said about PLoS One’s peer review- I had a paper rejected by the journal (in my opinion it was the wrong decision but this has nothing to do with the current discussion). Maybe it’s an exception but you can’t say they’d publish anything….

  18. Vitaly Citovsky says:

    i am very familiar with plos one and have been o its board for just less than a year i think. i handle all articles personally, but do not review them. this is not my job. i select reviewers from prominent scholars in the field with which i am familiar. there is absolutely no difference in how i handle papers for plos one or, for example, for plant physiology or virology or any other mainstream society journal on which i have served. the only difference is in the impact consideration. plos one does not consider potential impact as a criterion for acceptance whereas many other journals do. is it possible that one of the editors sent one of the papers to a wrong person to review? yes. everyone makes mistakes. labeling plos one a scientific spammer is wrong, however.

Leave a Reply -- All comments are subject to moderation, including removal.

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: