More Questionable Scholarly Metrics are Emerging

My previous blog post described a questionable new scholarly metric called International Impact Factor Services (IIFS). Now I’ve learned about three new additional questionable metrics.

The three additional and questionable metrics were mentioned in helpful comments left on the earlier blog post.

The first is Advanced Science Index (ASI).

Advanced Science Index

Completely bogus.

Just like with the predatory publishers, this company demonstrates a serious lack of transparency by not clearly stating its location. If you look at the “contact us” page, there’s only a web form, and the page is devoid of headquarters location information. This usually means that the website is operating out of an apartment in Asia somewhere. The main page makes mention of Germany, but this may be a sly attempt to associate a non-Western operation with a Western country.

The ASI website uses boastful and grandiose language to promote itself and its mission. Under a heading that states, “Making the Future of Science Evaluation,” the company betrays its unprofessional practices with its next line,

Advanced Science Index is an indexing service indexes scholarly journals in all areas and fields of science. It is aiming at rapid evaluation and indexing of all local and international journals.

I’ve copied this  verbatim, mistakes and all. This company (and the others described here)  is marketing its services to (predatory) journals – not to individual scholars. The journals will then use the bogus measure to promote themselves and to get increased submissions (and therefore article processing fees) from researchers.

Here is how the metric called the ASI-Impact Factor will be calculated, according to the website:

To calculate ASI-Impact Factor following relation would be considered:
Average number of citations to recent articles published in the journal.
For Example:
A = the number of times that articles published in that journal in 2006 and 2007, were cited by articles in indexed journals during 2008.
B = the total number of “citable items” published by that journal in 2006 and 2007. (“Citable items” are usually articles, reviews, proceedings, or notes; not editorials or letters to the editor.)
2008 impact factor = A/B.

Sound familiar? It’s copied from Wikipedia, without attribution.

Directory of Journal Quality Factor

Also completely bogus.

THE SECOND NEW METRIC is the ridiculously-named Directory of Journal Quality Factor. This profit-seeking company is from Tamilnadu, India, home to many scam publishers.

On its “About” page, this company says,

The Directory of Journal Quality Factor-DJQF is a gateway that includes the journals and developed from the information of the journals. These informations can be used to calculate Journal Quality Factor for the readers worldwide.

Translation: We are another company that wants to make easy money by charging predatory publishers to assign meaningless metrics to their meaningless journals.

DJQF devotes an entire webpage to explaining its evaluation method, a method that combines subjective analysis with some observational data. I don’t believe it for a second.

THE THIRD  NEW METRIC:

International Scientific Indexing

Clever, yet toxic.

I just learned of another likely bogus metric, International Scientific Indexing (ISI). The initialism “ISI” matches Institute for Scientific Information, the firm that originated the legitimate impact factor, now owned by Thomson Reuters.

This one purports to be headquartered in a village in the United Arab Emirates. Its website appears to have been set up in a single day. When will this end?

Discussion questions:

  • Is the altmetrics movement encouraging the emergence of questionable metrics companies?
  • Are these metrics just an extension of the expanding predatory publishing industry?

These new metrics often look (or claim to look) at citation data, but they use Google Scholar to gather this data. Google Scholar does not sufficiently screen for quality and includes thousands of predatory journals in its index. Therefore, these novel metrics — if they are really calculated at all rather than just made up — are largely based on citations in predatory journals, making them even less valid.

Don’t be fooled by bogus metrics.

Hat tip:  Claudia Holland, et al.

19 Responses to More Questionable Scholarly Metrics are Emerging

  1. behalbiotech says:

    few days back I checked site of DJQF, I was surprised by use of links given to various reputed organizations like Nobel prize forum and non Olympic sites as partner. But did not find any reference of DJQF on those sites.

  2. Harvey Kane says:

    Kind of reminds me of the explosion of boxing sanctioning bodies. Just who is the heavyweight champion?

  3. Ricardo Jiménez says:

    Reblogged this on comunicarbien and commented:
    Three new additional questionable metrics…

  4. Alex SL says:

    Sadly this flood of fraudsters does not change the fact that there are serious problems with the existing metrics.

    For example, taxonomists often need to publish in monograph series instead of journals because the works are too long for the latter. But a publication in such a monograph series will never have its citations appear in Web of Science no matter how useful it is to people because those series do not have enough papers per year to be acceptable for inclusion in the database. (They may pop up in Google Scholar though!) In addition, the results of taxonomic research (species names, identification keys) are cited much less often than they are used.

    In other words, taxonomists are at a continual disadvantage compared with other biologists, and the same will be true for other fields. It would be nice if we could get a bit more diversity in honest metrics to deal with the vast practical differences between fields of science.

    • Dr. Patil Shrish says:

      I agree with Alex. A metric needs to be specific for a particular field of research and particular type of research. All scientific studies cannot be evaluated on the same scale.

  5. Harvey Kane says:

    However these types of publications can appear on med line.

  6. Alex SL says:

    Yes, they can also appear elsewhere. What I am worried about is the myopic one-size-fits-all approach that considers the Web of Science Impact Factor, WoS citations and WoS based h-index to be the only useful metrics there are. And yes, that is what many science managers etc do.

  7. Albert Noel says:

    Every thing has been traded in this dinosaur materials economy. Oh bourgeoisie! forgive the science now! All the sources of international scams (CDO, More’s law, Cap n trade, credit crisis, citizen united vs FEC and more more…..), annihilated world’s economy were generated by Wall Street. And now these people are trading IMPACT FACTORS. I want to ask one think, where is the main street? Come and join the voice of truth because WE THE PEOPLE. [Note: This comment is not for common people. Only for people having sufficient background in classical political economy.]

  8. Julie Parmentier says:

    How about American Academy of Sciences, http://www.aasci.org/ ? They emailed me out of the blue saying my paper had been selected for oral presentation. Huh? Little info on their website but their mission statement includes “Elect AAS Academicians and AAS Fellows”. A little too close to AAAS or NAS if you ask me. And there was no list of fellows on their website.

    • Completely bogus. Don’t go there.
      This organization is not on my list because it doesn’t publish any journals, as least according to what I can see. It appears to be a predatory conference organizer.

  9. Imam Salehudin says:

    Dear Jeff,

    I have recently received an email claiming an IC Impact Factor of 7.17 by Journal of Islamic Management & Business (ISSN online:2222-2839 & printed: 2222-1905). I suspect this to be fraud, since the field of Islamic Management is relatively new and could not have generated such a high IF.

    Below is the complete email from the journal. Could you please look it up, is this claim be true?

    Best regards,

    ================================================
    Journal of– Islamic Management & Business(2nd&3rd)

    Dear Author/Researcher

    JIMB is a Special Issue peer reviewed International Journal which will be published under European Journal of Business and Management through online as well as print version. The Impact factor of this Journal is: 7.17.

    Index of this journal:

    · EBSCO (U.S.)

    · Index Copernicus (Poland)

    · Ulrich’s Periodicals Directory (Pro Quest, U.S.)

    · Journal TOCS (UK)

    · PKP Open Archives Harvester (Canada)

    · Bielefeld Academic Search Engine (Germany)

    · Elektronische Zeitschriftenbibliothek EZB (Germany)

    · SCI-Edge (U.S.)

    · Open J-Gate (India)

    · OCLC World Cat (United States)

    · Universe Digital Library (Malaysia)

    · New Jour (Georgetown University Library, U.S.)

    · Google Scholar

    Advantages of highest IC Impact Factor 7.17 and USA based ISSN(online:2222-2839) & printed: 2222-1905) and as the editorial body and research area cover its concerned hopefully this journal will be recognized from most universities of the Muslim world, like IIUM, IIUC, USIM, INSAIF, IUK, CU, UTM, KSA Universities & others.

    Email: ejbmspecialissue@gmail.com , islamicmanagement2010@gmail.com

    · Website:www.crimbbd.org; http://www.iiste.org/Journals/index.php/EJBM/announcement/view/35, or, http://www.crimbbd.org/call-for-paper-5/, http://www.drmohiuddin.net.

    Maassalam
    Dr Md Golam Mohiuddin
    Executive Director: Center for Research on Islamic Management and Business(CRIMB)
    Faculty of Management and HRD(Former),University Technology Malaysia(UTM)
    (Now)Deptt of Management, Islamic University Kushtia Bangladesh
    Email:ejbmspecialissue@gmail.com , islamicmanagement2010@gmail.com
    mmohiuddinutm@yahoo.com, http://www.crimbbd.org

    • This journal is published by the International Institute for Science, Technology and Education, and I have this publisher included on my list. The “IC Impact Factor” is not a credible or reliable measure. It doesn’t mean anything. I recommend that you not submit any papers to this journal or to any of the journals published by the International Institute for Science, Technology and Education.

  10. Barry Lee Reynolds says:

    I am now receiving emails from journals published by Elsevier promoting the articles that have received high ratings based on Altmetric.com. Dr. Beall, have you possibly written an earlier post about the Altmetric.com website and I have missed it?

    • I wrote about altmetrics (the concept) not Altmetric.com (the company). Altmetrics means “alternative metrics,” alternative to traditional metrics, that is, like the impact factor. Apparently Elsevier has retained Altmetric.com to measure the impact of articles published in its journals.

      • Barry Lee Reynolds says:

        Thanks very much for the reply! Your blog is a much needed source for junior researchers.

        The text from the e-mail I received states: ‘Have you heard about Altmetric.com? It provides a relatively new way of judging the quality of an article by generating a score based on relevant discussions around each article from Twitter, Facebook, science blogs, mainstream news outlets and many more sources.’

        From this e-mail I can understand that the metric is based highly on how much discussion it generates. However, if an article is discussed negatively, meaning that someone is critiquing the research as bogus, then that would not be a very reliable metric.

        In making the above statement I am in no way criticizing Altmetric.com or Elseiver. I am just noting my ignorance of the metric and how it can be a reliable measure of an article’s quality.

      • I agree — you point out one weakness. The metrics may also be easily gamed.

      • Imam Salehudin says:

        @Barry Lee: It is true. However, even citation based metrics may have this weakness since a paper with weak science may be cited repeatedly to discuss the weakness and refute the conclusions made in the paper.

      • Barry Lee Reynolds says:

        I don’t disagree. All metrics can be manipulated.

Leave a Reply -- All comments are subject to moderation, including removal.

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: