Humanities and Social Science Are Not Cited As Often

Citation counts have long been the tried and true measure of academic research usage and impact. Specifically, published articles in prominent journals citing other published articles in other prominent journals equate to prestige and tenure. This scheme for determining impact was developed in the 1960s, and while so much else about collecting and disseminating information has changed since that time, the citation count mechanism continues to dominate the way research is evaluated. There are many well-known problems with this system. This blog post is not going to delve into all of those problems; the purpose is to address the problem that citation rates vary by discipline — the Social Sciences and the Humanities do not get cited as much, thus disadvantaging researchers in these areas.

In a blog post on the Impact Blog author Dahlia Remler says:

Non-citation rates vary enormously by field. “Only” 12% of medicine articles are not cited, compared to about 82% (!) for the humanities. It’s 27% for natural sciences and 32% for social sciences (cite).

Using altmetrics data and analysis can tell the stories of interest, social reference and impact for social sciences and humanities.

Research Output Is Much More Than Articles

Some believe that altmetrics is only for articles, but Plum Analytics and our customers have identified 67 different kinds of research output, also known as artifacts, worth tracking.

This is important because the measurement of research can begin before it is published. Examples of this include if a researcher writes a blog post about his or her research ideas, presented research results at a conference prior to the published article, or there is a pre-print version of an article.

Additionally, some research is not destined to be an article. For example, one customer had a music faculty member that produced musical scores. This was a very important contribution to the field, yet very hard to measure.

Among the millions of outputs that Plum Analytics customers consider research, articles comprise only about 56 percent.

NOTE: For a full list of the 67 artifacts PlumX tracks, go here.

Books and Book Chapters

Books and book chapters are often the output of researchers and faculty in these disciplines. Citations do not do books justice.

Of the 50+ million total artifacts that PlumX tracks, about 4.1 million of them are books and book chapters.

The types of metrics you gather on these are different than what you find for articles and they can tell a different story.

• WorldCat – How many libraries hold this book?
• Wikipedia – How many Wikipedia articles references this book?
• Amazon.com – What are the reviews from Amazon?
• Goodreads – How many people place this book on their bookshelves?
• EBSCO eBooks – How many abstract views, downloads and clicks?

Example:

Plum Analytics’ customer, Mount Sinai Icahn School of Medicine and its Levy Library, has done a lot of work and thinking about the complexity of measuring books, and wrote this blog post about the subject. The author, Gali Halevi, PhD, Chief Director of the Icahn School of Medicine and The Mount Sinai Health System libraries wrote:

…studying the scientific and social impact of books still remains a challenge. The main reason is that books vary in format, content and genre. Books not only adhere to disciplinary or multidisciplinary foci, but also differ in categorical type and targeted audiences.

Dr. Halevi goes on to say:

Furthermore, in light of our analysis, we recommend using both traditional metrics such as citations and reviews as well as altmetrics such as social media mentions, downloads, reads and views. Using platforms such as PlumX can assist in capturing such metrics.

Below is an example of a group of books Mount Sinai has in PlumX about Cardiology. While none of these books have citations, they all have garnered a lot of interest and attention. This was a previously invisible form of research interest until they started tracking them in PlumX.

Identifying Research

Tracking anything is a challenge. Tracking a piece of research requires a way to identify it. The Digital Object Identifier (DOI) has been widely adopted to identify research output – mostly articles. But what about non-article output? Or articles that do not have a DOI? Plum Analytics includes as many identifiers as possible, including ISBNs for books, SlideShare IDs for presentations, YouTube IDs for videos, etc. PlumX was built to be web native so it can accommodate any way you identify your research online, including URLs.

Categorizing Altmetrics

To make sense of all of the altmetrics data available, it is important to group them into categories. Each category contains similar metrics to tell a different story.

• Usage: The most sought after metric after citations
• Captures: A leading indicator of citations
• Mentions: Where people are truly engaged with research
• Social Media: Tracks the buzz and attention of research
• Citations: The traditional measure of research

Who Does This Help?

• Researchers
• Department Heads
• Librarians
• Heads of Research, Grants, and Entrepreneurialism
• The Research—Itself

If an organization looks for citation counts and thinks: “No citations—game over—“ then this research needs rescuing.

Researchers: Especially early in their careers, researchers need to show that their work is used and appreciated.

Department Heads: Questions about who the top researchers are, who should the institution focus on and who is worthy of tenure and promotion can all be assisted by altmetrics.

Heads of Research and Grants: It’s important to measure efforts to ensure that the right decisions have been made and to put forth the research that best promotes the institution. Heads of research and grants have reporting requirements that can be greatly supported by altmetrics data.

Librarians: Of the librarians’ many missions, they are the custodians of their institution’s repository and they need to find ways to help faculty members understand the impact of their research. Both of these are supported with altmetrics data.

Altmetrics Tell the Stories of Research

As altmetrics have grown up, they hold the potential to help in a number of ways within an institution.

• Analyze the strengths of research at your institution
• Determine where research holds good potential for investment—or should be avoided
• Demonstrate ROI of research investments
• Identify rising stars among early career researchers
• Tell a better narrative about everything that is happening with research

Download a Poster

We created a poster about this topic that is available for download.

(Note: November 11, 2016 – updated the image in this blog post to refer to the metrics for the 67 artifact types in PlumX.)