Telling the Story of Clinical Impact: PubMed CPGs in PlumX

Plum Analytics PlumX Clinical Citations

We help researchers tell the stories of their research through metrics and analytics. One area that can be disenfranchised by traditional methods for analyzing impact is clinical research. In a 2013 article titled, “Citation Analysis May Severely Underestimate the Impact of Clinical Research as Compared to Basic Research” the authors noted that different areas of medical research — even within the same medical field — have different citation practices. They studied different research areas in three fields, Cardiac & cardiovascular systems, Clinical neurology, and Surgery. They found:

In each of these fields, there turn out to be large differences in citation practices between research areas. Low-impact research areas tend to focus on clinical intervention research, while high-impact research areas are often more oriented on basic and diagnostic research.

They go on to conclude:

As a consequence, the citation impact of clinical intervention research may be substantially underestimated in comparison with basic and diagnostic research.

It is this area that we’ve innovated with a new kind of citation known as a Clinical Citation. We first described this in May 2016 when we said that it hurts us all when our researchers don’t want to turn basic science into ways to treat illnesses and disease. Or, when our doctors pioneer new treatments but have no motivation to publish them. Now, we have expanded Clinical Citations to include Clinical Practice Guidelines in PubMed.

The American Academy of Family Physicians (AAFP) is an example of an organization that publishes evidence-based clinical practice guidelines (CPGs). In their Clinical Practice Guideline Manual they define CPGs as:

…statements that include recommendations intended to optimize patient care. They are informed by a systematic review of evidence, and an assessment of the benefits and harms of alternative care options. CPGs should follow a sound, transparent methodology to translate best evidence into clinical practice for improved patient outcomes.

PubMed indexes over 21,000 CPGs from many worldwide organizations. PlumX mines the guidelines for references to research and creates a Clinical Citation metric for every research article where our technology determines that the CPG references the underlying article. You can read more about our launch in our press release.

Below is an example of an article published in the journal, Gastrointestinal Endoscopy, “Sex and racial disparities in duodenal biopsy to evaluate for celiac disease.”

PlumX Metrics including Clinical Citations from PubMed
PlumX metrics including Clinical Citations from CPGs indexed in PubMed

You can see that this article has been included into two separate Clinical Practice Guidelines indexed in PubMed. One of these is from the American College of Gastroenterology and the other one is from British Society of Gastroenterology. In this example, these Clinical Citation metrics help tell the story of how this research is affecting clinical practice in both the US and the UK.

Clinical Practice Guidelines are a type of policy document that informs medical practice. It is important to understand these citations in that context and not group all types of policy documents together so that it is not clear how the research is being used and understood.



Jan van Eck, N., Waltman, L., Van Raan, A. F., Klautz, R. J., & Peul, W. C. (2013, April 24). Citation Analysis May Severely Underestimate the Impact of Clinical Research as Compared to Basic Research. PLOS ONE. doi:10.1371/journal.pone.0062395