If your New Year’s resolution was to learn a bit more about research metrics, we’re here to help.

Meaningful Metrics

The book Meaningful Metrics by Robin Chin Roemer & Rachel Borchardt was recently made available in open access and is 251 pages chock full of background reading about bibliometrics, altmetrics and research impact.  Although this is written as a “Librarian’s Guide,” even if you are not a librarian there is a lot of great information here.

One of the interesting aspects of this book is that the authors delve into the sorts of questions that these new metrics can help address.  For example, this is an excerpt from the book:

A graduating PhD student sends you an e-mail asking which journals are the most impactful to his field in hopes of publishing his most recent article in the best venue.

A researcher tells you that the funding agency for her grant is now asking for evidence of impact on the resulting research from the grant. The resulting research included several national and international conference presentations, journal publications, and write-ups of her research in blogs within her. She then asks you for suggestions on how to document that impact.

A tenure-track faculty member who has published articles and monographs is going up for promotion in six months and wants to know what impact metrics he can include to supplement his file.

By both posing questions like this, and then discussing how you might think of addressing them, the book provides an approachable way to learn more about metrics and how they might be applied.

Measuring the Societal Impact of Open Science

If you’re looking for something shorter, check out: Measuring the societal impact of open science – Presentation of a research project by researchers at the Research Unit for the Sociology of Eduction, University of Turku and Information Studies at the University of Oulu.  This article does a good job of explaining the landscape of traditional metrics and how embracing modern metrics might be used for measuring societal impact of Finnish research in Finland and beyond.  Here is an excerpt of how they frame the discussion:

Research assessment has become increasingly important—especially in these economically challenging times—as funders of research try to identify researchers, research groups, and universities that are most deserving of the limited funds. The goal of any research assessment is to discover research that is of the highest quality and therefore more deserving of funding. As quality is very difficult and time-consuming to assess and can be highly subjective, other approaches have been preferred for assessment purposes (especially when assessing big data). Research assessments usually focus on evaluating the level of impact a research product has made; impact is therefore used as a proxy for quality. Impact can, however, be difficult to identify, track, and quantify, and as the current methods to assess research impact are being increasingly criticized, new methods and data sources need to be investigated. For this purpose, the altmetrics movement is investigating what the online mentions of research products can disclose about the impact research has had and who has been influenced by research.

We’ve always believed that open metrics promote open science, and it’s great to see researchers analyzing this point of view.

Are Altmetrics Still Alternative?

Ok, a plug of some of our own thinking on the subject seems fair in our own blog.  Mike Buschman and I wrote this article titled Are Altmetrics still Alternative? back in 2013.  The main points in that article still hold true today:

• As the pace of scholarly communication and science advancement has increased, citation analysis is a lagging indication of prestige. Citations can take 3-5 years to accrue the critical mass necessary for meaningful analysis.
• Not all influences are cited in an article, thus leaving the whole measure incomplete. Research outputs other than a journal article are typically not cited.
• Securing research funding is getting more competitive. When applying for grants, researchers’ most highly cited work will typically be several years old, and not necessarily most relevant to the grant application at hand. If researchers can show that their recent research is generating a lot of interaction in the scholarly community, that information can provide an advantage in this tight funding environment.

Altmetrics for large multidisiplinary research groups  Comparison of current tools

For those looking to understand different tools in the altmetrics space, but want a deeper comparison, we earlier highlighted about an article entitled Altmetrics for large, multidisiplinary research groups:  Comparison of current tools.