An interview with Catherine Williams
by Cristina Rigutto
"Altmetrics is the creation and study of new metrics based on the Social Web for analysing, and informing scholarship." Jason Priem, 2010"Altmetrics are an indicator of engagement and a real-time record of the online attention surrounding an individual research output." Catherine Williams, 2016
Is altmetrics a suite of metrics for research impact evaluation based on data from the social web that is going to disrupt the primacy of citation-based metrics? Or is it an indicator of how a research is connecting with its target audiences after publication data that helps researchers to identify potential impact stories and gather evidence for case studies?
To answer this question (and a few more), I interviewed Catherine Williams, Chief Marketing Officer at Altmetric.
Dear Catherine, are altmetrics aiming to complement or to replace traditional bibliographic rankings?
Altmetrics are very much intended to complement traditional scholarly metrics, such as citations, to provide a more complete picture of the reach and potential broader impacts of a piece of research. Whereas citations reflect the scholarly influence of the work, altmetrics can help readers understand how it has been received and interpreted amongst a much broader audience - for example amongst policy makers, practitioners, and the general public.
Most work in altmetrics has been done comparing different social media indicators (e.g. mentions in tweets) to citation measures. Does this work give us information about success or about impact? Or both?
There have been several studies that have tried to determine if there is any correlation between online attention and subsequent scholarly citations of a publication. Most of these have concluded that if the correlation exists at all it is relatively weak - the one exception to this being the number of Mendeley readers (likely because if an academic saves an item to their Mendeley library it is not unusual that they would therefore later read and then cite it).
With any measure (including citations) what they are interpreted to indicate will really depend on the aims of the individual researcher - for example if your objective was to increase public understanding of a certain topic then altmetrics that showed a lot of public engagement on social media and references from Wikipedia would provide a positive indicator that you had achieved your objective to some extent.
In most cases, all research metrics (altmetrics, citations, downloads etc) simply act as an indicator - they are a way of identifying where there might in future be real-world impacts as a direct outcome of the research.
Altmetrics are very much intended to complement traditional scholarly metrics, such as citations, to provide a more complete picture of the reach and potential broader impacts of a piece of research. Whereas citations reflect the scholarly influence of the work, altmetrics can help readers understand how it has been received and interpreted amongst a much broader audience - for example amongst policy makers, practitioners, and the general public. Most work in altmetrics has been done comparing different social media indicators (e.g. mentions in tweets) to citation measures. Does this work give us information about success or about impact? Or both?
There have been several studies that have tried to determine if there is any correlation between online attention and subsequent scholarly citations of a publication. Most of these have concluded that if the correlation exists at all it is relatively weak - the one exception to this being the number of Mendeley readers (likely because if an academic saves an item to their Mendeley library it is not unusual that they would therefore later read and then cite it).
With any measure (including citations) what they are interpreted to indicate will really depend on the aims of the individual researcher - for example if your objective was to increase public understanding of a certain topic then altmetrics that showed a lot of public engagement on social media and references from Wikipedia would provide a positive indicator that you had achieved your objective to some extent.
In most cases, all research metrics (altmetrics, citations, downloads etc) simply act as an indicator - they are a way of identifying where there might in future be real-world impacts as a direct outcome of the research.
The only way to really know this is to go and read the actual mentions - to see who is saying what, and why. The details page provided by altmetrics provider Altmetric enables you to do exactly this; to view and click through to all of the original mentions and shares of a publication. Remember, attention, like citations, can be for both positive and negative reasons!
Does the use of social media metrics for evaluation create undesired incentives?
Social media metrics, such as counts of Tweets or Facebook shares, should never be used in isolation as part of an evaluation process. These counts instead should be used to identify where there is a bigger story worth investigating, and then the qualitative data (what people are actually saying) can be used to evidence how the work has been received and interpreted amongst a broader audience.
How can altmetrics be used to monitor the effects of science communication activities?
Journal article pages often include Altmetric badges which can be used to track the dissemination of research amongst many different sources, including the mainstream and social media, blogs and Wikipedia, as well as academic forums such as F1000 and post-publication peer review sites Pubpeer and Publons.
Clicking on the badges will show a record of all of the mentions and shares of the research. These mentions are picked up from the point of publication onwards, meaning that users can track the attention in real time and identify which channels are generating the most engagement. Exploring the altmetrics for other items in your field may also help you identify key influencers who could help increase the visibility of your work, or understand which sources and communities are likely to be most interested.
How can altmetrics help to develop future communication strategies?
Altmetrics can now be found (often via the Altmetric badge) on thousands of journal articles and other research items. Researchers can explore the altmetrics for other items published in their field, or in the same journal, to see where those publications are getting attention from – for example there may be particular groups or bloggers it would be worth engaging with to make them aware of your work. A big benefit of altmetrics data is that it updates in real time – so you quickly identify where any outreach efforts are having an effect, and what changes it might be worth making in future.
Can altmetrics be changed by self-promotion?
Yes, and that's not a bad thing! Getting your work noticed is something that should be encouraged - particularly if you can build a strategy to more directly target the people you are most concerned should see it (key policy makers or patient groups, perhaps).
Altmetric calculate an attention score that provides an indicator of the volume and likely reach of the attention an item has received (the more attention, the higher the score), but because all mentions of the work are so transparent (via the details page) users will be able to see where the author has shared their work and also where it is receiving attention independently of that.
Some tips for how you might start making your research more visible are available in this guide.
Yes, and that's not a bad thing! Getting your work noticed is something that should be encouraged - particularly if you can build a strategy to more directly target the people you are most concerned should see it (key policy makers or patient groups, perhaps).
Altmetric calculate an attention score that provides an indicator of the volume and likely reach of the attention an item has received (the more attention, the higher the score), but because all mentions of the work are so transparent (via the details page) users will be able to see where the author has shared their work and also where it is receiving attention independently of that.
Some tips for how you might start making your research more visible are available in this guide.
Are there any Countries where the acceptance of new technologies and models for measurement of scholarly influence is particularly low?
The majority of altmetrics are for the moment somewhat biased towards western sources and publications. This is because western networks (such as Twitter and Facebook) have processes in place for making their data available, and also because many of the text-mining mechanisms currently in place function best on English-language publications.
That said, altmetrics already offer a big opportunity for research that is underrepresented by scholarly citation measures alone; some South American journals, for example, receive a lot of engagement that is better reflected through altmetrics than through any later citations.
A study conducted by Digital Science in 2015 explored variations in the way geographies and networks shared medical research online – with some interesting conclusions!
Last but not least, what are the benefits to scholars of incorporating altmetrics in a CV?
Embedding altmetrics tools such as the Altmetric badges or even just summarising some of the data in your CV offers some great advantages:
The majority of altmetrics are for the moment somewhat biased towards western sources and publications. This is because western networks (such as Twitter and Facebook) have processes in place for making their data available, and also because many of the text-mining mechanisms currently in place function best on English-language publications.
That said, altmetrics already offer a big opportunity for research that is underrepresented by scholarly citation measures alone; some South American journals, for example, receive a lot of engagement that is better reflected through altmetrics than through any later citations.
A study conducted by Digital Science in 2015 explored variations in the way geographies and networks shared medical research online – with some interesting conclusions!
Last but not least, what are the benefits to scholars of incorporating altmetrics in a CV?
Embedding altmetrics tools such as the Altmetric badges or even just summarising some of the data in your CV offers some great advantages:
- Adding context to your work – showcasing what people are saying about your publication can help readers understand its potential future impacts.
- Demonstrating reach and influence – altmetrics insights can be particularly useful for finding examples to cite when applying for funding or reviewing previous outputs with your supervisor.
- Encouraging further engagement – making the altmetrics for your work available shows that you are both aware of and open to discussion of it, encouraging others to interact with it too.
- Building your professional profile – establishing yourself in your field, particularly as an early-career researcher, can be a challenge. Altmetrics provide a different way of getting yourself noticed and evidencing the reach of your work amongst your peers.
- Managing your reputation – because altmetrics data is tracked in real-time, it’s quick and easy to keep an eye on what’s being said – meaning you can step in early if anything is being misrepresented or misinterpreted.
Catherine Williams is Chief Marketing Officer at Altmetric, a Digital Science company that track mentions of research in non-traditional sources to help authors, publishers, institutions and funders monitor and report on the online activity surrounding their work.
Follow @catherinelucy
Follow @altmetric
Comments
Post a Comment