Research Culture Workshop: Towards Open Research

As part of Newcastle University’s Research Strategy, we are evolving our research culture in collaboration with the whole research community. We invite the research community across career stages, job families, and disciplines, to join this first Skills Academy Research Culture workshop: Towards Open Research.

The workshop will invite participants to consider open research practices and reflect on how they and the university can move towards a culture of more open research. In this workshop, we will consider open research principles and practices that increase transparency and rigour and accelerate the reach of our research.

Open research describes approaches to increase openness throughout the research cycle, including collaborative working, sharing and making research methodology, software, code, data, documentation and publications freely available online under terms that enable their reuse. Open research thereby increases the transparency, rigour and reproducibility of the research process and so can promote inclusivity, accelerate impact and improve public trust.  However, understanding and adopting open research practices can be challenging. This workshop therefore will explore strategies for culture change here at Newcastle University.

Workshop Details

Date: Thursday 30th September, 10.00 – 12.00.
Venue: Online.
Facilitators: Chris Emmerson and Steve Boneham.

Programme

  1. Introduction to open research
  2. Researcher perspectives on open research:
    1. Melissa Bateson – Professor of Ethology – Biosciences
    2. Greg Mutch – NU Academic Track Fellow – Engineering
    3. David Johnson – PhD Researcher – History, Classics and Archaeology
  3. Comfort break
  4. Breakout groups

    To discuss how the university can move towards a culture of open research by considering core aspects of the Center for Open Science strategy for culture change

    1. Systems and tools – what systems and research tools are needed to facilitate open sharing and documentation
    2. Support and training – what research support and training researchers require to undertake open research activities 
    3. Recognition and rewards – how open research behaviours can be encouraged, recognised and rewarded
    4. Policy – the role policy changes and interventions that require change to occur in open research practices at Newcastle

4. Reflections and next steps

*** This event is now fully booked. However please complete the booking form to go on the waiting list for this event and we will contact you should additional places become available ***

Secondary Data Is Out There

To researchers’ credit across the globe the amount of data being shared is growing and this will only increase over time as open research becomes ubiquitous. There are significant benefits to data sharing including increased rigour, transparency, and visibility.

But this post isn’t going to get blogged down in the benefits of data sharing as it is a path well-trodden. Instead, let’s consider that as researchers have been archiving and sharing data in archives and repositories there is a rich source of material that can be accessed, reworked, reanalysed and compared to recent data collections.

This secondary data analysis is a growing area of interest to researchers and funders, with the latter having calls focusing solely on reanalysis of data (e.g. UKRI). Accessing historic data also allows for research to be undertaken where costs are prohibitive, data is impossible or difficult to collect, and, possibly, reduce the burden on over researched populations. With the continuing challenges with collecting primary data during the pandemic there might not be a better time to consider what data is already out there.

And it is not only research that can benefit but also teaching and learning. Archived data sources can be accessed to introduce students to a fantastic range of existing data and code. Using secondary data can free students of data collection allowing them to focus on developing skills of research questions and analysis.

Based on data from re3data.org as of April 2021 there are over 2600 data repositories available for researchers to archive data, up from 1000 in November 2013. This isn’t a completely exhaustive list but is close enough to give an idea of the scale. Amongst these is our own data.ncl that now houses over 1200 datasets shared by university colleagues from across all disciplines and collected using a variety of methods and techniques.

However, finding the right dataset for your latest research project or teaching idea isn’t always straightforward. To help with that I have created guidance on how to find, reuse and cite data on the RDM webpages.

I would also be very keen to hear from users of secondary data to create case studies to inspire colleagues on this approach. If you would be interested in sharing your approach and experience, then please do get in touch.

Image Credit: Franki Chamaki on Unsplash

Guest post: Making Astronomy Research More Reproducible

Chris Harrison, as an astronomer who is a Newcastle University Academic Track Fellow (NUAct). Here he reflects on the good and bad aspects of reproducible science in observational astronomy and describes how he is using Newcastle’s Research Repository to set a good example. We are keen to hear from colleagues across the research landscape so please do get in touch if you’d like to write a post.

I use telescopes on the ground and in space to study galaxies and the supermassive black holes that lurk at their centres. These observations result in gigabytes to terabytes of data being collected for each project. In particular, when using interferometers such the Very Large Array (VLA) or the Atacama Large Millimetre Array, (ALMA) the raw data can be 100s of gigabytes from just one night of observations. These raw data products are then processed to produce two dimensional images, one dimensional spectra or three dimensional data cubes which are used to perform the scientific analyses. Although I mostly collect my own data, every so often I have felt compelled to write a paper from which I wanted to reproduce the results from other people’s observational data and their analyses. This has been in situations where the results were quite sensational and appeared to contradict previous results or conflict with my expectations from my understanding of theoretical predictions. As I write this, I have another paper under review that directly challenges previous work. This has been after a year of struggling to reproduce the previous results! Why has this been and what can we do better?

On the one hand most astronomical observations have incredible archives where all raw data products ever taken can be accessed by anyone after the, typically 1 year long, proprietary period has expired (great archive examples are ALMA and the VLA). These always include comprehensive meta-data and is always provided in standard formats so that it can be accessed and processed by anyone with a variety of open access software. However, from painful experience, I can tell you that it is still extremely challenging to reproduce other people’s results based on astronomical observational data. This is due to the many complex steps that are taken to go from the raw data products to a scientific result. Indeed, these are so complex it is basically not possible to adequately describe all steps in a publication. The only real solution for completely reproducible science would be to publicly release processed data products and the codes that were used both to reproduce these and analyse them. Indeed, I have even requested such products and codes from authors and found that they have been destroyed forever on broken hard drives. As early-career researchers work in a competitive environment and have vulnerable careers, one cannot blame them for wanting to keep their hard work to themselves (potentially for follow-up papers) and to not expose themselves to criticism. Discussing the many disappointing reasons why early career research are so vulnerable – and how this damages scientific progress – is too much to discuss here. However, as I now in an academic track position, I feel more confident to set a good example and hopefully encourage other more senior academics to do the same.

In March 2021 I launched the “Quasar Feedback Survey”, which is a comprehensive observational survey of 42 galaxies hosting rapidly growing black holes. We will be studying these galaxies with an array of telescopes. With the launch of this survey, I uploaded 45 gigabytes of processed data products to data.ncl (Newcastle’s Research Repository), including historic data from pilot projects that lead to this wider survey. All information about data products and results can also easily be accessed via a dedicated website. I already know these galaxies, and hence data, are of interest to other astronomers and our data products are being used right now to help design new observational experiments. As the survey continues the data products will continue to be uploaded alongside the relevant publications. The next important step for me is to find a way to also share the codes, whilst protecting the career development of the early career researchers that produced the codes.

To be continued!

Image Credit: C. Harrison, A. Thomson; Bill Saxton, NRAO/AUI/NSF; NASA.

2020 in review: data.ncl

This has been the first full calendar year data.ncl has been available for our researchers to archive and share data. And in the spirit of best of 2020 articles on film, TV shows and music I have dug into data.ncl’s usage statistics to pull out the headlines.

360 data deposits (718 in total)

118 different researchers archiving data (174 in total)  

154,630 views

47,190 data downloads

Our top three datasets based on views and downloads in 2020 were:

  1. Newcastle Grasp Library
  2. Handwritten Chinese Numbers
  3. EMG and data glove dataset for dexterous myoelectric control

The treemap below shows unsurprisingly that the most popular item uploaded was dataset (72%), then figure (15%) and media in a distant third (9%).

upload by item type

And the USA was the country that accessed our datasets the most with nearly 100,000 views from the stars and stripes alone.

As we move into 2021, I would love for this growth to continue and to see an increase in numbers across the board but in particular:

  • A greater number of records of datasets where the data is held elsewhere
  • An increase in code and software being archived and shared (currently 3% of all items but we have a GitHub plugin to make it easy to send snapshots to data.ncl)
  • The use of data.ncl as a platform to build dashboards upon that allows data to be manipulated and visualised

Let’s see what 2021 holds for data.ncl and we’ll be here to help archive and share the full variety of data and code from research at Newcastle.

Guest post: Can we be better at sharing data?

David Johnson, PGR in History, Classics and Archaeology, has followed up his post on rethinking what data is with his thoughts on data sharing. We are keen to hear from colleagues across the research landscape so please do get in touch if you’d like to write a post.

But if I wanted the text that much, the odds are good that someone else will, too, at some point.

Recently I wrote about how my perceptions of data underwent a significant transformation as part of my PhD work.  As I was writing that piece, I was also thinking about the role data plays in academia in general, and how bad we are at making that data available.  This is not to say we aren’t really great at putting results out.  Universities in this country create tremendous volume of published materials every year, in every conceivable field.  But there is a difference between a published result and the raw data that created that result.  I am increasingly of the mind that we as academics can do a much better job preserving and presenting the data that was used in a given study or article.

It’s no secret that there is a reproducibility crisis in many areas of research.  A study is published which points to an exciting new development, but then some time later another person or group is unable to reproduce the results of that study, ultimately calling into question the validity of the initial work.  One way to resolve this problem is to preserve the data set a study used rather than simply presenting the final results, and to make that data as widely available as possible.  Granted, there can be serious issues related to privacy and legality that may come into play, as studies in medicine, psychology, sociology, and many other fields may contain personal data that should not be made available, and may legally need to be destroyed after the study is finished. But with some scrubbing, I suspect a lot of data could be preserved and presented within both ethical and legal guidelines.  This would allow a later researcher to read an article, and then actively dig into the raw material that drove the creation of that article if desired.  It’s possible that a reanalyses of that material might give some hints as to why a newer effort is not able to replicate results, or at least give clearer insights into the original methods and results, even if later findings offers a different result.

In additional to the legal and ethical considerations, there are other thorny issues associated with open data.  There is the question of data ownership, which can involve questions about the funding body for the research work as well as a certain amount of ownership of the data from the researchers themselves.  There may also be the question of somebody ‘sniping’ the research if the data is made available too soon, and getting an article out before the original researchers do.  As with textual archives, there can also be specific embargoes on a data set, preventing that data from seeing the light of day for a certain amount of time.

Despite all the challenges, I still think it is worth the effort to make as much data available as possible.  That is why I opted last year to put the raw data of my emotions lexicon online, because to my knowledge, no one else had compiled this kind of data before, and it just might be useful to someone.  Granted, if I had just spent the several weeks tediously scanning and OCRing a text, I may be a little less willing to put that raw text out to the world immediately.  But if I wanted the text that much, the odds are good that someone else will, too, at some point.  Just having that text available might stimulate research along a completely new line that might otherwise have been considered impractical or impossible beforehand.  Ultimately, as researchers we all want our work to matter, to make the world a better place.  I suggest part of that process should be putting out not just the results we found, but the data trail we followed along the way as well.

Image credit: Kipp Teague (https://flic.kr/p/Wx3XYx)

Guest Post: Rethinking What Data Is

This is our first guest post on the Opening Research blog. We are keen to hear from colleagues across the research landscape so please do get in touch if you’d like to write a post. But the honor of debut guest blogger goes to David Johnson, PGR in History, Classics and Archaeology.


The trainings on open publishing and data storage fundamentally changed my perspective on what constitutes data.

Coming to start my PhD from a background in history and the humanities, I really didn’t give the idea of data much thought.  I knew I was expected to present evidence about my topic in order to defend my research and my ideas, but in my mind there was a fundamental difference between the kind of evidence I was going to work with and ‘data’.  Data was something big and formal, a collection of numbers and formulae that people other than me collated and manipulated using advanced software.  Evidence was the warm and fuzzy bits of people’s lives that I would be collecting in order to try and say something meaningful about them, not something to ‘crunch’, graph, or manipulate.  This was a critical misconception that I am pleased to say I have come to terms with now.

What I had to do was get away from the very numerical interpretation of the term ‘data’, and start to think in broader terms about the definition of the word.  When I was asked about a data plan for my initial degree proposal, I said I didn’t have one.  I simply didn’t think I was going to need one.  In fact, I had already developed a basic data plan without realising what it was called.  My initial degree proposal included going through a large volume of domestic literature and gathering as many examples of emotional language as I could find to create a lexicon of emotions words in use during the nineteenth century.  In retrospect, it’s obvious that effort was fundamentally based in data analysis, but my notion of what ‘data’ was prevented me from seeing that at the time. 

What changed my mind was some training I went to as part of my PhD programme, which demonstrates how important it is to engage with that training with an open mind.  The trainings on open publishing and data storage fundamentally changed my perspective on what constitutes data.  Together these two training events prompted me to reconsider the way I approached the material I was collecting for my project.  My efforts to compile a vocabulary of emotions words from published material during the nineteenth century was not just a list of word, but was a data set that should be preserved and made available.  Likewise, the ever-growing pile of diary entries demonstrating the lived emotional experiences of people in the nineteenth century constitutes a data set.  Neither of these are in numerical form, yet they both can be qualitatively and quantitatively evaluated like other forms of data.

I suspect I am not alone in carrying this misconception as far into my academic work as I have.  I think what is required for many students is a rethinking of what constitutes data.  Certainly in the hard sciences, and perhaps in the social sciences there is an expectation of working with traditional forms of data such as population numbers, or statistical variations from a given norm, but in the humanities we may not be as prepared to think in those terms.  Yet whether analysing an author’s novels, assessing parish records, or collecting large amounts of diary writings as I am, the pile of text still constitutes a form of data, a body of material that can be subjected to a range of data analysis tools.  If I had been able to make this mind shift earlier in my degree, I might have been better able to manage the evidence I collected, and also make a plan to preserve that data for the long term.  That said, it’s still better late than never, and I am happy say I have made considerable progress since I rethought my notions of what data was.  I have put my lexicon data set out on the Newcastle Data Repository, so feel free to take a look at https://doi.org/10.25405/data.ncl.11830383.v1.

Image credit: JD Handcock Photos: http://photos.jdhancock.com/photo/2012-09-28-001422-big-data.html

Did we share more data during Open Access Week?

To celebrate Open Access Week, 19– 25 October, data.ncl through Figshare ran a competition to encourage data to be uploaded and shared. We promoted Open Access Week on this blog, NUConnect, social media and in schools to help promote data.ncl and the merits of data sharing.

Anil Yildiz, Research Associate, in the School of Engineering has long embraced open data and has shared several datasets and supporting scripts from his research projects in data.ncl. The idea of a competition piqued his interest as an incentive for researchers to share data but also switched on his inquisitive nature as he wondered if it leads to an increase in uploads.

Figshare has an API that allows anyone to access a wide range of data and after we chatted Anil took an interest in the following four item types: figures; media; dataset; and software. He ran a query through the API between 06/07/2020 and 26/10/2020 on those four item types.

Number of figures, media, datasets and software uploaded to Figshare between 06/07/2020 and 26/07/2020

The graph above shows that the variation in uploads is not significant between the weeks examined but there were slight increases in media and software during open access week. Taking a deeper look into when these items are uploaded it indicated that Thursday are the most common day for researchers to archive and share data. And unsurprisingly weekends were found to be the quietest days.   

Conclusion

Open Access Week 2020 didn’t result in an upload frenzy. However, the sharing of these four item types is consistent across the timeframe analysed and Figshare is one of many data repositories that researchers can use to openly share their data. The bigger picture is that open research data is of growing importance as we look to increase transparency, reproducibility and reuse of data produced by our researchers. Data.ncl can archive all four item types and we are keen to see an increase in these deposits across all research data repositories. When data is archived elsewhere you can create a record of it in data.ncl to help increase the impact and visibility of the data.

At Newcastle this is the first time we have promoted the competition so it will take time for Open Access Week and data sharing to be on the radar of our researchers. It is interesting that Thursday is a particularly popular day to share data so perhaps we need a Thor inspired sharing initiative – data sharers assemble, anyone?

This blog was written in collaboration with Anil and his original blog Open Access Data: What do we Share can be found here: https://www.anilyildiz.info/blog/2020-10-26-blog-8. And a review of the data findings is available on data.ncl.

Celebrating Open Research Data with a DATA.NCL Upload Competition

For Open Access Week (October 19-25), Figshare is running a research data upload competition, offering prizes for participating institutions who upload the most items and researchers who upload during that week.

Data.ncl, Newcastle’s Research Data Repository, is powered by Figshare so all data uploaders – regardless of whether we are a winning institution – will have a chance to win one of five £100 Amazon gift vouchers, distributed virtually. Figshare will also be making a $500 donation to Resourcing Racial Justice, an organization that supports individuals and communities working towards racial justice.

Items must be uploaded to data.ncl between 12am on 19th October until 11:59pm on 25th October. Where possible we would encourage the data to be openly available, but it doesn’t necessarily have to be published if you require more time to prepare the dataset.

This is a little incentive to find some time during Open Access Week to prepare and share that dataset you been sitting on or meaning to archive. Some of the key benefits of sharing data through data.ncl are:

  • The data is assigned a persistent identifier (DOI) and a citation provided, so the data can be formally attributed
  • The persistent identifier helps to make the data discoverable through Google and other search engines to maximise visibility and impact of the research
  • Data can be located and accessed by you, without having to actively manage it

Since data.ncl was launched in April 2019, Newcastle researchers and PGRs have archived and shared 486 datasets, which have been viewed nearly 270,000 times across the world. ​Datasets have also been downloaded over 50,000 times and cited by researchers who have went on to reuse the data.

Data.ncl is not just for data but also code/ software and methodology so you can archive and share on the research process as well as any data outputs. There is guidance on how to archive data in data.ncl and you can get in touch with the Research Data Service on support in planning, managing and sharing research data at rdm@ncl.ac.uk

Happy uploading!