In several instances the gender of the animal used wasn't recorded which can have a bearing on the result as female mice have a stronger immune response to males. How the animals were housed will also impact on the results in experiments about the gut.
Dr Cruickshank says she was shocked at the lack of information provided in papers: "What our research has uncovered is that this lack of data makes it difficult to validate the experiment and the result. Crucially this is having an impact of the reproducibility of experiments, both in the animal model and when transferred to human trials."
The problem first became clear to the researchers when they were looking at building a knowledge base to help pull together research on inflammatory bowel disease (colitis). Working with computer scientists it became clear to the biologists that the data couldn't be understood by colleagues from different disciplines.
Professor Andy Brass says: "So much research is now being carried out across disciplines so it's vital that experiments can be understood by as many scientists as possible. Relying on the reader to make assumptions based on their own experience will only lead to errors, but this is what many papers are asking people to do when they come to replicate the research."
To address the issue the team have developed a critical checklist of what information should be included. It covers nine areas ranging from information about the animals, their housing condition, genetics, how colitis is induced, experiment design and monitoring. The checklist is included in their paper due to be published in the journal Inflammatory Bowel Diseases.
Dr Cruickshank explains: "Our checklist sounds like fairly basic information that should be in all papers. But over the past few years journals have asked for more and more abbreviated methods so information has stopped being included. Instead papers are focussed on the results and discussion and sometimes you have to go back to a paper from the sixties to find the last time a particular method was accurately recorded."
Whilst the researchers have been critical of the method reporting they are keen to point out that their criticism doesn't mean the research isn't valid or accurate. The experiments may well be sound, but the lack of detail in how they're reported makes that judgement more difficult to make and the reproducibility much harder.
As an example Dr Cruickshank points to the test case for the Reproducibility Initiative which was published in the journal PLOS One in December last year. The team wanted to reproduce an experiment which had found that peptide hormones made naturally by cattle are efficient killers of the parasite Leishmania major. When they followed the reported methodology they had to up the dose by ten times to get the same result. But what had actually happened was that the original paper had not precisely described the molecules involved.
Moving forward the Manchester team is recommending the adoption of their checklist as a requirement for publication to improve the quality, comparability and standardization of studies into inflammatory bowel disease. They believe it will make the interpretation and translation of data to human disease more reliable and ultimately contribute to making clinical trials more successful.
The paper "Quality of Methods Reporting in Animal Models of Colitis" will be published in the journal Inflammatory Bowel Diseases.