Sorry for the long absence from blog posts. I’m going to try to ease my way back in with a short post that I’d like to pose as a question as much as offering any obvious solution.
I’ve just been in contact with a colleague at another University who was the external marker on a student’s work. We disagreed on the mark awarded but managed to quickly come to an amicable compromise. The work was well presented and clear. The issue was the student had not been absolutely clear about the results.
Whilst I had to agree that implying there might be significance in a finding where the data suggests otherwise is a major problem. However, I couldn’t help but feel some sympathy for the student. The requirement to report positive findings is so pervasive in scientific research that it is quite difficult to expect academics to instil into students the need to report with complete clarity and honesty on their findings.
Until there is a way to report results with neutrality in terms of the significance this will always favour outputs which support the researcher’s hypothesis. From a broader perspective I would suggest that the pre-registration of studies is a good way to battle against this bias. If researchers were recognised for the quality of the ideas that generate their hypotheses rather than whether the hypotheses are supported by the evidence they seek, that would be a great start.
We do however still have a system in which many journal editors are looking for data driven positive evidence that confirms a hypothesis. From an educational standpoint, I think it’s fair to make students aware that recognition has this bias. It still should be clear in their formative stage of producing work for educational assessment that a well explained non-significant finding can be worth the best grade awarded. If we fail to make this clear students will end up undertaking research projects where the result is so heavily anticipated that the research is hardly even worth doing.