Perspectives on Scientific Error: Parsing History and Comparing Viewpoints

Recent studies reveal a high prevalence of errors in published research. Theoretical arguments suggest that many published findings are false, and reports across fields show that many experiments do not replicate. These errors raise important issues. From a practitioner’s perspective, errors mislead and slow down research projects. From a philosophical perspective, scientific error raises questions about the right forms of scientific inference, scientific progress and the reliability of science as a source of knowledge. From the perspective of the public, scientific error undermines the epistemic authority of science and the degree to which policy-makers trust scientific experts.

During the last decade, countless meta-scientific studies (i.e., research on science and scientists) have been conducted to investigate the extent of these problems, statistical tools have been developed to identify them, and many solutions have been proposed and implemented to combat them. In addition, we are seeing the first results of meta-scientific studies on the effectiveness of these interventions. In short, the time is ripe for those working on these problems to reflect on these investigations, evaluate their results, and provide guidance in plotting the future of methodological development and error prevention.

This workshop is a first step in that direction. During four days, both senior and junior researchers will present their perspectives on four themes: methodological reforms, statistical reforms, publishing reform, and institutional reform. We will focus on three main goals:

1. Obtaining an up-to-date overview of the research on each kind of reform from senior expert researchers.
2. Create a collaborative space for junior and senior researchers to identify gaps in knowledge and opportunities for improvement for each kind of reform.
3. Lay the foundations for future discussions on research plans to address the gaps.

Each day of the workshop will start with a talk in which the current status of the field is discussed by one of the invited speakers. The speakers will be asked to end their talks with a visual summary on one slide. After this talk, workshop participants are asked to improve and extend this visual summary.

The discussion of the knowledge gaps and opportunities will begin asynchronously before the workshop asking participants to brainstorm freely about their perceptions of potential gaps in shared documents. The second part of each day will focus on discussing them more systematically.. A junior scientist will talk about their research and the gaps that still remain. After this second talk, participants will be divided in focused groups and asked to identify additional knowledge gaps and place them in the meta-scientific landscape. After combining all this information, participants will be asked to rank the knowledge gaps. This workshop will therefore result in a visualisation of the meta-scientific landscape on the four themes and an identification and ordening of the knowledge gaps.