Past events: Caren M. Rotello. A “blinded” assessment of the validity of researchers’ inferences about cognitive processes

What: A “blinded” assessment of the validity of researchers’ inferences about cognitive processes

Where: BCBL Auditorium

Who: Prof. Caren M. Rotello, Psychological and Brain Sciences Department, University of Massachusetts Amherst, MA, US

When: 12 PM, noon

Science can only benefit society if people trust its conclusions, so we are obligated to demonstrate that our research methods are valid. In recent years, psychologists have become increasingly aware of potential methodological problems in their research, and they have initiated large-scale projects to test the replicability of published findings (e.g., Klein et al., 2014). These efforts are crucial, but establishing scientific credibility requires not only showing that data patterns are replicable, but also that the inferences made from data are valid. For example, cognitive psychologists use data to make inferences about unobserved cognitive processes, often through the application of a quantitative model or measurement procedure. If invalid inference techniques are used, then replicated data patterns simply increase confidence in inaccurate conclusions (Fiedler et al., 2012; Rotello et al., 2015).  To assess the quality of our inferences, recognition memory experts were given data from recognition experiments in which memory strength, response bias, both, or neither, had been manipulated across conditions. These experts were asked to infer – using their choice of strategy – which manipulation(s), if any, had been used. This “blinded” approach removes many “experimenter degrees of freedom,” allowing an assessment of the validity of various analytic approaches to interpreting the data.