A recent study by Virginia Commonwealth University (VCU) refutes how some psychological experiments are conducted. According to researchers, close to half of published experiments they probed lacked a valid foundation of empirical evidence.
Their findings were based on an examination of 348 psychological manipulations as part of countless peer-reviewed work. The research team zeroed in on the practice of experimental manipulations, inducing a desired mental state on participants by the implementation of certain complementary or insulting feedback.
In their new study, the VCU team determined that an estimated 42 percent of the experiments lacked evidence of validity. Some of the manipulations were face-valid, single item self-reports, and thus relied entirely on face validity. This represents replicability issues for the experimenters.
“These issues can be remedied by validating manipulations prior to implementation, using validated manipulation checks, standardizing manipulation protocols, estimating the size and duration of manipulations’ effects, and estimating each manipulation’s effects on multiple constructs within the target nomological network,” according to the findings.
“These findings call into question the accuracy of one of psychology’s most common practices and suggest that the field needs to strongly improve its practices in this methodological domain,” said David Chester, co-author of the study.
“These findings call into question the accuracy of one of psychology’s most common practices and suggest that the field needs to strongly improve its practices in this methodological domain,” said David Chester, co-author of the study.
The findings are set to be published in Perspectives on Psychological Science.
