Revisiting Old-Fashioned Reliability and Validity Concerns
Department of Psychology, New Mexico State University, Mexico
*Corresponding Author: David Trafimow, Department of Psychology, New Mexico State University, Mexico.
April 09, 2021; Published: July 31, 2021
Reported correlations in psychology research tend to be unimpressive. This would not be a problem if the underlying reason were that the phenomena under investigation really were not very related. However, a more troubling explanation pertains to the reliability and validity of the measures. As has been known since the seminal research by Spearman (1904), reliability sets an upper limit on predictive validity; unreliable measures result in unimpressive correlations even if all else is right. The present article briefly reviews the old literature on classical true score theory with an eye towards (a) reiterating long-known but rarely attended to prescriptions for obtaining more impressive correlations, (b) drawing lessons that contradict cliches in the field, and (c) expanding classical true score theory wisdom to cases where there are two predictor variables rather than a single amalgamated variable.
Keywords: Correlation; Multiple Correlation; Classical True Score Theory; Reliability; Validity
- Fishbein M and Ajzen I. “Predicting and changing behavior: The reasoned action approach”. Psychology Press, New York, NY (2010).
- Kraus SJ. “Attitudes and the prediction of behavior: A meta-analysis of the empirical literature”. Personality and Social Psychology Bulletin 21 (1995): 58-75.
- Fishbein M and Ajzen I. “Belief, attitude, intention and behavior: An introduction to theory and research”. Addison-Wesley, Reading, MA (1975).
- Spearman C. “The proof and measurement of association between two things”. American Journal of Psychology 1 (1904): 72-101.
- Lord F M and Novick M R. “Statistical theories of mental test scores”. Reading: Addison-Wesley (1968).
- Gulliksen H. “Theory of mental tests”. Hillsdale: Erlbaum (1987).
- Cronbach L J. “Coefficient alpha and the internal structure of tests”. Psychometrika3 (1951): 297-334.
- Morera O F and Stokes M A. “Coefficient α as a measure of test score reliability: Review of 3 popular misconceptions”. AJPH Methods3 (1963): 458-461.
- Trizano-Hermosilla I and Alvarado J M. “Best alternatives to Cronbach's alpha reliability in realistic conditions: Congeneric and asymmetrical measurements”. Frontiers in Psychology 7 (2016): 769.
- Mischel W. “Personality and assessment”. Mahwah, N. J.: Lawrence Erlbaum Associates (1968).
- Epstein S. “The stability of behavior: I. On predicting most of the people much of the time”. Journal of Personality and Social Psychology7 (1979): 1097-1126.
- Funder DC and Ozer DJ. “Behavior as a function of the situation”. Journal of Personality and Social Psychology 1 (1983): 107-112.
- Trafimow D., et al. “Evidence that perceived behavioral control is a multidimensional construct: Perceived control and perceived difficulty”. British Journal of Social Psychology1 (2002): 101-121.
- Pedhazur E J. “Multiple regression in behavioral research: Explanation and prediction (3rd edition)”. United States: Wadsworth (1997).
- Jussim L. “Social perception and social reality: Why accuracy dominates bias and self-fulfilling prophecy”. Oxford University Press (2012).