Item selection counts: A comparison of empirical key and rational scale validities in theory-based and non-theory-based item pools

Roni Reiter-Palmon, Mary Shane Connelly

Research output: Contribution to journalReview articlepeer-review

13 Scopus citations


Little explicit attention has been given to the impact of item pools on the validities and cross-validities of different background data scoring approaches. This study tests the idea that pools c f items theoretically related to the performance of interest will outperform pools of items with no hypothesized relationship with the criterion. Validities and cross-validities of rational scales and empirical keys created from theory- and non-theory-based item pools were compared for 3 criteria. When size of the item pools was held constant, theory-based empirical keys (correlational and vertical percent) and rational scales showed larger validities and cross-validities than non-theory-based empirical keys (correlational and vertical percent) and showed minimal shrinkage in cross-validities. Even when item pool for the non-theory-based keys was expanded to include all items in the instrument, the theory-based keys showed comparable or slightly better validities and cross-validities for 2 of the 3 criteria, including college GPA, which was separated from the predictors by 4 years.

Original languageEnglish (US)
Pages (from-to)143-151
Number of pages9
JournalJournal of Applied Psychology
Issue number1
StatePublished - Feb 2000

ASJC Scopus subject areas

  • Applied Psychology

Cite this