An assessment can be well designed, carefully administered, and psychometrically clean. The question predictive validity asks is whether any of that matters once the testing session is over: Do the scores tell you something useful about what happens next?
Predictive validity is the degree to which assessment scores forecast a future outcome. It requires 3 things: a predictor measured now, a criterion measured later, and enough time between the two for the prediction to mean something. A university examination predicting first-year grades, a licensing test predicting safe practice, a recruitment assessment predicting job performance—each is a claim about the future, and each needs evidence to back it up.
Predictive validity studies provide that evidence. They do not prove an assessment is valid in general. They establish whether scores support a specific use. This article will cover how predictive validity is evaluated, where it’s used, and the key limitations to consider when interpreting the evidence.
Why Does Predictive Validity Matter?
Assessment scores carry weight because people use them to make decisions. For example, a ministry sets a cut score for professional certification. A university admits one applicant and waitlists another. An employer screens candidates before an interview. Each time, the underlying assumption is the same: The score tells us something about future performance that we could not see as clearly without it.
Predictive validity is the evidence for that assumption. Without it, the score may be measuring something real, whether that’s knowledge, skill, or aptitude, while still failing to connect to the outcome the decision is meant to serve. A certification examination could produce reliable scores that bear no relationship to subsequent supervised practice, making the pass mark difficult to defend.
The question matters most in high-stakes contexts, where organizations must show that the scores they rely on have practical value beyond the test itself.
How Is Predictive Validity Evaluated?
Every predictive validity study follows these steps:
Choosing a defensible criterion
The study begins with the criterion, and this is where most predictive validity arguments are won or lost. The criterion must be a meaningful measure of the outcome the assessment is supposed to predict, and it must be measured independently rather than derived from the predictor.
First-year grade point average is the traditional criterion for university admissions tests. Workforce assessments may use supervisor ratings, while licensing studies might use later performance reviews or observed measures of safe practice.
A weak or poorly defined criterion limits the entire study. If the criterion is unreliable, even a genuinely predictive assessment will produce a low coefficient (i.e., a statistical measure of the strength of the relationship between the assessment score and the outcome). The test did not necessarily fail; the yardstick did.
Study design and the follow-up interval
A predictive validity study tracks a cohort forward, collecting scores at the point of assessment and measuring the criterion later. The length of the gap matters because it affects what the study can tell you: An admissions test validated against grades earned 6 weeks later is answering a different question from one validated against completion 4 years on.
The sample also matters: Findings from a narrow group of admitted students or a single institution may not generalize to other populations or settings.
Comparing the predictor and criterion
Once assessment scores and later outcomes have been collected, researchers compare the two sets of results to see how strongly they are associated. Did the students or candidates who scored highly tend to do well on the criterion? Did lower scorers tend to have weaker outcomes?
That relationship is usually expressed as a correlation coefficient: a single number indicating how strongly the assessment scores and the later outcome are associated. It runs from -1 to +1, with 0 meaning no association.
Interpreting the coefficient
In operational testing contexts, coefficients in the range of 0.30 to 0.50 are common and generally considered useful. For example, combining SAT scores with secondary school grades typically produces a correlation in the mid-0.60s with first-year university grades.
But raw coefficients can be misleading, for a reason worth understanding in some detail.
Where Is Predictive Validity Used?
Predictive validity is used across several assessment contexts. Here are some examples.
University admissions
Admissions testing is the most studied context for predictive validity. Large-scale validity studies consistently show that standardized test scores predict first-year university grades at moderate to strong levels, and that combining test scores with secondary school grades improves prediction beyond either measure alone.
However, the criterion used to assess predictive validity is evolving. First-year grades remain the most common benchmark, but more recent research also considers longer-term outcomes such as degree completion. This raises the question of whether an assessment validated against short-term performance remains valid for longer-term decisions.
Licensure and certification
Certification examinations exist to protect the public, and their predictive claim is that candidates who pass are more likely to practice competently. Unlike academic grades, however, the criterion is harder to pin down. Supervisor evaluations, clinical outcome measures, disciplinary records, and re-certification rates have all been used, each with its own reliability issues.
Because the stakes are high, even modest evidence of predictive validity carries weight, but so does its absence. A certification body that cannot show any relationship between test scores and subsequent practice outcomes faces a legitimacy question.
Workforce assessment
Pre-employment aptitude and skill assessments are often used to predict later job performance, using criteria such as supervisor ratings, productivity measures, and training completion. Substantial research shows how well these assessments predict job performance, but the quality of the criteria used to measure performance varies considerably.
Workforce prediction is harder because job performance is multidimensional and rarely captured well by a single measure. Criteria are often subjective, and relevance varies by role. Supervisor ratings, the most common criterion, show only moderate inter-rater agreement.
Range Restriction: The Limitation Hidden in Every Selection Study
Range restriction can distort how predictive validity evidence is interpreted, so it matters in any selection study.
Any assessment used for selection creates a restricted sample. A university that admits students above a certain score threshold will only have outcome data for those it admitted. The rejected applicants, who would have provided the lower end of the score distribution, are missing entirely. Because the remaining group is more similar on the predictor than the full applicant pool, the observed correlation between scores and outcomes is attenuated, meaning it is systematically smaller than the relationship in the population.
Statistical corrections exist, such as Thorndike’s formulae, and it is common practice to report both observed and corrected values. However, these corrections rely on assumptions about the full applicant distribution, so neither the raw figure nor the corrected one should be treated as the final word.
The implication for anyone commissioning or reading a validity study is straightforward: Ask whether the sample was selected on the predictor, and if so, expect the observed coefficient to understate the actual relationship.
Other common limitations and misconceptions
Beyond range restriction, several other factors can mislead:
- Criterion contamination. If assessors scoring the criterion know the predictor scores, their ratings may be influenced, inflating the observed relationship.
- Criterion unreliability. A noisy criterion depresses the coefficient even when the predictor genuinely forecasts the outcome. Job performance ratings, for instance, vary widely across raters.
- Correlation is not causation. A strong coefficient shows that scores and outcomes move together; it does not establish that the assessment caused the outcome.
- Validity can drift. The relationship between predictor and criterion may shift as populations, job requirements, or educational standards change. Evidence gathered a decade ago may no longer hold.
- A strong coefficient does not validate the assessment overall. Predictive evidence supports one specific use of scores. It says nothing about content coverage, fairness, internal structure, or the consequences of testing.
Predictive Validity Versus Concurrent and Criterion Validity
These terms overlap in ways that cause unnecessary confusion, so it’s worth clarifying them.
The Standards for Education and Psychological Testing, published jointly by the American Educational Research Association (AERA), the American Psychological Association (APA), and the National Council on Measurement in Education (NCME), treat validity as a unitary concept. There are not separate types of validity; there are different sources of evidence that support or weaken the interpretation of scores. Predictive and concurrent evidence both fall under evidence based on relations to other variables, an area formerly grouped under criterion-related validity.
The practical difference between predictive and concurrent evidence is timing. In a predictive study, you measure the criterion after the assessment. In a concurrent study, researchers measure the predictor and criterion at roughly the same time. An employment test correlated with current job performance is concurrent; the same test correlated with performance a year later is predictive.
The search term “predictive criterion validity” usually refers to predictive evidence gathered as part of a criterion-related investigation. The distinction matters because concurrent evidence cannot answer the forecasting question that predictive evidence addresses.
Where Predictive Validity Fits in a Validation Program
The Standards for Educational and Psychological Testing describe validation as an ongoing process drawing on multiple sources of evidence, not a one-time study. Predictive validity supplies one source: evidence that scores relate to a future outcome in a way that supports a particular decision. Content evidence, evidence from internal structure, evidence from response processes, and evidence from the consequences of testing each address different aspects of the same question: whether the interpretation of scores is justified for the use being made of them.
No single source settles the question on its own. Strong validation programs therefore plan predictive studies from the outset, with consistent assessment administration, structured and retrievable data, and clear links between assessment records and later outcomes. Digital assessment platforms built around open standards and structured data, such as TAO, support that work by making administration consistent, results portable, and follow-up linkage possible at scale.
Building a Defensible Validity Argument
Predictive validity asks a narrow question with wide consequences: Do these scores tell us anything dependable about what comes next? Answering it well means choosing a criterion that genuinely matters, allowing enough time for the outcome to emerge, reading the coefficient with range restriction in mind, and treating the result as one piece of evidence rather than a verdict on the assessment as a whole. Programs that plan for this from the outset, instead of commissioning a study once a decision is challenged, end up on much firmer ground.
Schedule a demo to see how TAO supports assessment programs designed for long-term validation.
