Validity of the control question polygraph test: The problem of sampling bias

Sampling bias is a potential problem in polygraph validity studies in which posttest confessions are used to establish ground truth because this criterion is not independent of the polygraph test. In the present study, criterion evidence was sought from polygraph office records and from independent police files for all 402 control question tests (CQTs) conducted during a 5-year period by federal police examiners in a major Canadian city. Based on blind scoring of the charts, the hit rate for criterion innocent subjects (65% of whom were verified by independent sources) was 55%; for guilty subjects (of whom only 2% were verified independently), the hit rate was 98%. Although the estimate for innocent subjects is tenable given the characteristics of the sample on which it is based, the estimate for the guilty subsample is not. Some alternatives to confession studies for evaluating the accuracy of the CQT with guilty subjects are discussed. The polygraph examination procedure used most frequently in criminal investigations is the control question test (CQT; cf. Barland & Raskin, 1973; Reid & Inbau, 1977), a procedure in which physiological responses to specific crime-relevant questions are compared with responses to control questions dealing with the subject's prior history of wrongdoing. The CQT relies on the assumption that guilty subjects should be more concerrted with and hence more reactive to the relevant questions, whereas innocent subjects should be more disturbed by the more broadly incriminatory control questions. This assumption is without a sound theoretical basis (Ben-Shakhar, Bar-Hillel, & Lieblich, 1986; Kleinmuntz & Szucko, 1982; Lykken, 1978,1981), however, and therefore carefully controlled empirical research is needed to establish the validity of the CQT. Ideally, an estimate of CQT validity would be derived from a representative sample of real-life examinations in which an unambiguous criterion of ground truth was available. Laboratory studies, although advantageous from the standpoint o fcriterion certainty, have generally failed to recreate the motivational and

Exhaustive archival analysis of all 402 CQTs from a 5-year period with blind rescoring by two independent evaluators and criterion verification from both polygraph office records and independent police files, distinguishing between confession-dependent and independently verified cases.

Patrick and Iacono's 1991 study addresses a fundamental methodological problem in polygraph validity research: sampling bias created when posttest confessions serve as the criterion for establishing ground truth. The researchers examined all 402 CQTs conducted during a 5-year period by federal police examiners, using independent police records rather than confession-dependent verification. Results revealed dramatic asymmetry: innocent subjects achieved 55% accuracy with 65% verified independently, while guilty subjects achieved 98% accuracy with only 2% verified independently. This gap exposes how confession-dependency obscures examiner errors—innocent people who fail tests don't confess and guilty people who pass tests don't confess, so both categories are systematically excluded from confession-based validity samples. The study demonstrates that widely-cited accuracy estimates for guilty subjects tell us more about confession rates than polygraph validity.

This study fundamentally challenged the validity of prior CQT research and established that field validity studies using confession-based criteria cannot reliably estimate polygraph accuracy for guilty subjects. The findings have critical implications for polygraph policy and practice, demonstrating the need for prospective designs with independent criterion verification rather than retrospective confession-dependent studies.

Background & Context

This landmark 1991 study addresses one of the most fundamental problems plaguing polygraph research: sampling bias in field validity studies. When posttest confessions are used to establish ground truth, the criterion is not independent of the polygraph test itself, creating a methodological circularity that artificially inflates accuracy estimates.

The problem is insidious: polygraph testing aims to solve crimes by extracting confessions from those who fail tests, and this benefit justifies polygraph use in the absence of compelling validity data. But this means confession-based field studies systematically exclude examiner errors—innocent people who fail tests don't confess, and guilty people who pass tests don't confess either, so these cases never enter validity samples. Patrick and Iacono's research exposed this circular logic that had undermined decades of CQT validation research.

Research Design & Methodology

The researchers sought criterion evidence from both polygraph office records and independent police files for all 402 control question tests (CQTs) conducted during a 5-year period by federal police examiners in a major Canadian city. This exhaustive approach was critical—rather than cherry-picking verified cases, they examined every single test administered during the study period.

Key methodological features included:

• Blind rescoring: Two independent evaluators blindly rescored 276 chart sets without knowledge of case outcomes or original examiner opinions
• High interscorer reliability: For DIR tests, R1/R2 reliability figures for respiration, cardiovascular, skin resistance, and total scores were .74, .87, .92, and .93 respectively
• Independent verification: Detailed criterion information was collected from police detachment files to identify cases verified with maximum certainty by information obtained independently of Polygraph Section files
• Criterion categories: Ground truth was established through confessions by the examinee, confessions by others exonerating the subject, or physical evidence recovery

Results & Key Findings

The study revealed a dramatic divergence in verification rates that exposed the sampling bias problem. Based on blind scoring of charts, the hit rate for criterion innocent subjects (65% of whom were verified by independent sources) was 55%; for guilty subjects (of whom only 2% were verified independently), the hit rate was 98%.

This striking asymmetry is the heart of the sampling bias problem:

• Innocent subjects: 55% accuracy with 65% verified independently—a credible if disappointing estimate
• Guilty subjects: 98% accuracy but only 2% verified independently—almost entirely confession-dependent
• Confession dependency: When confession-verified innocent suspects were examined, CQT accuracy was only 57%, and it was not possible to estimate accuracy for guilty subjects because police generally stopped investigating cases once a suspect failed a polygraph test

Polygraph opinions often incorporate extra-polygraphic information including case facts from police reports, discussions with investigators, and subject behaviors before and after testing; inconclusive opinions may be rendered when this information contradicts technically conclusive chart data. This finding revealed that real-world polygraph decisions aren't purely chart-based.

Discussion & Significance

Patrick and Iacono demonstrated that while the estimate for innocent subjects is tenable given the sample characteristics, the estimate for the guilty subsample is not. The near-total reliance on confessions to verify guilty subjects means these accuracy figures tell us more about which guilty people confess than about polygraph validity.

The research exposed a fundamental flaw in field validation methodology: selecting cases because they generated confessions systematically eliminates all cases where the original examiner made errors; if an innocent person failed, there would be no confession, and absent this ground truth criterion, the case would not be included in validity studies. This means published field study accuracy rates for guilty subjects are likely severe overestimates.

The study's impact on the field was profound. It became a cornerstone citation in critiques of CQT validity, referenced extensively in the 2003 National Research Council report on polygraph testing and continuing to shape debates about polygraph science decades later.

Limitations & Considerations

The study's most significant limitation is inherent to its design: it demonstrates the problem of sampling bias but cannot fully solve it. The researchers could identify that only 2% of guilty subjects were independently verified, but this same lack of independent verification prevented them from establishing a reliable accuracy estimate for guilty examinees.

Additional considerations include:

• The sample came from a single Canadian police agency, potentially limiting generalizability to other jurisdictions with different practices
• The 5-year retrospective design meant researchers had to rely on existing records rather than prospectively controlling data collection
• Laboratory studies fail to recreate the motivational and emotional characteristics of real-life polygraph examinations, but field studies suffer from the sampling bias problem—creating a methodological impasse

Practical Applications

This research fundamentally challenges the evidentiary use of polygraph results. If we cannot reliably estimate CQT accuracy with guilty subjects due to sampling bias, claims of 90%+ accuracy in detecting deception lack empirical foundation. The findings suggest polygraph results should not be admitted as evidence of deception in legal proceedings, though they may retain some utility as investigative tools for generating investigative leads.

For polygraph examiners and agencies, the study underscores the importance of documentation practices that enable independent verification beyond confessions. It also highlights how examiners may depart from strict chart-based decision making by incorporating extra-polygraphic information, suggesting the need for more rigorous standardization if scientific validity is the goal. The research ultimately calls for alternative validation methodologies that break the dependence on confession-based ground truth.