Rebuttal: The CQT Has Scientific Support

Point-by-point rebuttal of the Iacono and Ben-Shakhar (2019) criticisms of the Comparison Question Test, arguing their arguments lack merit when examined against the full body of evidence.

Systematic rebuttal analyzing each criticism raised by Iacono and Ben-Shakhar (2019) against the available empirical evidence.

Honts and Thurber directly addressed each criticism raised in the Iacono and Ben-Shakhar (2019) review, analyzing their arguments against the full body of CQT research evidence. Their point-by-point analysis found the criticisms to lack merit when examined in context. This rebuttal corroborated contemporary research reporting high CQT accuracy and laid the groundwork for the comprehensive meta-analysis that would follow (Honts et al., 2021), which provided definitive quantitative answers to the questions raised.

The published rebuttal demonstrates the scientific community's engagement with criticism through evidence-based discourse. For practitioners, the exchange reinforces that CQT validity has been repeatedly challenged and defended through legitimate scientific processes, strengthening rather than weakening confidence in the technique.

Background & Context

The scientific debate over polygraph validity intensified in 2019 when William Iacono and Gershon Ben-Shakhar published a critical review in Law and Human Behavior concluding that the scientific basis of the Comparison Question Test (CQT) remained weak, research quality was low, and accuracy claims were unfounded. This critique built upon decades of skepticism from prominent psychologists and challenged the polygraph profession's fundamental claims about test validity.

Honts and Thurber's rebuttal emerged as part of a broader response to these criticisms. The work was first presented as a conference paper at the American Psychology Law Society annual meeting in Portland, Oregon in March 2019, then published in Polygraph & Forensic Credibility Assessment: A Journal of Science and Field Practice, Volume 48. This rebuttal represented a critical moment in polygraph science—demonstrating the field's willingness to engage with criticism through systematic, evidence-based counterarguments rather than defensive rhetoric.

The exchange exemplifies how scientific controversies should unfold: through rigorous point-by-point analysis of claims, examination of empirical evidence, and transparent discourse. Honts and Thurber's analysis found the Iacono and Ben-Shakhar arguments to lack merit when examined against the full body of CQT research, setting the stage for their comprehensive 2021 meta-analysis that would provide definitive quantitative answers.

Research Design & Methodology

The rebuttal employed a systematic analytical approach, identifying six core arguments from Iacono and Ben-Shakhar (2019): (1) a thought experiment treating chance-level performance as evidence, (2) attacks on publication venues rather than research quality, (3) broad dismissal of experimental studies, (4) assumptions about motivation as a powerful moderator, (5) assertions about lack of theory, and (6) claims that the 2003 National Research Council report showed weak validity. Each criticism was then subjected to empirical examination.

The methodological approach involved several converging lines of evidence:

• Thought Experiment Analysis: Examining the assumptions and preconditions of Iacono's Thought Experiment (ITE), which attempted to show how a test with chance accuracy could produce high accuracy in confession-criterion field studies, finding these assumptions unrepresentative and highly unlikely in real-world settings
• Empirical Falsification: Using meta-analytic results, field studies without confession criteria, and wrongful conviction case data to test the Iacono hypothesis
• Comparative Analysis: Comparing laboratory versus field study accuracies to test whether field studies showed dramatically inflated results as predicted by critics
• Moderator Testing: Examining whether motivation, subject population, or setting (experiment vs. field) significantly moderated CQT accuracy

A critical finding undermining the thought experiment was that no studies showed any version of the CQT performing at chance levels—a necessary premise of Iacono's theoretical argument. The rebuttal emphasized that scientific criticism requires empirical grounding, not merely theoretical possibilities divorced from observed data.

Results & Key Findings

Iacono and Ben-Shakhar's six major arguments were characterized as "unsupported speculation" that "deserve critical and empirical examination," and the systematic analysis found them to lack merit.

Specific refutations included:

• Thought Experiment Refuted: The premise that polygraphs perform at chance level (like a coin flip) lacked any empirical support—no studies showed CQT performing at chance
• Field vs. Lab Accuracy: Existing reviews revealed no dramatically higher accuracies in field versus laboratory studies, contrary to what the Iacono hypothesis predicted
• No Moderator Effects: The comprehensive analysis reported no statistically detectable effects for moderators of motivation, subject population, or setting in CQT studies
• Generalizability Supported: Meta-analytic findings showed "lie detectability remains stable across contexts," with external validity patterns mirroring other psychological domains

The thought experiment's assumption that cases have only two suspects was identified as rarely met in real-world applications and not representative of field studies from the 1980s onward. The converging evidence approach produced "strong falsification evidence" leading to the conclusion that the Iacono hypothesis should be abandoned as a meaningful description of field polygraph research.

Discussion & Significance

The rebuttal's significance extends beyond defending polygraph validity—it demonstrates how scientific controversies should be resolved through systematic empirical examination rather than theoretical speculation. Iacono and Ben-Shakhar were criticized for stating their arguments "as fact" when they were largely "unsupported speculation", highlighting the importance of grounding scientific criticism in observable data.

The rebuttal addressed the lack-of-theory criticism by noting it represented "an illogical straw man argument"—there is no requirement that full explanatory theory be in place before using a technology, as evidenced by aspirin's clinical use since the late 1800s despite lacking complete theoretical explanation of its medical action. This pragmatic argument shifted focus from theoretical completeness to demonstrated utility.

The work laid critical groundwork for the 2021 meta-analysis. That subsequent study, analyzing 138 datasets, found a meta-analytic effect size of 0.69 and showed that motivation had a positive linear relationship with accuracy—opposite to critics' predictions. The rebuttal's systematic dismantling of methodological criticisms enabled this larger empirical synthesis to proceed with confidence.

Limitations & Considerations

As a rebuttal paper rather than primary empirical research, this work's limitations primarily concern the nature of argumentative analysis. The authors acknowledged that while they successfully falsified the Iacono hypothesis, results should not be interpreted as indicating all CQT tests have high accuracy, as there was a large range of results not explained by tested moderators, suggesting other factors at work.

The debate highlighted persistent challenges in polygraph field research. Analysis of FBI polygraph data indicated that under certain policies, 83% of actually innocent people could be needlessly subjected to interrogation risks including false confessions. The authors acknowledged "weaknesses in the field data for the CQT" and called for additional research using different and innovative approaches. While the rebuttal successfully defended CQT's scientific foundation, it recognized ongoing professional practice issues that could undermine real-world accuracy.

Practical Applications

For polygraph examiners and the broader forensic community, this rebuttal reinforces that CQT validity rests on solid empirical footing despite persistent criticism. The systematic refutation of methodological objections—particularly regarding laboratory versus field generalizability and the role of motivation—provides confidence that properly conducted CQTs can inform investigative and security decisions. The finding that motivation, subject population, and experimental versus field settings do not significantly moderate accuracy suggests that well-designed laboratory studies can meaningfully inform field practice expectations.

For consumers and legal stakeholders, the exchange demonstrates that polygraph science engages seriously with criticism through evidence-based discourse. The analyses showed "the CQT does work, albeit not perfectly" and "works much better at assessing credibility than unassisted humans doing interpersonal deception detection". This positions polygraph testing as a valuable but imperfect tool—neither the infallible "lie detector" of popular imagination nor the pseudoscientific practice critics claim. Understanding this nuanced reality enables appropriate evidentiary weight in legal, security, and investigative contexts.