PolyScore 3.3 and psychophysiological detection of deception examiner rates of accuracy when scoring examinations from actual criminal investigations

Abstract : A stratified random sample of 200 confirmed examinations were scored using PolyScore 3.3. Three experienced psychophysiological detection of deception (PDD) examiners scored the 100 Zone Comparison Test (ZCT) examinations, and three PDD examiners scored the 100 Modified General Question Test (MGQT) examinations, using a 7 position scale. The scores were converted to 3 position scale for comparison. PolyScore had an overall level of accuracy of 90.9% when scoring the ZCT examinations, but was less accurate when scoring the MGQT examinations. The PDD examiners had an overall level of accuracy of 82.3% and 73.3% using the 7 and the 3 position scoring scales, respectively when evaluating MGQT examinations, but were less accurate on ZCT examinations. A test for the significance of proportion differences was performed on the accuracy data and the differences between numerous comparisons were statistically significant. All computed Kappa values assessing the interrater agreement for the two groups of examiners were statistically significant. Finally, the proportion of concurrence between individual examiners, ranged from 76.8% to 81.0% for the ZCT examinations, and 78.8% to 92.0% for the MGQT examinations when using the 7 position scoring scale. Concurrence was lower when using the 3 position scoring scale.

The study utilized stratified random samples of 200 confirmed criminal investigation cases scored by PolyScore 3.3 and by three experienced human examiners per test format (ZCT and MGQT). Examiners scored examinations blind to ground truth using both 7-position and 3-position numerical scales, with statistical validation including Cohen's Kappa calculations and significance of proportion difference tests.

This field validity study evaluated the performance of PolyScore 3.3 computerized scoring and human examiner accuracy on 200 confirmed criminal cases, addressing a research gap that had relied primarily on laboratory simulations. The study compared three experienced PDD examiners on each of two test formats (ZCT and MGQT) using both 7-position traditional and 3-position simplified scoring scales. PolyScore demonstrated superior accuracy on the structured ZCT format (90.9%) but reduced effectiveness on MGQT, while human examiners showed the opposite pattern with higher accuracy on MGQT examinations. Accuracy declined significantly when scores were compressed from 7-position to 3-position scales, and interrater agreement among human examiners ranged from 76.8% to 92.0% depending on test format and scoring method.

The findings suggest that automated scoring algorithms and human examiners have differential strengths depending on test format, with important implications for standardizing scoring procedures and training practices in operational polygraph programs. The substantial information loss observed during score compression from 7-position to 3-position scales supports the use of more granular scoring methods in practice and raises questions about the ecological validity of simplified scoring approaches in real-world criminal investigations.

Background & Context

The late 1990s marked a pivotal era in polygraph science, as computerized scoring algorithms began challenging the supremacy of traditional human chart interpretation. PolyScore 3.3, developed by Johns Hopkins University Applied Physics Laboratory, represented one of the first automated scoring systems designed to objectively evaluate polygraph data. This technology promised to eliminate subjective bias and enhance the consistency of polygraph examinations across different examiners.

The field urgently needed empirical validation of both automated and human scoring methods using confirmed field cases rather than laboratory simulations. Prior research had primarily relied on mock crime scenarios, raising questions about ecological validity. This study addressed that gap by evaluating 200 confirmed examinations from actual criminal investigations, providing critical real-world performance data for both computerized algorithms and experienced human examiners.

Research Design & Methodology

The research employed a stratified random sample of 200 confirmed examinations scored using PolyScore 3.3, with three experienced psychophysiological detection of deception (PDD) examiners scoring 100 Zone Comparison Test (ZCT) examinations and three different PDD examiners scoring 100 Modified General Question Test (MGQT) examinations. The examiners were blind to ground truth, ensuring unbiased evaluations. All cases were drawn from actual criminal investigations where veracity had been independently confirmed through confession, corroborative evidence, or exoneration.

The study utilized two complementary scoring approaches to assess consistency and reliability:

• 7-position numerical scale — The traditional scoring method taught at the Department of Defense Polygraph Institute, allowing granular assessment of physiological responses
• 3-position scale conversion — Scores were collapsed into three categories (deceptive, no deception indicated, inconclusive) to enable direct comparison between human examiners and the PolyScore algorithm
• Statistical validation — Tests for significance of proportion differences and Cohen's Kappa calculations assessed interrater agreement and decision accuracy against ground truth

The research design separated test formats (ZCT vs. MGQT) and assigned different examiner panels to each, recognizing that these techniques employ distinct question structures and psychological assumptions that might differentially affect scoring performance.

Results & Key Findings

PolyScore achieved an overall accuracy level of 90.9% when scoring ZCT examinations, but demonstrated reduced accuracy on MGQT examinations. This substantial performance difference highlighted test format as a critical variable in automated scoring effectiveness. The algorithm excelled with the more structured ZCT format but struggled with the MGQT's different question architecture.

Human examiner performance revealed an inverse pattern:

• PDD examiners achieved 82.3% accuracy using the 7-position scale and 73.3% accuracy using the 3-position scale when evaluating MGQT examinations
• Human examiners demonstrated lower accuracy on ZCT examinations compared to their MGQT performance
• Accuracy declined significantly when converting from 7-position to 3-position scoring scales, suggesting information loss during score compression
• Concurrence between individual examiners ranged from 76.8% to 81.0% for ZCT examinations and 78.8% to 92.0% for MGQT examinations using the 7-position scale

Statistical analysis revealed that numerous accuracy differences between scoring methods and test formats reached statistical significance. All computed Kappa values assessing interrater agreement for both examiner groups were statistically significant, confirming reliability in the scoring process despite varying accuracy levels. Interestingly, concurrence decreased when using the simplified 3-position scale, contradicting assumptions that reduced complexity would improve agreement.

Discussion & Significance

The findings challenged prevailing assumptions about the universal superiority of either human judgment or algorithmic scoring. The performance crossover—PolyScore excelling on ZCT while humans performed better on MGQT—suggested that optimal polygraph evaluation may require matching the scoring method to the test format. This has profound implications for field practice, where one-size-fits-all approaches dominated.

The substantial accuracy drop when converting 7-position scores to 3-position decisions raises important theoretical questions about information retention and decision thresholds. While simplified scales offer clearer decision rules, the 9-percentage-point accuracy decrease (82.3% to 73.3%) represents an unacceptable trade-off. This finding influenced subsequent debates about optimal scoring granularity in polygraph practice and contributed to broader discussions within the National Research Council's 2003 comprehensive polygraph review.

The study provided the first large-scale field validation of PolyScore using confirmed cases, lending credibility to computerized scoring algorithms that would eventually become standard in modern polygraph practice. However, the format-dependent performance highlighted the need for algorithm refinement and validated concerns about premature adoption of automated systems without comprehensive testing across diverse examination types.

Limitations & Considerations

As a field study using archival data, the research faced inherent constraints in experimental control. Ground truth confirmation methods varied across cases (confession, corroboration, exoneration), potentially introducing heterogeneity in the criterion variable. The study did not report specific base rates of deceptive versus truthful examinees, which significantly affects accuracy interpretation and limits comparison with other research.

Additional methodological considerations include:

• Examiner experience levels and training backgrounds were not specified, potentially confounding human performance results
• The study did not control for case difficulty or complexity, which may have varied systematically between ZCT and MGQT samples
• Quality of original chart recordings could not be standardized in retrospective blind scoring
• Sample stratification methods were not detailed, raising questions about representativeness

Practical Applications

The research provides evidence-based guidance for polygraph program administrators deciding between manual and computerized scoring systems. Agencies using predominantly ZCT formats may achieve higher accuracy with PolyScore-type algorithms, while those relying on MGQT examinations should maintain emphasis on skilled human interpretation. Many contemporary practices employ hybrid approaches—computerized initial scoring with human quality control review—that leverage the complementary strengths revealed in this study.

For polygraph examiners and quality control supervisors, the findings underscore the importance of understanding scoring scale implications. The significant accuracy loss when collapsing 7-position to 3-position decisions suggests that examiners should retain granular scoring for internal analysis while recognizing that final call decisions may sacrifice some diagnostic precision for decisional clarity. Training programs should emphasize test format-specific scoring proficiency rather than generic chart interpretation skills.