A Validation Study of Polygraph Examiner Judgments

Pioneering field validity study using a panel of attorneys to establish ground truth, finding 92% agreement between polygraph results and panel verdicts.

Field study using independent panel of attorneys reviewing case evidence to establish ground truth criterion for polygraph accuracy assessment.

Bersh's pioneering field study introduced an innovative ground truth method — using a panel of attorneys to independently evaluate case evidence and establish whether suspects were truthful or deceptive. The finding of 92% agreement between polygraph results and panel verdicts provided strong early field validation.

The attorney panel method addressed the ground truth problem in field studies, providing a more rigorous criterion than confession-based methods alone.

Background & Context

Philip J. Bersh's 1969 field study emerged during a pivotal period in polygraph history. The federal government conducted around 20,000 polygraph exams in 1963 alone, yet the scientific validity of these examinations remained largely unproven. Field studies of polygraph accuracy faced a fundamental challenge: establishing "ground truth" about guilt or innocence without relying solely on confessions, which could be influenced by the polygraph examination itself.

Bersh introduced an innovative solution that would influence polygraph research for decades to come. Rather than using confessions or judicial outcomes as the criterion for actual guilt or innocence, the panel consisted of four Judge Advocate General (JAG) Attorneys who reviewed all available case evidence independently. This marked one of the first rigorous attempts to validate polygraph accuracy in real criminal investigations using an objective external standard.

Research Design & Methodology

Cases were initially drawn at random from a pool of criminal investigations conducted by the three military services over a period of 3 years (1963-66). The original sample included 323 cases, with 80 cases (one-quarter) eliminated because of insufficient evidence. Any cases judged "indeterminate" by the original polygraph examiner were also excluded from analysis.

The ground truth establishment process was rigorous. Each JAG attorney made an independent decision based upon perusal of case files from which all polygraph references were deleted. Attorneys were instructed to eliminate files lacking sufficient evidence and to disregard legal technicalities. Only those cases which resulted in a unanimous decision by the initial JAG panel were retained in the validation sample.

The examiners in the Bersh study used either GQT (a version of R/I) or the zone of comparison (ZOC) technique. Importantly, the examiner's judgment was considered the end product of his complete interrogation of a suspect, meaning the study evaluated the entire polygraph examination process, not just chart interpretation alone.

Results & Key Findings

The headline finding showed agreement was 92.4% when comparing polygraph examiner judgments to unanimous JAG panel decisions. However, this impressive overall figure concealed substantial variation across different testing techniques and outcome categories.

Breaking down the results by technique and outcome revealed critical differences:

• Correct guilty detections ranged from 70.6 percent in one condition of the Bersh study, representing one of the lower detection rates among field studies
• Correct innocent detections reached as high as 94.1 percent in one condition of the Bersh study, demonstrating strong specificity in certain applications
• Agreement decreased significantly to 74.6% when the criterion was a majority JAG panel decision rather than unanimous verdicts
• False negatives reached 29.4 percent in the Bersh study, indicating substantial rates of guilty suspects being classified as truthful

The general question test (GQT) portion of the Bersh study also detected deceptiveness and nondeceptiveness better than chance, providing early validation for the Relevant/Irrelevant technique in addition to control question methods.

Discussion & Significance

Bersh's attorney panel method represented a methodological breakthrough in polygraph field research. The method submits all of the case information except the polygraph results to a panel of experts who are asked to make judgments of guilt or innocence using the available information and disregarding legal technicalities. Accuracy of the polygraph tests is then determined by comparing the test outcomes to the composite judgments of the panel. This approach avoided the circular reasoning inherent in confession-based studies.

The study became one of the foundational works in polygraph validation research. The OTA included Bersh (29) among the 10 field studies that met minimal criteria for methodological adequacy when evaluating polygraph scientific validity for Congress. The wide range of accuracy rates within the study highlighted an important reality: polygraph accuracy depends heavily on the specific technique employed, the quality of evidence available, and the nature of examinees tested.

However, critics noted limitations in the panel approach. The problem with this method is the fallibility of panel judgments that are based on a vague evaluation of evidence of unknown and variable quality and quantity. The panel's decisions, while independent of polygraph results, still represented human judgment rather than absolute ground truth.

Limitations & Considerations

When random sampling is used (as in Bersh (29)), high rejection rates of cases selected for analysis create other sample bias problems. The elimination of indeterminate polygraph results and cases with insufficient evidence meant the final sample represented only a subset of actual field polygraph examinations—potentially the clearest, most straightforward cases.

The requirement for unanimous panel agreement further restricted the sample to cases where guilt or innocence appeared relatively unambiguous to legal professionals reviewing all available evidence. The two studies' results are strikingly different when comparing Bersh to Barland and Raskin's later panel study, suggesting that examiner skill, case selection, and panel composition significantly influenced outcomes. The study's focus on military criminal investigations may also limit generalizability to civilian contexts or other polygraph applications.

Practical Applications

The study supports the use by the military services of polygraph examiner judgments as an aid in determining whether to continue or to terminate criminal investigations. The high agreement rate with attorney panels suggested polygraph results could provide valuable investigative guidance, even if not suitable for courtroom evidence. The variation in accuracy across guilty versus innocent suspects underscored the importance of treating polygraph results as one investigative tool among many.

For modern practice, Bersh's findings emphasize that overall accuracy rates can mask important differences in test performance across different examinees and techniques. The study's demonstration that different polygraph formats (GQT versus ZOC) produced varying detection rates reinforces the need for technique-specific validation and examiner expertise in selecting appropriate methods for each investigation.