Reproducibility in lie detection research: A case study of the cue called complications

Abstract

Purpose
This review examined reproducibility in verbal lie detection research, wherein studies typically involve coding statements to identify deception cues. Such coding is prone to analytic flexibility that can invite false positives. I focused on the cue called complications as a case study. The variable emerged in the literature simultaneously with the availability of open science resources—providing a reasonable expectation that the relevant materials would be archived in accessible repositories if not in the publication.

Methods
I reviewed 30 relevant publications to assess whether complications research is amenable to auditing.

Results

The findings indicated sufficient consistency in the definitions of complications and little ambiguity regarding what the variable denotes. Additionally, numerical estimates indicated that the extant results in the literature might be replicable—but with a significant caveat. Such replicability entirely depends on acquiring the coding protocols and anonymized raw data of published studies. However, that critical information is
not
publicly available. I discuss the ramifications of this barrier to reproducibility: it prevents the auditing of published findings, which allows explaining null findings away with post hoc explanations that depend on inaccessible information.

Conclusions
At a minimum, journal editors and reviewers must insist on the codebooks of coding protocols. Providing the corresponding anonymized raw data should also be a requirement unless specific obstructions like grant agreements prevent data sharing. The nature of verbal lie detection research necessitates this policy.

Background & Context

Verbal lie detection research has grown rapidly over the past two decades, moving away from discredited nonverbal cues toward more diagnostic verbal content analysis. Studies in this domain typically involve coding statements to identify deception cues through a process prone to analytic flexibility that can invite false positives. Yet this methodological vulnerability has rarely been scrutinized from a reproducibility perspective.

Neequaye's 2026 review focuses on "complications"—a verbal cue believed to appear more frequently in truthful statements—which emerged in the literature simultaneously with the availability of open science resources, providing a reasonable expectation that relevant materials would be archived. This timing makes complications an ideal case study: if even recently developed cues lack reproducible documentation, the field faces a systemic transparency problem.

The study arrives at a critical juncture when psychology grapples with replication crises. This investigation should serve as a cautionary tale for anyone who edits, conducts, or consumes lie detection research, as coding verbal data is commonly used across Psychology and Law research, and the reproducibility call to action is needed across the entire field.

Research Design & Methodology

Neequaye reviewed 30 publications that met inclusion criteria, with 45 excluded because they either did not examine complications, were not about lie detection, or were retracted. The investigation employed multiple complementary analyses to assess reproducibility from different angles.

The methodology centered on extraction and thematic analysis of key elements:

• Definitions of complications and justifications explaining why the variable can be a veracity cue
• Information about coding processes and data availability to investigate whether independent analysts can reproduce original findings with ease
• Thematic analysis using strategies similar to Neequaye and Mac Giolla (2022), involving initial reading, code generation to saturation point, refinement, and application separately for definitions and justifications

The aim was to examine consistency of definitions and justifications, as consistent definitions provide an unambiguous benchmark to assess the presence or absence of complications. The author also examined whether coding protocols were preregistered and whether raw data were publicly available.

Results & Key Findings

Sufficient consistency exists in definitions of complications with little ambiguity, and numerical estimates indicated that extant results might be replicable—but this optimistic surface conceals a fundamental problem.

The critical findings reveal a severe reproducibility barrier:

• Replicability entirely depends on acquiring the coding protocols and anonymized raw data of published studies—yet that critical information is not publicly available
• The lack of accessibility to coding protocols and raw data prevents stakeholders from auditing published findings, which allows explaining null findings away with post hoc explanations that depend on inaccessible information
• Without actionable coding protocols to guide stakeholders, any independent operationalization of complications remains susceptible to post hoc explanations

The study exposes a paradox: while the conceptual definition of complications appears consistent across publications, the operational details necessary for verification remain locked away. Coding verbal cues is known to be prone to analytic flexibility, yet the field has not implemented safeguards against this vulnerability.

Discussion & Significance

Neequaye's findings illuminate a pernicious dynamic in verbal deception research. The absence of publicly available coding protocols creates an asymmetry of information: original researchers can dismiss failed replications by invoking coding nuances that no one else can verify. This effectively shields findings from falsification—the antithesis of scientific practice.

A popular recommendation in Psychology and Law research is that verifiability is a hallmark of credibility; the field must practice what it preaches by ensuring research is verifiable through reproducibility. This irony—that researchers who study credibility assessment have not made their own work credibly verifiable—carries both methodological and ethical weight.

The study's implications extend beyond complications to the entire verbal deception domain. If a recently emerged cue studied during the open science era lacks reproducible documentation, older research traditions likely face even greater transparency deficits. The findings suggest that meta-analytic estimates of effect sizes in this literature may be unreliable, as the underlying studies cannot be independently audited for coding consistency or analytic flexibility.

Limitations & Considerations

The review focuses exclusively on complications as a single case study, which limits generalizability to other verbal cues. However, this narrow focus also provides depth—if reproducibility problems exist for a well-defined, recently developed cue, concerns likely extend to the broader literature.

The study examines only whether materials are publicly available, not whether researchers would share them upon request. Some data-sharing barriers may result from legitimate privacy concerns, institutional restrictions, or grant agreements rather than intentional opacity. Specific obstructions like grant agreements might prevent data sharing, though these situations should be explicitly documented.

Finally, establishing that definitions are consistent does not guarantee that researchers apply those definitions consistently in practice—that question can only be answered through access to actual coding protocols and inter-rater reliability assessments using shared materials.

Practical Applications

For journal editors and reviewers, Neequaye delivers clear recommendations: At a minimum, insist on the codebooks of coding protocols. Providing corresponding anonymized raw data should also be a requirement unless specific obstructions prevent sharing, and such information must remain accessible to the scientific community post-publication, as the nature of verbal lie detection research necessitates this policy.

For practitioners and consumers of this research, the findings warrant appropriate skepticism. Until coding protocols become transparent and data auditable, claims about verbal cue diagnosticity should be treated as provisional. Legal decision-makers should be particularly cautious about admitting expert testimony based on research that cannot be independently verified. For researchers, the path forward involves embracing transparency not as an optional virtue but as a methodological necessity—preregistering coding schemes, archiving materials in accessible repositories, and conducting adversarial collaborations to test the robustness of coding decisions.