Interpretable Truth Detection: Interpretable Bimodal Within-Subject Truth and Deception Detection Models

Background & Context

Web-based questionnaires are ubiquitous in research, yet their validity is frequently undermined by deceptive responses driven by social desirability bias or intentional misrepresentation. Traditional approaches to identifying truthful answers in web-based questionnaires can drastically increase the validity of collected data, but most existing methods rely on single modalities that may not work equally well across all individuals.

Approaches based on cognitive load theory have been successfully applied, though they often rely on measuring cognitive load with one modality such as changes in pupil diameter. The fundamental premise is that deception requires greater cognitive effort than truth-telling, manifesting in measurable physiological and behavioral changes. This 2025 study addresses a critical limitation in prior cognitive load-based deception detection: the lack of interpretability and the failure to account for individual differences in how modalities reveal deception.

The research extends earlier work by the same authors (AnswerTruthDetector, 2023) by introducing an interpretable framework that automatically generates human-readable visualizations explaining why an answer was classified as truthful or deceptive—a crucial feature for research administrators who need to understand and justify data exclusions.

Research Design & Methodology

The study presents a bimodal approach that combines two modalities: pupil diameter and mouse movements. The system was designed as a within-subject model, meaning each participant serves as their own baseline, accounting for individual differences in physiological and behavioral responses.

Key design features include:

• Dual modality measurement: Eye-tracking captured pupil diameter changes as an index of cognitive load; mouse-tracking recorded spatial and temporal movement features including area under trajectory curve, maximum deviation, and coordinate flips
• Per-participant model training: Rather than applying universal thresholds, the system generates individualized truth-detection models for each participant
• Automated weighting system: The algorithm automatically determines optimal weighting factors for different modalities and features based on individual performance
• Controlled deception task: Participants completed web-based questionnaires where they were explicitly instructed to answer some questions truthfully and others deceptively, providing ground-truth labels for validation

The system automatically generates truth scores and weighting factors of the different modalities and produces human-readable graphs consisting of truth-score-graphs for each feature and optimal weightings of features and modalities. This interpretability component distinguishes the approach from "black box" machine learning methods.

Results & Key Findings

The study visualized truth scores for requested truthful and requested deceptive items using box plots and violin plots, demonstrating clear differentiation between truthful and deceptive responses. The bimodal system successfully distinguished between instructed truthful and deceptive answers across participants.

Critical findings include:

• Individual variability accommodation: The per-participant weighting system allowed the model to downweight modalities that performed poorly for specific individuals (e.g., eye tracking may fail for some participants due to technical issues)
• Cross-feature consistency: Even though model training was done per-participant, very similar generated graphs were observed, strengthening the assumption that universal as well as group-based models can be derived
• Interpretable outputs: Study administrators could examine truth scores for individual features, modalities, and combined scores, with visual graphs showing the distribution of cognitive load indicators for each question
• Enhanced diversity and inclusion: By weighting modalities individually, the system reduced the number of participants who would need to be excluded from studies due to poor performance on a single modality

The within-subject design proved effective at capturing the relative differences between truthful and deceptive cognitive load within individuals, rather than relying on absolute thresholds that may vary substantially across the population.

Discussion & Significance

This research represents an important evolution in automated deception detection for survey research, addressing two major limitations of prior work: lack of interpretability and failure to accommodate individual differences. The interpretable framework allows research administrators to understand not just which answers are flagged as potentially deceptive, but why, based on transparent physiological and behavioral indicators.

The adaptive weighting system is particularly significant because it acknowledges that no single modality works uniformly across all individuals. Eye tracking does not work equally well for every individual, and providing weightings for each modality enables different weights for different study participants, reaching higher diversity by decreasing the possible number of participants to be excluded from a study. This increases the inclusivity and generalizability of research findings.

Future analysis comparing per-participant models and graphs will be highly interesting, including comparison of weightings to determine how well some modality or feature works for some group of participants compared to other groups. The convergence of individually-trained models toward similar truth-detection graphs suggests underlying universal patterns in cognitive load during deception, even when individual baseline physiology varies substantially.

Limitations & Considerations

The study employed instructed lying in a laboratory-like web questionnaire setting, which differs from spontaneous deception in high-stakes real-world contexts. While instructed lying provides ground-truth labels necessary for validation, the cognitive processes may differ from naturalistic deception where participants spontaneously choose to misrepresent information.

The system requires specialized hardware (eye-tracking devices and mouse input) that may not be available in all research settings. Eye-tracking in particular can be sensitive to environmental conditions, lighting, and individual factors like glasses or contact lenses. The per-participant training approach requires each participant to answer both truthful and deceptive questions, which may not be feasible in all research designs where ground truth is unknown.

The paper does not report overall classification accuracy rates or compare performance to other deception detection methods, making it difficult to assess the system's effectiveness relative to alternative approaches. The system was evaluated in a systematic user study, but specific quantitative performance metrics were not detailed in the available materials.

Practical Applications

The InterpretableTruthDetection system offers practical value for researchers conducting web-based surveys on sensitive topics where response validity is crucial—such as studies on illegal behavior, stigmatized conditions, substance use, or politically sensitive attitudes. By providing transparent, interpretable classifications rather than opaque algorithmic judgments, the tool enables researchers to make informed decisions about data quality and participant exclusions with defensible justification.

The adaptive weighting approach is particularly valuable for diverse participant populations where physiological monitoring may work differently across groups. Rather than excluding participants with poor eye-tracking data, the system can rely more heavily on mouse-tracking features for those individuals. This enhances the external validity and generalizability of research findings by maintaining more representative samples. The human-readable visualizations also support post-hoc validation and quality control, allowing research teams to review flagged responses and apply expert judgment where algorithmic classification is uncertain.