Exploring Vision-Based Features for Detecting Deception in Well-Being: A Cross-Domain Comparison

Deception detection has been extensively studied using vision-based features in domains such as crime, finance, and social interaction. However, little attention has been given to how deception manifests visually in self-reported well-being-a critical area for behavioral health, where inaccurate reporting may affect treatment outcomes and the therapeutic alliance. While clinicians often rely on visual cues such as gaze, facial expressions, and body language to assess deception, these cues remain underutilized in AI-based deception detection in wellbeing scenarios. This study explores vision-based features of deception in the well-being domain and compares them with those from three other domains: biography, academics, and crime. Using mock interview data, we extract facial landmarks, body gestures, and facial action units (AUs) using four feature selection methods. We then visualize and analyze the spatial distribution of features associated with truthful and deceptive responses. Results show that well-being features are generally fewer and more localized-particularly around the nose ridge-with unique presence of eye landmarks and limited hand gestures. In contrast, biography and academics show broader facial and body engagement, while crime displays no differentiation between truth- and deception-related features and lacks emotional AU combinations. AUs associated with joy (AU 6 and AU 12) appear consistently across well-being, academics, and biography, suggesting some domain-agnostic cues. Overall, our findings indicate that most visual features relevant to deception are domain-specific. This highlights the importance of contextaware approaches in deception modeling and supports the development of more reliable, human-centered AI tools for wellbeing assessment and mental health applications.

Mock interview study with 50+ participants (primarily ages 18-25) who provided truthful and deceptive responses about well-being, biography, academics, and crime. Researchers extracted facial landmarks, body gestures, and facial action units using four feature selection methods, then visualized and compared spatial distributions of features across domains.

Background & Context

Deception detection has been extensively studied using vision-based features in domains such as crime, finance, and social interaction. However, the application of automated deception detection to mental health and well-being contexts remains largely unexplored, despite its potentially critical implications for therapeutic outcomes and treatment efficacy.

Deception in mental health settings can undermine therapeutic relationships, compromise treatment efficacy, and impact patient outcomes. Yet, research shows that mental health clinicians often perform no better than chance at detecting deceptive behavior in therapy. This creates a significant gap where AI-assisted tools could potentially support clinical practice, but the transferability of deception detection models trained on high-stakes domains like criminal interrogation to sensitive contexts like well-being remains unexamined.

This study addresses a fundamental question in affective computing and behavioral analysis: do the visual signatures of deception—facial expressions, body gestures, and micro-expressions—manifest similarly across different contexts? Understanding domain-specific versus domain-agnostic features is essential for developing reliable, context-aware deception detection systems that could be ethically deployed in mental health applications.

Research Design & Methodology

Over 50 participants (45 aged 18–25; 8 aged 30+) completed three video-recorded mock interviews involving deceptive and truthful responses to questions about personal background, job satisfaction, and well-being. The researchers interviewed each participant to confirm which answers were deceptive, creating a labeled dataset for analysis.

The study explores vision-based features of deception in the well-being domain and compares them with those from three other domains: biography, academics, and crime. The research team extracted three types of visual features from the recorded interviews:

• Facial landmarks — spatial positioning of key facial points
• Body gestures — hand movements and body positioning
• Facial Action Units (AUs) — specific muscle movements based on the Facial Action Coding System

Four different feature selection methods were applied to identify the most discriminative features for each domain. The researchers then visualized and analyzed the spatial distribution of features associated with truthful versus deceptive responses, comparing patterns across the four domains to identify domain-specific and domain-agnostic visual cues.

Results & Key Findings

The analysis revealed striking domain-specific differences in how deception manifests visually, with well-being deception showing unique characteristics compared to other contexts.

Well-being domain features were notably sparse and localized:

• Generally fewer features distinguished truth from deception compared to other domains
• Features concentrated particularly around the nose ridge
• Unique presence of eye landmarks not observed in other domains
• Limited hand gestures compared to biography and academic domains

Biography and academic domains showed broader facial and body engagement, with more distributed features across the face and greater involvement of hand gestures. The crime domain presented a surprising finding: no clear differentiation between truth- and deception-related features and a notable lack of emotional AU combinations.

Cross-domain patterns emerged for certain facial action units: AUs associated with joy (AU 6 - cheek raiser and AU 12 - lip corner puller) appeared consistently across well-being, academics, and biography domains, suggesting some domain-agnostic cues related to positive affect expressions may serve as deception indicators across contexts.

Discussion & Significance

The finding that well-being deception exhibits fewer and more localized visual features than other domains has important implications for both research and practice. The concentration of features around the nose ridge and eyes suggests that deception about one's mental or physical state may involve more subtle, microexpressive cues rather than the broader facial and bodily displays associated with biographical or academic lies.

As the lead author noted in related interviews, "lies across multiple contexts is challenging." This research provides empirical evidence for this claim, demonstrating that visual deception signatures are largely domain-dependent. The unexpected finding that crime-related deception showed no clear feature differentiation challenges assumptions about high-stakes contexts producing stronger deception cues and may reflect differences in interview structure or participant motivation between simulated crime scenarios and personally relevant topics.

The consistent appearance of joy-related AUs (6 and 12) across multiple domains is particularly intriguing. These action units are associated with smiling and may represent a universal deceptive strategy of masking negative emotions or discomfort with positive facial expressions—a phenomenon observed across cultures and contexts. However, the overall results emphasize that most visual features relevant to deception are domain-specific, supporting the need for context-aware AI models rather than one-size-fits-all deception detectors.

Limitations & Considerations

The study employed mock interviews in a controlled laboratory setting, which may not fully capture the dynamics of real-world deception in therapeutic contexts. Most existing research focuses on high-stakes legal contexts, limiting its applicability to mental health settings. While this study advances the field by examining well-being specifically, the laboratory paradigm still differs from genuine clinical encounters.

The sample was predominantly young adults (ages 18-25), with limited representation of older participants. Deception strategies and visual cues may vary across age groups and cultural backgrounds. Additionally, the study examined mock deception scenarios where participants were instructed to lie, which may differ psychologically from spontaneous deception about actual well-being concerns. The relatively small sample size (50+ participants) also limits the generalizability of findings for training robust machine learning models.

Practical Applications

As noted by the research team, "Understanding human behavior is what allows us to design AI that helps rather than harms." This study's findings suggest that any AI-assisted deception detection tools for mental health applications must be trained specifically on well-being data rather than repurposed from criminal justice or security screening models. The unique visual signature of well-being deception—characterized by subtle, localized features—requires specialized detection algorithms and feature extraction approaches.

For clinicians and researchers developing mental health technologies, the findings highlight critical ethical considerations: the sparse, subtle nature of well-being deception cues may make detection inherently uncertain, raising questions about appropriate confidence thresholds and the risks of false positives in therapeutic settings. Rather than replacing clinical judgment, such tools might best serve as decision support systems that alert clinicians to potential inconsistencies requiring compassionate follow-up rather than definitive deception judgments. The domain-specificity of deception cues also underscores the need for extensive validation in each application context before deployment.