Buyer–Seller-Deception-Game Dataset: A new comprehensive dataset for facial expression based deception detection in economic contexts

This 2026 study introduces the GAME (Buyer-Seller-Deception-Game) dataset, a new audiovisual dataset specifically designed for detecting deception in economic contexts like online sales meetings. Using experimental economics methodology, 500 annotated video samples capture naturally motivated deceptive and truthful behavior in incentivized buyer-seller interactions. The research compares OpenFace-based and CNN-based Action Unit detection across controlled and challenging recording conditions.

Participants engaged in incentivized buyer-seller game interactions where sellers attempted to persuade buyers to choose specific cards, creating naturally motivated deception. Researchers extracted gaze, head pose, facial Action Units, and prosodic features using both traditional OpenFace and deep learning CNN approaches, comparing performance on the GAME dataset against a high-stakes real-life trial dataset.

Background & Context

Automated deception detection using only video and audio modalities remains a challenging problem in computer vision, with deceptive behavior being highly context dependent and shaped by factors such as the specific scenario, cultural background, and associated stakes. This 2026 study addresses a critical gap in deception detection research by creating a dataset specifically designed for economic contexts, particularly online sales interactions.

Most existing deception detection datasets either use high-stakes real-world scenarios (like courtroom trials) or low-stakes laboratory settings where participants are simply instructed to lie. Knowing the ground truth and establishing endogenous lying are paramount for training algorithms, yet many existing datasets lack full control of the data generation process or fail to properly incentivize deception. The GAME (Buyer-Seller-Deception-Game) dataset fills this void by combining experimental economics methodology with computer vision, creating naturally motivated deceptive behavior within a controlled setting.

To explore the potential of future deception detection tools in online interactions, such as virtual sales meetings, this research presents a new high-quality low-stakes dataset specifically tailored for this task. This focus on economic deception represents a novel application domain with practical implications for digital commerce, online negotiations, and remote business interactions.

Research Design & Methodology

The study employed an experimental economics approach to create ecologically valid deception scenarios. Participants engaged in incentivized online interactions, in which sellers attempted to persuade buyers to choose a specific card, creating naturally motivated deceptive and truthful behavior. This buyer-seller game structure was specifically designed to simulate online sales meetings.

The dataset currently includes 500 annotated video samples, with a planned extension to around 1000 before public release. The experimental design creates situations where sellers have private information and sometimes have financial incentives to mislead buyers, naturally generating both honest and deceptive communications without artificial instructions to lie.

For feature extraction and analysis, the researchers evaluated multiple approaches:

• Visual and audio based feature sets including gaze, head pose, facial Action-Units (AUs), and prosodic features
• OpenFace-based AU extraction for traditional feature-based analysis
• CNN-based AU predictors using deep learning approaches
• Multimodal fusion combining visual and acoustic modalities

The GAME dataset was compared against a high-stakes in-the-wild deception dataset to assess generalizability across different contexts and recording conditions.

Results & Key Findings

OpenFace based AU features performed best on the clean and controlled recordings, while CNN based AU predictors outperformed others in the more challenging dataset with lower video quality and unstable head pose. This finding has important implications for selecting appropriate feature extraction methods based on recording quality.

Key performance outcomes included:

• Superior performance of OpenFace on controlled data: The clean, stable recordings of the GAME dataset allowed traditional OpenFace AU extraction to achieve optimal results
• CNN advantages in challenging conditions: Deep learning-based AU prediction proved more robust when handling real-world variability in video quality and head movement
• Multimodal approaches slightly outperformed the best unimodal features in both cases, demonstrating the value of combining visual and acoustic cues

Low-stake datasets, including the rolling dice Experiment, presented more challenges for deception detection compared to the high-stake Real-Life trials dataset, though various evaluation measures showed deception detection performance surpassing chance levels. This contextual difference highlights how stakes influence the detectability of deception.

Discussion & Significance

This research makes several important contributions to the deception detection field. First, it demonstrates how experimental economics can inform machine learning by creating datasets with known ground truth and naturally motivated deception. Experimental economics can help provide controlled, standardized, well-documented, and incentivized experiments in laboratories with a good no-deception reputation, contributing to providing automatically annotated data sets for training algorithms.

The finding that different feature extraction methods perform optimally under different conditions has practical implications. In commercial applications with controlled video call environments, traditional methods like OpenFace may suffice. However, for real-world deployment where video quality varies, CNN-based approaches offer greater robustness. Combining multiple modalities and feature selection consistently enhances detection performance, with observed variations in expressed features across datasets affirming the importance of scenario-specific training data for effective deception detection.

The economic context of this dataset addresses a previously underserved application domain. While most deception detection research focuses on criminal justice or security screening, online commerce represents a massive real-world use case where automated deception detection could have immediate practical value.

Limitations & Considerations

The study acknowledges important methodological constraints. As a low-stakes laboratory experiment, the GAME dataset may not fully capture the emotional intensity and behavioral patterns seen in high-stakes deception. The participants know they are in an experiment, which may influence their behavior differently than actual commercial transactions.

The dataset size of 500 annotated video samples (expanding to ~1000) is relatively modest compared to typical deep learning requirements. This limitation is common across deception detection research, where labeled datasets from real-life data are scarce. The controlled laboratory setting, while allowing ground truth verification, sacrifices some ecological validity compared to actual online sales interactions.

The focus on a specific cultural and linguistic context (the study appears to use German-speaking participants based on the institutional affiliations) may limit generalizability across different cultural norms around deception and sales communication.

Practical Applications

This research has direct applications for online commerce platforms, video conferencing systems, and digital negotiation tools. The dataset and methods could inform the development of decision-support systems that flag potentially deceptive claims in online sales meetings, virtual auctions, or remote business negotiations—without requiring intrusive physiological sensors.

The dataset will be made freely available to support future research in automated deception detection, enabling the broader research community to develop and validate new approaches for economic deception detection. For practitioners, the key takeaway is that multimodal analysis combining facial expressions, gaze, head pose, and vocal cues offers the most reliable detection, with method selection depending on recording quality: OpenFace for controlled environments, CNNs for variable real-world conditions.