Suen, H. Y., Hung, K. E., Liu, C., Su, Y. S*., & Fan, H. C. (2024).
IEEE Transactions on Computational Social Systems, Accepted. (*Corresponding Author) (SCIE)
Abstract
Whether an interviewee’s honest and deceptive responses can be detected by the signals of facial expressions in videos has been debated and called to be researched. We developed deep learningmodels enabled by computer vision to extract the temporal patterns of job applicants’ facial expressions and head movements to identify self-reported honest and deceptive impression management (IM) tactics from video frames in real asynchronous video interviews. A 12- to 15-min video was recorded for each of the N = 121 job applicants as they answered five structured behavioral interview questions. Each applicant completed a survey to self-evaluate their trustworthiness on four IM measures. Additionally, a field experiment was conducted to compare the concurrent validity associated with self-reported IMs between our modeling and human interviewers. Human interviewers’ performance in predicting these IMmeasures from another subset of 30 videos was obtained by having N = 30 human interviewers evaluate three recordings. Our models explained 91% and 84% of the variance in honest and deceptive IMs, respectively, and showed a stronger correlation with self-reported IMscores compared to human interviewers.