Multimodal Emotion Recognition in Cinema: A "Synthetic" Human- and ChatGPT- Driven Analysis of Emotional Dynamics in Sophie's Choice

Recently, growing research has focused on analyzing the emotional impact of films, driven by the potential to enhance audience engagement, improve content recommendation systems, and deepen narrative understanding. Advances in artificial intelligence (AI) have opened new avenues for automated emotion recognition across various domains. This study explores the integration of AI, specifically ChatGPT, with human expertise to analyze complex emotional dynamics in cinema. Using a key scene from Sophie's Choice, we investigated ChatGPT's ability to recognize and interpret emotions through a multimodal approach combining visual, auditory, and textual inputs. Two human judges segmented 56 sequences based on characters' facial expressions, and AI tools were employed for emotion analysis of facial recognition, dialogues, audio cues, and cinematographic elements, through an approach that simulated a real-world analytical workflow and the output of several libraries. Results highlight ChatGPT's ability in detecting nuanced emotional cues, such as micro-expressions and contextual elements, while acknowledging challenges in low-visibility conditions. This study demonstrates AI's capability to decode emotions and narrative dynamics and highlights the importance of multimodal frameworks in improving AI-driven emotion recognition. Future research should examine fully autonomous AI analyses and integrate subjective viewer experiences to bridge the gap between AI interpretations and human emotional responses.