A Multimodal Framework for Assessing Mental Health Strain among IT Industry Workers Using Machine Learning

Abstract

The growing prevalence of work-related stress highlights the need for unobtrusive, continuous monitoring tools. This paper proposes a multimodal system for real-time detection of employees’ emotional states during video conferences, with a focus on stress. The framework integrates Facial Emotion Recognition (FER), speech transcription using OpenAI’s Whisper, NLTK VADER. The results from experimental show encouraging performance: Whisper gives accurate transcription, NLTK VADER sentiment analysis achieves great classification accuracy across five emotion categories, and multimodal fusion extents up to 88% accuracy in stress detection. This work founds a basis for real-time, automated valuation of employee well-being, allowing adaptive interferences and associate healthier workplace surroundings.