Facial Emotion Recognition (FER) is crucial for applications in human-computer interaction and mental health. However, existing FER datasets often suffer from limitations such as class imbalance and limited diversity, hindering the development of robust models. This paper introduces EmoSet, a novel dataset designed to address these challenges. Comprising 29K images across seven basic emotions (anger, disgust, fear, happiness, neutral, sadness, and surprise), EmoSet integrates diverse sources, including movies, TV shows, GIFs, internet images, and AI-generated content, ensuring balanced representation across categories. We evaluate EmoSet using a Vision Transformer (ViT) architecture, comparing its performance against established datasets (FER2013, RAF-DB, and RAVDESS). Our results demonstrate EmoSet’s balanced performance, particularly excelling in challenging emotions like disgust, and significantly improved accuracy when combined with existing datasets (e.g., achieving 65.27% on AffectNet and 69.32% on FER2013). These findings highlight EmoSet’s contribution to developing more generalizable and robust FER systems.