Skintoned Emojis Are Crucial For Fostering Personal Identity And Social Inclusion In Online Communication As Ai Models Particularly Large Language Models Llms Increasingly Mediate Interactions On Web Platforms The Risk That These Systems Perpetuate Societal Biases Through Their Representation Of Such Symbols Is A Significant Concern This Paper Presents The First Largescale Comparative Study Of Bias In Skintoned Emoji Representations Across Two Distinct Model Classes We Systematically Evaluate Dedicated Emoji Embedding Models Emoji2vec Emojisw2v Against Four Modern Llms Llama Gemma Qwen And Mistral Our Analysis First Reveals A Critical Performance Gap While Llms Demonstrate Robust Support For Skin Tone Modifiers Widelyused Specialized Emoji Models Exhibit Severe Deficiencies More Importantly A Multifaceted Investigation Into Semantic Consistency Representational Similarity Sentiment Polarity And Core Biases Uncovers Systemic Disparities We Find Evidence Of Skewed Sentiment And Inconsistent Meanings Associated With Emojis Across Different Skin Tones Highlighting Latent Biases Within These Foundational Models Our Findings Underscore The Urgent Need For Developers And Platforms To Audit And Mitigate These Representational Harms Ensuring That Ais Role On The Web Promotes Genuine Equity Rather Than Reinforcing Societal Biases
“This paper presents the first large-scale comparative study of bias in skin-toned emoji representations across two distinct model classes.”