Paper Title: A multimodal deep learning model integrating visual features and early comment sentiment for predicting audience acceptance of Tibetan-motif tourism advertisements on Xiaohongshu
Authors: Lu Hong, Mohd Nor Shahizan Ali, Mohd Azul Mohamad Salleh
Corresponding Author: Mohd Nor Shahizan Ali (shahizan@ukm.edu.my)/Malaysia
Abstract
Tibetan art’s ornate designs are widely used in social media promotions in the tourism industry, but predicting consumer reactions to ethnic advertising remains difficult. For some people, the elaborate ornamentation symbolizes cultural authenticity; for others, it seems staged. Most existing models analyze imagery and user commentary in isolation. Whether these two sources offer complementary predictors of audience response therefore remains unclear. Here we present a gated bidirectional cross-attention model to predict thirty-day audience acceptance of Tibetan-motif tourism advertisements on Xiaohongshu from visual representations and early twenty-four-hour comment sentiment. We first train the motif-recognition backbone and two interpretable descriptors (motif density and circular hue chromatic conformity) on 19,142 archived pattern images, then transfer them to 4,860 promotional posts using a month-stratified partitioning scheme. Acceptance is treated as a three-category ordinal target derived from engagement residuals adjusted for covariates. Sentiment measures calculated from 149,300 preliminary comments feed into an entropy-dependent adaptive gate that shifts the balance toward visual information as discussion threads become scarce. Our proposed model achieves 0.754 accuracy and 0.734 macro-F1 in experiments on hand-crafted, unimodal, concatenation, CLIP, and ViLT baselines. It beats standard late fusion by 3.5 and 3.7 percentage points on these two metrics and cuts ordinal Mean Absolute Error (MAE) by 12.3 percent relative (from 0.318 to 0.279). Cross-modal complementarity is validated by Accumulated Local Effects (ALE) and Ablation Analyses, with chromatic conformity and sentiment polarity increasing acceptance, while motif density exhibits an inverse-U relationship.
