Paper Title: An explainable deep learning framework for intelligent documentation of silk textile heritage: cross-cultural pattern recognition and shortcut auditing on open museum collections
Authors: Yanrong Li, Lin Zhu, Penpisut Sikakaew
Corresponding Author: Lin Zhu(linzhu0928@163.com)/Thailand
Abstract
Open museum collections allow deep learning to support the documentation of historical textiles, but photographs taken over decades carry conventions that a model can learn in place of the objects. This study presents an explainable deep learning framework for the intelligent documentation of silk textile heritage and applies it to 11,388 open silk textile images from The Metropolitan Museum of Art and the Cleveland Museum of Art. Images of nine cultural origins were represented with frozen features from a residual network (ResNet-50), a vision transformer (ViT-B/16), and two foundation models, DINOv2 and CLIP. Ridge multinomial classifiers were evaluated with five repetitions of cross-validation grouped by object and near-duplicate photograph, and the ridge estimates for the main settings and backbones were reproduced in R. Gradient-weighted class activation mapping (Grad-CAM), supported by a deletion test, showed that the initial model attended in some classes to accession labels, color calibration charts, and the black-and-white format of archive photographs. After restricting the data to color photographs and trimming image borders, ResNet-50 features recognized origin with a balanced accuracy of 0.577 (95% CI 0.561 to 0.593) (chance 0.111). DINOv2 and CLIP features raised this to 0.655 and 0.641, respectively, and partial fine-tuning of ResNet-50 to 0.624. Baselines built on date, museum, image size, or background reached at most 0.389, and weaving technique was recognized with a balanced accuracy of 0.722 across four classes (0.709 after the same audit). Explainability auditing should therefore precede the use of open museum images in heritage recognition and design tools.