UniDis: Universal Distillation for Efficient and Personalized Pathology Diagnosis

Recent advances in pathology foundation models have demonstrated remarkable capabilities in feature representation through large-scale pre-training. Despite their success, these models often suffer from substantial computational overhead, posing challenges for deployment in practical clinical settings with resource limitations. In addition, their generalizability is limited by distributional biases inherent in the pretraining datasets, often resulting in suboptimal performance when transferred to rare or underrepresented cancer types. To overcome these limitations, we introduce Universal Distillation (UniDis), a novel framework that distills knowledge from multiple large foundation models into a lightweight model with significantly fewer parameters. UniDis supports efficient and personalized fine-tuning on private, institution-specific datasets, enabling tailored adaptation to various downstream cancer-type classification tasks. Extensive experiments on the TCGA-Lung and CPTAC-Lung datasets demonstrate that UniDis achieves state-of-the-art performance.