An Explainable Vision Transformer-Based Web Application for Medical Decision-Making: Case of Colon Cancer

Abstract

Despite the impressive research related to the application of artificial intelligence in the medical field, its adoption in real clinical settings, especially in medical decision-making, remains very limited. Therefore, our objective in this work is to develop a deep learning-based web application that supports medical decision-making. In addition to enabling efficient interaction and knowledge sharing among medical professionals, our web application also provides an accurate prediction system for colon cancer. This system is based on a Vision Transformer (ViT) deep learning model, which is characterized by its attention mechanism that ensures rich contextual representations and captures long-distance dependencies within images. To promote physicians’ confidence in the intelligent system, our approach provides clear visual explanations of the ViT predictions using the XAI method LIME. The validation of our model was conducted on a merged dataset of LC25000 and DigestPath images, with an additional external evaluation on the EBHI-Seg dataset. The experimental results demonstrate the competitive performance of the proposed ViT-based approach, which achieved perfect accuracy on the LC25000 dataset, 94.90% on the challenging merged dataset, and a robust accuracy of 92.17% on the unseen EBHI-Seg dataset. This remarkable performance makes the model suitable for real-world clinical applications.