Challenges in Medical Image Segmentation: Insights from Transformer-CNN Integration

Abstract

After the success of Transformers in the field of computer vision, via the VIT architecture, several hybrid approaches have emerged combining Transformers and CNN to leverage the strengths of both. This hybrid architecture marks a significant advance in medical diagnostics, with numerous research projects highlighting its potential to improve diagnostic accuracy thereby enhancing patient care outcomes. The primary aim of this paper is to present these advances by focusing on the segmentation of medical images, we provide the fundamentals of transformer-based architectures and we will present the different ways to combine transforms with CNNs while showing the importance of each. In addition, we categorize the various challenges faced by this type of architecture and present the specific architectures that address each issue. This paper distinguishes itself from other reviews by specifically focusing on the integration methods of transformers for segmentation tasks, encompassing both spatial and frequency domains. In conclusion, we highlight the major challenges and explore various potential directions for future research.