MARATTO

article · Artificial Intelligence Review

A comprehensive investigation of multimodal deep learning fusion strategies for breast cancer classification

202481 citationsOpen accessMohammed V University

In plain language

Breast cancer research uses a wide variety of information types, including radiological imaging, clinical records, histology, and gene expression data. Relying on a single data source is rarely sufficient for complex medical evaluations. Integrating multiple data streams through multimodal deep learning fusion can enhance diagnostic and predictive performance. This systematic review analyses 47 studies published between 2018 and 2023 across six digital libraries, examining the architectures, models, datasets, and integration strategies used in the field. The findings indicate an emergence of novel data combinations that broaden the capabilities of predictive tools across screening, diagnosis, and prognosis. Combining diverse modalities consistently boosts model accuracy compared to traditional methods. However, key barriers remain, notably the requirement for larger reference datasets, the need for effective ensemble methods, and difficulties in interpreting multimodal model outputs.

Key takeaways

  • Combining multiple data sources such as medical images, clinical records, and gene expression improves the accuracy of breast cancer classification.
  • Recent research has introduced previously unexplored modality pairings that expand predictive modelling across screening, diagnosis, and prognosis.
  • Multimodal deep learning surpasses traditional machine learning by autonomously learning intricate patterns across varied datasets.
  • Major ongoing challenges include the need for larger datasets, integration of ensemble techniques, and model interpretability.

Why it matters

Breast cancer is complex, and single tests often provide an incomplete picture. By reviewing how modern artificial intelligence combines images, genetic details, and clinical histories into unified assessments, this work highlights pathways toward more accurate cancer detection and prognosis. Clarifying the current limitations also guides future development toward safer, more explainable diagnostic tools.

Commercialisation angle

The work synthesises research relevant to developers of clinical decision support systems and medical diagnostic software targeting breast cancer screening, diagnosis, and prognosis. As a literature review of academic studies published up to 2023, the underlying technologies remain largely in the research and development phase. Commercial deployment will require overcoming stated hurdles, notably access to larger datasets and the development of interpretable multimodal models suitable for clinical use.

AI-generated from the published abstract. Always read the original work before citing.

Abstract

In breast cancer research, diverse data types and formats, such as radiological images, clinical records, histological data, and expression analysis, are employed. Given the intricate nature of natural phenomena, relying on the features of a single modality is seldom sufficient for comprehensive analysis. Therefore, it is possible to guarantee medical relevance and achieve improved clinical outcomes by combining several modalities. The presen study carefully maps and reviews 47 primary articles from six well-known digital libraries that were published between 2018 and 2023 for breast cancer classification based on multimodal deep learning fusion (MDLF) techniques. This systematic literature review encompasses various aspects, including the medical modalities combined, the datasets utilized in these studies, the techniques, models, and architectures used in MDLF and it also discusses the advantages and limitations of each approach. The analysis of selected papers has revealed a compelling trend: the emergence of new modalities and combinations that were previously unexplored in the context of breast cancer classification. This exploration has not only expanded the scope of predictive models but also introduced fresh perspectives for addressing diverse targets, ranging from screening to diagnosis and prognosis. The practical advantages of MDLF are evident in its ability to enhance the predictive capabilities of machine learning models, resulting in improved accuracy across diverse applications. The prevalence of deep learning models underscores their success in autonomously discerning complex patterns, offering a substantial departure from traditional machine learning approaches. Furthermore, the paper explores the challenges and future directions in this field, including the need for larger datasets, the use of ensemble learning methods, and the interpretation of multimodal models.

Research topics

  • AI in cancer detection
  • Brain Tumor Detection and Classification
  • Gene expression and cancer classification

Read the original research

This page summarises published work. The authoritative version sits with the publisher.

DOI: 10.1007/s10462-024-10984-z

Is something wrong with this record? Report it or request removal.

Discussion

Discuss this research

Have you built on this work, tried to replicate it, or seen it applied in practice? Share what you know. Verified researchers and MARATTO™ domain experts can open a discussion, and any member can reply. Contributions are reviewed before they appear.

No discussion yet. Open the first thread.