GENERATIVE BRAIN-TO-TEXT DECODING: INTEGRATING NEURAL REPRESENTATIONS WITH LARGE LANGUAGE MODELS (LLMs)

Abstract

Rebuilding language directly from brain activity is a cutting-edge topic in neuroscience and Artificial Intelligence (AI). Conventional decoding approaches have relied on classification methods that translate neural signals into discrete words or semantic categories. However, these techniques often yield rigid and context-limited outputs. We propose a generative framework that leverages recent advances in Large Language Models (LLMs) and neural representation learning to translate brain activity into coherent natural language. This study synthesises evidence from functional Magnetic Resonance Imaging (fMRI) and Electrocorticography (ECoG)-based decoding work, rather than introducing new empirical results. It emphasises the promise of transformer-based architectures and multimodal embedding spaces such as GPT and CLIP to serve as bridges between neural and linguistic representations. The framework exploits alignment between brain-derived semantic embeddings and generative model representations to flexibly and interpretably reconstruct open-ended language. The paper, based on literature review and theoretical synthesis, argues that a generative paradigm improves the scalability, semantic precision and cross-modal decoding as compared to traditional classification approaches. Finally, it considers the theoretical, ethical and practical implications of such models for non-invasive Brain-Computer Interfaces (BCIs), cognitive neuroscience and assistive communication systems.