Direct English-to-Yoruba speech Translation model using Transformer with Augmented Attention Mechanism

Abstract

This study explores English-to-Yoruba speech-to-speech translation using a transformer neural network with 36 encoder-decoder pairs and 42 multi-head attention mechanisms. Unlike traditional cascaded models, this approach aligns speech directly, reducing dependency on large spectrogram datasets. Trained on a curated dataset of 200,000+ data points, the model achieved a BLEU score of 0.7, an MOS score of 2.8, and 98% translation accuracy, demonstrating its potential for practical speech translation applications.