Enhance big data security based on HDFS using the hybrid approach

Abstract

Hadoop has emerged as a prominent open-source framework for the storage, management, and processing of extensive big data through its distributed file system, known as Hadoop distributed file system (HDFS). This widespread adoption can be attributed to its capacity to provide reliable, scalable, and cost-effective solutions for managing large datasets across diverse sectors, including finance, healthcare, and social media. Nevertheless, as the significance and scale of big data applications continue to expand, the challenge of ensuring the security and safeguarding of sensitive data within Hadoop has become increasingly critical. In this study, the authors introduce a novel strategy aimed at bolstering data security within the Hadoop storage framework. This approach specifically employs a hybrid encryption technique that leverages the advantages of both advanced encryption standard (AES) and data encryption standard (DES) algorithms, whereby files are encrypted in HDFS and subsequently decrypted during the map task. To assess the efficacy of this method, the authors performed experiments with various file sizes, benchmarking the outcomes against other established security measures.