Resource-Aware Deep Learning Deployment for IoT–Fog Environments: A Novel BSIR and RAG-Enhanced Approach

Abstract

The proliferation of Internet of Things (IoT) devices challenges deep learning (DL) deployment due to their limited computational power, while cloud offloading introduces high latency and network strain. Fog computing provides a viable middle ground. We present a resource-aware framework that intelligently partitions DL tasks between fog nodes and the cloud using a novel Binary Search-Inspired Recursive (BSIR) optimization algorithm for rapid, low-overhead decision-making. This is enhanced by a novel module that fine-tunes deployment by analyzing memory at a per-layer level. For true adaptability, a Retrieval-Augmented Generation (RAG) technique consults a knowledge base to dynamically select the best optimization strategy. Our experiments demonstrate dramatic improvements over established metaheuristics. The complete framework boosts memory utilization in fog environments to a remarkable 99%, a substantial leap from the 85.25% achieved by standard algorithms like Genetic Algorithms (GA), Simulated Annealing (SA), and Particle Swarm Optimization (PSO). The enhancement module alone improves these traditional methods by over 13% without added computational cost. Our system consistently operates with a CPU footprint under 3% and makes decisions in fractions of a second, significantly outperforming recent methods in speed and resource efficiency. In contrast, recent DL methods may use 51% CPU and take over 90 s for the same task. This framework effectively reduces cloud dependency, offering a scalable solution for DL in the IoT landscape.