CloudX: A dual-layered cloud native architecture for GPU resource management using interactive notebook

Abstract

This study proposes CloudX, a two-layer framework that integrates Docker-based container provisioning with a GPU-aware scheduler embedded within Jupyter Enterprise Gateway (JEG) and dynamically allocates interactive notebook sessions using a weighted multi-resource scheduling policy. It extends JupyterHub and JupyterLab through a lightweight service discovery layer that monitors CPU, RAM, and GPU telemetry in real time. Experiments are conducted on an 8-node heterogeneous cluster consisting of three GPU-enabled (NVIDIA RTX A4000) and four non-GPU worker nodes using four workload types: GPU-intensive training (W1), memory-bandwidth-constrained inference (W2), CPU-based preprocessing (W3), and mixed heterogeneous sessions (W4). Baseline comparisons use Round Robin (RR), Least Connection (LC), and DRF-GPU, a multi-resource-aware scheduler. The evaluation showed that CloudX was able to provision new capacity in less than six minutes, increase average GPU utilization by 25%–40%, reduce makespan by 18%–23%, and maintain efficient Efflb load balancing for > 1.5 to 20 concurrent GPU sessions while consistently keeping startup latency below five seconds. On the Jain Fairness Index (JFI), CloudX achieved a score of 0.996, equal to DRF-GPU and higher than LC (0.981) and RR (0.956), indicating balanced workload distribution and GPU utilization across all cluster nodes. From a tail latency perspective, CloudX maintained P95 and P99 startup latencies of 4.7 s under a peak load of 20 concurrent sessions, remaining below the 5-second interactivity threshold. Meanwhile, CloudX recorded P99 makespan of 309.6 s, compared to 411.9 s for RR. Overall, CloudX is efficient for multi-tenant GPU sharing in cloud-based AI environments.