A Hardware-Aware Analysis of PTQ and QAT Quantized CNNs for Object Detection on FPGA

Abstract

Real-time object detection on embedded platforms is critical for safety-critical and industrial applications, but FPGA deployment remains challenging due to constraints on numerical precision, latency, and hardware resources. Although quantization is widely used to enable efficient FPGA inference, its impact on object-detection models combining classification and bounding-box regression has not been systematically analyzed within an hls4ml based workflow. This work compares Post-Training Quantization (PTQ) and Quantization-Aware Training (QAT) for deploying a lightweight CNN-based detector on an FPGA. An FP32 model is quantized to INT16 and INT8 using QKeras and subsequently converted to fixed-point hardware representations with hls4ml. The results show that PTQ severely degrades detection performance, reducing classification accuracy to approximately 10% and mean IoU below 0.30. In contrast, QAT preserves near-floating-point performance, achieving ≈94% accuracy and ≈0.89 IoU at the software level for both INT16 and INT8. However, default HLS fixed-point configurations introduce software-hardware discrepancies, particularly in classification. A regression-aware refinement that increases fractional precision in the bounding-box head restores hardware-level localization accuracy (IoU ≈0.89), while residual classification gaps remain due to fixed-point constraints. These findings demonstrate that reliable FPGA-based object detection requires both QAT and hardware-aware fixed-point design, providing practical guidelines for low-precision deployment using hls4ml.