Optimizing Confidence Thresholds for Cooperative Inference in Edge-AI Surveillance Systems: Avoiding the Fate of 'The Boy Who Cried Wolf'

In this paper, we propose a new cooperative infer-ence method between the end device and the edge server for intelligent surveillance services. In this method, the end device with a small neural network (NN) model operates with dual confidence thresholds to filter ambiguous input images, which are then forwarded to the edge server and reevaluated by a large NN model. We numerically analyze the performance of the proposed method in terms of accuracy and end-to-end latency, taking into account the confidence scores derived from both positive images and negative images that induce false alarms. Subsequently, we identify the optimal confidence thresholds for both the end device and the edge server to minimize the end-to-end latency while ensuring the required accuracy. The simulation and analysis results show that a tradeoff exists between accuracy and latency according to the confidence thresholds, and the selection of optimal confidence thresholds significantly reduces the latency while satisfying the required accuracy. Accordingly, the proposed method achieves higher accuracy than the device-only inference and lower latency than the server-only inference. This highlights the importance of employing cooperative inference with optimal confidence thresholds in surveillance systems to avoid the fate of 'The Boy Who Cried Wolf.'