Optimal Confidence Thresholds for Cooperative Inference in Intelligent Surveillance Systems

To overcome the limitations of standalone inference that relies on either an edge device or a server, this study proposes a new cooperative inference method between the edge device and the edge server for intelligent surveillance services. In this method, the edge device equipped with a small neural network (NN) model operates with dual confidence thresholds to filter ambiguous input images, which are then forwarded to the edge server and reevaluated by a large NN model. We numerically analyze the performance of the proposed method in terms of inference accuracy and end-to-end latency, taking into account the distribution of confidence scores resulting from positive images as well as negative images that may induce false alarms. Subsequently, we formulate an optimization problem to minimize the end-to-end latency while ensuring the required accuracy, and propose a greedy search algorithm to find the optimal confidence thresholds with low complexity in a nonconvex problem. We also present an operational framework to utilize the proposed cooperative inference method in a practical on-site environment. The simulation and analysis results show that a tradeoff exists between accuracy and latency according to the confidence thresholds, and the selection of optimal confidence thresholds significantly reduces the latency while satisfying the required accuracy. Therefore, the proposed cooperative inference achieves higher accuracy than the device-only inference and much lower latency than the server-only inference across various system parameters. This verifies the importance of optimizing confidence thresholds when applying a cooperative inference method to mobile edge networks.