Network state estimation using smart experts

Several core network protocols and applications adjust their operation dynamically based on current network conditions. TCP and IEEE 802.11 are notable examples, both of which periodically adapt the retransmission timeout and the contention window size depending on the average round trip time and the degree of collisions, respectively. Consequently, accurate network state estimation is critical to the performance of networks and their applications. In this paper, we present a novel mechanism to estimate "nearfuture" network state based on past network conditions. Smart Experts for Network State Estimation, or SENSE, uses a simple, yet effective algorithm combining a machine-learning method known as Fixed-Share Experts and Exponentially Weighted Moving Average (EWMA). SENSE introduces novel techniques that improve the performance of the basic Fixed-Share Experts framework by: (1) making SENSE's accuracy considerably less sensitive to the number of experts; and (2) making SENSE more responsive to network dynamics at different time scales, i.e., long- And medium-term fluctuations as well as short-lived variations. We evaluate SENSE using synthetic and real datasets. Our results show that it yields superior performance for all datasets we used in our experiments when compared to "pure" Fixed- Share Experts and EWMA. We confirm that the performance of EWMA is quite sensitive to its "smoothing" factor, which specifies how much weight will be placed on the "past" versus the "present" when predicting the "future". Another key advantage of SENSE is that, unlike Fixed-Share Experts, it needs no a-priori information about the dataset. In our experiments, SENSE yields up to 24% and 30% prediction accuracy improvement over the Fixed-Share algorithm and EWMA, respectively.