Communication-Efficient Online Detection of Network-wide Anomalies
Overview:
In this project, we design a system and propose a novel approximation scheme for
continuously online anomaly detection that dramatically reduces the burden on
the production network. Our algorithm leverages intelligent data filtering at
distributed monitors and Principal Component Analysis (PCA) for detection at NOC.
We derive analytical results based on stochastic matrix perturbation theory to
effectively balance the tradeoff between detection accuracy and the amount of
data communicated over the network. By avoiding the expensive step of
centralizing all traffic data, our solution enables tracking PCA-based
anomalies in real time with minimal data communications. This overcomes
the key scalability limitations of the state-of-the-art network-wide
anomaly detection solution. Experiments with traffic data from an ISP-backbone
network demonstrate that our methods yield significant
communication benefits while simultaneously achieving high detection accuracy.
Publications:
Talks:
People: