+61 3 5227 1266
Centre for Pattern Recognition and Data Analytics
School of Information Technology
Locked Bag 20000
GEELONG VIC 3220
D. S. Pham, S. Venkatesh, M. Lazarescu and B. Saha. Anomaly detection in large-scale data stream networks. Data Mining and Knowledge Discovery , 2012.
This paper addresses the anomaly detection problem in large-scale data mining applications using residual subspace analysis. We are specifically concerned with situations where the full data cannot be practically obtained due to physical limitations such as low bandwidth, limited memory, storage, or computing power. Motivated by the recent compressed sensing (CS) theory, we suggest a framework wherein random projection can be used to obtained compressed data, addressing the scalability challenge. Our theoretical contribution shows that the spectral property of the CS data is approximately preserved under a such a projection and thus the performance of spectral-based methods for anomaly detection is almost equivalent to the case in which the raw data is completely available. Our second contribution is the construction of the framework to use this result and detect anomalies in the compressed data directly, thus circumventing the problems of data acquisition in large sensor networks.We have conducted extensive experiments to detect anomalies in network and surveillance applications on large datasets, including the benchmark PETS 2007 and 83GB of real footage from 3 public train stations. Our results show that our proposed method is scalable, and importantly, its performance is comparable to conventional methods for anomaly detection when the complete data is available.
Deakin University CRICOS Provider Code: 00113B
27th February 2015