Manifold learning visualization of network traffic data

ABSTRACT

When traffic anomalies or intrusion attempts occur on the network, we expect that the distribution of network traffic will change. Monitoring the network for changes over time, across space (at various routers in the network), over source and destination ports, IP addresses, or AS numbers, is an important part of anomaly detection. We present a manifold learning (ML)-based tool for the visualization of large sets of data which emphasizes the unusually small or large correlations that exist within the data set. We apply the tool to display anomalous traffic recorded by NetFlow on the Abilene backbone network. Furthermore, we present an online Java-based GUI which allows interactive demonstration of the use of the visualization method.

REFERENCES

• 1: Anukool Lakhina , Mark Crovella , Christophe Diot, Diagnosing network-wide traffic anomalies, Proceedings of the 2004 conference on Applications, technologies, architectures, and protocols for computer communications, August 30-September 03, 2004, Portland, Oregon, USA
• 2: Anukool Lakhina , Mark Crovella , Christiphe Diot, Characterization of network-wide anomalies in traffic flows, Proceedings of the 4th ACM SIGCOMM conference on Internet measurement, October 25-27, 2004, Taormina, Sicily, Italy [doi>10.1145/1028788.1028813]
• 3: J. B. Tenenbaum, V. de Silva, and J. C. Langford, "A global geometric framework for nonlinear dimensionality reduction," Science, vol. 290, pp. 2319--2323, Dec 2000.
• 4: S. T. Roweis and L. K. Saul, "Nonlinear dimensionality reduction by local linear embedding," Science, vol. 290, pp. 2323--2326, Dec 2000.
• 5: J. A. Costa, N. Patwari, and A. O. Hero III, "Distributed multidimensional scaling with adaptive weighting for node localization in sensor networks," IEEE/ACM Trans. Sensor Networks, submitted May 2004, (revised Jan. and May 2005). {Online}. Available: http://www.eecs.umich.edu/~hero/comm.html
• 6: A. C. Gilbert, Y. Kotidis, S. Muthukrishnan, and M. J. Strauss, "QuickSAND: Quick summary and analysis of network data," DIMACS, Tech. Rep. 2001-43, Nov. 2001. {Online}. Available: http://www.math.lsa.umich.edu/~annacg/ps.files/quickdimacstr.ps
• 7: Nitin Thaper , Sudipto Guha , Piotr Indyk , Nick Koudas, Dynamic multidimensional histograms, Proceedings of the 2002 ACM SIGMOD international conference on Management of data, June 03-06, 2002, Madison, Wisconsin [doi>10.1145/564691.564741]
• 8: Cristian Estan , Stefan Savage , George Varghese, Automatically inferring patterns of resource consumption in network traffic, Proceedings of the 2003 conference on Applications, technologies, architectures, and protocols for computer communications, August 25-29, 2003, Karlsruhe, Germany [doi>10.1145/863955.863972]
• 9: Dave Plonka, FlowScan: A Network Traffic Flow Reporting and Visualization Tool, Proceedings of the 14th USENIX conference on System administration, December 03-08, 2000, New Orleans, Louisiana
• 10: Stephen Lau, The Spinning Cube of Potential Doom, Communications of the ACM, v.47 n.6, June 2004 [doi>10.1145/990680.990699]
• 11: Kiran Lakkaraju , William Yurcik , Adam J. Lee, NVisionIP: netflow visualizations of system state for security situational awareness, Proceedings of the 2004 ACM workshop on Visualization and data mining for computer security, October 29-29, 2004, Washington DC, USA [doi>10.1145/1029208.1029219]
• 12: Cooperative Association for Internet Data Analysis (CAIDA). http://www.caida.org/.
• 13: Steve Romig, The OSU Flow-tools Package and CISCO NetFlow Logs, Proceedings of the 14th USENIX conference on System administration, December 03-08, 2000, New Orleans, Louisiana
• 14: Michael J. Swain , Dana H. Ballard, Color indexing, International Journal of Computer Vision, v.7 n.1, p.11-32, Nov. 1991 [doi>10.1007/BF00130487]
• 15: W. Cleveland, "Robust locally weighted regression and smoothing scatterplots," J. American Statistical Assoc., vol. 74, no. 368, pp. 829--836, 1979.
• 16: Map-tools online supplement. http://www.engin.umich.edu/~npatwari/mnd05.
• 17: Internet2: Abilene Observatory. http://abilene.internet2.edu/observatory/.

INDEX TERMS

Primary Classification:
  C. Computer Systems Organization
  C.2 COMPUTER-COMMUNICATION NETWORKS
      C.2.3 Network Operations
          Subjects: Network monitoring

General Terms:
Algorithms, Measurement

Keywords:
data mining, internet traffic anomaly detection & forensics