Top seven network traffic monitoring challenges
Network traffic monitoring is often touted as a way for enterprises to meet performance, security and compliance goals. But implementing network traffic monitoring tools can also pose a series of challenges that range from difficulty in creating network baselines to trouble finding the right tools and strategies for monitoring content in a proxied environment.
Here are the top seven networking traffic monitoring challenges:
Challenge 1: Network baselines. Frequently network and security practitioners hear that the start of any network-centric project is to baseline the network. Just what is this supposed to mean? Simplistic approaches concentrate on bandwidth utilization over time, typically focusing on spikes and troughs. Some try to describe traffic in terms of protocols and port numbers. More advanced approaches try to classify traffic according to flows or even content. Regardless, there is no single accepted taxonomy for creating a network traffic baseline.
 | ||
| ||
| ||
Challenge 2: Topology, locating the problem. If the network baseline challenge is related to traffic passing a single monitoring point, this involves multiple locations. By placing instruments in enough locations, it should be possible to visualize the network based on observed traffic patterns. Doing this in an automated way would prove extremely useful to network administrators and defenders.
Challenge 3: Visualization at scale. Trying to meet the two previous challenges is likely to be possible when the networks involved are small to midsized. In truly large networks, analysts are likely to begin reaching the limits of some tools to digest and render network data. Tools which comfortably depict dozens or hundreds of nodes face severe limitations when working with thousands or millions of nodes.
| ||
| ||
| ||
Challenge 4: Knowledge management. As techniques and tools derive information from network data, it's often the analyst's responsibility to derive knowledge from the information. But how should the analyst capture that knowledge? Consider the "simple" problem of applying tags to network flows. Depending on the data set and the classification involved, tagging individual items in a packet or flow record can be difficult. Still, analysts should have a way to annotate network information for their benefit and the benefit of their teams.
Challenge 5: Privacy. Too many network tools assume the user is fully privileged. In other words, rarely do tools recognize that analysts might have to limit their activities in order to meet privacy or other regulations. Historically, lawful intercept tools have tried to honor these restrictions by applying filters to include or exclude certain traffic. That approach is too crude to handle modern protocols, especially when a large percentage of traffic is carried using HTTP. Entire methods for meeting privacy concerns are needed.
Challenge 6: Mixing and matching record types. IP addresses are an important element of network traffic but, increasingly, content is becoming more significant. Anyone working in a heavily proxied enterprise will appreciate this problem. Network flows between proxies are almost useless. With the rise of proxy-in-the-cloud solutions, network tools will need to spend more time looking at HTTP requests in traffic to the proxy. Associating these "level 7" records with the mixed "level 3" records from the original host can complicate analysis.
Challenge 7: Not another platform. The final obstacle involves how to extract value from network traffic. Countless vendors are likely to read this article and reply: "Drop my box on your network!" Unfortunately, this response reflects a lack of appreciation of the limits imposed by many IT organizations on deploying new equipment. Often, IT staff must cajole and plead to deploy the hardware currently watching network links. Some of those same deployments also required signing elaborate agreements concerning the nature of the work done at those sites. Ultimately, it can be unrealistic simply to add yet another appliance to a link of interest. Rather, networking teams should be willing to consider deploying their tools and techniques to open platforms so they can devise and deploy their own network appliances. In fact, they should be unwilling to spend any effort installing closed vendor platforms.
