This might seem a strange subject for a blog post from a company like Byte25, whose main focus is in fact …. network performance monitoring, but it is an interesting question for consideration. I wouldn’t necessarily be calling the undertaker just yet, but there is no question that the role of network performance monitoring within IT operations is changing.
For those that don’t have time to read the whole blog …. Network Performance Monitoring is not dying, but it definitely needs to adapt to meet the need of modern network environments.
Way back in the early 2000’s when network bandwidth was small and expensive, network performance monitoring was an essential part of maintaining end user experience. SNMP (the Simple Network Management Protocol) was flavour of the month for collecting network performance metrics with a range of commercial and freeware tools (remember MRTG or HP OpenView?) dedicated to collecting information from switches and routers across the network.
Ahhhhh, the good old days! We could see how much traffic was running through the network pipes, gauge capacity and ensure the links were correctly provisioned – and life as a network engineer was good! But just as we thought we had it made, business started throwing more applications at the network, the links became more heavily utilised and questions were asked as to why end user performance was degrading.
SNMP, whilst good, simply wasn’t sufficient to answer these questions. We could see the volume of data but not the who or what was actually generating the traffic. If only we had tools that would identify the breakdown of actual traffic!
Fortunately flow based protocols such as NetFlow and IPFIX evolved to answer this very question. Using NetFlow, we could identify both the type of traffic (at least up to Layer 4) as well as the source and destination. Now we could carefully monitor network links and see exactly what applications and what users were contributing to bandwidth and act accordingly to ensure performant links.
But still the world did not stand still. Applications started to use common transport layer protocols, like HTTP. So too new dynamic port applications like VoIP emerged. This rendered application identification difficult using available flow based protocol techniques. To address this new shortfall, vendors turned toward sophisticated deep packet inspection (DPI) engines that examined data payloads to identify the actual application within the data stream.
DPI was network performance monitoring nirvana, presenting a detailed picture of which applications and which users were generating traffic. Many DPI implementations also had the added advantage of providing other performance metrics such as latency, jitter and round trip time further enhancing their usefulness. DPI provides a comprehensive view of network performance which persists today in many of the major network performance monitoring products.
So what’s the problem? Why the question on the relevance of network performance monitoring as a whole? Put simply, network performance monitoring has lost significance due to the availability of higher speed and lower cost network bandwidth. Organisations can ‘throw’ bandwidth at networks relatively cheaply making justification for expensive network monitoring tools difficult. Performance is no longer the main problem.
Network performance monitoring isn’t dying, but is does need to adapt to meet the need of business and modern IT operations.
First up, and most importantly, as a network visibility vendor, we need to deliver monitoring products that address the needs of both business and IT operations. Performance metrics like throughput and errors, latency information and application level statistics gained from DPI are still very relevant in modern networks (especially so as we move toward more distributed environment like SD WAN and cloud based applications) but they are only part of the solution.
Rather than focus of network link monitoring , performance monitoring tools need to leverage and correlate other data sources to provide a wholistic picture of network activity. Rather than be labelled as ‘network performance monitoring’ tools, we should be thinking of ‘network visibility’ tools that derive and correlate information from a range of disparate sources.
Network visibility tools should be integrating with other data sources in a similar fashion to what SIEM tools do in the cyber security space. In addition to techniques like DPI, network visibility tools need to also extract data from the range other systems that rely on or contribute to network activity. This includes a wide range systems such as Active Directory, Office365, cloud platforms like AWS, Intrusion Detection Systems, server logs and even end point agents. In this way, we can build a precise and useful picture of network activity. Or put more simply a complete picture of network visibility.
This is precisely what we are doing at Byte25. To maintain relevance, we are starting to leverage other datasets to present a picture of network visibility. We already have an integrated IDS engine sitting alongside our DPI engine and are working toward other integrations to be delivered later this year.
For sure there will always be a place for traditional network performance monitoring, but moving toward comprehensive network visibility provides a stronger product set that actually meets the need of IT operations and business as a whole.