Magic Quadrant untuk Network Performance Monitoring and Diagnostic

Magic Quadrant for Network Performance Monitoring and Diagnostics

10 February 2015 ID:G00263101

Analyst(s): Vivek Bhalla, Colin Fletcher, Gary Spivak

VIEW SUMMARY

Network professionals support an increasing number of technologies and services. With adoption of SDN and network function virtualization, troubleshooting becomes more complex. Identify the right NPMD tools to detect application issues, identify root causes and perform capacity planning.

Market Definition/Description

This document was revised on 23 February 2015. The document you are viewing is the corrected version. For more information, see the Corrections page on gartner.com.

Network performance monitoring and diagnostics (NPMD) enable network professionals to understand the impact of network behavior on application and infrastructure performance, and conversely, via network instrumentation. Other users and use cases exist, especially because these tools provide insight into the quality of the end-user experience. The goal of NPMD products is not only to monitor network components to facilitate outage and degradation resolution, but also to identify performance optimization opportunities. This is conducted via diagnostics, analytics and debugging capabilities to complement additional monitoring of today's complex IT environments. At an estimated $1.1 billion, the NPMD market is a fast-growing segment of the larger network management space ($1.9 billion in 2013), and overlaps slightly with aspects of the application performance monitoring (APM) space ($2.4 billion in 2013).

Magic Quadrant

Figure 1. Magic Quadrant for Network Performance Monitoring and Diagnostics

Source: Gartner (February 2015)

Vendor Strengths and Cautions

AppNeta

Founded in 2011, Boston-based AppNeta focuses on application performance monitoring. It acquired significant intellectual property and an installed base from the now-defunct Apparent Networks. Much of the heritage of the company comes from NPMD. AppNeta is focused on easy-to-deploy, inexpensive SaaS-based solutions. Its NPMD technologies have changed during the past 12 months, and AppNeta is now more focused on application visibility.

AppNeta's core NPMD features come from its AppView product (for synthetic application availability monitoring), enhanced with SaaS-specific monitoring for Salesforce, ServiceNow, Microsoft Dynamics and Office 365, Google Docs, NetSuite and athenahealth. FlowView provides traffic analysis including both deep packet inspection and flow analysis. The unique component of the offering is PathView, which provides deep path analysis along with patented capacity measurement.

Most of AppNeta's improvements to date have been focused on application measurement, and the efforts are largely seen in the TraceView product, which provides APM for modern Web applications. While the deployment characteristics of each solution component vary between little (small physical or software appliances) to no footprint, the core analysis is done via a SaaS-delivered portal. AppNeta does support on-premises deployments for core NPMD features as well.

AppNeta's NPMD revenue is between $5 million and $10 million per year. AppNeta did not respond to requests for supplemental information and/or to review the draft contents of this document. Gartner's analysis for this vendor is therefore based on other credible sources, including previous vendor briefings and interactions, the vendor's own marketing collateral, public information and discussions with more than 200 end users who either have evaluated or deployed each NPMD product.

Strengths

• Its full-featured offering, including APM and NPMD within lightweight SaaS delivery, appeals to those wishing to experiment with technologies without large capital investment.
• PathView's capabilities, including multiple appliance form factors and hardware/software deployment options, appeal to those wishing to measure distributed networks and branch offices.
• AppNeta has benefited from the growth in SaaS use among enterprises; this differentiates it from offerings that focus on packet or network buyers only.

Cautions

• Management changes across several executive areas have resulted in a decreasing focus on NPMD.
• AppNeta is shifting its product and marketing focus away from network monitoring buyers who represent a significant (if slower growing) portion of its current customer base.
• The modules within AppNeta's offering are linked together, but they do not share a common user interface or back-end data store, creating some fragmentation and inconsistency within the offering.

Automic (Orsyp)

Automic acquired Orsyp in May 2014. Automic has U.S. headquarters in Bellevue, Washington, with global headquarters in Austria. While the company is focused on workload automation, application release automation and automated service orchestration, the acquisition of Orsyp provided the means to build a dedicated business unit focused on NPMD. Orsyp's NPMD tool, while originally rooted in quality of service (QoS)-oriented deep packet inspection capabilities developed internally, is focused on the Sysload and Streamcore products. Orsyp's Sysload 5.80 provides performance and endpoint monitoring capabilities, as well as capacity planning. Streamcore 6.2 adds WAN optimization and network performance monitoring capabilities. Recent feature inclusions are enhanced NetFlow collections (NetFlow v.9 collection capabilities on the central management console), 10 Gbps support and a renovated service-oriented scorecard dashboard. Streamcore probes can manage up to 10 Gbps of traffic for classification and QoS, and probes can be stacked to manage up to 40 Gbps in monitoring only. The improved scorecard dashboard provides the ability to consolidate large numbers of sites and dozens of applications with drill-down capabilities through site, application, flow and packets.

Automic (Orsyp) NPMD revenue is between $11 million and $20 million per year.

Strengths

• Complementary technologies have been consolidated to provide a well-rounded overall solution, including the ability not only to monitor, but also to depict the QoS and application delivery.
• The visually compelling user interface captures typical operator workflows well, including troubleshooting and optimization use cases, and demonstrates an innovative approach to present multiple dimensions of status easily.
• As part of Automic, the NPMD solution is now part of a larger company (combined revenue is roughly triple that of Orsyp alone) with greater resources, and which boasts a solid network of partners (35 channel partners outside of the Americas).

Cautions

• The former Orsyp solutions are now part of a larger organization that has its primary focus on other technologies, making NPMD an even smaller focus within Automic than it was under Orsyp.
• While Automic (Orsyp) conducts business globally, it has not yet broadly expanded its NPMD installed base outside of EMEA.
• Resources devoted to NPMD are limited for this midsize organization.

CA Technologies

CA's extensive history and product set for NPMD was built over many years of acquisitions and organic development. Founded in 1976 and based in Islandia, New York, CA's R&D centers span multiple locations, but are centralized in Fort Collins, Colorado, for core NPMD functions. CA has the largest focused revenue within the IT operations management (ITOM) area, including an extensive product portfolio. CA has been undergoing rapid change, including shifts in strategy around long-term plans.

CA's offerings for this Magic Quadrant are based on the CA Performance Management solution, and completed with other offerings including CA Network Flow Analysis, CA Unified Communications Monitor, and CA Application Delivery Analysis (ADA), which is focused on packet-level inspection. If additional depth is needed, CA has a referral relationship and integration with Network Instruments' Observer GigaStor product for packet storage and analysis. CA has other products for physical and virtual systems and operating system monitoring. Data from the CA components is rolled into CA Performance Management, which provides a single user interface and reporting system across the components. When depth is needed, there are drill-downs into the specific products.

CA's NPMD revenue is between $151 million and $300 million per year.

Strengths

• CA's broad portfolio includes many functional areas, with a strong partner and internal service capability to implement and maintain the solutions.
• CA changed its development organization during the past two years in order to become more agile to increase release velocity, providing point releases with fixes and new functionality every three to four months.
• CA's financial viability, and the fact that the company is public, shows there is limited risk to business stability involved in doing business with the company. New leadership with a proven track record of innovation is a departure from previous leadership.

Cautions

• Complexity and size of the CA product portfolio continues to be an issue.
• Scalability of packet analysis and management of the ADA probes are common user complaints; the product doesn't have the same scale issues when dealing with SNMP or flow data.
• The product is currently designed and used by customers primarily for reporting and dashboard purposes versus guided diagnostic troubleshooting.

Cisco

Cisco's large hardware installed base across network environments gives it a large addressable set of customers across verticals. With extensive software, Cisco provides what customers need to manage a Cisco-centric environment. Founded in 1984, with headquarters in San Francisco, California, Cisco is focused on moving from a network-hardware-centric company to include unified communications, server hardware (Unified Computing System [UCS]), management software and extensive SaaS capabilities for collaboration.

Cisco's offerings include Cisco Prime Infrastructure (PI) for SNMP and flow monitoring, configuration management, and provisioning Cisco hardware devices. Cisco Prime Network Analysis Module (NAM) is an appliance-based solution (physical server, physical module within a device, virtual machine) for packet analysis. The data can be analyzed via PI, and the NAM devices can be managed centrally. Cisco Prime Collaboration (PC) provides monitoring, configuration management and provisioning of Cisco's unified communications solutions. Cisco provides workflows for troubleshooting, and focuses on integration with Cisco technologies along with identity and access management.

Cisco's NPMD revenue is between $51 million and $150 million per year.

Strengths

• Cisco is in a unique position to support its pervasive networking installed base. Its NPMD solution goes very deep within Cisco technologies and has industry-leading support for Day 1 releases of Cisco hardware and software within the management technologies.
• Flow generation by Cisco devices (such as Next Generation Network-Based Application Recognition [NBAR2] included in Application and Visibility Control [AVC] 2.0) are among the best and most useful implementations in the industry, supporting recognition and basic performance monitoring of thousands of application types. This is supported in both the devices and the NPMD tools, which leverage NBAR2.
• The uniform UI across the Prime products makes navigation consistent across modules and implementations.

Cautions

• Central packet analysis is limited in Cisco solutions, since data is summarized off the NAM devices. This prevents the capability for historical, distributed or deep packet analysis after packets have been processed. The limited capability in this space is done via a dated user interface off a specific NAM instance.
• Cisco Prime product sets are often included in deals, and used for point solution management; unfortunately, the support for non-Cisco devices is minimal today, requiring buyers to purchase other management tools for availability and performance monitoring, including NPMD.
• Analytics capabilities are limited; most of the product is based on reporting and presenting large reports of metric data.

Corvil

Corvil's focus has historically been on the need for short-time-scale network monitoring, such as in the high-frequency trading market. Founded in 2000, Corvil is privately held and headquartered in Dublin, Ireland. It recently refocused on improving its streaming analytics platform for IT operations, which brought some initial activity from customers outside of the financial markets for the first time. The company and product transition began in the summer of 2014, with the release of the Corvil streaming analytics platform, called Giga (aka CorvilNet 8.2), in what is planned to be a 12- to 18-month transition. The company is also focused on go-to-market expansion via the indirect sales channel to include two-tier distribution, system integrators and service providers (for example, Dimension Data, BT, CenturyLink, IBM and CSC). The platform includes several new capabilities, including autodiscovery of data sessions on the network, plug-ins to analyze applications and protocols, support for big data platforms and the ability to publish analytic streams to big data and business intelligence (BI) systems. However, Corvil still lacks 40G support.

Corvil's NPMD revenue is between $21 million and $50 million per year.

Strengths

• The offering can monitor large amounts of data at rapid speeds. This will appeal to companies beyond Corvil's high-speed trading installed base, as concepts such as the Internet of Things (IoT) are adopted into the mainstream.
• Corvil's NPMD solution excels in terms of multisegment analysis and accuracy of packet analysis.
• Increased capabilities on analytics and integration with BI systems are a differentiator and unique selling point.

Cautions

• The user interface is geared to the highly technical network engineer supporting high-speed data environments, limiting the speed of adoption and appeal to those less technically proficient.
• There are few "greenfield" opportunities in the company's financial vertical core. This will invariably inhibit growth.
• The company is still relatively small, but has fairly large typical deal sizes, making it somewhat dependent on large deals as it moves to expand to a broader audience.

Fluke Networks

Fluke Networks emerged from Fluke Corp.'s network testing equipment for diagnostics capability in 2000. Fluke Corp. and Fluke Networks are wholly owned subsidiaries of the Danaher Corp. since 1998, but this will change given Danaher's October 2014 announcement of the proposed sale of its communications business (including Fluke Networks) to NetScout Systems, scheduled to conclude in 1H15. The first Fluke Networks product was the OptiView NetFlow Tracker, a network troubleshooting tool. Fluke Networks' offerings have expanded beyond this initial focus, upselling new capabilities to its large footprint of existing OptiView users. Acquisitions, such as Visual Networks in 2006 and ClearSight Networks in 2009, expanded its NPMD portfolio, forming its core solution, Visual TruView 9.5, which features distributed voice over IP (VoIP) and video monitoring support, and advanced path analysis. TruView is designed to serve the needs of large enterprises and carrier-grade service providers. Network Time Machine 9.1 (a scale-out leveraging TruView) and OptiView XG 12.1 (a handheld application and network diagnostic tool) complete the NPMD portfolio. TruView's broad data source support includes packet analysis, flow analysis and SNMP polling. Applications, such as Microsoft Lync and Citrix environments, are also supported. TruView appliances are sized based on network storage and processing requirements. OptiView XG provides endpoint and additional data to TruView to enable centralized troubleshooting analysis. In 2014, Fluke Networks launched the beta version of TruView Live, a SaaS-based delivery solution with monthly and yearly subscription options.

Fluke Networks' NPMD revenue is between $51 million and $100 million per year.

Strengths

• Strong contextual drill-down capabilities help simplify diagnostic and troubleshooting workflows.
• Fluke has a well-defined global sales organization demonstrating success via direct sales (in the U.S.) and channel partners (in EMEA and Asia/Pacific).
• TruView is a single modular appliance-based product offering that allows the customer to repurpose and use each appliance for multiple types of data acquisition.

Cautions

• Fluke Networks' NPMD strategic road map remains unclear in light of the impending merger of Danaher's communications business with NetScout.
• Questions surround the extent to which Danaher has achieved technology sharing among the acquired companies that form its communications business (VSS Monitoring, Arbor Networks, Tektronix and Fluke Networks), translating to risks of resource inefficiencies and capability overlap.
• While the vendor continues to innovate, its ability to demonstrate product enhancements and overall capabilities has regressed.

Genie Networks

Founded in 2000, Genie Networks targets its solutions for the communications service provider (CSP) industry. Based in Taiwan, the company looks to provide scalable yet competitively priced solutions.

Building from a competency in traditional SNMP-based network fault and performance monitoring capabilities, Genie Networks also incorporates deep packet inspection and flow analysis into its current offering, GenieATM 6.3.1. The latest features include support for MPLS VPN networks and extended traffic analysis reports.

Reporting templates for several CSP-oriented use cases are packaged with the default product, and these were augmented in the past year to include greater forensic and BGP routing analysis. The product has also been developed to appeal to security buying centers with the inclusion of spoofed IP detection and other anomaly mitigation features.

Genie Networks' NPMD revenue is between $11 million and $20 million per year.

Strengths

• Prebuilt reporting templates support key CSP customer support and operations use cases out of the box.
• Attractive pricing and carrier-grade scalability should continue to stimulate enterprise customer interest.
• An appliance-based product form factor simplifies implementation, particularly in highly distributed environments.

Cautions

• Committed CSP and Asia/Pacific prioritization compromises adoption beyond these vertical and geographical focal points, although the EMEA client base has almost doubled in the past 12 months.
• The product's user interface, while demonstrating some new features, remains perfunctory compared to the most sophisticated examples in the market space.
• Analytics capabilities are basic, centered on automated baselining to support anomaly detection and on security-focused use cases.

HP

One of the first vendors to offer enterprise network monitoring tools, HP has a long history of investment in this market, and a substantial portfolio and customer base, particularly in network fault management. Even as HP continues to maintain a strong market position, its customers continue to express concern to Gartner regarding HP's focus, strategy and execution. HP's recently announced split will likely result in little to no direct impact on NPMD strategy and execution. HP's NPMD product packaging and pricing has been radically simplified over the past year. A key component of HP's Business Service Management (BSM) offering, the NPMD solution is joined by APM, event correlation and analysis, and other HP technologies to support availability and performance monitoring. HP's NPMD solution is composed of the Network Node Manager i (NNMi) v.10.0, available in Community, Premium and Ultimate editions, with each edition differentiated by the number of types of included Smart Plug-ins, as well as HP Real User Monitor (RUM) v.9.24 (also available in Premium and Ultimate editions) for deep packet inspection.

HP's NPMD revenue is between $151 million and $300 million per year.

Strengths

• HP's vision for the increasing role of IT operations analytics (ITOA) technologies in NPMD is strong.
• Data interchange, modeling and process-level integration with other HP solutions is robust.
• The vendor offers a broad portfolio of complementary availability and performance monitoring products.

Cautions

• Usability is hindered by multiple, varied and dated user interfaces across the NPMD solution.
• Customers report difficulty in implementation and management of the complex solution.
• Implementation and integration of ITOA capabilities is not pervasive, both within NPMD solution components and across the HP BSM portfolio.

InfoVista

InfoVista's history as a carrier-grade and large-enterprise-focused NPMD provider began in 1995, based in Les Ulis, France, with a heavy physical presence in France. Offerings span NPMD, wireless network design and planning, mobile network optimization, and configuration assurance. Product offerings focused on NPMD include VistaInsight for Networks (which includes the Vista360 UI) for visualization across the components in the solution. Those components are 5View Service Data Manager (for data aggregation and analysis of the raw metric data, before feeding up to VistaInsight for Networks); 5View Netflow (appliances that provide flow collection, supporting NetFlow, sFlow, JFlow and Internet Protocol Flow Information Export [IPFIX] data sources); 5View Applications (appliances that provide deep packet inspection capabilities); and the final module, 5View Mediation (for collection of data from appliances doing packet and flow analysis for reporting purposes). InfoVista is active within carrier-specific marketing activities, and contributes to standards bodies for carrier services, such as carrier-Ethernet-focused Metro Ethernet Forum (MEF), in addition to TeleManagement Forum (TMF) and next-generation mobile network (NGMN). It focuses on the service assurance component of the TMF framework, and integration with operations support system (OSS) architectures.

InfoVista's NPMD revenue is between $51 million and $150 million per year.

Strengths

• InfoVista remains focused on solution scalability, meeting the needs of some of the largest carrier and large-enterprise networks.
• InfoVista does carrier focus well, expanding offerings specific to those buyers, and increasing support and capability of the products.
• Flexibility of the InfoVista solution allows it to meet sophisticated use cases, which many other vendors cannot meet.

Cautions

• Complexity within the InfoVista product line may result in long lead times in implementation, difficulty with product usability, and typically high levels of required consulting — often driven by the need to meet customer-specific use cases.
• Growth has been slow, since most customers are carriers and large enterprises, although InfoVista has been introducing new offerings to complement existing product lines.
• InfoVista has a heavy reliance on customers in EMEA, which make up 60% of its installed base.

JDSU (Network Instruments)

In 2014, we have witnessed the completion of JDSU's acquisition of Network Instruments, its subsequent integration into JDSU's Network and Service Enablement business segment, the recent release of updates to its NPMD offering, and announced plans to separate JDSU into two entities in 2015. While this action could provide additional efficiencies and focus in the future, the preceding business integration and sales enablement efforts are only now beginning to bear fruit and will have to shift once more in response to the coming changes. The Network Instruments unit has followed a well-established, vertically integrated technology development strategy, designing and manufacturing most of its product components and software. An OEM relationship with CA Technologies, which had Network Instruments providing its GigaStor products to CA customers, devolved into a referral relationship, but no meaningful challenges have been voiced by Gartner clients as a result of this. Two key parts of the NPMD solution have new product names (Observer Apex and Observer Management Server) and a new, modern UI that is a significant improvement. Network Instruments' current NPMD solution set is now part of the Observer Performance Management Platform 17, and includes Observer Apex, Observer Analyzer, Observer Management Server, Observer GigaStor, Observer Probes and Observer Infrastructure (v.4.2).

JDSU's (Network Instruments) NPMD revenue is between $51 million and $150 million per year.

Strengths

• Data- and process-level integration workflows are well-thought-out across the solution's component products.
• Network Instruments' recent addition of a network packet broker product (Observer Matrix) to its offerings may appeal to small-scale enterprises looking for NPMD and NPB capabilities from the same vendor.
• Packet capture and inspection capability (via GigaStor) is well-regarded by clients.

Cautions

• While significant business integration activities have not, to date, had a perceptible impact on support or development productivity, this process is ongoing and now part of a larger business separation action that could result in challenges in the near future.
• The NPMD solution requires multiple components with differing user interfaces that are not consistent across products.
• The solution is focused on physical appliances, with limited options beyond proprietary hardware.

NetScout Systems

Founded in 1984 and headquartered in Westford, Massachusetts, NetScout Systems focuses almost exclusively on providing network performance monitoring solutions to customers in enterprise, service provider and public-sector markets. Thanks to NPMD market growth, lengthy market tenure and acquisition execution, NetScout maintains a sizable market share. NetScout is no stranger to chasing growth and reactively extending its capabilities via acquisition (acquisitions include Network General, Simena, Fox Replay, Psytechnics and Accanto Systems), of particular interest is its recent announcement of its intention to acquire Danaher Corp.'s sizeable communications business (which includes NPMD competitor Fluke Networks) in exchange for $2.6 billion of NetScout stock. Upon completion, the acquisition will increase NetScout's NPMD market share, but at the expense of significant, potentially lengthy portfolio rationalization efforts due to substantial technology overlap, complexity and the size of the overall acquisition. These efforts will join ongoing and, to date, successful consolidation of acquired adjacent and developed capabilities into NetScout's nGeniusONE v. 5.3 NPMD solution. The products comprising the NPMD solution include nGeniusONE, the nGenius Service Assurance Solution (composed of nGenius Performance Manager, nGenius Service Delivery Manager, nGenius Enterprise Intelligence, nGenius Forensic Intelligence, nGenius Trading Intelligence, nGenius Voice/Video Manager and the Sniffer Analysis suite), and nGenius Intelligent Data Sources (including both hardware InfiniStream Appliances and Virtual Probes for data collection).

NetScout's NPMD revenue is between $301 million and $500 million per year. NetScout did not respond to requests for supplemental information and/or to engage in Gartner's standard procedures to address the contents of this document. Gartner's analysis for this vendor is therefore based on other credible sources, including previous vendor briefings and interactions, the vendor's own marketing collateral, public information and discussions with end users who either have evaluated or deployed each NPMD product. NetScout has sued Gartner over the content of the 2014 NPMD Magic Quadrant, and that lawsuit is pending. In addition, as noted above, NetScout has refused Gartner's invitation to engage in Gartner's standard procedures to address the content of this Magic Quadrant.

Strengths

• NetScout has a large and loyal customer base.
• Once third-party market innovators have proven market demand, NetScout has successfully acquired and integrated such complementary technologies.
• NetScout's patented Adaptive Session Intelligence technology provides technical differentiation in support of growing scalability requirements.

Cautions

• NetScout's planned, large acquisition of significantly overlapping technologies may result in road map delays, changes in market and product strategy, and leadership churn that impacts customer value in the near and long term, as evidenced by other acquisitions.
• NetScout's ability to derive and use application context from network data should not be confused with full APM capabilities.
• The hardware-dependent solution is priced at a premium, with limited upgrade paths.

Niksun

Founded in 1997, headquartered in Princeton, New Jersey, Niksun offers both network performance and security products. Products exist under the broad umbrella of the recently announced Alpine 4.5, which is designed specifically for large-scale network monitoring. While offering appliances and architectures to suit network infrastructures of all sizes, the vendor's NetVCR Alpine appliance supports high-fidelity data capture and analytics at speeds exceeding 100 Gbps. Other products include Virtual NetVCR (NetVCR for virtual environments), NetTradeWatch (which offers visibility into the trading network environment), NetVoice Alpine (for VoIP oversight and administration), IntelliNetVCR (portable NetVCR for field professionals), FlowAggregator Alpine (a flow traffic collector) and NetBlackBox Pro (a scaled-down version of the appliance that is designed for high-performance data capture and simple analytics).

Niksun has primarily focused on global service providers and other highly network-dependent industries, including large financial institutions and defense and intelligence communities. The vendor's current NPMD solution consists of the NetVCR family of products (for line-rate packet capture and analytics).

Niksun's NPMD revenue is between $21 million and $50 million per year. Niksun did not respond to requests for supplemental information and/or to review the draft contents of this document. Gartner's analysis for this vendor is therefore based on other credible sources, including previous vendor briefings and interactions, the vendor's own marketing collateral, public information and discussions with more than 200 end users who either have evaluated or deployed each NPMD product.

Strengths

• Packet capture in excess of 100G and broad analysis support in recent releases allow Niksun to maintain a leading position at high speeds.
• Niksun provides interfaces to a broad array of large IT vendors including Microsoft, Cisco, IBM Tivoli, HP ArcSight, Juniper Networks, Gigamon, VMware and Intel (McAfee), providing customers with flexibility when using complementary solutions.
• The vendor has a long history and remains an innovator at the high end of the market by introducing new features before competitors.

Cautions

• Niksun is not positioned as a pure-play NPMD vendor despite innovation in the space. Its marketing collateral leads first with security solutions, then with its NPMD solutions.
• Market presence and messaging are limited, inhibiting growth potential.
• Niksun's focus on the premium high end inhibits its ability to aggressively expand its installed base.

Riverbed

Riverbed, founded in 2002 and headquartered in San Francisco, California, focuses on application delivery, acceleration and performance monitoring. During the past five years, Riverbed has assembled a broad set of performance monitoring capabilities primarily through acquisition. The acquired technologies (from Mazu Networks, Cace Technologies and Opnet Technologies, now all branded as SteelCentral products) have seen recent brand and product name updates to better indicate support of both NPMD and APM, as well as relationships to each other. Further integration and consolidation remains incremental and focused largely on the data interchange and analysis dimensions, as the solution still comprises nine individual core products of varying delivery methods; however, the solution continues to provide more than competitive technical depth and breadth. The Riverbed NPMD solution includes SteelCentral Dashboards (previously RPM Dashboards) v.2.3, SteelCentral NetProfiler (previously Cascade Profiler) v.10.7, SteelCentral NetShark (previously Cascade Shark) v.10.7, SteelCentral Packet Analyzer (previously Cascade Pilot) v.10.7, SteelCentral AppResponse v.9.5, SteelCentral Transaction Analyzer (previously Opnet AppTransaction Xpert) v.17.0, SteelCentral NetSensor (previously Opnet AppSensor Xpert) v.2.0, SteelCentral Web Analyzer (previously Opnet AppResponse Xpert BrowserMetrix) v.2.2, and SteelCentral UCExpert (previously Opnet Unified Communication Xpert) v.5.0.

Riverbed's NPMD revenue is between $151 million and $300 million per year. The pending acquisition of Riverbed by Thoma Bravo and Teachers' Private Capital was not factored into this Magic Quadrant evaluation, as it occurred after the assessment cutoff.

Strengths

• Riverbed's technical breadth and depth is often cited as a key positive differentiator in competitive evaluations.
• A focus on application delivery quality across all solution components is evident and useful, particularly for those invested in other Riverbed offerings.
• Multiple ITOA capabilities are utilized across solution components.

Cautions

• Riverbed's NPMD solution is a visibly complicated combination of several unique and complex hardware and software technologies.
• Continued activist investor involvement negatively impacts corporate stability and ability to execute.
• Product line and naming changes continue to create confusion.

SevOne

Founded in 2005 and headquartered in Wilmington, Delaware, by a serial entrepreneur and a technologist, SevOne has maintained rapid growth to date. SevOne's NPMD solution consists of the SevOne Performance Appliance Solution (SevOne PAS) and Performance Log Appliance (PLA). The $150 million venture capital funding from Bain Capital in January 2013 remains the largest in the history of the NPMD market. The SevOne PAS has gained a reputation in the market for its scalability and flexible distributed architecture. Monitoring of Citrix infrastructure through the ingestion of AppFlow records is another notable capability, although Microsoft Lync support is limited.

In April 2014, SevOne acquired RapidEngines, a provider of log analytics solutions. SevOne recognized the opportunity that such technology presented to network domain specialists, and, in June 2014, brought to market its PLA product that enabled a single-click metric-to-log correlation view. Other feature improvements include enhanced WLAN traffic analysis reporting, which provides sound logical representation through an intuitive UI.

SevOne's NPMD revenue is between $51 million and $150 million per year.

Strengths

• Integration of PLA has been rapidly executed upon, successfully adding value to the portfolio.
• The patented distributed peer-to-peer architecture is highly scalable and facilitates a buy-as-you-grow deployment approach.
• A rapid flow analysis engine enables the highest reported flows and polling instances on a single appliance.

Cautions

• Rapidly expanding reseller agreements will dilute focus of resources dedicated to supporting each channel partner, which, in turn, may affect the end users they serve. SevOne has demonstrated an evolved channel strategy in anticipation of this impact.
• While support for software-defined networking (SDN) is a strategic goal, no current capabilities exist to support the SDN controller or SDN environments.
• Depth into packet data is comparatively limited due to this component being an OEM offering of third-party technology and based on flow analysis.

Vendors Added and Dropped

We review and adjust our inclusion criteria for Magic Quadrants and MarketScopes as markets change. As a result of these adjustments, the mix of vendors in any Magic Quadrant or MarketScope may change over time. A vendor's appearance in a Magic Quadrant or MarketScope one year and not the next does not necessarily indicate that we have changed our opinion of that vendor. It may be a reflection of a change in the market and, therefore, changed evaluation criteria, or of a change of focus by that vendor.

Added

Cisco — Inclusion criteria concerning flow-based technology support have changed this year, which now qualifies Cisco for this Magic Quadrant.

Dropped

Lancope — This vendor has reduced its focus on IT operations and is increasingly aligned more closely with security-oriented use cases and scenarios.

Paessler — This vendor does not meet the scalability criterion of a single instance of the tool supporting 10G environments at full line rate.

Please Note: While Ixia's Spyke product does support NPMD, the company announced in October 2014 that it would no longer be selling it. Therefore, it does not qualify for inclusion in this Magic Quadrant.

Inclusion and Exclusion Criteria

Product-Related Criteria

Vendors were required to meet the following criteria to be considered for the 2015 NPMD Magic Quadrant:

• Offer the ability to monitor, diagnose and generate alerts for:
  • Network endpoints — Servers, virtual machines, storage systems or anything with an IP address, by measuring these components directly in combination with a network perspective.
  • Network components — Routers, switches and other network devices. This includes SDN and network function virtualization (NFV) components.
  • Network links — Connectivity between network-attached infrastructure.
• Offer the ability to monitor, diagnose and generate alerts for dynamic end-to-end network service delivery as it relates to:
  • End-user experience — The capture of data about how end-to-end application availability, latency and quality appear to the end user from a network perspective. This is limited to network traffic visibility and not within components, such as what APM is able to accomplish.
  • Business service delivery — The speed and overall quality of network service and/or application delivery to the user in support of key business activities, as defined by the operator of the NPMD product. These definitions may overlap as services and applications are recombined into new applications.
  • Infrastructure component interactions — The focus on infrastructure components as they interact via the network, as well as the network delivery of services or applications.
• Provide support for analysis of:
  • Real-time performance and behaviors — Essential for troubleshooting in the current state of the environment. Analysis of data must be done within three minutes under normal network loads and conditions.
  • Historical performance and behaviors — To help understand what occurred or what is trending over time.
  • Predictive behaviors by leveraging IT operations analytics technologies — The ability to distill and create actionable advice from the large dataset collected (see the data sources listed below).
• Leverage the following data sources:
  • Network-device-generated data, including flow-based data sources inclusive of NetFlow and IPFIX.
  • Network device information collected via SNMP.
  • Network packet analysis to identify application types and performance characteristics.
• Demonstrate the ability to support the following scalability and performance requirements:
  • Real-time monitoring of 10GbE networks at full line rate via a single instance of the product.
  • Ingest sampled flow records at a rate of 75,000 flows per second via a single instance of the product.

Non-Product-Related Criteria

• A minimum of 10 NPMD customer references must be included at the time of survey submission.
• Customer references must exclude security-oriented use cases and scenarios.
• Customer references must be located in at least two of the following geographic regions: North America, South America, EMEA and/or the Asia/Pacific region/Japan.
• Total revenue (including new licenses, updates, maintenance, subscriptions, SaaS, hosting and technical support) must have exceeded $7.5 million in 2013, excluding revenue derived from security-related buying centers.
• The vendor should have at least 75 customers that use its NPMD product actively in a production environment.
• The product and specific version submitted for evaluation must have been shipping to end-user clients for production deployment and designated with general availability by 31 October 2014.

Evaluation Criteria

Ability to Execute

Product/Service: Gartner makes judgments from a variety of inputs in order to evaluate the capabilities, quality, usability, integration and feature set of the solution, including the following functions:

• Data source support, including application and network device log data
• Behavioral analysis
• Diagnostic workflows
• Real-time monitoring
• Day-to-day maintenance of the product
• Ease and management of deployment and configuration
• Ease of use and richness of functions within the product
• Product deployment options and usability

Overall Viability (Business Unit, Financial, Strategy and Organization): We consider the vendor's company size, market share and financial performance (such as revenue growth and profitability). We also investigate any investments and ownership, and any other data related to the health of the corporate entity. Our analysis reflects the vendor's capability to ensure the continued vitality of its NPMD offering.

Sales Execution/Pricing: We evaluate the vendor's capability to provide global sales support that aligns with its marketing messages; its market presence in terms of installed base, new customers and partnerships; and flexibility and pricing within licensing model options, including packaging.

Market Responsiveness and Track Record: We evaluate the execution in delivering and upgrading products consistently, in a timely fashion, and meeting road map timelines. We also evaluate the vendor's agility in terms of meeting new market demands, and how well the vendor receives customer feedback and quickly builds it into the product.

Marketing Execution: This is a measure of brand and mind share through client, reference and channel partner feedback. We evaluate the degree to which customers and partners have positive identification with the product, and whether the vendor has credibility in this market.

Customer Experience: We evaluate the vendor's reputation in the market, based on customers' feedback regarding their experiences working with the vendor, whether they were glad they chose the vendor's product and whether they planned to continue working with the vendor. Additionally, we look at the various ways in which the vendor can be engaged, including social media, message boards and other support avenues.

Table 1. Ability to Execute Evaluation Criteria

Evaluation Criteria	Weighting
Product or Service	High
Overall Viability	Medium
Sales Execution/Pricing	High
Market Responsiveness/Record	High
Marketing Execution	Medium
Customer Experience	Medium
Operations	Not Rated

Source: Gartner (February 2015)

Completeness of Vision

Market Understanding: This criterion evaluates vendor capabilities against future market requirements. The market requirements map to the market overview discussion and look for the following functionality:

• Data source support, including application and network device log data
• Behavioral analysis
• Diagnostic workflows
• Real-time monitoring
• Virtualization (NFV and SDN)

Marketing Strategy: We evaluate the vendor's capability to deliver a clear and differentiated message that maps to current and future market demands, and, most importantly, the vendor's marketing effectiveness to the NPMD market through its website, advertising programs, social media, collaborative message boards, tradeshows, training and positioning statements.

Sales Strategy: We evaluate the vendor's approach to selling NPMD in the appropriate distribution channels, including channel sales, inside sales and outside sales.

Offering (Product) Strategy: We evaluate product scalability, usability, functionality, and delivery model innovation. We also evaluate the innovation related to the delivery of products and services.

Business Model: This is our evaluation of whether the vendor continuously manages a well-balanced business case that demonstrates appropriate funding and alignment of staffing resources to succeed in this market. Delivery methods will also be evaluated as business model decisions, including the strength and coherence of on-premises and SaaS solutions.

Vertical/Industry Strategy: We evaluate the targeted approaches in marketing and selling into specific vertical industries.

Innovation: This criterion includes product leadership and the ability to deliver NPMD features and functions that distinguish the vendor from its competitors. Specific considerations include resources available for R&D and the innovation process.

Geographic Strategy: This is our evaluation of the vendor's ability to meet the sales and support requirements of IT organizations worldwide. In this way, we assess the vendor's strategy to penetrate emerging markets.

Table 2. Completeness of Vision Evaluation Criteria

Evaluation Criteria	Weighting
Market Understanding	High
Marketing Strategy	Medium
Sales Strategy	Medium
Offering (Product) Strategy	High
Business Model	Low
Vertical/Industry Strategy	Low
Innovation	High
Geographic Strategy	Low

Source: Gartner (February 2015)

Quadrant Descriptions

Leaders

The Leaders quadrant represents those vendors that are pushing the NPMD market forward, including those with comprehensive portfolios and the ability to handle multiple application and technology types. They offer a choice of hardware or software appliances for optimum flexibility. Additionally, the use of SaaS delivery methods within portfolios gives enterprise IT teams more choices, while making formerly premium priced NPMD solutions attainable by midsize organizations. All Leaders offer a high degree of application-aware insight and visibility.

Challengers

Challengers consist of those vendors with high market reach and large deployments. Once leaders in the network performance monitoring market, they are currently struggling to deal with new technical demands and rising expectations. These established NPMD vendors bring a substantial installed base, but also architectures, feature sets and pricing structures that require modernization (often in progress) to better compete with those in the Leaders quadrant.

Visionaries

Visionaries have built a compelling plan to competitively address current and future NPMD customer demands. The Visionaries are combining elements of APM, infrastructure performance monitoring (IPM) and NPMD in ways that provide deeper visibility than is currently available from other vendors. Presently, execution is limited either by insufficient market reach or by the extent of existing tools and technology capabilities that are not initially designed, or able, to meet these needs.

Niche Players

Niche Players are those vendors with solutions catering to specific audiences or with limited use-case support today. They have often been unable to address the needs of larger enterprises or have only done so within specific verticals or market segments. Each of these vendors is working to appeal to the broader NPMD buying community, versus the targeted use cases they serve today. With the right changes to their product plans, positioning and/or business execution strategies, any of these vendors could successfully shift their differentiated technologies to address use cases in ways that today's Leaders might have a hard time matching.

Context

NPMD solutions should be considered as part of a larger network management initiative, that is, part of an overall availability and performance monitoring strategy. Utilizing these additional points of reference will yield further unique criteria (such as existing investments, investment plans, vendor relationships) that, when combined with Gartner analysis, can prove critical to proper solution selection.

In the course of this research, several key observations emerged that should be carefully considered during NPMD strategy formulation and solution selection, including:

• Products have been mostly stagnant. The use of IT operations analytics has not fully materialized to date.
• The potential for market disruption via alternative delivery models and innovation is high.
• Ease of use remains an area that needs improvement and varies significantly both across vendors and within solutions.
• Virtualization and SDN support is minimal in most NPMD solutions.
• Flow protocol support and actual flow data utilization vary significantly across vendors.
• Pricing and product/capability packaging vary significantly across vendors.
• On-premises appliances are the dominant delivery model, but software and SaaS offerings are available (see Note 1).
• Integrations with other ITOM solutions (even other availability and performance solutions) are an afterthought for most, thus encouraging the use of these tools within the network silo.
• Many NPMD solutions are assembled from multiple products, which can enable modular adoption of NPMD capabilities, but also can add significant complexity to procurement and ongoing maintenance.

It remains imperative that organizations purchase tools that closely match their current maturity levels. It is now a reality that many network monitoring teams have yet to successfully make the leap from basic, reactive network availability management to proactive performance management. While tool investment can play a part in this maturation, it is clear that simply investing in NPMD tooling without similar investments in training, integration and processes will yield limited results at best. Gartner recommends that network teams assess their current state of maturity on a regular basis, both individually and at the organizational level, to provide this perspective. To help, teams can utilize Gartner's ITScore for infrastructure and operations (ITSIO; see "How to Improve I&O Maturity by Using the ITScore").

Organizations should not utilize the Leaders quadrant as a shortlist of appropriate vendors, but instead should build a list of criteria that describe their current and future needs, and then select from vendors that best meet those requirements. Organizations should select a vendor that has both a history of and future plans for focusing on this market. Careful consideration should be given to required skills, training, process and deployment investments, because these factors will have a much greater impact on the overall value realized from an NPMD investment than any specific functional capability found in a given tool.

Market Overview

NPMD: A Mature Market Evolves Rapidly in Response to New Demands

For decades, the well-established practice of network management has enjoyed no shortage of available monitoring technologies, tools and vendors; however, the vast majority of those solutions, both acquired and implemented over the years, have been designed to support isolated, reactive resolution of availability issues by network specialists. There have also been many investments in tools and skills, with the specific goal of monitoring the performance of network infrastructure in addition to its availability; however, these efforts have typically been hampered by technology limitations and isolated implementation. This approach, while delivering moderately satisfactory results for many years, has proven inadequate in the face of several key shifts, including:

• Increasing recognition that network traffic is a critical source of information about the behavior of the holistic IT stack
• Rising demand for network services and end-user expectations of their quality
• Growing appreciation of the network as a critical component of IT services and as an agnostic, trusted source of cross-domain availability and performance data
• Exponential growth in application and infrastructure dynamism and complexity

Each of these shifts has pressured network teams to rethink their tooling strategy, so that they can get the visibility they need to truly monitor and troubleshoot the performance of their network resources in the context of the applications and services they support. A subset of the overall network performance monitoring market, NPMD has emerged as a unique market designed to address these very needs in today's increasingly complex environments. A fast-growing subsegment, the NPMD tool market is currently estimated by Gartner to be approximately $1.1 billion in size.

NPMD tools provide this required breadth and depth of visibility in both real-time and historical perspectives by uniquely analyzing data from all three of the following sources. Previous approaches that only take into account one or two of the following data sources have proven to be inadequate, so all three must be supported for a tool to be considered an NPMD solution (see Note 2).

SNMP Polling

Period polling is one method that looks to quantify network usage of network elements to gauge the requirements of the infrastructure. Each network device has embedded agents that "speak SNMP." These agents can then be interrogated with a polling-based approach, returning metrics from the embedded agent. These collected metrics can be stored, reported on, analyzed for troubleshooting or used for capacity planning. SNMP polling can also be used to gather information about hardware or software errors (faults) and capacity data (for example, triggering an alert when a hardware fault occurs, the device CPU is above a threshold or the interface capacity is abnormal when compared to a baseline). Based on the metrics gathered, the network team can estimate the delta between existing and required bandwidth needs on a per-location basis. A limitation with this method is the minimal granularity it offers, which matches the frequency of the polling. In most NPMD technologies, SNMP is used during troubleshooting to collect additional data, whereas in availability monitoring, it's used more regularly to understand the health of the network devices.

Flow-Based Technology

Summarized data is generated by the network devices, including characteristics of the IP conversation between two network nodes, and these characteristics are embedded within flows. Flow data is exported from the network devices to the NPMD technologies, which then collect and process this data stream to provide insight into which devices and applications are consuming bandwidth, how long the conversations are lasting, and who is participating in them. Since the data is summarized, a degree of detail is removed to simplify processing and extract meaning from the actual network data. Flow-based data does not provide details down to a specific set of network packets going between the source and destination, and can have a performance impact on the devices exporting flow data.

There are several flow collection standards, such as Cisco's NetFlow (v.5/v.9), Juniper's JFlow, Huawei's NetStream, Citrix's AppFlow, Riverbed's Cascade Flow, the Internet Engineering Task Force's (IETF's) IPFIX (which is based off NetFlow v.9) and sFlow from the sFlow.org consortium. Vendor-derived standards are predominant, which hinders integration and comparisons. Flow data collection is a function embedded in the network devices themselves. The device analyzes the network traffic traversing from one interface to another, with the primary purpose of assessing bandwidth consumption, and the level of data being sent and received between various source and destination ports across the network. That data is then summarized into a stream-of-flow record that is sent to the monitoring tools that collect and assess the flow records.

Additionally, the quality and granularity of flow information are always evolving. Many vendors embed additional data within their flows, especially those implementing flexible record types, such as Cisco's IOS Flexible NetFlow, which allows the user to configure the exported data format. Example data embedded in flows contains wireless protocols, link aggregation, URLs, latency information, and other application or infrastructure monitoring data. With such open standards in flow technologies, the architecture varies between network equipment vendors, but most tools collect and process the data regardless of the network equipment implementation.

Packet-Based Technology

Examining the current infrastructure in detail on a per-packet basis provides the necessary real-time and historical visibility into volatile traffic behavior from "bursty" modern application types, like today's chatty Web applications, unified communications services (such as voice and video delivery), and the growing footprint of hosted virtual desktop (HVD) and virtual desktop infrastructure (VDI) technologies. Because only raw, unmanipulated packet data is being collected, a vendor-agnostic view of performance can be preserved throughout the analysis. This approach affords far greater insight and precision, but it comes with potentially costly (and for some, impossibly costly) appliance or "probe" implementation requirements.

The modern packet analysis technologies were pioneered more than 20 years ago with Network General's packet Sniffer in 1986. Sniffer was designed to help troubleshoot network issues after they occurred. This technology was acquired several times, and now finds its home with NetScout as of 2007. Through the years, these high-end proprietary packet analysis technologies have commoditized and moved into open source with entrants such as Wireshark, tcpdump and libpcap providing the underpinnings of this technique. These particular open-source technologies have, in turn, been incorporated into countless numbers of other critical open-source projects, such as Snort (intrusion detection system), Nmap (port scanning) and ngrep. These technologies continue to evolve, most recently into enabling real-time visibility and, in many cases, supporting the archiving of packet data for forensics and debugging without requiring the issue to be reproduced for diagnosis.

As the NPMD market continues to grow and develop, Gartner expects that future tool enhancements will center on usability, advanced IT operations analytics and virtualization/SDN support (see "Introducing the Network Performance Monitoring and Diagnostics Market"). Additional vendors are expected to enter the NPMD market, as well as over 30 vendors participating in the larger network performance monitoring market, but they did not meet the criteria specific to the 2015 NPMD Magic Quadrant.

Adjacent and Overlapping Markets

NPMD is, and will likely continue to be, frequently confused with adjacent and component technologies, as it is both a reasonably recent addition to the dynamic availability and performance monitoring market and a superset of multiple network performance monitoring technologies. Because vendors will both intentionally and unintentionally exacerbate this confusion to their benefit, IT leaders are advised to utilize the following definitions to add clarity to their evaluation efforts.

Application-Aware Network Performance Monitoring

A wholly contained subset of NPMD, application-aware network performance monitoring (AA NPM) is a necessary maturation step that adds some degree of application visibility accomplished via packet-based monitoring, which can provide varying degrees of application context to monitored network traffic.

APM

APM tracks the end-user performance of application components, and provides granular troubleshooting tools for the application and its components. It provides this insight by monitoring on five main functional dimensions: end-user experience (EUE) monitoring, application topology discovery and visualization, user-defined transaction profiling, application component deep dive, and IT operations analytics. APM differs from NPMD primarily in its focus on monitoring the quality of the end-user's experience via application interactions across all application and infrastructure tiers, including, but not limited to, the network perspective. See "Magic Quadrant for Application Performance Monitoring" for further details.

IPM

IPM tools focus on the infrastructure topology as defined by storage, server and virtualization professionals and IT generalists who leverage the network to deliver applications to IT end users. The difference between these tools and APM products is that they look at server-level metrics and processes, while also looking at the way servers interact with one another, versus living within the application logic and seeing the code execute. Unlike NPMD products, which cater to network professional use cases and speak in protocols and packet data that those buyers best understand, IPM products focus on IT operations generalists. These generalists need to determine which part of the infrastructure is contributing to poor performance, as well as understand the application's topology from an infrastructure perspective. As agent-based NPMD solutions evolve, they will clash with IPM technologies and how these IPM tools can better enable distributed troubleshooting use cases for network professionals.

NPBs

NPBs assist with traffic aggregation, visibility and overall management of the data being sent to monitoring tools. Vendors in the NPB space often partner and collaborate with NPMD and security vendors as a go-to-market strategy, resulting in marketing messages that can make it difficult to determine which tool is actually performing the monitoring (network performance monitoring or security) and which is facilitating the monitoring by managing the data to be monitored (network packet brokering). We have also seen an increasing number of acquisitions in this space. NetScout and Network Instruments, both of which are in this Magic Quadrant, pair NPBs and NPMD tools together in deals. Other popular NPB vendors include Gigamon, VSS Monitoring, JDSU (Network Instruments), Interface Masters Technologies, Apcon, Datacom, and switch provider Arista Networks. See "Market Guide for Network Packet Brokers" for further details.

Network Fault Monitoring

Network fault monitoring tools indicate the status of network components, such as routers and switches. These tools isolate, aggregate, deduplicate, filter, prioritize and resolve faults/alerts on the network. In some cases, the tools discover and visualize the topology of physical and logical relationships and dependencies among network elements. This helps depict the up/down status of those elements in a contextual map, provide basic root cause analysis (RCA), and enhance error deduplication and suppression capabilities. These tools have been widely deployed, primarily to address the reactive nature of network monitoring in IT operations. They provide network teams with a single location to monitor, alert and coordinate diagnosis of all network-related fault and availability information. These events are useful to help ensure that critical network devices remain available to support the business applications and services that rely on them.

Unified Monitoring

Unified monitoring is a subset of infrastructure monitoring that relies on primarily agentless and API-integration-based data collection methods to monitor the availability of servers, networks, storage and virtualization layers. These tools also do minimal discovery, and acquire physical, virtual and logical topologies and the relationships between them for the purposes of monitoring. They monitor common application instances on OSs, perform synthetic transactions for checking Web application availability, support service and infrastructure grouping, and are multitenant by design. Unified monitoring differs from NPMD primarily in its pure focus on infrastructure availability versus NPMD's focus on performance. Unified monitoring should be supplemented by log analytics, NPMD and APM tooling to support complex troubleshooting and performance monitoring.