Gartner Magic Quadrant untuk Application Security Testing

Magic Quadrant for Application Security Testing

6 August 2015 ID:G00268424

Analyst(s): Neil MacDonald, Joseph Feiman

VIEW SUMMARY

Highly publicized breaches in the past 12 months have raised awareness of the need to identify and remediate vulnerabilities at the application layer. Enterprise application security testing solutions for Web, native, cloud and mobile applications are key to this strategy.

Market Definition/Description

Application security testing (AST) products and services are designed to analyze and test applications for security vulnerabilities. Ideally, an application would be tested using multiple approaches, and many providers in this Magic Quadrant offer multiple styles of AST, including:

• Static AST (SAST) technology analyzes an application's source, bytecode or binary code for security vulnerabilities typically at the programming and/or testing software life cycle (SLC) phases (see "Hype Cycle for Application Security, 2015").
• Dynamic AST (DAST) technology analyzes applications in their dynamic, running state during testing or operational phases. It simulates attacks against an application (typically Web-enabled applications and services), analyzes the application's reactions and, thus, determines whether it is vulnerable.
• Interactive AST (IAST) technology combines elements of SAST and DAST simultaneously. It is typically implemented as an agent within the test runtime environment (for example, instrumenting the Java Virtual Machine [JVM] or .NET CLR) that observes attacks and identifies vulnerabilities.
• Mobile AST uses a combination of traditional SAST and DAST and behavioral analysis using static and dynamic techniques to discover malicious or potentially risky actions the app may be taking unbeknown to the user (for example, activating the user's address book or GPS).

All of the above technology approaches can be delivered as a tool or as a subscription service. Many of the larger vendors offer both options to reflect enterprise requirements for both a product and service. Collectively, AST is adopted by the majority of enterprises, but the various technologies differ in adoption and maturity. DAST, followed by SAST, is the most widely adopted, while IAST and mobile AST have only recently emerged. This Magic Quadrant focuses on a vendor's maturity in offering SAST and DAST features as tools or as security as a service, and has weighted more heavily vendors' innovation in AST for mobile applications, IAST and emerging runtime application security protection (RASP) capabilities.

Magic Quadrant

Figure 1. Magic Quadrant for Application Security Testing

Source: Gartner (August 2015)

Vendor Strengths and Cautions

Acunetix

Acunetix is a Malta-based provider of DAST tools, and it has extensive technical capabilities in its DAST offering. Its primary offering, Acunetix Web Vulnerability Scanner (WVS), can be run on-premises or from its console in the cloud (built on Amazon Web Services), Acunetix Online Vulnerability Scanner (OVS). Approximately 20% of its installed base utilizes Acunetix's integrated IAST capability that it calls AcuSensor, and which is included at no additional cost with WVS and OVS with support for .NET and PHP. Acunetix should be considered by organizations looking for leading-edge Web application security testing and manual Web penetration testing capabilities with integrated IAST to be placed into the hands of their security testing professionals.

Strengths

• Acunetix WVS and OVS test REST, JSON and XML-based interfaces of modern and mobile-enabled Web applications; and for advanced testers, it includes an integrated HTTP sniffer and HTML5 support.
• Acunetix DeepScan engine can parse complex client-side JavaScript applications to test for vulnerabilities.
• Both WVS and OVS can detect malware or phishing links in websites and Web applications being scanned; and its OVS offering can also test systems for network- and OS-level vulnerabilities.
• Acunetix has introduced an innovative cloud-based service, AcuMonitor, for WVS and OVS, which can monitor an application over time for vulnerabilities (such as blind cross-site scripting [XSS], XML external entity [XXE], server-side request forgery [SSRF], blind out-of-band SQL injection and remote code execution) that do not provide any response to a scanner.

Cautions

• While Acunetix offers a "console in the cloud" option for automating Web testing with its standard support (result interpretation and so on), human-augmented testing is available at a separately negotiated cost.
• Acunetix does not share policies and configurations across WVS and OVS for organizations that may use both.
• Acunetix does not have SAST capabilities; however, information from SAST scanners can be correlated with Acunetix DAST results via intermediary vulnerability management solutions, such as ThreadFix and LockPath.
• Although it has IAST capabilities, these are not available for Java, and it has not yet evolved its IAST capabilities into RASP capabilities.
• Acunetix has no specific mobile application testing capabilities other than testing the HTTP-based interfaces to and from the mobile application.

Appthority

Appthority is a U.S.-based startup that was founded to provide mobile AST, risk analysis and policy management. It offers a stand-alone portal to upload private or public (third party) apps for analysis, or it can automatically import apps from the customers' enterprise mobility management (EMM)-managed devices. The portal may also be used to query its database for security ratings of about 3 million analyzed public mobile apps. This process is fully automated and uses Amazon Web Services (AWS) to achieve the high scalability of its services. Appthority's technology is for organizations of any size that are concerned about the security and risk of their own mobile apps or purchased mobile apps, as well as those that are seeking an app security analysis, a reputation risk/security rating of mobile apps, and app policy enforcement on mobile devices. Appthority provides testing as a cloud service with the ability to integrate into customers' on-premises EMM deployments.

Strengths

• Appthority provides static analysis of the binary code of mobile apps on iOS and Android devices for security issues such as lack of position-independent executables (PIE) or lack of encryption of sensitive data. It tests native and HTML5 apps. It also tests apps that include client-side JavaScript and Ajax, and apps written with frameworks such as PhoneGap and Appcelerator, or dynamic programming languages such as MacRuby.
• It provides behavioral analysis of mobile apps by executing tested apps in the mobile device emulator (or in a real device) with a purpose of detecting malicious/risky behavior exhibited in the background (such as unexpected retrieval of the contact list and transmission of it to an outside IP address), even if that application exhibits manifested benign behavior in the foreground.
• Appthority provides a mobile device agent that ensures that all apps on the device (homegrown or downloaded from app stores) are submitted for the test, and notifies the enterprise/owner if the app fails the test. It also integrates with EMM technologies (from AirWatch and MobileIron with BlackBerry BES12 planned), enabling protection for mobile devices based on detected application risk and vulnerability analysis.
• It analyzes commercial apps from app stores, provides its proprietary Appthority Trust Score, and allows corporations to build a customized mobile policy with customized application scoring — whitelisting or blacklisting application behaviors and taking automated remediation actions for compliance.

Cautions

• Appthority doesn't analyze mobile applications for source code security vulnerabilities, such as buffer overflows. Further, Appthority does not offer security testing of nonmobile (for example, Web or legacy) applications, and it does not test the back-end interfaces of mobile apps.
• It does not offer an on-premises mobile AST tool, only a cloud-based testing service.
• Appthority does not offer out-of-the-box integration with application development environments (integrated development environments [IDEs]) and bug-tracking systems, although it does provide APIs for such integration, leaving integration to the customer.
• It does not support Windows Mobile, Windows Phone or BlackBerry mobile application platforms, although Appthority plans to support Windows 10 Mobile in a future release.

Checkmarx

Checkmarx is a well-established AST vendor based in Israel that is selling its technology in North America, Europe and Asia/Pacific. It has earned a strong reputation for the quality of its SAST tools and services. Checkmarx appeals to application development and security organizations that are seeking a comprehensive SAST tool for a variety of programming languages and frameworks, including mobile applications. The SAST tool can test composite applications and provide scalability and quick turnaround times via incremental and parallel tests. Checkmarx is moving into new application security areas, specifically IAST and RASP, where it has beta offerings of these two technologies.

Strengths

• Checkmarx offers one of the strongest SAST technologies, which tests a broad variety of programming languages and offers a large number of integration options into the software life cycle (SLC), including source code repositories, build systems, bug-tracking systems, IDEs and QA testing tools.
• Checkmarx offers a universal application model that can be queried to discover vulnerabilities, and to check for code adherence to secure programming best practices. The model also enables incremental scans and analysis across components of composite applications that are written in different programming languages and with the use of different frameworks.
• Checkmarx offers SAST as a tool and as a cloud service. In addition to testing code written in various programming languages, it can test Apex, and is also a major provider of SAST for Salesforce, its partners and users. Checkmarx also offers support for many cloud platforms and frameworks, such as CloudSpokes, MediaMind and topcoder.
• Checkmarx offers SAST for Android, iOS and Windows Phone platforms, and also tests mobile apps. It tests native apps and hybrid apps using the WebView component or other browser-based components, and it is adding support for cross-platform development tools such as PhoneGap.

Cautions

• Checkmarx does not offer its own DAST for Web applications, but rather partners for DAST with Trustwave and Rapid7 (which acquired NT OBJECTives [NTO]), but it is unclear if or how the recent acquisitions will affect these partnerships.
• Its IAST for .NET is in beta, and its IAST for Java is planned for YE15. Its RASP technology is also in beta, and is planned to be generally available by the end of 2015.
• Its SAST integration with Web application firewalls (WAFs) supports only ModSecurity and not leading commercial WAFs.
• For mobile testing, Checkmarx provides only SAST. It provides partial behavioral analysis via heuristic queries. The testing of communication between the mobile app and Web services can be performed by its partner Rapid7 (NTO). It is also missing some important features for comprehensive coverage (for example, mobile agent for proactive app testing and risk/security reputation ratings of commercial apps).

Cigital

Cigital, based in Dulles, Virginia, is best-known as an application security consultancy that entered the DAST-as-a-subscription-service market with its October 2014 acquisition of India-based iViZ Security, and which also offers less well-known SAST-as-a-subscription-service testing. Cigital offers three levels of SAST and DAST as a service to its customers, all of which include a human review of the results. For DAST, Cigital's application security testing types are as follows: Dynamic Scanning Services (DSS) — automated testing; Automated Ethical Hacking (AEH) — some human-augmented testing of standardized areas, human readout; and Manual Ethical Hack (MEH) — human-augmented testing with areas of focus at the discretion of the tester. Similarly, for SAST it offers three levels: automated, standard and advanced. For both DAST and SAST, fully automated tests take less time than the advanced tests. Under its 3D licensing program, at any given point in time, organizations license how many applications can be concurrently tested (typically one). Cigital should be considered by organizations looking for human-augmented DAST and SAST as a service, and that want choice in how deeply applications are tested within a flexible licensing model.

Strengths

• Cigital is a well-known brand name in North America for application security consulting services.
• Cigital has introduced an innovative 3D licensing model where, for a fixed cost per year, customers can choose among three levels of testing for any application, and it has recently expanded this to include SAST-as-a-service testing.
• Cigital has an innovative SecureAssist tool that can check for a limited number of SAST coding issues by integrating directly within the developer's IDE (Visual Studio or Eclipse).
• In April 2015, IBM announced that it was partnering with Cigital to deliver its human-augmented DAST-as-a-service capabilities (IBM still performs its fully automated testing).

Cautions

• Although it has offices in London, Amsterdam and India, Cigital has limited brand recognition and revenue outside of North America.
• Cigital's SAST as a service is a lesser-known offering, has not yet been widely adopted and has only recently been added to its 3D licensing program.
• Cigital has no IAST or RASP testing capabilities, nor does it offer WAF integration.
• Mobile application security testing is not yet integrated with 3D licensing, human-augmented (AEH/MEH) testing is recommended, only very limited SAST is included (primarily to the extent that it improves the DAST testing), limited behavioral testing is provided, and there is no integration with EMM tools for the automated submittal of mobile apps for security tests.

Contrast Security

Contrast Security, based in the U.S., offers a single solution that provides IAST and RASP functionality. Its instrumentation approach aims to provide highly accurate vulnerability and attack detection across the SLC. The vision of Contrast's distributed approach is to bring AST closer to developers and testers, and to make AST transparent to them, with no need to buy, install and learn security testing tools, and without requiring application security experts. IAST technology is at the adolescent phase of the application security Hype Cycle; therefore, Contrast is recommended to enterprises that want to utilize the advantages of IAST and application self-testing, and yet understand that IAST has not reached its maturity. Enterprises, especially Type A (advanced IT adopters), should also start evaluating Contrast Security's RASP offering, with the understanding that this transformative security technology is still emerging.

Strengths

• Contrast Security offers a self-testing model, where security testing is driven by any application test (typically QA) that is executed automatically or manually. This process is transparent to the interested parties (that is, developers and security specialists) and does not require training.
• Its self-testing model is highly scalable. Instrumenting every test server with an IAST agent enables testing of all applications throughout the software development life cycle (SDLC).
• It enables analytics of production applications at runtime (if the production server is instrumented).
• Contrast has announced its RASP technology for Java and .NET platforms.

Cautions

• Contrast Security's IAST is currently limited to test applications written in Java, .NET and ColdFusion, although it is working on Node.js and PHP.
• It cannot observe and analyze client-side logic executed in the browser only (for example, JavaScript or Java applets). As a result, it will miss attacks such as JavaScript-based Document Object Model (DOM) XSS.
• Contrast does not have a mobile AST offering.
• Its RASP is a new offering announced in June 2015. Therefore, customers will have to evaluate their requirements against this product's capabilities.

HP

HP, headquartered in Palo Alto, California, is a worldwide provider of SAST, DAST, IAST and RASP products and services. Its flagship SAST offering, Fortify Static Code Analyzer (SCA), anchors its solutions, and the Fortify brand has been extended to its other capabilities, including WebInspect DAST and IAST. It also offers all of its AST products as services under Fortify on Demand branding, with four levels of dynamic testing, one level of static testing and three levels of mobile testing. In 2014, HP announced its intention to split into two companies, but this is expected to have no impact on HP Software (where HP security resides), which is a part of the Enterprise Group that will stay with Hewlett-Packard Enterprise. HP's AST solutions should be considered by enterprises looking for a comprehensive set of AST capabilities, either as a product or service or both combined, with enterprise-class reporting and integration capabilities.

Strengths

• HP Fortify is a well-known brand worldwide. It frequently appears on clients' shortlists, and HP is the only AST vendor that provides capabilities in all four areas: SAST, DAST, IAST and RASP. HP was one of the first to ship a commercial RASP offering, and, in late 2014, it released HP Application Defender, on-premises and as a service for Java and .NET.
• HP's SAST has the broadest language support of any of the SAST providers, and its WebInspect IAST agent for Java and .NET is included at no cost for WebInspect DAST tool customers.
• HP has a comprehensive set of enterprise capabilities, such as full SDLC integration (IDE, QA, bug tracking), Selenium support, role-based access control (RBAC), full authentication integration, SOAP, REST and JSON Web services testing, extensive WAF integration, mobile device management (MDM) integration and Sonatype integration for software composition analysis.
• HP is the only large AST vendor that provides SLAs with financial penalties on the turnaround time for its AST-as-a-service offerings.

Cautions

• Some AST capabilities are only available with the Fortify on Demand offering, such as malware detection and Universal Description, Discovery and Integration (UDDI) testing for DAST.
• Its IAST offering does not yet support PHP.
• Although it offers security testing for mobile device languages on Android, iOS and Windows Phone, it has limited behavioral mobile application security testing and its database of tested apps is relatively small (100,000 applications), and doesn't yet have EMM integration.
• The cost of equipping every developer with Fortify's SAST capabilities can be high, if an organization chooses to equip individual developers.

IBM

IBM is a global vendor based in the U.S. Its security testing solutions are primarily composed of products and services from various vendor acquisitions (including SAST [via its acquisition of Ounce Labs] and DAST [via its acquisition of Watchfire]). Over the past several years, IBM has extended its testing capabilities into IAST, and more recently into mobile AST. In addition, IBM has a large portfolio of security technologies, which, besides AST, include security information and event management (SIEM), identity and access management, data masking, database activity monitoring, endpoint protection, EMM and Web fraud prevention. In April 2015, IBM announced that it was partnering with Cigital to deliver all of IBM's human-augmented, managed DAST-as-a-service capabilities moving forward. IBM will appeal to enterprises seeking a single provider of AST technologies and technologies in adjacent security areas.

Strengths

• IBM Security AppScan is well-known for its enterprise application security testing capabilities (for example, user-customizable risk ratings), and IBM is one of the larger application security testing tool vendors.
• IBM offers SAST, DAST and IAST technologies. Its IAST for Java and .NET applications is integrated with its DAST offering, thus making IAST available at no cost to DAST users.
• IBM provides SAST analysis of JavaScript within the context of a DAST scan for testing Web applications that use JavaScript. It also offers innovative taint analysis.
• It has begun offering mobile AST as a cloud service. Its IAST for Android offering includes a version of its Java Glassbox testing technology to improve DAST scanning results when testing mobile applications. IBM is the first vendor to apply IAST to mobile applications. To expand its mobile security offerings, IBM partners with Arxan, which provides mobile application hardening technology.

Cautions

• Although it offers SAST tools, IBM has not offered SAST as a service; however, it plans a 2015 release of a fully automated SAST-as-a-service offering (initially only for Java and .NET), augmented with Watson-powered machine learning to improve testing results.
• IBM, as part of its AST as a service, provides a fully automated DAST as a service, but relies on Cigital for delivery of managed, human-augmented DAST services.
• IBM's IAST works for Java and .NET only, and has not earned brand recognition in this space compared with vendors such as Contrast Security or Quotium.
• IBM does not provide RASP and it does not offer software composition analysis (SCA).
• Its mobile AST capabilities do not include commercial application reputation ratings, proactive testing or integration with EMM technologies. Its mobile behavioral analysis for Android is in beta, and its iOS offering is planned by YE15.

NSFOCUS

NSFOCUS, based in Beijing, China, is new to the 2015 Magic Quadrant. NSFOCUS offers a variety of security solutions, including intrusion prevention system (IPS), next-generation firewalls, distributed denial of service (DDoS) protection and vulnerability assessment. For AST, NSFOCUS offers two products: NSFOCUS Web Vulnerability Scanning System (NSFOCUS WVSS) for DAST scanning of websites, and Web Security Monitoring System (NSFOCUS WSMS) for the monitoring of website vulnerabilities, malicious content, defacement and sensitive content. It also offers, WebSafe, a cloud-based service that combines WVSS and WSMS capabilities, priced on a monthly or weekly basis for scanning. NSFOCUS should be considered by organizations looking for a basic, competitively priced Web application security testing service where local language console and support for Chinese and regional languages is desired with full support for all of mainland China, or where there is a regulatory requirement to use local China-based security providers for security services.

Strengths

• NSFOCUS is a well-known security provider in China. For example, it has the largest revenue of any IPS provider in China.
• Being a Chinese-owned-and-operated company will help NSFOCUS address the growing market for application security testing solutions in China, while its North American and EMEA-based competitors may have difficulty getting traction and permission to sell in China.
• NSFOCUS also sells its own WAF product and managed WAF service offering, providing "virtual patching" for vulnerabilities discovered via WVSS.
• It has a new English language product interface, but it hasn't yet created an English language version of its cloud-based service portal.

Cautions

• Although it has offices in the U.S. and EMEA, NSFOCUS is not well-known outside of China and it will have difficulty selling into security-sensitive industries such as defense, aerospace, critical infrastructure and government outside of China.
• It offers no other WAF integration other than its own WAF.
• It offers no capability to test Web services, REST, JSON or XML-based application interfaces for more advanced Web applications.
• It offers no SAST or mobile application security testing capabilities.

N-Stalker

N-Stalker, based in Sao Paulo, Brazil, is a regionally known provider of DAST products and services. N-Stalker has several offerings: N-Stalker Free Edition (a community edition with a limited set of security checks), N-Stalker Enterprise Edition and N-Stalker Cloud Web Scan (DAST as a service that, interestingly, adds SAST capabilities for comprehensive Web application security testing). It has also recently introduced basic mobile AST capabilities, but only as a service. N-Stalker should be considered by organizations looking for easy-to-use, reasonably priced, enterprise-class Web application security testing in South America and Latin American that prefer the regional expertise that N-Stalker provides, as well as local language support for Portuguese and Spanish.

Strengths

• In addition to discovering unknown vulnerabilities, N-Stalker supports the identification and scanning of more than 1,900 commercial off-the-shelf (COTS) and open-source software (OSS) packages for more than 5,000 Common Vulnerabilities and Exposures (CVEs) related to these packages, thus delivering software composition analysis.
• N-Stalker has a broad array of enterprise features not typically found from smaller providers, such as RBAC, Selenium support, IDE integration, OAuth and OpenID support, SOAP- and REST-based Web services testing, as well as JSON remote procedure call (RPC) and Extensible Messaging and Presence Protocol (XMPP) support.
• The N-Stalker Cloud Web Scan service integrates SAST testing of the Web application. For PHP and Java Platform, Enterprise Edition (Java EE), it partners with Dognaedis (a smaller regional SAST provider) and for Python, .NET, ASP and Node.js it uses its own internally developed engine.
• N-Stalker is well-known in Brazil and neighboring countries in South America, with local language support and consoles in Spanish and Portuguese.

Cautions

• N-Stalker has limited brand awareness outside of South America.
• N-Stalker's integrated SAST capabilities are only available via its Cloud Web Scan platform, and are limited to the context of testing Web applications.
• N-Stalker has no IAST or RASP capabilities.
• N-Stalker has very limited mobile AST capabilities available only as a service, and its mobile testing service doesn't yet use Dognaedis. N-Stalker acquired and introduced Android-only capabilities from its acquisition of a small local provider in 2015.

PortSwigger

PortSwigger is a U.K.-based, privately owned vendor. Its offering called Burp Suite is a set of integrated tools which together provide advanced DAST and client-side SAST capabilities. PortSwigger offers free editions of Burp Suite, and an aggressively priced (at approximately $300 per user per year) Burp Suite Professional edition. It offers a proxy for the real-time capture of Web interactions, including back-end interfaces for dynamic testing. This technology is highly popular, and other vendors — even some of PortSwigger's competitors — support the use of Burp Suite's proxy recorder. Burp Suite Professional should be considered by organizations seeking a powerful DAST tool with advanced testing capabilities, which, as of yet, lacks enterprise-class features (for example, SLC integration or RBAC console access and reporting).

Strengths

• PortSwigger's Burp Suite is one of the most widely adopted DAST tools in the DAST market.
• PortSwigger's products are highly customizable and extensible, and can be API-driven. The community of Burp users have developed a number of useful extensions/additions to Burp (such as SAML Editor, WSDL Wizard and Payload Parser) that are available to Burp users.
• Burp Suite enables JavaScript code static analysis when it conducts DAST.
• PortSwigger introduced the Burp Collaborator service component that interacts in real time with a running Burp DAST tool to improve detection of such vulnerabilities as blind XSS, XXE and SSRF.

Cautions

• PortSwigger does not offer SAST, IAST and RASP technologies.
• PortSwigger does not offer mobile application code analysis, behavioral analysis, integration with EMM or commercial app reputation ratings; although some organizations and vendors use its Burp Suite proxy to analyze traffic between Web services and mobile apps.
• PortSwigger does not offer its DAST as a service.
• PortSwigger does not offer integration with WAFs, IDEs or QA systems.

Pradeo

Pradeo is a privately held startup based in France. Its mobile AST technology is delivered as three separately purchasable components: (1) AuditMyApps — a platform for app security testing; (2) CheckMyApps — a platform for mobile applications' security policy management; and (3) CheckMyApps API — a set of APIs. Pradeo offers mobile AST as a service for iOS, Android, and Windows 8 and Windows Phone platforms. Pradeo's technology is for organizations looking to conduct comprehensive code and behavioral analysis of their mobile applications.

Strengths

• Pradeo's testing services use a combination of static code analysis (reverse-engineered bytecode/binary code analysis) and dynamic, behavioral analysis of mobile applications.
• The dynamic behavioral analysis executes the app in a virtual, emulated environment, and monitors the network connections to see if an app is accessing someone's contact information and if that data is subsequently being sent somewhere else.
• It offers its technology as a service, either directly from the cloud or using an optional on-premises virtual appliance to keep scanning local to the enterprise.
• Pradeo offers its own EMM agent, which can act on the results from its mobile AST. It also integrates with EMM tools from AirWatch, MobileIron, Good Technology and SAP.

Cautions

• Pradeo doesn't analyze mobile applications for security vulnerabilities such as buffer overflows, only for risky behaviors (such as unexpected retrieval of the contact list and transmission of it to an outside IP address). Further, Pradeo does not offer AST for nonmobile (for example, Web and legacy) applications, and although it can identify them, it does not test the back-end interfaces of mobile apps.
• Binaries can be automatically retrieved from iOS and Android, but not Windows Phone.
• It does not offer a tool, nor does it offer built-in integration with application development environments and bug-tracking systems, although it has an API for IDE integration, leaving the integration work to the customer.
• Pradeo's smaller size and low visibility inhibits its appearance on customers' shortlists outside of EMEA.

Qualys

Qualys, based in Redwood City, California, is a provider of cloud-based security services and offers DAST-as-a-service capabilities. Like the rest of the Qualys' offerings, its Web Application Scanning (WAS) service offering is completely automated and is integrated with the other Qualys services in its Web-based customer portal. A consistent portal, platform, users/roles and workflow is used for WAS as well as its WAF- and Vulnerability Management-as-a-service capabilities. To access internal applications for testing, Qualys uses a physical or virtual appliance to establish secure VPN connectivity. Because of its low cost, in many cases, enterprises using a more expensive competitive offering for their critical applications will supplement with Qualys' scanning for the rest of their application portfolio. Qualys should be considered by any organization looking for basic, automated Web application security testing as a service at an extremely competitive price.

Strengths

• Qualys offers one of the lowest costs per application scanned of any of the DAST-as-a-service providers, and its WAS business continues to grow significantly year over year.
• Qualys DAST scanning also scans for the presence of malware on websites.
• Qualys has introduced progressive scanning, enabling it to pick up scanning where it left off, useful for large sites where scanning can't complete in a given time window.
• All subscriptions include 24/7 technical support.
• Qualys has extensive WAF integration, including its own WAF-as-a-service offering.

Cautions

• Without human augmentation, there are limits as to the types of vulnerabilities that can be discovered using a fully automated approach. Qualys will refer customers to its partners for additional professional services, including having results reviewed by a human.
• Although Qualys offers basic Web Services Description Language (WSDL) and SOAP Web services fuzzing, it doesn't support the rest of the WS-* standards, nor does it test RESTful application interfaces or test the content within JSON messages.
• Qualys has no SAST-as-a-service capabilities and no mobile AST capabilities other than testing the Web-services-based interfaces used by the mobile application.
• Qualys offers no IAST or RASP capabilities.
• Qualys provides no out-of-the-box trouble ticketing system integration for WAS vulnerabilities discovered, although this is scheduled for 2015.

Rapid7 (NTO)

In 2015, Rapid7 acquired NT OBJECTives (NTO), a provider of DAST products and services based in Irvine, California. Rapid7 is best-known for its network vulnerability scanner capabilities, and with the acquisition, extends its vulnerability scanning solutions to the application layer. Its offerings include AppSpider Pro (formerly NTOSpider; its completely automated Web app scanner), AppSpider Enterprise (its enterprise portal), and AppSpider Enterprise OnDemand (DAST as a service with five levels of testing). Rapid7 should be considered by organizations looking for enterprise-class DAST products and services as a competitive alternative to the larger providers. The acquisition of NTO by Rapid7 occurred during the research process for this Magic Quadrant, and we will be watching the integration of NTO into Rapid7's culture and business.

Strengths

• Innovative "universal translator" technology normalizes how requests are handled in the application for specific attacks across HTML forms, SOAP, JSON, REST, and Action Message Format (AMF) and other formats as they emerge. Further, customers can create their own parsers for custom protocols using C# programming.
• Rapid7 has a broad array of enterprise AST capabilities: enterprise console, RBAC, one-click vulnerability verification, bug-tracking integration and extensive WAF integration.
• AppSpider Enterprise and AppSpider Enterprise OnDemand use an innovative cloud platform behind the scenes to automatically create new scan engines from Rapid7's cloud to handle increased workloads with cloud servers in the U.S. and Europe.
• Rapid7 has added workflow-based sequence attacking for testing complex workflows while maintaining session state (such as filling a shopping cart with multiple items then checking out).

Cautions

• Rapid7's AppSpider solutions have limited brand name recognition in EMEA and Asia/Pacific.
• Rapid7 has no SAST capabilities (even for client-side JavaScript within Web applications), although it has partnerships with Checkmarx and Coverity for SAST and mobile SAST testing.
• Rapid7 has no IAST or RASP capabilities.
• Review of DAST-as-a-service results for basic vulnerability verification by a human being is charged separately (as are business logic testing and penetration testing).

SiteLock

SiteLock, based in Scottsdale, Arizona, is a new entrant to the AST Magic Quadrant for 2015 and is focused on Web application security testing. SiteLock is best-known as a result of its partnership with Web hosters such as GoDaddy. SiteLock offers three tiers of completely automated Web application scanning services (application scan, application pen testing and its SecureVIP solution) using a combination of its own tools and commercial tools for Web hosting customers, as well as those customers that come to SiteLock directly. It also has integrated network vulnerability scanning of the Web server, as well as SAST capabilities specifically for Web applications developed in Java or PHP. It has no tool offerings and sells its DAST with integrated SAST solutions as a service only. SiteLock should be considered by midsize organizations seeking comprehensive Web application security testing combining both DAST and SAST analysis, and that includes network vulnerability scanning as well.

Strengths

• The use of Web hosters as a channel has helped SiteLock grow its installed base and reach midsize customers that the larger AST vendors have difficulty reaching.
• SiteLock is one of a few vendors that provides both DAST and SAST capabilities, and that combines both approaches in a single-priced comprehensive Web application security scan. For SecureVIP customers, SAST scanning of binaries (in a service SiteLock calls TrueCode) is delivered behind the scenes via SiteLock's relationship with an undisclosed OEM partner.
• All of SiteLock's Web security testing services include automated malware detection and removal capabilities.
• SiteLock Web security testing scans the Web platform software for known vulnerabilities, as well as for the use of open-source components with known vulnerabilities, thus providing a basic form of software composition analysis.
• For DDoS and WAF services, SiteLock has partnered with an undisclosed cloud-based provider.

Cautions

• SiteLock does not yet have a strong brand in AST, and is not well-known outside of the U.S.
• SiteLock's AST-as-a-service capabilities are basic. For example, it supports forms-based authentication only (no SAML, OAuth or others), and it has no support for testing Web services, REST or JSON interfaces of the Web application. It has no IDE, QA, bug-tracking system or SIEM integration.
• SiteLock does not offer IAST, RASP or mobile AST capabilities. SiteLock has no integration with WAFs, and integration with its OEM-powered WAF service requires SiteLock's professional services at extra cost (but is included for SecureVIP customers).
• SiteLock only tests Web applications (not stand-alone native applications). Human-augmented Web scanning services are available at extra cost from SiteLock's professional services but only with the SecureVIP level of service.
• SiteLock's relationship with its SAST OEM partner could end, leaving SiteLock with no longer-term SAST capabilities.

Synopsys

In July 2015, during the creation of this research, Synopsys closed the acquisition of Quotium's Seeker product (following acquisitions of static code analysis vendor Coverity in 2014, and of software composition analysis and dynamic testing of connected devices vendor Codenomicon, in 2015). Headquartered in France, with R&D in Israel, Quotium is a point solution vendor that created an IAST product called Seeker. Seeker should be considered by enterprise security and application development organizations that are seeking to adopt an innovative IAST technology that provides effective vulnerability detection, and that can be reasonably easy to embed into the software life cycle. Gartner will be watching the integration of Seeker into Synopsys' portfolio of quality and security testing technologies.

Strengths

• Seeker's IAST includes JavaScript analysis, and analysis of stored procedures and database transactions.
• Seeker is one of the most broadly adopted IAST technologies in the IAST market.
• Seeker's Selenium support, quick and accurate results, and out-of-the-box integration with build servers and bug-tracking solutions fit well into agile and DevOps/DevOpsSec approaches.
• Seeker includes IAST for Java, .NET and PHP application server platforms, as well as support for PL/SQL and T-SQL.
• As part of the reporting, Seeker provides visualization of the exploit, thus making it easier for the user to fix the detected vulnerability.

Cautions

• Quotium (and Seeker) lacked the brand-name recognition of the AST market leaders, and its market presence was stronger in EMEA than in North America. The technology acquisition by Synopsys should help these issues.
• Seeker does not include mobile AST. However, it can observe and learn how the mobile application interacts with the back-end servers (for servers that Seeker supports), and test these back-end servers.
• Seeker does not include RASP capabilities, although Synopsys is working on evolving the Seeker DataHound feature (now in beta) into a RASP offering.
• Its WAF support is limited to F5 only.

Trustwave

Trustwave, based in Chicago, is a worldwide provider of security-related products and services. Trustwave expanded its application security testing business with its 2014 acquisition of Cenzic. In 2015, Singtel announced its intention to acquire Trustwave, which will remain a stand-alone business unit. Trustwave offers a portfolio of application-layer products and services, many of them required to achieve PCI compliance, such as Web application firewalling, Web application vulnerability assessment, network vulnerability scanning and database activity monitoring, and it is a dominant player in the PCI assessment market. In its AST offerings, Trustwave is focused on offering DAST products (App Scanner Enterprise) and cloud-based services. In its Managed Security Testing (MST) offering, there are options for application penetration testing, managed application scanning, self-service application scanning and mobile application security testing. Trustwave should be considered by organizations looking for an enterprise-class DAST solution with product and service options at competitive pricing, or a "one-stop shop" for PCI-compliance-related products and services. The announcement of the acquisition of Trustwave by Singtel happened during the research process for this Magic Quadrant, and we will be watching the process of Trustwave integration into Singtel's culture and business once the acquisition closes.

Strengths

• In addition to its WAF service, Trustwave offers several relevant Web monitoring services, including Web malware monitoring and Web content monitoring services.
• For fuzzing, Trustwave's abstraction layer was improved in 2014 to handle XML REST services and JSON requests (as well as SOAP/WSDL, which it already handled) for the testing of modern Web services and Google Web Toolkit (GWT) format.
• Trustwave offers a broad array of enterprise capabilities, including IDE integration, bug-tracking integration, quality testing tool integration, vulnerability replay, RBAC, proprietary Hailstorm Application Risk Metric (HARM) risk scoring and a large selection of WAF integrations, including Trustwave's own WAF.
• Trustwave offers three tiers of mobile application security testing services for iOS, Android, Windows Mobile and BlackBerry that focus on the dynamic interfaces for these applications.

Cautions

• Trustwave does not offer a SAST product or service, nor does it partner to provide this.
• Trustwave does not have a mobile AST product, although it offers mobile AST services. However, it doesn't perform a detailed static analysis of the mobile application.
• Trustwave does not offer IAST or RASP capabilities.
• Its stand-alone desktop version of its DAST tool (used primarily by stand-alone security testers) is no longer being improved. All new development is focused on the enterprise-console-based and cloud-based versions of Trustwave's testing platform.

Veracode

Veracode is a U.S.-based, well-established and rapidly growing provider of SAST and DAST cloud services, software supply chain testing and mobile AST. For SAST, Veracode has been a pioneer in the analysis of binary code, not requiring the source code for testing. Its 2012 acquisition of Marvin Mobile Security accelerated its mobile AST capabilities, where it was also an early innovator. In 2014, Veracode added integrated software composition analysis capabilities into its AST services for the identification of vulnerable open-source components. Veracode's AST services will meet the requirements of organizations looking for a broad set of AST services — SAST, DAST and mobile AST — that want to delegate their AST and SCA to a third-party expert with a strong reputation for the quality of its services and with demonstrated innovation in application security.

Strengths

• Veracode offers SAST, DAST and mobile AST all as cloud services. Results of the different types of testing can be integrated into a single dashboard to simplify vulnerability management and remediation.
• Veracode offers scalable AST as a service and tests tens of thousands of applications per year. It includes an innovative "Web application perimeter monitoring" service that discovers and tests Web applications on the public Internet.
• Veracode's mobile AST as a cloud service includes static bytecode and binary code analysis, as well as behavioral analysis in the mobile device emulator or in a physical device. It also offers a Mobile Application Reputation Service (MARS) for commercial application risk/security ratings for the most frequently downloaded apps from app stores, with EMM integration for MobileIron, IBM (Fiberlink) and AirWatch. Veracode mobile testing supports iOS, Android, BlackBerry and Windows Mobile platforms.
• For integration into SLC processes, Veracode offers built-in integration with multiple IDEs, bug-tracking systems and APIs for integration into QA environments and build servers.

Cautions

• Veracode does not offer AST tools, only AST as a service. An on-premises option is not generally available; thus, for SAST and mobile AST an organization's binary code must be uploaded to be analyzed in Veracode's platform, but the code is not retained following the analysis.
• For compiled languages such as Java, C/C++ and Objective-C, applications must be in a compiled state to be analyzed. Veracode doesn't permit customers to scan stand-alone C, C++, or Objective-C code that hasn't been compiled with debug mode. However, Veracode will scan non-debug code if the customer has a non-debug library as a part of an application (for example a commercial library). However, in these cases for C, C++ and Objective-C there is some degradation in accuracy.
• Veracode does not yet offer IAST, although it has announced an OEM partnership with Contrast Security to deploy a version of Contrast technology in Veracode's testing centers and to offer IAST as a service to Veracode's customers.
• Veracode does not offer RASP, although it shows this as a roadmap item.
• Veracode's WAF integration is limited to Imperva and ModSecurity.

Virtual Forge

Virtual Forge is a German-headquartered SAST solution provider with a specific focus on the security testing of SAP's Advanced Business Application Programming (ABAP) language. Virtual Forge offers its solution, CodeProfiler, as a product or as a service. Virtual Forge can perform dynamic testing of the secure configuration of the SAP environment with its SystemProfiler. It also offers SAP penetration testing services. IBM and Checkmarx resell Virtual Forge's ABAP testing capability as a part of IBM's AppScan and Checkmarx's SAST solutions. Virtual Forge should be considered by security-sensitive organizations that have extended and customized their SAP environments.

Strengths

• One of only two vendors capable of SAST of ABAP code (the other is HP), Virtual Forge has deep SAP expertise, is SAP-certified, and leverages the native SAP trouble-ticketing workflow in SAP's Solution Manager and other environments.
• In addition to SAST of ABAP, Virtual Forge has also added SystemProfiler that scans the SAP environment for secure configuration and up-to-date patching.
• CodeProfiler is integrated into the SAP development environment (ABAP Workbench, Eclipse and Web IDE) and will be familiar to SAP developers without them having to learn a separate console; and it includes automated code correction capabilities (a capability it refers to as "quick fix") within SAP's IDE.
• In 2015, Virtual Forge integrated the output from its CodeProfiler and SystemProfiler results, improving the efficacy of CodeProfiler's findings, and the output from its profiling tools can also feed other security monitoring systems.

Cautions

• As SAP UI5 is phased in, Virtual Forge needs to improve its HTML5-testing capabilities, as well as migrate its UI over to UI5.
• CodeProfiler is integrated into the SAP development environment, making it unfamiliar to most security testing professionals.
• Virtual Forge doesn't provide other SAST language support, such as Java, which is also often used in SAP environments, but it does offer this via its partners Checkmarx and IBM, as well as via integration with ThreadFix.
• While Virtual Forge has correlated the results of CodeProfiler and SystemProfiler, it has no RASP capability and no true IAST, although it is working on an IAST offering for SAP.
• Virtual Forge offers no mobile AST or out-of-the-box WAF integration.

WhiteHat Security

U.S.-based WhiteHat Security is a global, well-established security provider of DAST as a service, and it was an early pioneer for this type of service. It has also added SAST as a service using an on-premises appliance to keep scanning local. Its SAST and DAST services are scalable and are capable of testing tens of thousands of applications a year with broad WAF integration. WhiteHat Security has introduced a number of capabilities to help customers shift from focusing only on vulnerabilities to building security programs that manage risk, such as Factor Analysis of Information Risk (FAIR)-based quantification of application risk, the WhiteHat security index for comparisons with peers and, with its Elite offering, a money-back guarantee, including a dedicated customer success manager. WhiteHat Security should be considered by organizations looking to delegate their DAST and, to a lesser degree, SAST and mobile AST to an expert third-party testing service provider, as well as organizations looking to evolve their AST programs to more of a risk-based approach.

Strengths

• As a differentiator, the results of all of WhiteHat's DAST and SAST scans are reviewed by a human expert before delivery to the customer.
• It has an innovative architecture for SAST scanning that doesn't require the code to compile (it can be in development), performs the scanning on-premises for the customer and submits only small pieces of code to WhiteHat for human review, so that only a very small percentage (typically less than 1%) of the code leaves the organization.
• It offers correlation between its SAST and DAST when both are used to test an application; specifically, SAST discoveries can be submitted for DAST execution to confirm or disprove suspected vulnerabilities.
• It offers insurance of its efficacy: If WhiteHat fails to identify a vulnerability it should have found, which is later exploited by hackers, then WhiteHat Security pays damage fees up to $500,000 in addition to the customer's subscription fee.

Cautions

• WhiteHat Security does not sell DAST and SAST tools, but rather testing services only. However, its on-premises virtual appliance can keep scanning and scanning results local.
• WhiteHat Security provides SAST for a limited number of programming languages: Java, C#, JavaScript, PHP and Objective-C. Its SAST has the lowest adoption among SAST vendors.
• For mobile AST, WhiteHat Security provides source code analysis for Objective-C and Java, but does not offer automated mobile behavioral testing, reputation service, proactive testing or integration with EMM (behavioral testing is available via a manual assessment using WhiteHat professional services).
• WhiteHat Security does not offer IAST and RASP; however, RASP is on its roadmap.

Vendors Added and Dropped

We review and adjust our inclusion criteria for Magic Quadrants and MarketScopes as markets change. As a result of these adjustments, the mix of vendors in any Magic Quadrant or MarketScope may change over time. A vendor's appearance in a Magic Quadrant or MarketScope one year and not the next does not necessarily indicate that we have changed our opinion of that vendor. It may be a reflection of a change in the market and, therefore, changed evaluation criteria, or of a change of focus by that vendor.

Added

• Cigital
• NSFOCUS
• SiteLock

Dropped

• Trend Micro

Inclusion and Exclusion Criteria

We included in this Magic Quadrant vendors that met the following criteria:

1. Vendors that provide a dedicated application security testing solution (product, service or both; with SAST, DAST, IAST or mobile application security testing capabilities).
2. Vendors that provide AST as a service using a repeatable, cookie-cutter subscription-based model using at least some of its own testing tools to enable its testing capabilities.
3. The vendor must have a 2014 revenue of at least $4 million specific to application security testing, or be a provider of a significant and new AST capability, such as mobile AST or IAST.
4. The vendor must be capable of providing at least eight production references that can be surveyed.

We did not include in this Magic Quadrant:

• Vendors that provide services, but not on a repeatable, predefined subscription basis — for example, providers of custom consulting application testing services, contract pen testing, professional services and other nonsubscription services.
• Vendors that provide network vulnerability scanning, but do not offer a separately purchasable AST capability, or vendors that offer only some Web-application-layer dynamic scanning.
• Vendors that offer only penetration testing products and services.
• Vendors that offer network protocol testing and fuzzing solutions.
• Consultancies that offer AST services.
• Vendors that are focused on application code quality and integrity testing solutions, which have some limited AST capabilities.
• Open-source offerings, because they do not offer enterprise-class capabilities and security-as-a-service delivery.

Note that there are several smaller players with specific technical or regional expertise that did not meet the inclusion criteria and so fall outside the scope of this Magic Quadrant, but that may be worth considering (see Note 1).

Evaluation Criteria

Ability to Execute

Product or Service: This criterion evaluates the vendor's core AST products and services. It includes current product/service capabilities, quality and feature sets. We give higher ratings for proven performance in competitive assessments. We also give higher ratings to vendors that appeal to an expanse of users in the SLC process (such as information security specialists as well as development and QA/testing specialists), and that offer a choice of AST products and AST testing services.

Overall Viability (Business Unit, Financial, Strategy and Organization): This is an assessment of the organization's or business unit's overall financial health, the vendor's focus on AST, and the likelihood that the company will continue investing in the AST market. We also evaluate a vendor's estimated AST market share, AST revenue amount, the number of AST customers, the number of installed AST tools, and AST expertise.

Sales Execution/Pricing: We account for the company's global reach, pricing model (including multiple types of offerings addressing different buyers and use cases) and product/service/support bundling. We review the vendor's capabilities in all presales activities and the structure that supports those activities. This includes customer feedback on deal management, pricing and negotiation, and presales support, as well as the overall effectiveness of and customer receptiveness toward the sales and partner channels worldwide.

Market Responsiveness and Track Record: We look at the vendor's ability to respond, change directions, be flexible, and achieve competitive success as opportunities develop, competitors act, customer needs evolve and market dynamics change. We evaluate AST market awareness, the vendor's reputation and clout among security specialists, the match of the vendor's broader application security capabilities with enterprises' functional requirements, and the vendor's track record in delivering innovative features when the market demands them. We also account for vendors' appeal with security technologies other than AST but that complement AST.

Marketing Execution: We look at the clarity, quality, creativity and efficacy of programs designed to deliver the organization's message to influence the market, promote the brand and business, increase awareness of the products, and establish a positive identification with the product/brand and organization in the minds of buyers. This "mind share" can be driven by a combination of publicity, promotional initiatives, thought leadership, word of mouth and sales activities, including the number of times the vendor appears on Gartner clients' shortlists.

Customer Experience: This is an evaluation of the AST solution's functioning in production environments. The evaluation includes customer experiences with ease of deployment, operation, administration, stability, scalability and vendor support capabilities. It also includes relationships, products and services/programs that enable clients to be successful with the products evaluated. Specifically, this includes the ways customers receive technical support, as well as the vendor's willingness to work with its clients to customize the product or service, to develop specific features requested by the client and to offer personalized customer support.

Table 1. Ability to Execute Evaluation Criteria

Evaluation Criteria	Weighting
Product or Service	High
Overall Viability	High
Sales Execution/Pricing	Medium
Market Responsiveness/Record	High
Marketing Execution	High
Customer Experience	High
Operations	Not Rated

Source: Gartner (August 2015)

Completeness of Vision

Market Understanding: We evaluate the vendor's ability to understand buyers' needs and translate them into products and services. AST vendors demonstrating the highest degree of market understanding have responded to emerging customer requirements in areas such as providing comprehensive DAST, SAST, IAST and mobile capabilities. Higher ratings are given to techniques and approaches that are proven to improve accuracy. We evaluate the ease of an AST solution's native integration with multiple popular IDEs, source code management systems, bug-tracking systems and quality assurance testing environments. The enterprise console is an important element in providing enterprisewide consolidation, analysis, reporting and rule management across a number of installed scanners; user-friendliness; and ease of identifying and enabling customers to focus on the most severe and high-confidence vulnerabilities. We give higher ratings to the vendors that have the ability to provide AST product options and testing as a service options, with unified visibility and reporting across both.

Marketing Strategy: We look for a clear, differentiated set of messages consistently communicated throughout the organization and externalized to customers through the website, advertising, customer programs and positioning statements. The visibility of the vendor's security research labs is also a consideration.

Sales Strategy: Here, we assess the vendor's strategy for selling its AST solutions, including worldwide direct sales presence, channels and partners to target a worldwide installed base, including local sales offices to support regional sales efforts.

Offering (Product) Strategy: We assess the vendor's approach to product development and delivery. This includes the vendor's focus on AST with tools and services, as well as leading-edge capabilities such as IAST, RASP and mobile AST. We consider the vendor's ability to provide a balance in satisfying the needs of Type A (leading-edge), Type B (mainstream) and Type C (risk-averse) enterprises, and the needs of typical enterprises and specialized clients. We give higher scores to the vendors that offer a variety of solutions to meet different customer requirements and testing program maturity levels.

Innovation: Here, we evaluate the vendor's development and delivery of a solution that is differentiated from the competition in a way that uniquely addresses critical customer requirements. We give a higher rating to vendors that are evolving toward the vision of enterprise security intelligence with DAST/SAST interaction, integration and correlation (including offering IAST), thus enabling higher accuracy and breadth of security coverage, as well as advanced analytics, contextual assessments, and support for optimal security and risk management decisions across the enterprise. We also give a higher rating to vendors that develop methods to make security testing more accurate (for example, decreasing false-positive and false-negative rates). In addition, we give higher ratings to vendors with the ability to innovate in mobile AST; to provide static, dynamic and behavioral testing; and to provide security/risk reputation scoring of commercial mobile applications, and integration with protection (for example, MDM) technologies. Other areas of innovation include application protection features (for example, RASP); out-of-the-box integration with application protection mechanisms, such as WAFs and IPSs; integration with governance, risk and compliance (GRC) and SIEM technologies; offering software composition analysis; innovative ways of delivery (for example, security testing as a service); support for DAST testing of SOAP and RESTful HTTP applications and cloud services; testing of and integration with cloud applications and platforms (such as Salesforce, Azure and Amazon); and AST for modern rich Internet applications (RIAs).

AST solutions should provide a variety of options for testing — for example, stand-alone engines for security professionals, integration into development tools for developers, and integration into QA for QA testers. The AST solution should provide the options to submit jobs to on-premises testing engines and to a testing service provider, while also providing a unified view and reporting across all these testing options.

Geographic Strategy: Here, we evaluate the worldwide availability of and support for the offering, including local language support for tools, consoles and customer service. Ideally, the vendor would provide worldwide availability, with local language and local service and support options.

Table 2. Completeness of Vision Evaluation Criteria

Evaluation Criteria	Weighting
Market Understanding	High
Marketing Strategy	High
Sales Strategy	Medium
Offering (Product) Strategy	High
Business Model	Not Rated
Vertical/Industry Strategy	Not Rated
Innovation	High
Geographic Strategy	High

Source: Gartner (August 2015)

Quadrant Descriptions

Leaders

Leaders in the AST market demonstrate breadth and depth of AST products and services. Leaders should provide mature, reputable SAST, DAST, mobile AST and, desirably, IAST techniques in their solutions. Leaders also should provide organizations with AST-as-a-service delivery models for testing, or with a choice of a tool and AST as a service, using a single management console and an enterprise-class reporting framework supporting multiple users, groups and roles.

Challengers

Challengers in this Magic Quadrant are vendors that have executed consistently, typically by focusing on a single technology (for example, SAST or DAST) or a single delivery model (for example, on AST as a service only). In addition, they have demonstrated substantial competitive capabilities against the Leaders in this particular focus area, and also have demonstrated momentum in their customer base in terms of overall size and growth.

Visionaries

Visionaries in this Magic Quadrant are vendors that are advancing the emerging areas of IAST, mobile AST and RASP. The goal of IAST is fast and accurate security testing that is suitable for use in development, where minimal security expertise is present and accurate results are needed quickly (for example, to support agile development and DevOps development models). Mobile testing is a set of existing and new technologies and methods for ensuring the security of mobile applications. RASP is an emerging capability for the real-time protection of production applications that complements IAST.

Niche Players

Niche Players offer viable, dependable solutions that meet the needs of specific buyers. Niche Players are less likely to appear on shortlists, but fare well when considered for business and technical cases that match their focus. Niche Players may address subsets of the overall market, and often can do so more efficiently than the Leaders. Enterprises tend to pick Niche Players when the focus is on a few important functions, on specific vendor expertise, or when they have an established relationship with the vendor. Niche Players typically focus on a specific type of AST technology, language or delivery model, or on a specific geographic region.

Context

Attackers have increased the sophistication and frequency of their attacks, motivated financially by the theft of monetary assets, intellectual property and sensitive information. In most cases, users and systems interact with an enterprise's sensitive data using applications — and it is within this application code that attackers are finding and exploiting vulnerabilities to get at the underlying data. To use an analogy, the sensitive information is the crown jewel and the applications act as the treasure chest. To get at the jewels, attackers need to attack vulnerabilities in the application container. These vulnerabilities have included SQL injection, cross-site request forgery (XSRF) and XSS, which are focused on manipulating applications and stealing or tampering with sensitive data. Hackers easily gain access to open-source technologies that enable remote application inspection and probing. New application delivery models and platforms (such as a cloud and mobile) and technologies (such as RIAs, mobile app programming languages and frameworks) pose new security risks, because application security technologies and processes have not been developed or matured for them.

Enterprises are increasingly understanding the need to implement a comprehensive, life cycle approach to application security. Today's application security markets offer a variety of reasonably mature technologies, and demonstrate innovations that are capable of deterring new threats brought to life by new social and business phenomena, such as the adoption of cloud computing services and mobile applications.

Market Overview

DAST and then SAST technologies emerged in the early 2000s as two isolated silos. They gained initial adoption in the 2004 to 2006 time frame, with organizations typically adopting DAST first, then SAST. Since 2006, Gartner research has stated that the future of AST was in using SAST and DAST. Since that time, leading AST vendors have adopted Gartner's vision, and have evolved isolated technology silos that featured combined, then correlated, and now interactive solutions.

The market and solutions have continued to evolve to support enterprise-class requirements, such as an enterprise console with RBAC providing consolidated risk-based reporting. Solutions now provide robust integration with SLC technologies such as application development IDEs and QA testing. Most provide integration between AST and adjacent security technologies (for example, SIEM and GRC reporting systems). They have also been building integration capabilities with protection technologies, specifically with WAFs or mobile device management for mobile platforms.

To make adoption even easier and broader, many vendors now offer AST as a subscription service. As a result, these technologies have reached the point where cost and risk of adoption are well-balanced. Market innovation continues, and we have witnessed the emergence of such technologies as IAST (which increases the accuracy and breadth of vulnerability detection) and RASP (which is capable of detecting and preventing real-time attacks). Also, we have witnessed startups and established vendors innovating in the mobile applications security space. Today's market is filled with numerous vendors, ranging from innovative startups to established large companies that offer a variety of AST solutions.

AST Technologies

At a high level, AST capabilities fall into four broad categories — SAST, DAST, IAST and mobile AST:

• SAST technology (sometimes referred to as white-box testing) analyzes applications for security vulnerabilities at programming and/or testing SLC phases. SAST technology's advantages include the following: (1) Vulnerability analysis typically starts early in the SLC, thus making remediation less expensive; and (2) SAST determines the exact address of the suspected vulnerability because it analyzes applications' source code, or bytecode or binary code. However, at the same time, SAST technology has a serious weakness/limitation: A detected vulnerability may not be real (false positive) or may never be executed (to say nothing of being exploited) in the application's "real" life, during the operation phase of the SLC. SAST technologies differentiate on their support for multiple languages and frameworks. In addition, a small number of vendors have the capability to test native binary code, useful for testing third-party libraries and applications where access to the source code is not available.
• DAST technology analyzes applications dynamically in an assembled running state — typically, during operation or testing phases, which is an important advantage. DAST can often accurately identify the exploitability of the potential vulnerabilities it finds, because it analyzes a real application response to the dynamic tests. However, when a vulnerability is detected, DAST technology cannot point to the line of code where it originates, because DAST is a black-box technology that does not have access to source code. Although DAST technologies are relatively mature, there is significant change to the nature of Web applications occurring that require DAST solution vendors to evolve in order to support widespread adoption of HTML5, complex client-side JavaScript implementations, and the extensive use of interfaces and Web services using RESTful and JSON-based implementations.
• IAST technologies instrument an application or its runtime environment and then monitor the application's behavior under testing conditions. During the test, an inducer typically executes test/attack scenarios as inputs for vulnerability testing in order to observe applications' input and output, application logic execution, execution of libraries and data flow during the test. As a result, IAST determines whether a vulnerability is exploitable with increased accuracy, and can identify where specifically the vulnerability is located in the code (see "Evolution of Application Security Testing: From Silos to Correlation and Interaction" [Note: This document has been archived; some of its content may not reflect current conditions]).
• Mobile AST is, in fact, a specific combination of all of the techniques targeted at the comprehensive testing of mobile applications. Ideally, a mobile application is tested statically, its Web services interfaces are tested dynamically for security vulnerabilities, and its behavior is also analyzed both statically and dynamically to uncover potentially risky behavior that the enterprise needs awareness of. Mobile AST requires explicit knowledge of the programming languages and frameworks specific to the mobile environment. Proper mobile AST requires six key capabilities (see "Six Principles of Mobile App Security Testing").

AST Delivery Models

AST technologies can be delivered as a testing tool or testing as a service (a delivery model in which application security is delegated to third-party professional security providers that conduct their services remotely, typically via the Internet). Most AST vendors have the option to deliver their capabilities as a service, and offer these alongside their application security products. Some vendors have exclusively focused on AST as a service and do not offer products at all.

Many organizations will use a combination of on-premises tools and AST as a service. For example, testing their most critical applications with their limited testing professionals and using AST as a service for the rest of their application portfolio. Maturity of the service differs for different technologies. For example, DAST services are more mature, while SAST services are less mature, because they often require uploading of the application's code into the service provider's site — a requirement that complicates clients' willingness to adopt SAST as a service. We have also witnessed IAST and mobile AST delivered via a subscription service model.

AST and WAF Integration

Application security detection and protection technologies have inherent limitations that impact their accuracy and risk assurance capabilities. These limitations could be substantially mitigated if AST solutions designed for detection and application security protection technologies interacted and shared knowledge.

The accuracy of a WAF increases when an AST technology (typically, DAST but also SAST and IAST) passes it detected security vulnerabilities and attack patterns, so that the WAF can terminate sessions that match malicious patterns. Even if the WAF, in its log or alert mode, has identified a suspicious traffic pattern, the correlation with DAST analysis results provides greater confidence that the pattern can be safely used in WAF protection mode. Detected mismatches between discoveries made by a WAF and DAST should be forwarded to DAST for further analysis (see "Magic Quadrant for Web Application Firewalls").

There is also benefit in sharing intelligence in the other direction — when a WAF provides input for an AST technology. For example, a WAF becomes an integral part of the AST conducted by DAST. Even a WAF placed into "observation mode" can provide a great deal of information about the size, boundaries and content of the Web application it is monitoring. For example, a WAF can provide lists of reachable URLs and "real" parameters from its logs. DAST could use this for its crawler, or to infer proper page flow and fuzzing parameters. This information can also be used for code coverage confirmation by comparing the WAF visibility to the DAST analyses. If the analyses do not match, then DAST test scenarios should be expanded based on the information received from the WAF. Using a WAF as part of the DAST process also enables better prioritization of DAST scans. A WAF can provide such information as the frequency of which content is requested, as well as which parts of the application/website are the most popular — and, therefore, might be more important to test than others (see "Application Security Detection and Protection Must Interact and Share Knowledge" [Note: This document has been archived; some of its content may not reflect current conditions]).

AST and RASP

Over the past two years, some vendors have been working on a new technology — RASP — to offer an enhanced way to protect applications (see "Runtime Application Self-Protection: A Must-Have, Emerging Security Technology"). RASP is an emerging technology that, like IAST, "instruments" the application runtime environment. In other words, it extends the functionality by additional functionality — namely, security detection and protection to the application. RASP is conceptually similar to a WAF, but logically closer to the application by becoming an integral part of an application runtime environment (for example, the JVM). As such, RASP monitors the execution from the "inside" of an application by using the application runtime environment, gets controls when specified security conditions are met, and takes the necessary protection measures. Actions taken could include the following, for example: user session termination, application termination (without bringing down other applications on the server), or allowing the requested action but issuing an alert sent to security personnel or a warning sent to the user.

AST for Mobile Platforms

Mobile AST aims to analyze applications for coding, design, packaging and deployment conditions that are indicative of security vulnerabilities and risky app behavior. Testing can also point to application functions that conflict with an enterprise's security policies (for example, testing can raise warnings that an application accesses the corporate calendar or contact list, or transmits corporate information to external locations). As enterprises are adopting mobile AST, two challenges are typically being encountered: It is getting increasingly difficult to analyze obfuscated apps from official app stores, even if the license allows it; and, administrators often are overwhelmed by hundreds of reports for apps that their workforce wants to use.

As such, the following actions should be performed:

• Tests should include: (1) static analysis of the code for security vulnerabilities; (2) dynamic testing of the UI and the application's Web services interfaces for security vulnerabilities; and (3) comprehensive behavioral analyses.
• Two layers of the mobile application should be tested: (1) the mobile client side; and (2) the server side.
• Enterprises can choose from two testing delivery models, or combine them. They can: (1) acquire testing tools to conduct their own tests; and/or (2) procure testing as a cloud service from specialized vendors.
• When an enterprise plans to use third-party mobile applications that it cannot test on its own, it should consider the security/risk reputation score of the tests conducted by independent, reputable security testing vendors.
• Technology must automatically ensure that all applications on the mobile device have been detected and submitted for tests.
• Mobile AST should enable integration of its results with mobile protection technologies — for example, with MDM and EMM solutions — and focus administrators on the truly gray apps, allowing users to block (or automatically blocking) known bad applications.