Comparison and integration of genetic algorithms and dynamic symbolic execution for security testing of cross-site scripting vulnerabilities

Context: Cross-site scripting (XSS for short) is considered one of the major threat to the security of web applications. Static analysis supports manual security review in mitigating the impact of XSS-related issues, by suggesting a set of potential problems, expressed in terms of candidate vulnerabilities. A security problem spotted by static analysis, however, consists of a list of (possibly complicated) conditions that should be satisfied to concretely exploit a vulnerability. Static analysis, instead, does not provide examples of what input values must be used to make the application execute the (sometimes complex) execution path that causes a XSS vulnerability. Runnable test cases, however, consist of an executable and reproducible evidence of the vulnerability mechanics. Test cases represent a valuable support for developers who should concretely understand security problems in detail before fixing them.Objective: This paper evaluates various strategies to automatically generate security test cases, i.e. test cases that expose a vulnerability by making the application control flow satisfy vulnerability conditions.Method: A combination of genetic algorithms and concrete symbolic execution is presented for the automatic generation of security test cases. This combined strategy is compared with genetic algorithms and with concrete symbolic execution alone, in terms of coverage and productivity on four case study web applications.Result: While genetic algorithms require less time to generate security test cases, those generated by concrete symbolic execution cover a higher number of vulnerabilities. The highest coverage, however, is achieved when the two approaches are combined and integrated.Conclusion: The integrated approach that we propose has shown to be effective for security testing. In fact, genetic algorithms have shown to be able to generate test cases only for few and simple vulnerabilities when not combined with other approaches. However, their contribution is fundamental to improve the coverage of test cases generated by concrete symbolic execution. (C) 2013 Elsevier B.V. All rights reserved.