An Experimental Comparison of Two Risk-Based Security Methods

A significant number of methods have been proposed to identify and analyze threats and security requirements, but there are few empirical evaluations that show these methods work in practice. This paper reports a controlled experiment conducted with 28 master students to compare two classes of risk-based methods, visual methods (CORAS) and textual methods (SREP). The aim of the experiment was to compare the effectiveness and perception of the two methods. The participants divided in groups solved four different tasks by applying the two methods using a randomized block design. The dependent variables were effectiveness of the methods measured as number of threats and security requirements identified, and perception of the methods measured through a post-task questionnaire based on the Technology Acceptance Model. The experiment was complemented with participants' interviews to determine which features of the methods influence their effectiveness. The main findings were that the visual method is more effective for identifying threats than the textual one, while the textual method is slightly more effective for eliciting security requirements. In addition, visual method overall perception and intention to use were higher than for the textual method.