Hackers vs. Testers: A Comparison of Software Vulnerability Discovery Processes

@article{Votipka2018HackersVT,
  title={Hackers vs. Testers: A Comparison of Software Vulnerability Discovery Processes},
  author={Daniel Votipka and Rock Stevens and Elissa M. Redmiles and Jeremy Hu and Michelle L. Mazurek},
  journal={2018 IEEE Symposium on Security and Privacy (SP)},
  year={2018},
  pages={374-391}
}
Identifying security vulnerabilities in software is a critical task that requires significant human effort. [...] Key Result The results suggest that hackers and testers follow similar processes, but get different results due largely to differing experiences and therefore different underlying knowledge of security concepts.Expand
Understanding security mistakes developers make: Qualitative analysis from Build It, Break It, Fix It
TLDR
This paper investigates how and why programmers, despite a baseline of security experience, make security-relevant errors, and conducts an in-depth analysis of 94 submissions to a secure-programming contest designed to mimic real-world constraints: correctness, performance, and security. Expand
A Case Study of Software Security Red Teams at Microsoft
TLDR
The results of this work are applicable to practitioners, researchers, and toolsmiths who wish to understand how offensive security teams operate, situate, and collaborate with partner teams in their organization. Expand
Toward a Field Study on the Impact of Hacking Competitions on Secure Development
The ability to find and fix vulnerabilities is critical to producing secure software. Previous research has shown that the main difference between experts who specialize in finding security flaws andExpand
"You've Got Your Nice List of Bugs, Now What?" Vulnerability Discovery and Management Processes in the Wild
TLDR
It is found that organizations often struggle with vulnerability remediation and that vulnerability discovery efforts are hindered by significant trust, communication, funding, and staffing issues. Expand
Bug Bounty Marketplaces and Enabling Responsible Vulnerability Disclosure: An Empirical Analysis
TLDR
The research findings suggest that BBPs are valuable opportunities to source vulnerabilities in software; nevertheless, the rate of disclosure and hacker participation marginally increases with vendor's rewards and other incentives. Expand
A Survey on Ethical Hacking: Issues and Challenges
TLDR
The main focus of this paper is to explain the technical and non-technical steps of penetration tests, to make existing systems and their corresponding data more secure, efficient and resilient. Expand
Mitigating Remote Code Execution Vulnerabilities: A Study on Tomcat and Android Security Updates
The security of web-applications has become increasingly important in recent years as their popularity has grown exponentially. More and more web-based enterprise applications deal with sensitiveExpand
API Blindspots: Why Experienced Developers Write Vulnerable Code
TLDR
The presence of blindspots correlated negatively with the developers’ accuracy in answering implicit security questions and the developers' ability to identify potential security concerns in the code, and has the potential to advance API security in design, implementation, and testing of new APIs. Expand
SPIDER: Enabling Fast Patch Propagation In Related Software Repositories
TLDR
This paper is the first to define safe patches (sps), a patch that does not disrupt the intended functionality of the program (on valid inputs), meaning that it can be applied with no testing; it is argued that most security fixes fall into this category. Expand
Understanding the Reproducibility of Crowd-reported Security Vulnerabilities
TLDR
The first empirical analysis on a wide range of real-world security vulnerabilities with the goal of quantifying their reproducibility suggests that there is not only a necessity to overhaul the way a security forum collects vulnerability reports, but also a need for automated mechanisms to collect information commonly missing in a report. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 108 REFERENCES
Game of detections: how are security vulnerabilities discovered in the wild?
TLDR
This first study on vulnerability repositories that targets the reporters of the most common security vulnerabilities, thus concentrating on the people involved in the process, finds that the communities differ based on the security vulnerability they target; but within a specific community, reporters follow similar approaches. Expand
Questions developers ask while diagnosing potential security vulnerabilities with static analysis
TLDR
An exploratory study with novice and experienced software developers equipped with Find Security Bugs and observed their interactions with security vulnerabilities in an open-source system that they had previously contributed to found that they asked questions not only about security vulnerabilities, associated attacks, and fixes, but also questions about the software itself, the social ecosystem that built the software, and related resources and tools. Expand
Discovering buffer overflow vulnerabilities in the wild: an empirical study
TLDR
An empirical study on reporters of buffer overflow vulnerabilities to understand the methods and tools used during the discovery found that in spite of many apparent choices, reporters follow similar approaches. Expand
Crowdsourced Security Vulnerability Discovery: Modeling and Organizing Bug-Bounty Programs
Despite significant progress in software-engineering practices, software utilized for desktop and mobile computing remains insecure. At the same time, the consumer and business information handled byExpand
An Empirical Study on the Effectiveness of Security Code Review
TLDR
None of the subjects found all confirmed vulnerabilities, more experience does not necessarily mean that the reviewer will be more accurate or effective, and reports of false vulnerabilities were significantly correlated with reports of valid vulnerabilities. Expand
Improving software security with static automated code analysis in an industry setting
TLDR
A case study was conducted to evaluate static code analysis in industry focusing on defect detection capability, deployment, and usage of static automated code analysis with a focus on software security, finding that the tool was capable of detecting memory related vulnerabilities, but few vulnerabilities of other types. Expand
Comparing the Effectiveness of Penetration Testing and Static Code Analysis on the Detection of SQL Injection Vulnerabilities in Web Services
  • Nuno Antunes, M. Vieira
  • Computer Science
  • 2009 15th IEEE Pacific Rim International Symposium on Dependable Computing
  • 2009
TLDR
This work used several commercial and open source tools to detect vulnerabilities in a set of vulnerable services and suggested that, in general, static code analyzers are able to detect more SQL Injection vulnerabilities than penetration testing tools. Expand
Market for Software Vulnerabilities? Think Again
TLDR
It is demonstrated that an active unregulated market-based mechanism for vulnerabilities almost always underperforms a passive CERT-type mechanism, and it is extended to show that a proposed mechanism--federally funded social planner--always performs better than a market- based mechanism. Expand
One Technique is Not Enough: A Comparison of Vulnerability Discovery Techniques
  • Andrew Austin, L. Williams
  • Engineering, Computer Science
  • 2011 International Symposium on Empirical Software Engineering and Measurement
  • 2011
TLDR
The most effective vulnerability discovery technique in terms of vulnerabilities discovered per hour was automated penetration testing, and the results suggest that if one has limited time to preform vulnerability discovery one should conduct automated penetrationTesting to discover implementation bugs and systematic manual penetration testing to discover design flaws. Expand
You Get Where You're Looking for: The Impact of Information Sources on Code Security
TLDR
Analyzing how the use of information resources impacts code security confirms that API documentation is secure but hard to use, while informal documentation such as Stack Overflow is more accessible but often leads to insecurity. Expand
...
1
2
3
4
5
...