Loading…
THE MUST ATTEND EVENT FOR CYBERSECURITY PROFESSIONALS
← Back to schedule
Hackuracy: Boosting AST accuracy through hacking
Friday September 27, 2024 1:15pm - 2:00pm PDT
Room: Grand Ballroom
How may a system's exposure to successful cyberattacks be detected more accurately? The short answer is that it is not possible with vulnerability scanning alone; expert manual evaluation by ethical hackers is also necessary.

While automated tools, due to their processing capacity and speed, have become indispensable in identifying potential vulnerabilities, they report high rates of false positives and false negatives, finding only 45% of systems' risk exposure. Tools cannot find vulnerabilities when their discovery involves an external user having come up with a complex, unexpected use of the application. This is where the expertise of ethical hackers comes into play.


In this talk, we will explain what accuracy in AppSec entails and specify three different measures that we used to assess security testing accuracy of scanning alone and the combination of scanning and hacking. We will characterize the insecure-by-design web application that was used as the target of evaluation (ToE), and then compare the performance of the different conditions both in reporting vulnerabilities and risk exposure (identified with a metric designed to accurately show the severity of vulnerabilities to help prioritize them for remediation).


We will present our research findings, which highlight that the combination of scanning and hacking dramatically outperformed all of the assessed tools' scanning in all three accuracy measures in identifying both the amount of vulnerabilities and associated risk exposure. Specifically, the combined approach achieved accuracy scores ranging from 78.9% to 93.7% in detecting the amount of vulnerabilities in the ToE, and from 94.3% to 98.5% in identifying its risk exposure. In contrast, the most accurate tool's scores ranged from 26.4% to 58.4% and 8.5% to 27.0%, respectively. Notably, the overall performance of application security testing in our research was better for guaranteeing few false negatives in reports than for providing reports containing all legitimate vulnerabilities.


In summary, we will demonstrate that achieving accurate detection of a system's risk exposure related to its vulnerabilities requires more than just automated security testing. It necessitates the involvement of expert hackers who can perform manual evaluations, understand the nuances of application logic and identify sophisticated security flaws. Further, we mention how the accuracy of vulnerability scanners can be enhanced. Ultimately, the goal is to equip developers, security professionals and organizations with the knowledge and tools needed to enhance the security of their applications and protect against threats.

Speakers
AR

Andres Roldan

VP of Hacking, Fluid Attacks
Andres Roldan is Fluid Attacks’ VP of Hacking. He leads the company's research team and has identified and ethically disclosed 110 CVEs in open-source software. He has over 20 years of experience in cybersecurity, is a GIAC Advisory Board member, and holds 29 certifications in offensive... Read More →
Friday September 27, 2024 1:15pm - 2:00pm PDT
Room: Grand Ballroom
  Breakout: Breaker Track

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

Tweet Share