PoCGen: Generating Proof-of-Concept Exploits for Vulnerabilities in Npm Packages

Security vulnerabilities in software packages are a significant concern for developers and users alike. Patching these vulnerabilities in a timely manner is crucial to restoring the integrity and security of software systems. However, previous work has shown that vulnerability reports often lack proof-of-concept (PoC) exploits, which are essential for fixing the vulnerability, testing patches, and avoiding regressions. Creating a PoC exploit is challenging because vulnerability reports are informal and often incomplete, and because it requires a detailed understanding of how inputs passed to potentially vulnerable APIs may reach security-relevant sinks. In this paper, we present PoCGen, a novel approach to autonomously generate and validate PoC exploits for vulnerabilities in npm packages. The approach combines the complementary strengths of LLMs (e.g., understanding informal vulnerability reports), static analysis (e.g., identifying taint paths), and dynamic analysis (e.g., validating generated exploits). PoCGen successfully generates exploits for 71% of the vulnerabilities in the SecBench.js dataset. This success rate significantly outperforms a recent baseline (by 38 absolute percentage points), while imposing an average cost of only $0.02 per generated exploit. Moreover, PoCGen generates successful exploits for 60% of 126 recent real-world vulnerabilities, which helped augment five recent vulnerability reports in the GitHub Security Advisories database with PoCGen-generated PoC exploits.