FlagThis

Abstract

Defenders cannot patch every newly disclosed vulnerability at once, so they must prioritize, quickly, under a fixed compute budget, and without deceiving themselves about how well they are doing. We make a security-evaluation argument: naive exploit-prediction studies leak the future. A vulnerabilitys eventual exploitation generates public chatter, proof-of-concept code, and advisories, and feeding those later signals to a model inflates its apparent accuracy. We instead specify a _leakage-safe prospective triage protocol_ that, for each CVE, admits only public evidence visible by a fixed decision time, and we attach to every risk score an auditable _evidence certificate_ naming the public signals that justified it. Public evidence around CVEs (advisories, exploit archives, fix commits, and hacker-community discourse) is organized as a temporal evidence graph and selected per CVE under an explicit budget. On 12012 prospective CVEs from public sources, budgeted evidence selection raises leakage-safe prospective recall@50 from 0.010 (severity-only baseline) to 0.026, a 150% relative gain, while a budget of only 2 evidence documents per CVE captures most of that value, so triage is cheap. A result of direct interest to practitioners is that a strong cross-encoder reranker _lowers_ prospective recall (to 0.016), because semantic relevance to a CVE is not evidence of its exploitation. Most consequentially, a naive random split with unfiltered evidence inflates apparent prospective recall by 8.5 __ and EPSS-high recall by 5.0 __ , the temporal leakage that our protocol removes. All embeddings, candidate edges, certificates, and metrics are released as reproducible artifacts.