The short version. LLM-driven engines can now find real, previously-unknown vulnerabilities in production code. That is not hype anymore; it has happened, including in our own lab. But a machine that finds bugs also generates plausible bugs that are not there, and a flood of unconfirmed reports is worse than useless to the maintainers who have to answer them. The winning systems will not be the ones that find the most. They will be the ones whose every report comes with a proof a human can check in minutes.
The capability is real now
For years, "AI finds zero-days" was a demo that fell apart on contact with real code. That has changed. Coverage-guided fuzzing, patch-diff variant hunting, and LLM hypothesis generation have matured to where autonomous engines are surfacing genuine, previously-unreported memory-corruption bugs in widely-used open-source libraries. Google's Project Big Sleep and the OSS-Fuzz-Gen work have public examples. So do we.
Our discovery engine found a previously-unreported heap out-of-bounds write in CWPack, a widely-used MessagePack serialization library for C. A truncated stream is treated as a successful read, the parser advances past the end of its buffer, and the next refill calls memmove with a negative size. The engine minimized it to a 74-byte trigger, root-caused it to the stream-refill handler, and it went into coordinated disclosure with the maintainer. Found by fuzzing the public source, no third-party systems touched.
So the capability is here. Which immediately creates a different, harder problem.
The coming flood, and why it is dangerous
If one engine can find and report a bug, a thousand engines can find and report a hundred thousand. Most of those reports will be generated by systems that are good at producing plausible bug narratives and bad at being sure. Maintainers of critical open-source projects, who are often a handful of unpaid people, are about to be buried.
We have already seen the early version of this. Bug bounty programs and open-source security inboxes have started receiving AI-generated vulnerability reports that are fluent, confident, and wrong. A model describes a data flow that does not exist, cites a function that was refactored two years ago, and asks for a CVE. Each one costs a human an hour to disprove. Scale that to machine speed and you do not get safer software. You get maintainers who stop reading the security inbox, which is the worst possible outcome, because that is where the real report is too.
This is the same trap as noisy scanning, moved one layer down. When finding is cheap and proving is skipped, the cost lands on the humans downstream, and eventually they stop trusting the channel.
The bottleneck moved. Build for the new one.
The old bottleneck was discovery: can the machine find the bug at all. That bottleneck is falling. The new bottleneck is trust: can the machine convince a skeptical human, in minutes, that this specific bug is real, without the human re-doing the analysis. Everything worth building now sits on the trust side.
The mechanism is a proof ladder. Instead of one binary "is it a bug," you decompose exploitability into ordered rungs, each with its own independent check:
- A crash the input reliably reproduces.
- A typed fault, confirmed by a sanitizer that names the class: heap overflow, use-after-free, the specific negative-size memmove.
- Controllability, proof that attacker-supplied input drives the fault, for example that the faulting address tracks bytes you planted.
- A primitive, a demonstration of what the attacker actually gets: a controlled write-what-where, not just "it crashed."
The report states the highest rung the engine actually reached, and downgrades honestly. A finding that reached "typed fault" is reported as a typed fault, not dressed up as a working exploit. The LLM narrates the story; it never asserts the verdict. The verdict comes from the oracle. This is the difference between a system that hallucinates a bug and a system that proves one.
Machine-speed discovery needs machine-speed disclosure
There is a second, less-discussed problem. If discovery runs at machine speed and disclosure runs at human speed, the gap between "the bug is found" and "the maintainer is safely notified" becomes a window of exposure, and a governance mess. A responsible autonomous engine has to solve disclosure, not just discovery: deduplicate across its own fleet so a maintainer gets one report and not two hundred, respect the receiver's capacity, sign its provenance so the report is attributable, and carry a machine-checkable proof so triage takes minutes instead of a day.
The proof is what makes the whole thing humane. A maintainer who receives a signed, self-contained bundle, the 74-byte trigger, the sanitizer output, the minimized reproduction, can confirm the bug themselves in the time it takes to run one command. They do not have to trust the sender at all. That is the only version of machine-speed disclosure that maintainers will not learn to ignore.
How Nemesis Forge is built
Nemesis Forge is a verifier-backed zero-day discovery engine. It generates candidate bugs the hard ways, coverage-guided fuzzing, patch-seeded variant hunting that generalizes a known fix into a signature for its undiscovered siblings, and LLM-synthesized harnesses, and then it proves each one up the ladder. Nothing is reported unless a deterministic oracle, not the model, confirms it. A clean run reports nothing.
The output is a disclosure packet, not a claim: a reproducing trigger, the sanitizer-typed crash, a trace-grounded root cause, the exploitability grade it actually reached, and a suggested fix, assembled as a byproduct of the proof rather than as a separate writing step. On CWPack, that packet took the finding from a raw crash to a reproducible, disclosure-ready artifact the maintainer could verify independently.
We are precise about what we claim. We distinguish a novel discovery from a reproduced known-CVE. We report the highest proven rung, not the highest imaginable one. And we treat the maintainer's inbox as a scarce resource to be respected, not a channel to be flooded.
What to ask of any autonomous discovery system
Whether you are a maintainer receiving reports or a security leader deploying an engine, ask:
- What proof comes with the finding, and can I check it without trusting you? A report with no reproducible artifact is a suggestion, not a vulnerability.
- What is the highest rung, and how honestly does it downgrade? "Crash" and "working exploit" are very different claims. A system that blurs them is managing perception.
- What is your false-report rate to real maintainers? If a vendor cannot answer, they are not measuring the thing that matters.
- How do you handle disclosure at scale? Dedup, provenance, and receiver capacity are part of the product, or the product is a denial-of-service on open source.
The machines can find the bugs now. The organizations that matter will be the ones that made their findings trustworthy, and treated the humans on the other end as people who deserve proof, not volume.
Nemesis Forge is the verifier-backed zero-day discovery engine from Nemesis Labs. LLM sub-agents propose; deterministic oracles prove; every finding ships with a reproducible packet. nemesislabs.xyz