Last week a two-stage research pipeline processed a submitted document. Stage one — extraction — returned a structured information-point array. Length: zero. Every metadata field read "not provided." Stage two — analysis — then executed all nine of its dimensions exactly as designed: technical, tokenomics, market, ecosystem position, regulatory, team and governance, risk, narrative, and supply-chain transmission.
Field-completion rate: 0%. Output length: roughly 4,000 words. Every substantive conclusion slot held the same string: "N/A — insufficient information."
I archived that document. It is the most honest research report I have read this year. It is also a live grenade, and the pin is already out.
The architecture that produced it is standard now. Stage one decomposes a source into numbered, citable information points: P1, P2, P3. Stage two draws conclusions and tags each one to the points it rests on. The rule is non-negotiable: no conclusion without a citation. If a claim cannot be traced to a numbered fact, it is not analysis — it is fiction wearing a lab coat.
The reason is not academic purity. It is failure economics. In my 2020 stress-testing work on Aave v1 and Compound v1, I ran 1,000 simulated liquidity crunches against a $50 million book. The value of each scenario was never the number it produced. It was the audit trail: which assumption moved, by how much, and why. When the May drawdown came, we could reconstruct every decision — and that reconstruction is what let me argue leverage down from 3x to 1.5x and survive it. A conclusion you cannot trace is a conclusion you cannot defend.
But the pipeline has a seam. Stage one and stage two are usually owned by different people, sometimes different systems, occasionally different companies. That seam is where the empty shell gets through.
Look at what actually happened. Stage one returned nothing. Stage two did not stop. It executed. The schema was valid, the loop ran, and the fields populated with the only legally true value available: N/A.
Three distinct failures live inside those 4,000 words, and the fixes for each are different.
The first is analysis pollution. If a human or a model insists on filling the conclusion slots, the output becomes structurally indistinguishable from research and factually indistinguishable from a dream. It will have a technical assessment and a risk matrix, with a "probability" column populated by numbers I invented. A report with a filled risk matrix and no source data is more dangerous than no report at all, because it manufactures false confidence at scale.
The second is traceability chain rupture. The framework's hard constraint is that every conclusion cites an information point. With an empty array, there is no P-number to cite. The report becomes unauditable. If a client later asks why technical risk was rated medium, there is no answer — because there was never a cause. You cannot audit a ghost.
The third is field-ownership drift. "Publication timestamp" and "source quality" are metadata. They belong to stage one, because they exist in the original document. Stage two cannot reverse-engineer a publication date from text it was never given. Yet both fields were punted downstream with the instruction "assess yourself." That is not a division of labor. It is a division of blame.
Here is the number that matters. The minimum viable input for this pipeline is roughly five structured information points plus a precise publication date. With those, you unlock about 70% of the analytical surface. Without them, you unlock zero — and every hour spent generating the rest is spent manufacturing noise.
I learned the shape of this in 2017, auditing a token called EtherFund. Forty hours a week for three months, hand-tracing ERC-20 transfer logic and reading the vesting contract at the bytecode level. What saved 12% of that fund was not a narrative. It was a specific line number in the EVM. The source is the asset. Everything downstream is opinion — and opinion, unlike a ledger, can be fabricated to order.
I met the same seam in 2022, dissecting Arbitrum's Nitro upgrade and the OP Stack through the bear market. One hundred fifty hours on fraud proofs and sequencer centralization. I found a latency issue in the dispute-resolution phase that could, under extreme load, push withdrawals out to seven days. That finding only meant something because I could point to the mechanism. Strip the mechanism and you are left with a number a stranger wrote on a napkin.
Consider the asymmetry a null input actually creates. A false positive — publishing analysis on a project that turns out fine — costs nothing visible. A false negative — refusing to publish — costs a filing. Left unguarded, the pipeline always drifts toward the cheaper error. It hallucinates a risk matrix because an empty risk matrix makes the desk look lazy.
Tight input validation has a price. A rejection gate means fewer reports per quarter, and reports per quarter is a number a manager can see. A hallucinated report is a number nobody sees until the trade is already on. We optimize for the metric we can measure and ignore the failure we cannot.
Now the part nobody wants to hear.
The empty-shell report did exactly what it was built to do. It refused to answer questions it had no data to answer. It left nine dimensions blank rather than bluffing. That is not failure. That is the integrity the framework exists to enforce.
The contrarian claim is this: the real bug is not in the data pipeline. It is in the incentive that punishes the honest N/A. Desks are measured on throughput — reports filed, dimensions covered, clients reassured. Nobody gets promoted for returning a blank page. So the rational move, under that incentive, is to fill the blanks. To invent a probability. To write "medium risk" because the column looked empty.
Code is law, but human greed is the bug. The schema can be perfect and the output still lies, because the human at the seam is rewarded for lying. Yield is the interest paid for ignorance — and here the ignorance is structural. We are paying for a filled report while pretending we bought a true one.
The blind spot is systemic. Everyone blames stage one for the missing data. Almost nobody audits the pressure that made stage two fill it in.
So the question for the next build cycle is not how to make extraction smarter. It is whether we are willing to route a null result to the reject pile instead of the publish bin.
Ledgers do not lie, only their auditors do. The next hallucinated research report is already in someone's folder, fully formatted, risk matrix and all. You will not be able to tell it from a real one. Not until the position is on.
