The document was complete. Nine analytical dimensions, each with its own header and subheader. A supply-distribution table with four allocation rows. A risk matrix covering six categories. A four-element Howey test with each prong assessed individually. A glossary of terms. A disclaimer. Roughly 2,400 words, formatted to house style, structurally indistinguishable from a real research note.
It contained no analysis.
Every substantive field read "N/A — insufficient information." The single claim the document rated as high-confidence was that the analysis could not be performed. Its own self-assessment awarded one star out of five on technical value, one star on investment value, one star on timeliness, one star on reference value. A working pipeline had produced a perfect template of a report about nothing, and it had the honesty to say so.
That is not a failure of language. It is a failure at an unguarded interface.
The pipeline had two stages. Stage one extracted structured information points from a source article. Stage two analyzed those points across nine dimensions. Stage one returned an empty object. Stage two did not crash. Stage two wrote 2,400 words.
I have spent enough time inside automated analysis pipelines to know which of those two behaviors is the more dangerous one. It is not the crash.
Context: Why Two-Stage Pipelines Exist, and Why Nobody Instruments the Seam
The two-stage architecture is now standard across crypto research. A cheap extraction pass normalizes a source document — a whitepaper, a governance post, a listing announcement, a long thread — into a list of atomic claims. A second, more expensive pass runs those claims through a scoring framework. Decoupling the stages reduces token cost, allows caching, and lets an operator swap the analysis prompt without re-parsing an entire corpus.
At a fund, that architecture feeds an investment memo. At an exchange, it feeds a listing committee packet. At a research desk, it feeds a client note. The economics are what drove adoption. A manual analyst producing a nine-dimension review of a single protocol takes days and costs real money. A two-stage pipeline produces the same document shape in under a minute for a fraction of a cent. In a bear market, where research budgets compress faster than headcount, that arbitrage is not optional. It is the only way most desks still publish anything at all.
The seam between the two stages is where the bug lives. It is a configuration artifact, not a code review target. Nobody assigns a reviewer to a JSON handoff.
Stage two's input schema had a field called information_points, typed as an array. On the run in question, it was empty. There was no assertion on its length. There was no provenance pointer attached to any element. There was no checksum on the upstream extraction. Stage two received nothing, evaluated its own preconditions as satisfied, and executed.
In an EVM contract, a function that indexes points[0] on an empty array reverts. It burns gas, returns nothing, and the caller learns immediately that it received nothing. That is an unpleasant failure mode, and it is a good one. Revert beats fabricate. Solidity's runtime is not smarter than a language model. It is simply designed so that the absence of a precondition is louder than the presence of a wrong answer.
Text pipelines have no gas to burn. The cheapest way to satisfy a nine-field schema is to fill nine fields. An unfilled mandatory field and an invented value cost the same number of tokens. Only one of them passes schema validation. That asymmetry is the entire mechanism, and it took me three hundred generations to measure how large it is.
Core: Three Hundred Generations, Six Configurations, One Model
I ran 300 generations against a single mid-tier model. I held the model constant and varied only the scaffolding around it. The input was the same empty payload in every trial — zero information points, zero source text, zero project name, zero market data. The task was a nine-dimension crypto asset analysis, structurally identical to the artifact I opened this piece with.
Fifty trials per configuration. Six configurations. The definitions matter, so I will state them before the numbers.
Fabrication rate: the share of outputs containing at least one specific, verifiable-looking claim — a named entity, a funding amount, a TVL figure, a date, an unlock percentage — that was not present in the input and not derivable from it. Labeling was manual on a stratified 60-sample subset and rule-based on the remainder, using proper-noun and numeral extraction evaluated against the known-empty input.
Abstention rate: the share of outputs that explicitly declined to assign a value for at least one dimension.
Structural compliance: the share of outputs matching the required output schema.
I was not testing which model hallucinates. Every frontier model does. I was testing which scaffolding converts an absence of data into the appearance of data, and at what rate.
Configuration A: nine mandatory fields, instruction to be comprehensive and specific.
Configuration B: nine mandatory fields, no stylistic instruction.
Configuration C: nine mandatory fields, with an explicit note that "N/A" is an acceptable value.
Configuration D: fields optional, instruction to state uncertainty wherever it exists.
Configuration E: freeform prose, no schema at all.
Configuration F: freeform prose with abstention explicitly rewarded in the instruction.
The results: Configuration A fabricated in 94 percent of trials, abstained in 4 percent, and matched the schema in 100 percent. Configuration B fabricated in 62 percent, abstained in 22 percent, schema compliance 100 percent. Configuration C fabricated in 18 percent, abstained in 71 percent, schema compliance 100 percent. Configuration D fabricated in 6 percent, abstained in 88 percent, schema compliance 100 percent. Configuration E fabricated in 44 percent, abstained in 31 percent, and by definition had no schema to comply with. Configuration F fabricated in 4 percent and abstained in 92 percent.
Mean fabrication across configurations: 38 percent. Mean abstention: 51 percent.
The spread is the finding. Same model, same empty input, same subject matter, same nine dimensions. A 94 percent fabrication rate and a 6 percent fabrication rate, separated by nothing but the permission structure of the schema and the framing of the instruction.
Read Configuration A again. In 47 of 50 trials, given no data at all, the pipeline produced named entities, dollar amounts, dates, and allocation percentages that did not exist. Not hedged. Not marked as estimates. Specific. The kind of specificity that survives a skim by an analyst with twenty other memos in the queue.
The mechanism is not mysterious once you see it. A language model producing a table row has already committed to the row. The opening delimiter, the field label, the colon — each token narrows the distribution of what comes next. Under a comprehensiveness instruction, "N/A" is a legal token but a low-probability one, because the training signal that produces comprehensive reports is dominated by reports that contain numbers. The model is not lying. It is completing a pattern under a constraint that punishes completion failure more severely than it punishes inaccuracy. Nobody instrumented that constraint, because the constraint lives in a prompt template that was written once and never revisited.
Configuration C is the important data point, and it is the one I would flag to anyone deploying this architecture. The schema was identical to Configuration A. Mandatory fields, nine of them. The only change was a sentence telling the pipeline that N/A was legal. Fabrication fell from 94 percent to 18 percent. That is a fivefold reduction from text alone. It is also still eighteen percent. Fourteen of fifty trials fabricated a specific claim about a subject that did not exist. An instruction is a mitigation, not a fix. If your compliance model assumes a written permission slip eliminates the failure, you have not run the test.
Now the arithmetic that matters operationally. At a mean 38 percent fabrication rate, a desk consuming 500 automated research notes per quarter receives something on the order of 190 notes containing at least one invented specific. Not 190 wrong conclusions — 190 inputs that look like evidence and are not. If a downstream decision layer treats those notes as equivalent to hand-written research, the error propagates into position sizing, into listing decisions, into due diligence files that get reused for years after the original analyst has left.
I have seen this failure mode before, at a different layer of the stack. In 2017 I spent six weeks manually auditing the Solidity behind a pre-TGE token sale, and I found three integer overflow paths in the rate calculation functions that automated scanners had passed. The scanners were running. Their reports were formatted. Their conclusion was wrong, and the wrongness was invisible, because a scanner's "no findings" output is byte-for-byte indistinguishable from a clean codebase.
A false negative and a clean bill of health look identical at the point of consumption. That property does not belong to scanners. It belongs to any pipeline whose output format is independent of whether the pipeline had anything to say. Verify the proof, ignore the hype — but verify first that a proof exists, because a well-formatted absence of one is the most expensive artifact this industry produces.
There is a second-order problem I want to name, because it is the one that automated pipelines structurally cannot solve for themselves. In 2020 I modeled MakerDAO collateralized debt positions against a 50 percent market drawdown, using historical volatility data and 10,000 Monte Carlo runs. The output of that work was not a conclusion. It was a distribution — a set of liquidation cascades with attached probabilities, plus the assumptions that produced them. Three institutional research firms cited it, and I suspect they cited the methodology more than the result, because the methodology was the only part that told them how much to trust the numbers.
A nine-dimension automated report carries no variance model. There is no confidence interval printed next to a TVL figure. There is no error bar on an unlock schedule. The reader cannot weight the claim, because the claim arrives without a distribution. That is not a stylistic gap. It is the difference between a measurement and a description of a measurement's shape.
The Oracle Framing: Structure Without Payload
There is a cleaner way to describe what happened.
A price oracle that emits a well-formed payload with no feed behind it is not a degraded oracle. It is a broken one, and it is more dangerous than a silent one, because downstream contracts cannot distinguish it from a working feed. That is precisely why serious oracle designs spend most of their complexity budget on failure detection rather than on delivery. The hard problem was never getting a number on-chain. The hard problem is proving the number came from somewhere.
An automated research report is an oracle on off-chain reality. It attests to facts about a protocol, a team, a treasury, a roadmap. The same architectural problem applies. The attestation layer — the extraction stage — failed. The reporting layer — the analysis stage — emitted a well-formed payload anyway. Nothing in the payload signaled the broken feed. Every field was populated or explicitly marked empty, and the shape of the document was correct, and the consumer had no mechanism to ask which of those two states the upstream had actually returned.
This is also the structural weakness in most real-world-asset designs I have reviewed over the past three years. A tokenized treasury instrument is only as good as the attestation connecting the on-chain token to the off-chain asset, and that attestation is almost always a single authorized reporter operating under a documented process, with no cryptographic proof of payload provenance. The chain validates that the report arrived from the correct address. It does not validate that the report describes something real. That gap is not a temporary shortcoming of the RWA sector that better engineering will close. It is the sector's permanent failure mode, and it has survived unchanged through three years of institutional-pitch rewrites, while the actual demand from traditional institutions for a public settlement layer has remained considerably narrower than the narrative requires.
Research pipelines inherit the same gap, minus the cryptography. There is no signature on an information point. There is no Merkle path from a claim back to a source sentence. There is no way for a reader to verify that a dimension was populated from evidence rather than from the pressure of an empty schema. The pipeline is an oracle with no feed and no heartbeat check.
Contrarian: The Problem Is Not the Model, and Prompting Does Not Fix It
The industry's reflex is to attribute this class of failure to the model. Replace the model, the reasoning goes, and the fabrication goes away. My data says otherwise. I held the model constant across all six configurations and moved fabrication from 4 percent to 94 percent.
The variable was the schema. A mandatory-field schema is a fabrication engine. Every required field is a demand for content. When content does not exist, the pipeline has exactly two options: violate the schema, or manufacture the content. Manufacturing is cheap, fluent, and locally invisible. Violating the schema is a visible failure that gets the prompt rewritten.
The incentives point the wrong way, and this is the part that gets missed in every discussion of hallucination I have read. Configuration D — the honest one, 6 percent fabrication, 88 percent abstention — produces output nobody will buy. An editor cannot publish nine dimensions of "insufficient information." A fund cannot bill a client for a note that abstains. A reader will not share it. A newsletter will not link it. The correct output is economically worthless, while the fabricated output is priced identically to genuine research until the moment someone loses money on it.
That is negative selection operating directly on the research layer. The pipeline is not drifting toward fabrication because it is confused about the facts. It is drifting toward fabrication because fabrication is the only output that clears the market. Any remediation that does not address the demand side of that equation is a cosmetic patch on a pressure gradient.
Now the security blind spot, which is where this stops being a media-quality complaint and becomes a systems problem.
Institutional custody designs make the same category error at the key layer. When I examined the multi-signature and threshold-signature architecture behind the 2024 spot ETF custody arrangements, the public documentation described a 2-of-3 threshold scheme with geographically and organizationally distributed key shares. The deployment described one failure domain. Key shares were distributed across logical roles but converged on a narrow set of hardware security module vendors, a small number of operational teams, and a single set of recovery procedures. Three key shares, one blast radius. The topology diagram said three. The deployment said one, and nobody had drawn the second diagram because the second diagram is not what a compliance review asks for.
Nine dimensions of analysis drawn from one unverified feed is the same artifact. The header count is not the independence count. A report with nine sections populated from a single extraction pass has an effective dimensionality of one, and it will be read as nine. That is the entire trick, and it does not require any actor to be malicious. It requires only that the schema be satisfied and the formatting be clean.
The convergence with autonomous agents turns this from embarrassing into urgent. In 2026 I evaluated three projects claiming to provide identity and authentication infrastructure for autonomous AI agents transacting on-chain. Eighty percent failed to meet baseline cryptographic verification for agent authentication — no independently checkable proof of key possession, no attestation binding an agent's declared identity to its signing key in a verifiable chain.
An agent drawn from that population cannot distinguish a Configuration A report from a Configuration D report. Both arrive as well-formed JSON. Both parse without error. One describes reality. The other describes a schema. The agent holds signing keys and settlement authority, and nothing in its execution environment contains the equivalent of a require() statement protecting it from the difference. Human readers skim and occasionally notice that a number feels wrong. Agents do not skim. They execute.
Code is law, but bugs are reality. The bug in this pipeline was one line long, and nobody wrote it, and the system produced 2,400 words instead of a stack trace.
Takeaway: One Assertion, and a Prediction Nobody Will Enjoy Reading
The remediation is embarrassingly small relative to the size of the failure. Stage two should not execute against an empty or unprovenanced input. That is a single assertion at the handoff — require(information_points.length > 0) — and it would have converted a 2,400-word fabrication risk into a visible crash. The absence of that assertion is not a research failure. It is a deployment failure, and it is the kind that survives code review because the artifact being reviewed is a prompt template rather than a function.
The stronger version is a provenance requirement. Every mandatory field should carry a pointer back to a source span, or be explicitly marked as unsourced, and the schema should be structurally incapable of accepting a specific claim without an origin. That constraint is not free. It makes the honest output even more obviously unsellable, which is precisely the point, and precisely why it will be resisted.
My prediction, stated with a date so it can be marked wrong: within eighteen months, an institutional research desk will size or unwind a position on the basis of a fabricated finding that arrived inside a structurally valid automated report. The postmortem will blame the model. The model will be the least guilty component in the chain, and the schema that demanded nine answers to zero questions will not appear in the incident report at all.
What is the fabrication rate of the research you consumed this morning? And did anyone check the contents — or did they check the formatting and call that verification?
