First-hand information for everyone
Large language models (LLMs) are good at answering questions when the needed facts are nearby in their input. But when relevant information is scattered across many long or loosely organized documents, the models — and the retrieval systems that feed them — can become slow, repetitive, and error-prone. A new approach, presented in a preprint titled “Librarian Agents,” asks whether we can simply do some of that search-and-synthesis work ahead of time.
The idea is straightforward in concept. Instead of treating retrieval as always reactive — search for passages after a query arrives — train a “builder” agent to read a raw corpus and reorganize it into a compact, hierarchical workspace that anticipates a broad family of future questions. That workspace is made up of synthetic summary files, higher-level merges, and links back to the original sources. A second, frozen “answering” agent then uses that prebuilt filesystem to find evidence and generate answers for individual queries.
To make the builder learn useful behaviors, the authors frame construction as a decision process and train it with reinforcement learning. The builder gets rewarded for work that helps the answering agent produce correct responses, and penalized only in configurations that explicitly account for construction cost. In other words, the agent is taught to create artifacts that a downstream model can actually use, rather than produce summaries judged only by surface metrics.
On two challenging benchmarks for long-context document reasoning, the results are notable but should be read cautiously. On BrowseComp-Plus, a dataset of deep-research web tasks, the trained Librarian Agents improved answer accuracy by roughly 33 percentage points over the strongest retrieval and precomputation baselines in the authors’ experiments. On BRIGHT — a benchmark made of diverse domains including math and biology — a builder trained only on the BrowseComp-Plus-style data transferred zero-shot and improved average accuracy by about 31 points. The precomputation is paid for up-front: in the reported BrowseComp-Plus runs the builder generated roughly 179,000 tokens of synthetic content per corpus, which is then amortized across multiple queries. With reuse across many questions, the per-query token cost can become reasonable and still yield higher correctness than reactive methods.
Several features make the approach appealing. The workspace preserves links to raw documents so the answering agent can fall back to original evidence when details are missing from summaries. The builder learns to produce hierarchical artifacts intended to be easy for an answering agent to traverse, rather than relying on rigid, hand-designed indices such as entity graphs. And because construction happens once for a corpus, the approach can amortize expensive cross-document reasoning across repeated queries.
But there are important caveats. The evaluation is limited to question answering over static corpora; when documents change frequently, maintaining and updating the synthetic workspace introduces new engineering and research challenges. The upfront construction cost can be substantial, so gains depend on serving many downstream queries over the same document set. The builder itself must reason over large contexts to create good summaries, which raises scalability questions for much larger corpora. Finally, synthesized summaries can omit rare but critical details; while the reported system preserves provenance links to raw files, deciding when the answering agent should bypass the summaries remains nontrivial.
The authors release code and a project website to support follow-up work. More broadly, Librarian Agents joins a growing line of work that explores shifting some computation earlier in a system’s lifecycle — doing more at build time to make inference faster and more accurate — and suggests that learned, anticipatory curation can be a useful tool in that toolbox. As with many promising lab results, more tests on changing collections, varied downstream tasks, and larger-scale deployments will be important to understand how this approach performs in real-world production settings.