For the past few years, the default architecture for a knowledge agent has been clear: chunk documents, generate embeddings, store them in a vector database, retrieve relevant chunks and pass them to a language model. Vercel’s recent write-up challenges that default. Their production knowledge-agent architecture drops the vector pipeline in favour of direct filesystem and shell access, and the result is a simpler system with a substantially lower cost per query.
The vector default has hidden costs
Vector search is powerful for large, unstructured corpora where semantic similarity matters. But it also adds operational complexity: embedding models to maintain, vector stores to operate, chunking strategies to tune, and retrieval pipelines that can return stale or irrelevant context. For many business agents, especially those operating over well-structured codebases, documentation and internal wikis, this overhead may not be necessary.
The Vercel example focuses on a sales-support agent. Instead of embedding every document, the agent reads files directly from a filesystem, runs shell commands to explore repositories and calls existing APIs. The cost per sales call drops from roughly $1.00 to roughly $0.25. More importantly, the system is easier to reason about because the data path is direct.
When simpler architectures win
A non-vector approach works best when the knowledge source is structured, version-controlled and accessible through deterministic queries. Code repositories, structured documentation, configuration files and API schemas are good candidates. In these environments, an agent can often find what it needs by listing directories, reading files and running commands that humans would run.
The approach is less suitable for massive, heterogeneous document collections where semantic search is genuinely valuable, or where the user asks vague questions that do not map cleanly to file names or paths.
Implications for agent design
The broader lesson is to match the architecture to the problem. Start with the simplest possible data access pattern and add complexity only when it is justified by performance or cost. Ask whether embeddings solve a real retrieval problem or whether they are simply the pattern everyone assumes.
For UK SaaS teams building internal agents, this is a useful corrective. A filesystem-aware agent with a small context window and clear tool use may outperform a vector-heavy pipeline at lower cost and with fewer moving parts.
Governance considerations
Direct filesystem and shell access introduce their own risks. The agent needs least-privilege access, clear audit logging and strict boundaries on which commands it can run. Treat it as a service account with a well-defined role, not as a general-purpose user.
The goal is not to reject vector databases entirely. It is to make them a deliberate choice rather than the automatic starting point.