The context problem

When architecting an analyst agent on enterprise data, LLM context limits constrain how much you can actually process at once. RAG is the natural solution, but it breaks down with tabular data - so instead, you let the agent write SQL, retrieve results, and use them as the basis for analysis. The problem is that out-of-the-box, the agent expects clean, well-documented data. Without proper guidance on your specific schema, join keys, and domain rules, it will stumble and hallucinate. So how do you provide that guidance at scale? This post outlines a repeatable pattern for bootstrapping agent context from artifacts most organizations already have.

A two pronged approach

In breaking down this issue, I defined two sub-issues which guided the planning. The first of these is the response usefulness. Can the agent understand what you are asking? Does it understand your internal business jargon and definitions? The second is response accuracy, specifically in the returned data. Even if it understands what you are asking, can the agent actually find the correct data to answer your query?

There are several things required to give an AI agent proper context of your data and its role. In general, the idea is to give the agent a sort of knowledge bank to draw from when providing answers, almost like an employee handbook. It needs to know all of the jargon that the organization's members know, all of that jargon that wasn't used in the LLM's training data but is crucial for providing useful answers.

The obvious first approach is to build this by hand: a glossary table in the warehouse holding business terms, their definitions, synonyms and some other metadata, whose contents get appended to the bottom of the user prompt. Sure, this helps make the answers from the agent more useful, but this human-written context-injection approach is highly error-prone and it risks introducing bias in the domain knowledge excluded through oversight.

Next, before writing any SQL, the agent needs more context about the data itself. It needs to know all of the ins and outs of the schema, the required joins, the specific table names, the actual formulas used to calculate specific pertinent values. The agent needs some examples of querying your data, a prompting technique known as few-shot generation. This helps to direct the agent's output towards more correct results. Call this list of examples the "validated query repository." This repo is injected directly into the prompt so the agent always has a recent reference to it. However, manually building a custom business glossary is a lot of work. Gaps in personal knowledge produce gaps in the LLM's knowledge and traditional semantic layering doesn't provide enough signal to ward off hallucinations. Looking forward, data discovery must also be documented and maintained for future sources in order to guarantee a robust product.

The framework in four steps

1. Mine the query history. Pull historical SQL that carries human-written comments. The analysts already documented the schema, they just did it inline.
2. Derive the glossary. Group those queries by the table they read from and have an LLM extract organization-specific vocabulary, definitions, and formulas.
3. Derive the validated query repository. Run the same grouped queries through an LLM again, this time asking for a unified, canonical query per table or subject area.
4. Inject both at runtime. Retrieve glossary entries semantically as needed; pass the validated queries wholesale.

The search for signal

So, where can you find sufficient signal for the semantics of the schema to generate a knowledge base for the agent to use? Employees' internal communications would technically carry that signal, but mining internal chat and email archives is not a real option - it disregards privacy and shouldn't be seriously considered. Instead, the best source is your warehouse's query history metadata. ACCOUNT_USAGE.QUERY_HISTORY in Snowflake, INFORMATION_SCHEMA.JOBS in BigQuery.

The idea is very simple: query that history table where there are strings that match the comment syntax of SQL and that are longer than some threshold, say 300 characters. Take the full list of queries and group them by the table that they are actually selecting data from, then feed that whole list into an LLM, and ask it to derive as much organization specific vocabulary from it as it can. In spot checks against the source queries, it consistently extracted correct formulas and definitions, automatically building that business glossary and solving the first sub-issue - a more systematic accuracy eval is still on the roadmap.

And elegantly, building the validated query repository is nearly the exact same process. The difference here is that instead of instructing the LLM to extract vocabulary, you ask for a unified query that spans the queries it saw as examples.

Finally, in the agent loop, text embeddings pass the business glossary entries contextually as needed. The validated queries, however, get passed all at once. I found that giving the agent the full repo, rather than trimming it down, meaningfully improves its ability to join tables and build queries that span multiple sources. That tradeoff is easy to make today: with frontier models routinely offering context windows over 1M tokens, spending a few thousand tokens on the query repo barely moves the needle.

Prerequisites and failure modes

This pattern is only as good as the exhaust it feeds on, so it is worth checking two things before committing to it. First, query history retention: Snowflake's ACCOUNT_USAGE views retain a year, BigQuery's INFORMATION_SCHEMA.JOBS only 180 days, and a warehouse that was migrated recently may hold far less real history than the retention window suggests. Second, commented SQL culture: if your analysts write bare SQL with no prose, there is no semantic signal to mine and the glossary comes back thin and generic. The approach also degrades where the history is dominated by BI tool output. Machine-generated SQL from a dashboard layer is voluminous, uncommented, and will drown out the handwritten queries unless you filter it out by user or client. Finally, mined context inherits whatever is wrong upstream: deprecated tables that analysts still query, stale formulas, and one analyst's idiosyncratic definition of a metric all get promoted into the glossary as though they were canon. Treat the generated glossary and query repo as a first draft that a domain owner reviews, not as ground truth, and regenerate them on a schedule so they track the schema as it changes.