---
title: "Data context in governance: where it lives, how to harvest it — Datagist"
description: "Where business, technical and operational data context lives, from query logs to tickets and people, and how to harvest it for governance and AI."
url: https://datagist.ai/insights/data-context-in-governance-where-it-lives-how-to-harvest-it
---

[Insights](https://datagist.ai/insights) Article

# Data context in governance: where it lives and how to harvest it

Most of the context a governance program needs already exists in query logs, pipeline code, tickets, wikis and people's heads. Where to find it, how to collect it, and how to put it to work for governance and AI.

Datagist November 3, 2026 7 min read

[Governance & Operations](https://datagist.ai/services/governance-operations) [GenAI & Agents](https://datagist.ai/services/genai-agents)

Open almost any data warehouse and you will find a table like `cust_mstr_v2`. The table can tell you its columns and their types. It can’t tell you whether it holds every customer or only active ones, why there is a v2, who to ask when a number looks wrong, or whether finance stopped using it last spring. That information is the table’s context, and without it the table is hard to trust.

Governance programs usually try to fix this by asking people to write context into a catalog. Stewards get a list of tables and a deadline. A few weeks later the popular tables have half-filled descriptions and the rest are blank. The effort stalls because it treats context as something to author from scratch.

Most of it already exists. It sits in query logs, pipeline code, tickets, wiki pages, chat threads and the memory of a few experienced people. The job is to find it, collect it, check it and keep it current. This article covers where each kind of context lives, how to harvest it, and how to use it for governance and for AI.

## Three kinds of context

Business context says what the data means. It covers definitions, the rules behind a metric, the purpose a dataset was built for, who is accountable for it and which policies apply. “Active customer means a paid order in the last 12 months” is business context.

Technical context says how the data is built. It covers schemas, keys, data types, lineage from source to report, refresh schedules and quality checks. “This column comes from the billing system and is rebuilt at 2 a.m.” is technical context.

Operational context says how the data is actually used. It covers who queries it, how often, with which joins and filters, what broke last quarter and what changed since. “Nobody has read this table in 90 days” is operational context. It is the kind teams forget to collect, and it is often the most honest, because it records behavior where documentation records intent.

## Where context lives

Each source holds a different slice of context and needs a different way in. The table below is the map we use at the start of a governance engagement.

| Source                               | What it tells you                                          | How to collect it                                       |
| ------------------------------------ | ---------------------------------------------------------- | ------------------------------------------------------- |
| System tables and information schema | Columns, types, keys, comments, grants                     | Scheduled metadata pull                                 |
| Query logs and access history        | Who reads what, common joins and filters, unused tables    | Parse and aggregate 90 days of queries                  |
| Pipeline code and orchestration      | Lineage, transformation logic, tests, schedules            | Parse SQL and dbt manifests, collect OpenLineage events |
| BI semantic models                   | Metric formulas, approved joins, which dashboards use what | API export of models and usage                          |
| Tickets and change records           | Why something changed, known defects, workarounds          | Language model extraction, linked by asset name         |
| Wikis, design docs, data contracts   | Business meaning, rules, intended use                      | Language model extraction with source links             |
| Chat threads                         | Warnings and tribal rules ("use the \_v2 table")           | Search for asset names, extract, then review            |
| People                               | Intent, history, exceptions nobody wrote down              | Short interviews driven by gaps                         |

The first four sources are structured and can be collected by software with little human effort. The last four are text and people. They hold most of the business context, and they need a mix of language models and human review.

## Harvesting structured sources

Start with system metadata. Every platform exposes its tables, columns, comments and grants through an information schema or catalog API. Pull it on a schedule into one place. This gives you the inventory that every other source attaches to.

Query logs are the richest source most teams ignore. Snowflake keeps them in `ACCOUNT_USAGE.QUERY_HISTORY` and `ACCESS_HISTORY`. Databricks keeps them in system tables such as `system.query.history` and `system.access.audit`. BigQuery has `INFORMATION_SCHEMA.JOBS`. Parse 90 days of queries and you can answer questions no one documented. Which tables are read daily and which are dead. Which columns are always joined together, which tells you the real join keys. Which filters always appear, which often reveals a business rule. If every query on `orders` filters out `status = 'TEST'`, that rule belongs in the table’s description and probably in a view.

Usage also points to owners. The person or service account that writes a table, and the few people who query it most, are your candidate owner and stewards. A suggested owner backed by evidence gets confirmed far faster than a blank field.

Lineage comes from code. If you use dbt, its `manifest.json` already holds model dependencies, descriptions and tests. For stored procedures and hand-written SQL, a SQL parser can recover column-level lineage. Orchestrators that emit OpenLineage events give you runtime lineage, including jobs that run outside the warehouse. Use all three where you can, since each one misses something the others catch.

BI tools hold the most reviewed definitions in the company, because people argued over them before they shipped a dashboard. Export metric formulas and model joins from the semantic layer, and pull dashboard usage to see which definitions people actually rely on.

## Harvesting text and people

Tickets, wikis and chat hold the why. A Jira ticket explains that `region_code` changed format after an acquisition. A Confluence page defines churn. A Teams thread warns that the March data was loaded twice. None of this reaches the catalog unless someone goes looking.

A language model is good at this first pass. Search each source for the names of tables, columns and metrics in your inventory, then ask the model to pull out candidate definitions, caveats and rules, each with a link back to the ticket, page or message it came from. Treat the output as a draft. Models merge two teams’ definitions into one, and they state old caveats as current facts. A steward approves, edits or rejects each item, and nothing reaches users or an AI assistant until they do.

Mind access while you harvest. A ticket about a salary table or a restricted project can reveal more than the table itself. Keep the harvested text under the same permissions as its source.

People come last, and they should be asked the fewest questions. By now you know the gaps: the 20 most-read tables with no owner, the metrics with two competing definitions, the columns whose names mean nothing. Book 30 minutes with the person who knows, show them what you already have, and ask only about the gaps. A specific question such as “Is `cust_mstr_v2` all customers or active ones?” gets an answer in seconds. A blank template gets put off for weeks.

[Where data context lives and how it is harvested. Sources on the left: system tables, query logs, pipeline code, BI models, tickets, wikis and chat, and the people who know the data. Each source has its own way in: connectors for structured metadata, log mining for usage, code parsing for lineage, language models for text, and short interviews for people. Everything lands in one context store where each fact keeps its source, the date it was captured and who approved it. Approved context then feeds governance work and AI assistants.](https://datagist.ai/media/img/714baaec0259fb0a3394833dff8cc84f5e0ce8fc-1960x1138-96b27e96f7.webp)

## Store each fact with its source

Harvested context is only useful if people can judge it. Store every fact with four things next to it: where it came from, when it was captured, who approved it and whether it is still current. A definition from a certified BI model and one pulled from a 2021 chat thread should not look the same to a reader.

Model the context as connected records. Tables, columns, metrics, business terms, people, policies and source documents are the nodes. “Defined by”, “derived from”, “owned by” and “used by” are the links. Most modern catalogs support this shape. A spreadsheet doesn’t, and that is usually where harvesting efforts die.

Expect conflicts. Sales and finance will each have a definition of active customer, both in use and both written down. Don’t pick one quietly. Show both, mark the conflict and send it to the data owner. Each settled conflict is a governance decision with a record behind it, which is what auditors want to see.

## Using context for governance

With context in place, routine governance work gets faster and better grounded. Ownership reviews start from usage evidence. Sensitive data classification improves when column names, sample values and lineage are read together, and a tag such as PII can follow a column downstream through lineage, so nobody has to tag each table by hand.

Access reviews change too. Compare who holds a grant with who has actually read the table in the last 90 days, and the list of access to remove writes itself. Impact analysis before a schema change becomes a lineage query plus a list of the people who use each affected report. Quality rules come from the filters and business rules you found in query logs, so they test what people really depend on.

## Using context for AI

AI assistants that answer questions over data fail in a predictable way. They pick the wrong table, join on the wrong key or use a definition nobody in the business agrees with. A larger model makes the same mistakes, because what it lacks is context.

The context you harvested is what an assistant needs: approved definitions, real join paths from query logs, known caveats, and example queries that analysts have already run and checked. Feed it in as retrieval content or as part of a semantic model the assistant queries. Only approved facts should go in. Drafts and conflicting definitions stay out until a person settles them, or the assistant will repeat whichever one it saw last.

Apply the same access rules to context as to data. If a user can’t query a table, the assistant shouldn’t describe that table’s contents to them either.

## Keeping it current

Context goes stale as fast as the data changes. Re-run the harvest on a schedule and compare it with the last run. A new column with no description, a table whose readers dropped to zero, or a definition whose source page was edited should all raise a flag for the owner.

Tie context to the change process. A pull request that adds a dbt model needs a description and an owner before it merges. A ticket that changes a column’s meaning updates the definition when it closes. Measure coverage by usage: the share of your most-read tables that have an owner, a definition and lineage tells you far more than the share of all tables.

## Where to start

Pick one business domain where wrong numbers hurt, such as revenue or customer. Pull 90 days of query logs and rank that domain’s tables by reads. Take the top 30.

Harvest system metadata and lineage for those tables, then run the text extraction over that domain’s tickets and wiki pages. Draft owners, definitions and caveats from what you find. Hold a few short sessions with the domain’s stewards to approve, fix or reject the drafts. Publish the approved context to your catalog and to any AI assistant that answers questions about that domain.

Then watch what happens. Count how many questions the catalog or assistant answers without someone asking an analyst. When that number moves, take the same steps to the next domain.

---

We help companies set up data governance that runs inside their platforms, including the catalog, lineage and context that AI assistants depend on. See our [Governance & Operations](https://datagist.ai/services/governance-operations) work, or [talk to us](https://datagist.ai/contact) about where your data context lives today and how to collect it.
