From software factory to data factory

An agentic software factory is a framework of loops that gives agents goals and context, lets them build, test, review, learn from feedback, and refine the software until it meets the standard for release.

Software factories became necessary because code generation alone did not solve software delivery. Agents needed quality checks, shared context, validation, memory, and accumulated learning before teams could trust them with more autonomous work. That infrastructure lets agents carry more of the software lifecycle while humans remain focused on setting goals, applying judgment, and shipping the product.

Data is becoming the same constraint for every agent expected to understand or act on a business. Access to a database is not access to meaning. Raw tables lack the definitions, relationships, history, quality controls, and company context agents need, forcing each one to rediscover the business before it can produce trustworthy work. The opportunity is to make company data prepared, validated, and reusable for every human and agent that depends on it.

The next logical step is an agentic data factory built on the same principles of context, refinement, quality, feedback, and learning. It refines raw data into trusted metrics, queryable datasets, relationships, and knowledge that agents can use reliably. This gives teams of humans and their agents a shared data layer for running the business, while the knowledge it produces empowers the judgment and wisdom no system can generate on its own.

Raw data must be refined

Without prepared analytical context, every agent can approach a data request like it is the first one. It inspects schemas, searches for tables, generates new SQL against production systems, loads rows into context, recovers definitions, and rebuilds metrics another agent has already calculated.

Connecting, sorting, joining, and applying semantics to data are necessary steps, but they are not the finished product. Raw data becomes useful when it is refined into the units a business actually runs on: a trusted metric, a defined KPI, or a valuable queryable dataset.

A refinery separates, purifies, combines, and concentrates raw materials into something more valuable. The Data Refinery inside a Data Factory does the same with the raw material of a business.

What a Data Factory unlocks

Dreambase Data Factory turns raw company data into reusable analytical datasets, trusted metrics, and a growing Knowledge Layer for people, teams, and agents.

Specialized agents source data from product databases, billing platforms, analytics tools, CRMs, APIs, and MCP servers. They join and refine it, adding definitions, annotations, historical events, useful comparisons, anomaly context, and the relationships that form a Knowledge Graph.

Datasets and metrics are the core products of the factory. The Knowledge Layer preserves how those products relate to the business and how they should be used. Together, they give humans and agents the context required to reason over company data without reconstructing its meaning inside every prompt.

Data Refinement

Refinement is the defining function of the Data Factory.

For a business question, the factory can join multiple sources into a purpose-built analytical dataset. The result is materialized as a Parquet dataset cache and analyzed with DuckDB, giving agents an analytical engine without repeatedly querying production systems or loading every row into their context windows.

A refined dataset can support one investigation, power dashboards and reports, or become a durable data product that other agents and workflows reuse. KPIs retain their definitions, current values, history, and important business events. Day-over-day changes and rolling 7-day, 14-day, and 30-day comparisons are pre-calculated, while anomaly detection identifies changes that deserve attention.

The factory does not simply move or create more data. It concentrates raw data into the metrics, datasets, relationships, and context that carry business value.

One shared data layer

For business leaders and operators, refinement turns activity across the company into refreshed KPIs with consistent definitions, historical context, annotations, and anomalies. The results can be delivered through dashboards, reports, alerts, digests, and mobile experiences. The value is getting the same number twice and knowing what changed around it.

For product, finance, operations, and other internal teams, Dreambase provides a Data Analyst that works from refined datasets and the company Knowledge Layer. It can create dashboards, reports, presentations, and investigations without forcing each team to recover the joins, definitions, history, and business context behind the question.

For developers and agent builders, one MCP gives Claude, Codex, ChatGPT, Cursor, Gemini, Grok, agents in Slack, and internal systems access to the same refined datasets, metrics, and Knowledge Layer. Those agents work from governed analytical products instead of querying live production databases directly or filling their context windows with unrefined rows.

Wisdom, the holy grail of agentic data

Raw data can become information. Refined information, connected through definitions, relationships, history, and context, becomes knowledge.

Wisdom requires judgment, experience, and responsibility. You cannot generate wisdom on demand, but you can empower it.

Dreambase builds a semantic layer around the data and a context layer around the organization. Schemas, relationships, definitions, annotations, rules, memories, prior decisions, and usage patterns improve what the factory produces over time.

People get the history and context behind a metric before making a decision. Agents get governed analytical products instead of rediscovering the business. Teams of humans and their agents get a shared foundation for applying judgment without starting over.

Software factories give agents the environment required to produce and verify software. A Data Factory provides the equivalent environment for refining raw company data into the analytical products and knowledge used to run the business.

This is why we built the Dreambase Data Factory for agents.