Overview
An application starts with a database and a model of the data it needs. As requirements change, migrations reshape that model. Some discard information, while others require rebuilding the stored data. Updating and deleting records also removes the previous state unless the application preserves it separately.
An audit requirement introduces history tables and the code that maintains them. Those tables grow, so they need retention and compaction rules. An anonymization requirement then reaches beyond the current records into that history and the backups. Restoring a backup must account for information that should no longer be kept.
Each addition changes the responsibilities of the others. The team has to keep their behavior consistent before addressing N+1 queries, cache invalidation, or a move to another language or platform. Much of this work concerns the machinery around the data, repeated across applications. Stardust exists to make that machinery part of a shared data engine.
Open ontologies
An application learns more about its domain as people use it. New facts may extend the existing model without changing the meaning of facts already stored. Requiring a complete model before accepting those facts ties new knowledge to structural changes in storage. A migration can then become necessary even when the application only needs to describe something more precisely.
Stardust starts with an open ontology: entities can acquire additional fields and relationships without a declaration of every permitted shape. Schemas add constraints where a particular operation needs them. An entity can participate in different uses, each with its own requirements. This gives the model room to develop while allowing validation at the places that require a specific structure.
An open model still needs decisions about meaning. Adding a field cannot determine whether it duplicates an existing concept or conflicts with one. The application defines the requirements for using those fields, including validation and the operations that change them. In Stardust, those definitions are themselves data.
Reification
Reification gives something an identity and a representation as data. In Stardust, this applies to the machinery that acts on records as well as the records themselves. Stored query and mutation definitions, schemas, and transactions can be addressed and examined. Changes to stored definitions have transaction history, so earlier behavior can be examined alongside earlier data.
This matters when behavior belongs close to the data it uses. Moving data between processes introduces work, and repeated round trips can dominate an otherwise small computation. Database triggers can keep work near storage, but introduce behavior that a team must also source control and update. Reifying behavior makes its definition available to the same storage and query machinery as the data it operates on.
That shared representation lets an application inspect the definitions it depends on and keep their revisions. An incorrect definition can affect many records, so its changes need review just as application code does. Keeping revisions also requires a decision about how much history to preserve. Reification gives the application access to that history and the definitions it needs to manage.
A data engine
A game engine supplies systems that many games need and makes them work together. The game developer can build on their common model instead of integrating each system independently. Stardust applies that idea to applications built around data. Open ontologies let the model grow, while reification brings its behavior and changes into that model.