Queries#

Schemas define selected forms of correctness, but stored knowledge remains more flexible than any one model. Projects need to ask new questions without first converting facts into fixed tables or generated classes. Stardust uses declarative query documents with a Datalog-style relational core. The query remains data, just like the knowledge that it examines.

Logic before tables#

Stardust does not follow the common database approach of exposing SQL. SQL organizes questions around tables, while Stardust records dynamic entity-field-Component facts and logical structures. A SQL boundary would make a secondary tabular representation more important than the data Stardust produces. Object-relational mappers (ORMs) are a symptom of this mismatch because applications must convert between tabular results and the structures that they use.

Datalog provides a closer model for Stardust's facts. It is part of logic programming, which started before SQL. The Datalog name came later, but its less frequent use does not identify a technical limit. This choice fits Stardust's data instead of following language popularity.

A Datalog-style clause matches an entity, field, and Component. Variables that appear in more than one clause connect facts without a declared foreign key or fixed table layout. This DUST query joins each temperature to the unit recorded on the same device:

find  [?device ?temperature ?unit]
where [[?device temperature ?temperature]
       [?device temperatureUnit ?unit]]

Result shape creates another mismatch for Stardust. SQL results are tabular, while applications and agents frequently need nested data. GraphQL shows the demand for client-selected nested shapes. Nestable query data connects facts to their consumers without an object mapper.

A question can become knowledge#

A query document uses the same JSON-compatible value model as the rest of Stardust. People can author it as DUST, while agents and other clients can exchange it as JSON. No SQL parser, string builder, or language-specific callback becomes the authority. The document can be examined and compared with the same tools as other project knowledge.

Ad hoc queries permit experiments while a question is still changing. A repeated question can become a stored query under definitions/queries. Stardust validates the document against the query engine before it commits, so the stored tree holds only queries that compile. This reification records what the project asked instead of keeping that knowledge in application code alone.

Stored queries become typed routes in the generated Go SDK. Parameter types are inferred from how each parameter is used. A variable that appears only in a predicate, such as ?floor in [> ?age ?floor], becomes a required input without a parameters block. Declared parameters supply defaults that a caller can override.

Current and historical state#

A query reads current committed state by default. A read never takes Stardust's writer lock, so reads run beside other reads and ordinary writes. The executor reads facts through read-only store transactions.

The same language reads earlier state without a separate archive interface. bind {with {db {asOf 3}}} reads state as of one transaction, and an interval reads a window between two transactions. A what-if overlay under bind.with.facts layers hypothetical facts over the read without committing them.

Queries can use Macros from the current callable definitions or pin a Macro title to one historical revision with a definitions entry. Current mappings let shared calculations stay current without edits to every dependent query. Pins keep fixed semantics for the definitions that need them.

Relationships and calculations#

Fact clauses find relationships by sharing variables. A not clause excludes solutions whose inner fact matches. A schema clause keeps only entities that validate against a stored JSON Schema. Predicates keep rows when expressions are true, and derivations bind calculated values to new variables.

Aggregates such as count, sum, mean, and median calculate group values with groupBy and having. Window functions such as rowNumber, rank, and lag calculate across related result rows. These forms stay in one declarative document, and every expression compiles through Stardust's expression language.

A query can give object rows, positional rows, or a nested project shape that follows entity references along dotted paths. Positional results are useful for direct analysis and composition. A projection makes the selected consumer shape explicit without an additional client conversion.

Full-text search is another query clause rather than a separate search service. This clause binds each entity matching fox and its relevance score:

where [[fts fox ?e ?score]]

The analyzer applies Unicode NFKC normalization and case folding, divides the text into words, removes English stop words, and applies the English Snowball stemming algorithm. A score is a BM25 relevance value. A text query matches an entity only when the entity contains every query word. The phrase, prefix, and, or and not forms combine terms in other ways. Each commit updates the full-text index in the same transaction, and the index is derived and rebuildable.

Bounded answers#

Every execution has explicit bounds because a finite service cannot do unlimited work. maxCost sets a CPU time ceiling in nanoseconds, and without one a query receives a dynamic budget derived from its structure and rows handled. maxResults, maxBytes, and maxWorkRows bound rows, result size, and intermediate work. A query that exceeds a bound fails instead of returning a silently truncated answer. Each result reports its measured CPU time.

Large ordered results use keyset pages. A page requires orderBy, and each reply gives hasMore and an opaque cursor for the next page. The cursor names the last row by value, so rows inserted or deleted outside the window do not shift later pages. The cursor also carries a fingerprint that binds it to the query that created it. Pages do not share one snapshot, so a row changed inside the window between pages can still be skipped or repeated.

Live queries#

Live is not a kind of query. Any query can be run once or subscribed to, and the query document is the same either way:

parameters {?floor 18}
find       [?entity]
where      [[?entity age ?age]
            [> ?age ?floor]]

A subscription binds the parameter values once, when it starts, and holds them for its life. It yields the complete current result immediately. As facts change, Stardust re-runs the query with those bound values and yields the complete result again. Nothing is materialized or persisted, and the question stays exactly the one that was asked. A subscription always describes current facts, so it has no keyset page state and no historical bind.with.db read.

After every committed write, the mount sends each connected client a stardust/entitiesChanged notification with the touched entity ids and whether any entity was created. A client that falls behind receives coalesced notifications, because each one only means "re-read". The signal is useful on its own, and it is also what tells a subscription that its facts may have changed.

A query's where fact clauses name the fields it reads. A commit that touches none of those fields cannot change the result, so it does not need to re-run the query. This routing can over-trigger, but it never misses an affecting commit. Queries turn durable facts into answers without changing those facts. A Mutation uses the same relational model to select facts for a change.