Language values and documents#

Your application keeps its own structs, records, dataclasses and maps. The typed layer copies them into native documents and copies documents back into them. The native document API stays available beside it for work that needs native fidelity or no conversion.

Choose an interface#

Example#

The example stores three people from language values, runs a query written in the selected format, and decodes each row into a value of the language. The captured result is the decoded rows, encoded again for display.

Store Python dataclasses and read typed rows#

query_text holds this document:

find    [?name]
where   [[?e name ?name]]
orderBy [?name]

@dataclass
class Person:
    name: str
    role: str
    age: int


@dataclass
class Name:
    name: str


people = {
    "#_ada": Person("Ada", "engineer", 36),
    "#_bob": Person("Bob", "engineer", 41),
    "#_cy": Person("Cy", "designer", 29),
}
async with (await stardust.open(database_path)) as database:
    (await stardust.merge_patch_value(database, people))
    async with (await stardust.parse(query_text, "dust")) as definition, (await database.query(definition)) as query, \
            (await query.run()) as rows:
        names = [row async for row in stardust.values(rows, Name)]
result = (await stardust.encode(names))

[{name Ada} {name Bob} {name Cy}]

Standard JSON values#

Any value that the standard JSON encoder of your language accepts also converts here, and decode reads back what encode writes: encoding/json in Go, json.dumps in Python, serde_json in Rust, cheshire or data.json in Clojure and core:encoding/json in Odin. Native tags come first, so instants, durations and links become #utc, #dur and #link values instead of JSON text. Several languages accept more, such as Python sets and UUIDs, or Clojure java.time values. A value is refused only where that encoder refuses it, or where a document cannot hold it: a non-finite number, an integer outside i64, a decimal, instant or duration without an exact native form, or a cycle. The host mapping gives each type.

Ownership and errors#

Encode returns a new owned document that holds a snapshot of the value. Close it as you close any document. Decode borrows a document only for the call and returns independent values. It never closes or keeps the document. Typed row iteration borrows the rows and reads one page at a time. Close the rows yourself. Every page of one cursor reads the same store snapshot, so iteration never repeats the query.

A value-argument operation accepts a language value or a document. It passes a document through unchanged. It encodes a language value into a temporary document, calls the native operation once, and closes the temporary document on success and failure.

Await encode, decode and value-argument operations. Use async with for document owners and async for for typed rows. Conversion also accepts dataclasses, sets, Enum, UUID, exact Decimal and standard date and time types. We welcome Python developer feedback on which interface works best in applications.

A conversion failure is invalid_argument with a code and a value path such as people[1].age. A failed decode returns no partly filled value.

Costs#

Encode and decode copy every value. A document that you pass directly is not copied. Type inspection belongs only to the typed layer, and each language keeps per-type plans where it inspects types at run time. No conversion writes or reads DUST, EDN or TRON text, and JSON text appears only where a Go value supplies its own MarshalJSON or UnmarshalJSON method. Those formats are optional boundaries, not an intermediate transport.

The conversion rules and the host mapping below give each type, the null and missing-field rules and the strict mode.

Operations#

from_value#

def from_value(value, *, library: str | None = None) -> Document

Parameters#

Errors#

Notes#

Host maps cannot preserve duplicate keys. Use native documents when member order or duplicate keys matter.

Examples#

Copy a host integer into a native document#

result = (await stardust.from_value(41))

41

C exports: stardust_document_builder_create, stardust_document_builder_add_many, stardust_document_builder_seal, stardust_document_builder_error, stardust_document_builder_release

Document.to_value#

def to_value(self)

Parameters#

Errors#

Notes#

Host map semantics determine how duplicate keys are represented. The copied value survives document closure.

Examples#

Copy a native input into a host value#

input_text holds this document:

x 41

async with (await stardust.parse(input_text, "dust")) as input:
    value = (await input.to_value())
result = (await stardust.from_value(value))

{x 41}

C exports: stardust_document_count, stardust_document_read_many, stardust_document_i64, stardust_document_f64

Typed operations#

These helpers convert language values or call native operations. The nesting limit for typed documents is 128 levels.

duration_from_ticks#

def duration_from_ticks(ticks: int) -> timedelta:

duration_from_ticks(ticks) converts signed native 100 ns ticks to the language's standard duration. This conversion performs no native call. Precision loss and values outside the host range fail. Failures: invalid_argument when the value has the wrong type, loses precision or exceeds the target range.

duration_to_ticks#

def duration_to_ticks(value: timedelta) -> int:

duration_to_ticks(duration) converts a standard duration to signed native 100 ns ticks. This conversion performs no native call. Precision loss and values outside the native range fail instead of truncating or saturating. Failures: invalid_argument when the value has the wrong type, loses precision or exceeds the target range.

instant_from_ticks#

def instant_from_ticks(ticks: int) -> datetime:

instant_from_ticks(ticks) converts signed native 100 ns ticks since the Unix epoch to the language's standard timestamp. This conversion performs no native call. Precision loss and values outside the host range fail. Failures: invalid_argument when the value has the wrong type, loses precision or exceeds the target range.

instant_to_ticks#

def instant_to_ticks(value: datetime) -> int:

instant_to_ticks(instant) converts a standard timestamp to signed native 100 ns ticks since the Unix epoch. This conversion performs no native call. Precision loss and values outside the native range fail. The datetime must be timezone-aware. Failures: invalid_argument when the value has the wrong type, loses precision or exceeds the target range.

encode#

def encode(value, *, library: str | None = None) -> Document:

encode(value) copies a language value into a new owned document that holds an immutable snapshot. The caller closes it. Conversion walks the value directly into one preorder builder batch and serializes no text; only a Go json.Marshaler's JSON output is parsed, by the native JSON codec. Failures carry the value path, such as people[3].age, and leave no document: the temporary builder always closes. Failures: invalid_argument when the value has no document form (a type the language's standard JSON encoder also refuses), a number is outside i64 or finite f64, a decimal, instant or duration has no exact native form, a conversion method of the value fails, the nesting exceeds the depth limit, or the value contains a cycle. out_of_memory when native memory is exhausted.

decode#

def decode(document: Document, type_=None, *, strict: bool = False):

decode(document, type) copies a document into an independent language value of the requested type. It borrows the document only for the call and never closes or retains it. Reads use bounded bulk node reads. A failure reports the value path and returns no partially filled value. Typed objects refuse duplicate keys; unknown keys are ignored unless strict mode is on. A missing field keeps the language default; explicit null needs a nullable target. Failures: invalid_argument when a node does not match the target type, a number is outside the target range or, as an integer, has no exact form in it, a text does not parse as the target or its key type, a union has several matching variants, a conversion method of the target fails, an object repeats a key or, where a strict mode is offered and on, has an undeclared key. Narrow floating targets round to nearest. closed when the source is closed.

values#

def values(rows: Rows, type_=None, *, strict: bool = False):

values(rows, type) lazily decodes the rows of a query cursor one at a time, reading one native page at a time. It borrows the rows and never closes them: close the rows yourself. Each internally read page closes when iteration advances, ends, fails or stops early. All pages read the snapshot of the run that created the cursor, so typed iteration never requeries between pages. A failure is reported once, and then iteration ends. Failures: invalid_argument when a node does not match the target type, a number is outside the target range or, as an integer, has no exact form in it, a text does not parse as the target or its key type, a union has several matching variants, a conversion method of the target fails, an object repeats a key or, where a strict mode is offered and on, has an undeclared key. Narrow floating targets round to nearest. closed when the source is closed.

build_value#

def build_value(expression, explain=None, *, library: str | None = None) -> Program:

build_value(expression) accepts a language value or a borrowed document in place of the document argument of the low-level operation. A document is passed through unchanged. A language value is encoded into a temporary document, the low-level operation runs once, and the temporary document closes on success and failure. A string is a string value; it is never parsed as text. The result is the low-level operation's native owner or record. It calls the low-level build operation. Failures: those of encode for a language value, then those of the low-level operation.

evaluate_value#

def evaluate_value(program: Program, input=None, explain=None) -> Document:

evaluate_value(program, input) accepts a language value or a borrowed document in place of the document argument of the low-level operation. A document is passed through unchanged. A language value is encoded into a temporary document, the low-level operation runs once, and the temporary document closes on success and failure. A string is a string value; it is never parsed as text. The result is the low-level operation's native owner or record. It calls the low-level evaluate operation. Failures: those of encode for a language value, then those of the low-level operation.

query_value#

def query_value(database: Database, definition) -> Query:

query_value(database, definition) accepts a language value or a borrowed document in place of the document argument of the low-level operation. A document is passed through unchanged. A language value is encoded into a temporary document, the low-level operation runs once, and the temporary document closes on success and failure. A string is a string value; it is never parsed as text. The result is the low-level operation's native owner or record. It calls the low-level query operation. Failures: those of encode for a language value, then those of the low-level operation.

expression_value#

def expression_value(database: Database, definition, explain=None) -> Program:

expression_value(database, definition) accepts a language value or a borrowed document in place of the document argument of the low-level operation. A document is passed through unchanged. A language value is encoded into a temporary document, the low-level operation runs once, and the temporary document closes on success and failure. A string is a string value; it is never parsed as text. The result is the low-level operation's native owner or record. It calls the low-level expression operation. Failures: those of encode for a language value, then those of the low-level operation.

run_query_value#

def run_query_value(query: Query, parameters=None, *, page_size: int | None = None, max_cost: timedelta | None = None, as_of: int | None = None, explain: _core.QueryExplain | None = None) -> Rows:

run_query_value(query, parameters, options) accepts a language value or a borrowed document in place of the document argument of the low-level operation. A document is passed through unchanged. A language value is encoded into a temporary document, the low-level operation runs once, and the temporary document closes on success and failure. A string is a string value; it is never parsed as text. The result is the low-level operation's native owner or record. It calls the low-level run operation. Failures: those of encode for a language value, then those of the low-level operation.

mutation_value#

def mutation_value(database: Database, definition) -> Mutation:

mutation_value(database, definition) accepts a language value or a borrowed document in place of the document argument of the low-level operation. A document is passed through unchanged. A language value is encoded into a temporary document, the low-level operation runs once, and the temporary document closes on success and failure. A string is a string value; it is never parsed as text. The result is the low-level operation's native owner or record. It calls the low-level mutation operation. Failures: those of encode for a language value, then those of the low-level operation.

run_mutation_value#

def run_mutation_value(mutation: Mutation, parameters=None, *, page_size: int | None = None, max_cost: timedelta | None = None, as_of: int | None = None, explain: _core.MutationExplain | None = None) -> _core.Commit:

run_mutation_value(mutation, parameters, options) accepts a language value or a borrowed document in place of the document argument of the low-level operation. A document is passed through unchanged. A language value is encoded into a temporary document, the low-level operation runs once, and the temporary document closes on success and failure. A string is a string value; it is never parsed as text. The result is the low-level operation's native owner or record. It calls the low-level run operation. Failures: those of encode for a language value, then those of the low-level operation.

merge_patch_value#

def merge_patch_value(database: Database, patch, explain=None) -> _core.Commit:

merge_patch_value(database, patch) accepts a language value or a borrowed document in place of the document argument of the low-level operation. A document is passed through unchanged. A language value is encoded into a temporary document, the low-level operation runs once, and the temporary document closes on success and failure. A string is a string value; it is never parsed as text. The result is the low-level operation's native owner or record. It calls the low-level merge_patch operation. Failures: those of encode for a language value, then those of the low-level operation.

patch_definition_value#

def patch_definition_value(database: Database, kind: str, path: str, patch) -> _core.Commit:

patch_definition_value(database, kind, path, patch) accepts a language value or a borrowed document in place of the document argument of the low-level operation. A document is passed through unchanged. A language value is encoded into a temporary document, the low-level operation runs once, and the temporary document closes on success and failure. A string is a string value; it is never parsed as text. The result is the low-level operation's native owner or record. It calls the low-level patch_definition operation. Failures: those of encode for a language value, then those of the low-level operation.

Conversion rules#

Host mapping#

Typed encode and decode, and the untyped from_value and to_value where the language offers them, use this mapping. Each conversion walks values directly; no text format is involved.