Formats and conversion#

Convert between JSON, DUST, EDN and TRON with the library's parse and serialize operations, through the C ABI or any binding. Serialization writes a document as text, or as binary bytes for TRON. The example parses the people document and serializes it in the same format.

Example#

Serialize the people document#

people_text holds this document:

async with (await stardust.parse(people_text, "tron")) as people:
    output = (await people.serialize("tron"))
result = (await stardust.parse(output, "tron"))

Choosing a format#

EDN query variables#

EDN symbols such as ?name parse as native strings. Serialization writes a native string such as "?name" as the EDN symbol ?name. The string must start with ?, contain a name, and form one valid EDN symbol token. Other string values remain quoted.

Query variables therefore keep their symbol form through EDN, JSON, DUST and TRON conversions. Clojure users can quote a query map to pass its symbols as data:

(with-open [query (sd/from-value
                   '{:find [?name ?age]
                     :where [[?e :name ?name] [?e :age ?age]]
                     :orderBy [?name]})]
  (sd/serialize query :edn))

DUST layout#

An object on one line, {op a args {t 1} e 1}, separates its members with whitespace. A top-level document, and any object that spans lines, holds exactly one member per line, and the rest of the line is the value of that member:

{name Ada Lovelace
 role engineer}

This object is {"name":"Ada Lovelace","role":"engineer"}. In a multi-line object, {op a args 1 is the member op with the string a args 1. A line that cannot be one bare value, such as {op a args {t 1}, fails with Members_On_One_Line and the offset of its second key. Start each member on its own line, or keep the whole object on one line. Serialization never writes several members on one line of a multi-line object.

Operations#

Document.serialize#

def serialize(self, format: str, width: int = 0) -> str | bytes

Parameters#

Errors#

Examples#

document/serialize#

expression_text holds this document:

async with (await stardust.parse(expression_text, "tron")) as expression:
    result = (await expression.serialize("tron"))

document/serialize#

number_text holds this document:

async with (await stardust.parse(number_text, "tron")) as number:
    result = (await number.serialize("tron"))

document/serialize#

name_text holds this document:

async with (await stardust.parse(name_text, "tron")) as name:
    result = (await name.serialize("tron"))

document/serialize#

expression_text holds this document:

async with (await stardust.parse(expression_text, "tron")) as expression:
    result = (await expression.serialize("tron"))

document/serialize#

expression_text holds this document:

async with (await stardust.parse(expression_text, "tron")) as expression:
    result = (await expression.serialize("tron"))

C exports: stardust_serialize, stardust_text_data, stardust_text_error, stardust_text_release

convert#

def convert(
    text: str | bytes, from_format: str, to_format: str, width: int = 0, *, library: str | None = None
) -> str | bytes

Parameters#

Errors#

Examples#

convert#

expression_text holds these bytes:

// One expression built once and evaluated against several inputs. The
// evaluate examples load this file, build `expression`, evaluate it with each
// of `inputs` in order and compare each result with `expected`.
expression [+ ?x 1]
inputs     [{x 1} {x 41}]
expected   [2 42]

result = (await stardust.convert(expression_text, "dust", "tron"))

convert#

edn_text holds this document:

async with (await stardust.parse(edn_text, "tron")) as edn:
    output = (await edn.read((await edn.root()))).text
try:
    (await stardust.parse(output, "edn")).close()
except stardust.StardustError as error:
    result = error

convert#

instant_text holds this document:

async with (await stardust.parse(instant_text, "tron")) as instant:
    result = (await instant.serialize("edn"))

convert#

container_text holds this document:

async with (await stardust.parse(container_text, "tron")) as container:
    output = (await container.serialize("tron"))
result = (await stardust.parse(output, "tron"))

convert#

order_text holds this document:

async with (await stardust.parse(order_text, "tron")) as order:
    output = (await order.serialize("tron"))
result = (await stardust.parse(output, "tron"))

C exports: stardust_parse, stardust_serialize, stardust_text_data, stardust_text_error, stardust_text_release

Fidelity#

Each cell shows the result after parsing and serialization in that format. The notes below the table explain each result.

Fidelity notes#

• null_boolean
  • JSON: Round trip. Null and booleans keep their values.
  • DUST: Round trip. Null and booleans keep their values.
  • EDN: Round trip. EDN nil maps to null.
  • TRON: Round trip. Binary tags preserve null and booleans.
• integer
  • JSON: Round trip. The codec preserves integer spelling.
  • DUST: Changed. Leading-zero tokens are strings.
  • EDN: Changed. Integer spelling is canonical.
  • TRON: Changed. Integers keep their value and use canonical spelling.
• float
  • JSON: Round trip. The codec preserves float spelling.
  • DUST: Round trip. The codec preserves float spelling.
  • EDN: Round trip. The codec preserves this float spelling.
  • TRON: Changed. Integral floats in containers become integers.
• non_finite
  • JSON: Refused. JSON has no non-finite numbers.
  • DUST: Changed. NaN is a string.
  • EDN: Refused. The parser rejects non-finite numeric values.
  • TRON: Refused. The parser rejects non-finite binary values.
• decimal_ratio
  • JSON: Round trip. The codec preserves finite decimal spelling.
  • DUST: Round trip. The codec preserves finite decimal spelling.
  • EDN: Changed. Ratios reduce to native numbers.
  • TRON: Changed. Decimals use f64 precision.
• string
  • JSON: Round trip. Strings require UTF-8.
  • DUST: Round trip. Number-like strings require quotes.
  • EDN: Changed. A lone surrogate becomes a question mark.
  • TRON: Round trip. Decoded strings require UTF-8.
• member_order
  • JSON: Round trip. The codec preserves object order.
  • DUST: Round trip. The codec preserves object order.
  • EDN: Round trip. The codec preserves map order.
  • TRON: Changed. Members use trie order.
• duplicate_keys
  • JSON: Round trip. The codec preserves all duplicate members.
  • DUST: Round trip. The codec preserves all duplicate members.
  • EDN: Refused. The parser rejects equal map keys.
  • TRON: Changed. The codec keeps the last duplicate value.
• empty_object
  • JSON: Round trip. The empty object is braces.
  • DUST: Round trip. The empty root object serializes to zero bytes.
  • EDN: Round trip. The empty map is braces.
  • TRON: Round trip. The empty object has a binary footer.
• utc
  • JSON: Round trip. UTC remains a tagged object.
  • DUST: Round trip. UTC uses a tagged object.
  • EDN: Changed. Millisecond instants use the EDN inst tag.
  • TRON: Round trip. UTC remains a plain map.
• link
  • JSON: Round trip. Links remain tagged objects.
  • DUST: Round trip. Links remain tagged objects.
  • EDN: Round trip. Links remain maps.
  • TRON: Round trip. Links remain maps.
• binary
  • JSON: Round trip. Base64 remains a tagged object.
  • DUST: Round trip. Base64 remains a tagged object.
  • EDN: Round trip. Base64 remains a map.
  • TRON: Changed. Valid padded base64 in a sole member becomes a binary node.
• edn_forms
  • JSON: Not applicable. EDN forms have no JSON syntax.
  • DUST: Not applicable. EDN forms have no DUST syntax.
  • EDN: Changed. The parser converts keywords, symbols, characters, sets and lists to native values. It omits metadata and discarded forms. It reads only the first form.
  • TRON: Not applicable. EDN forms have no TRON syntax.

The host mapping on the Language values and documents page shows how language values become documents without a text format.