Explain and cost

Explain output shows the work that a call completed. The example executes the names query with an explain record and reads its completed stages.

Example

Explain a query

query_text holds this document:

edn

66 B

{:find    [?name]
 :where   [[?e "name" ?name]]
 :orderBy [?name]}

670 B

stardust_document *definition = NULL;
stardust_db_query *query = NULL;
stardust_db_rows *rows = NULL;
stardust_db_query_explain explain = {0};
uint32_t status = stardust_parse(query_text, STARDUST_EDN, &definition);
stardust_db_source source = {.kind = STARDUST_SOURCE_DOCUMENT, .document = definition};
if (status == STARDUST_OK) status = stardust_db_query_open(db, &source, &query);
if (status == STARDUST_OK) status = stardust_db_query_run(query, NULL, NULL, &explain, &rows);
stardust_db_rows_release(rows);
stardust_db_query_release(query);
stardust_document_release(definition);
if (status != STARDUST_OK) return status;
stardust_db_query_explain result = explain;

Result / 1.1 kB

{:value {:load_definition   {:cpu         1788
                             :wall        1800
                             :arena_bytes 1680}
         :parameters        {:cpu         10
                             :wall        11
                             :arena_bytes 0}
         :run               {:cpu         10411
                             :wall        10526
                             :arena_bytes 7424}
         :total             {:cpu         12252
                             :wall        12379
                             :arena_bytes 9104}
         :arena_capacity    4194304
         :work              7122
         :rows              3
         :compile           337
         :plan              621
         :execute           0
         :group             0
         :order             0
         :memory_bytes      5995
         :memory_refusals   0
         :budget_bytes      10489856512
         :budget_used_bytes 10194
         :budget_peak_bytes 14267
         :budget_refusals   0
         :retry             0
         :stages_completed  3
         :retries           0}}

Failure output

A failed call can still supply explain output for the stages that completed.

Read stages_completed.
Then read the fields of the stages that completed.

The record tables below give the fields, absent values, and time units of each operation.

A call's cost record describes that execution. It does not load stored plan history. Use the plan operation in Definitions with the explain action to read accepted plans and their history. Each revision reports its version, source and reason. Trial decisions include candidate proposals and measured verdicts.

Set page_size and max_bytes to bound one diagnostic response. The response includes the current accepted transaction, the complete plan state and a history page from one snapshot. Pass a nonzero next_after as the next page's argument.

The adaptive query planner can retry a capped plan with the statistics plan. Query explain reports those attempts in retry and retries.

stardust_cost

This record measures one explain stage. The cpu and wall fields are durations. The ABI gives 100-ns ticks. Thus, a time less than one tick is zero.

Field	C type	Unit	Range	Meaning
`cpu_ticks`	`int64_t`	100 ns ticks	Signed i64	Thread CPU time. The call measures this value. Independent calls can differ.
`wall_ticks`	`int64_t`	100 ns ticks	Signed i64	Wall time. The call measures this value. Independent calls can differ.
`arena_bytes`	`int64_t`	Bytes	Signed i64	Bytes the stage allocated from the scratch arena of the call. Heap-owned outputs and the LuaJIT heap are not included. The call measures this value. Independent calls can differ.

stardust_expr_build_explain

This record contains explain output for build.

Field	C type	Unit	Range	Meaning
`prepare_expression`	`stardust_cost`	None	C type range	—
`compile`	`stardust_cost`	None	C type range	—
`total`	`stardust_cost`	None	C type range	The cost of all stages. `arena_bytes` records the peak scratch-arena allocation.
`arena_capacity`	`int64_t`	Bytes	Signed i64	Bytes the scratch arena reserved. The call measures this value. Independent calls can differ.
`stages_completed`	`uint32_t`	Count	Unsigned u32	Stages that ran.
`cache_hit`	`uint32_t`	None	Unsigned u32	The leased runtime holds a compiled copy of the expression. The call measures this value. Independent calls can differ.

stardust_expr_eval_explain

This record contains explain output for evaluate.

Field	C type	Unit	Range	Default	Meaning
`prepare_variables`	`stardust_cost`	None	C type range	The call sets it	—
`evaluate`	`stardust_cost`	None	C type range	The call sets it	—
`adopt_document`	`stardust_cost`	None	C type range	The call sets it	—
`total`	`stardust_cost`	None	C type range	The call sets it	The cost of all stages. `arena_bytes` records the peak scratch-arena allocation.
`arena_capacity`	`int64_t`	Bytes	Signed i64	The call sets it	Bytes the scratch arena reserved. The call measures this value. Independent calls can differ.
`document_bytes`	`int64_t`	Bytes	Signed i64	The call sets it	Used bytes of the native value, string and source buffers of the result. The call measures this value. Independent calls can differ.
`stages_completed`	`uint32_t`	Count	Unsigned u32	The call sets it	Stages that ran.
`reserved`	`uint32_t`	None	Unsigned u32	0	Must be zero.

stardust_db_query_explain

This record contains explain output for query run. The compile, plan, execute, group and order fields divide the wall time of run into stages.

Field	C type	Unit	Range	Meaning
`load_definition`	`stardust_cost`	None	C type range	—
`parameters`	`stardust_cost`	None	C type range	—
`run`	`stardust_cost`	None	C type range	Planning, execution and aggregation. Pages render later, in next.
`total`	`stardust_cost`	None	C type range	The cost of all stages. `arena_bytes` records the peak scratch-arena allocation.
`arena_capacity`	`int64_t`	Bytes	Signed i64	Bytes the scratch arena reserved. The call measures this value. Independent calls can differ.
`work_ticks`	`int64_t`	100 ns ticks	Signed i64	Thread CPU duration consumed by the query. The call measures this value. Independent calls can differ.
`rows`	`int64_t`	Count	Signed i64	Result rows.
`compile_ticks`	`int64_t`	100 ns ticks	Signed i64	The call measures this value. Independent calls can differ.
`plan_ticks`	`int64_t`	100 ns ticks	Signed i64	The call measures this value. Independent calls can differ.
`execute_ticks`	`int64_t`	100 ns ticks	Signed i64	Production of solutions. A streamed result produces solutions while pages render. Thus, execute measures only the solutions that run reads. The call measures this value. Independent calls can differ.
`group_ticks`	`int64_t`	100 ns ticks	Signed i64	Aggregates, windows and computed columns. The call measures this value. Independent calls can differ.
`order_ticks`	`int64_t`	100 ns ticks	Signed i64	The call measures this value. Independent calls can differ.
`memory_bytes`	`int64_t`	Bytes	Signed i64	The most working memory the run held reserved in the database's memory budget at once. The call measures this value. Independent calls can differ.
`memory_refusals`	`int64_t`	Count	Signed i64	Reservations of the run that the memory budget granted less than asked. The call measures this value. Independent calls can differ.
`budget_bytes`	`int64_t`	Bytes	Signed i64	The database's memory budget (`memory_budget_bytes`).
`budget_used_bytes`	`int64_t`	Bytes	Signed i64	Bytes every query of the database held reserved when run returned. The call measures this value. Independent calls can differ.
`budget_peak_bytes`	`int64_t`	Bytes	Signed i64	The most bytes the database's queries held reserved at once since it opened. The call measures this value. Independent calls can differ.
`budget_refusals`	`int64_t`	Count	Signed i64	Reservations the database's budget granted less than asked since it opened. The call measures this value. Independent calls can differ.
`retry_ticks`	`int64_t`	100 ns ticks	Signed i64	Thread CPU duration of the attempts given up because the planner's plan passed its cap. work includes it. The call measures this value. Independent calls can differ.
`stages_completed`	`uint32_t`	Count	Unsigned u32	Stages that ran.
`retries`	`uint32_t`	Count	Unsigned u32	Attempts given up because the planner's plan passed its cap; the query then ran the statistics' plan within the rest of its budget.

stardust_db_mutation_explain

This record contains explain output for mutation run.

Field	C type	Unit	Range	Meaning
`load_definition`	`stardust_cost`	None	C type range	—
`parameters`	`stardust_cost`	None	C type range	—
`run`	`stardust_cost`	None	C type range	—
`total`	`stardust_cost`	None	C type range	The cost of all stages. `arena_bytes` records the peak scratch-arena allocation.
`arena_capacity`	`int64_t`	Bytes	Signed i64	Bytes the scratch arena reserved. The call measures this value. Independent calls can differ.
`stages_completed`	`uint32_t`	Count	Unsigned u32	Stages that ran.
`attempts`	`uint32_t`	Count	Unsigned u32	Evaluations. More than one when a concurrent write forced a retry.

stardust_db_patch_explain

This record contains explain output for merge_patch.

Field	C type	Unit	Range	Default	Meaning
`copy_patch`	`stardust_cost`	None	C type range	The call sets it	—
`commit`	`stardust_cost`	None	C type range	The call sets it	Validation and the commit.
`total`	`stardust_cost`	None	C type range	The call sets it	The cost of all stages. `arena_bytes` records the peak scratch-arena allocation.
`arena_capacity`	`int64_t`	Bytes	Signed i64	The call sets it	Bytes the scratch arena reserved. The call measures this value. Independent calls can differ.
`stages_completed`	`uint32_t`	Count	Unsigned u32	The call sets it	Stages that ran.
`reserved`	`uint32_t`	None	Unsigned u32	0	Must be zero.

Explain examples

The generator captured each result from the native library.

Inspect one measured cost stage

expression_text holds this document:

edn

10 B

["+" ?x 1]

445 B

stardust_document *expression = NULL;
stardust_expr_build_explain explain = {0};
stardust_expr_program *program = NULL;
uint32_t status = stardust_parse(expression_text, STARDUST_EDN, &expression);
if (status == STARDUST_OK) status = stardust_expr_build(expression, &explain, &program);
stardust_expr_program_release(program);
stardust_document_release(expression);
if (status != STARDUST_OK) return status;
stardust_cost result = explain.total;

Result / 75 B

{:value {:cpu         12
         :wall        13
         :arena_bytes 0}}

Explain expression compilation

expression_text holds this document:

edn

10 B

["+" ?x 1]

453 B

stardust_document *expression = NULL;
stardust_expr_build_explain explain = {0};
stardust_expr_program *program = NULL;
uint32_t status = stardust_parse(expression_text, STARDUST_EDN, &expression);
if (status == STARDUST_OK) status = stardust_expr_build(expression, &explain, &program);
stardust_expr_program_release(program);
stardust_document_release(expression);
if (status != STARDUST_OK) return status;
stardust_expr_build_explain result = explain;

Result / 511 B

{:value {:prepare_expression {:cpu         2
                              :wall        2
                              :arena_bytes 0}
         :compile            {:cpu         8
                              :wall        7
                              :arena_bytes 0}
         :total              {:cpu         12
                              :wall        12
                              :arena_bytes 0}
         :arena_capacity     65536
         :stages_completed   2
         :cache_hit          true}}

Explain patch validation and commit

seed_text holds this document:

edn

287 B

{:#_ada {:name "Ada"
         :role "engineer"
         :age  36}
 :#_bob {:name   "Bob"
         :role   "engineer"
         :age    41
         :mentor {:#link "#_ada"}}
 :#_cy  {:name "Cy"
         :role "designer"
         :age  29}
 :#_tx  {:author "smoke"
         :reason "seed"}}

425 B

stardust_document *patch = NULL;
stardust_db_patch_explain explain = {0};
stardust_result *written = NULL;
uint32_t status = stardust_parse(seed_text, STARDUST_EDN, &patch);
if (status == STARDUST_OK) status = stardust_db_merge_patch(db, patch, NULL, &explain, &written);
stardust_result_release(written);
stardust_document_release(patch);
if (status != STARDUST_OK) return status;
stardust_db_patch_explain result = explain;

Result / 484 B

{:value {:copy_patch       {:cpu         1435
                            :wall        1438
                            :arena_bytes 3096}
         :commit           {:cpu         317
                            :wall        28296
                            :arena_bytes 8008}
         :total            {:cpu         1866
                            :wall        29849
                            :arena_bytes 11118}
         :arena_capacity   4194304
         :stages_completed 2}}

Explain mutation execution

birthday_text holds this document:

edn

82 B

{:query {:find  [?e]
         :where [[?e "name" "Ada"]]}
 :patch {:?e {:age 37}}}

710 B

stardust_document *definition = NULL;
stardust_db_mutation *mutation = NULL;
stardust_result *written = NULL;
stardust_db_mutation_explain explain = {0};
uint32_t status = stardust_parse(birthday_text, STARDUST_EDN, &definition);
stardust_db_source source = {.kind = STARDUST_SOURCE_DOCUMENT, .document = definition};
if (status == STARDUST_OK) status = stardust_db_mutation_open(db, &source, &mutation);
if (status == STARDUST_OK) status = stardust_db_mutation_run(mutation, NULL, NULL, &explain, &written);
stardust_result_release(written);
stardust_db_mutation_release(mutation);
stardust_document_release(definition);
if (status != STARDUST_OK) return status;
stardust_db_mutation_explain result = explain;

Result / 647 B

{:value {:load_definition  {:cpu         1150
                            :wall        1157
                            :arena_bytes 2320}
         :parameters       {:cpu         10
                            :wall        11
                            :arena_bytes 0}
         :run              {:cpu         7174
                            :wall        35893
                            :arena_bytes 43856}
         :total            {:cpu         8427
                            :wall        37161
                            :arena_bytes 46176}
         :arena_capacity   4194304
         :stages_completed 3
         :attempts         1}}

Explain expression evaluation

expression_text holds this document:

edn

10 B

["+" ?x 1]

input_text holds this document:

edn

7 B

{:x 41}

760 B

stardust_document *expression = NULL;
stardust_document *input = NULL;
stardust_expr_program *program = NULL;
stardust_result *output = NULL;
stardust_expr_eval_explain explain = {0};
uint32_t status = stardust_parse(expression_text, STARDUST_EDN, &expression);
if (status == STARDUST_OK) status = stardust_parse(input_text, STARDUST_EDN, &input);
if (status == STARDUST_OK) status = stardust_expr_build(expression, NULL, &program);
if (status == STARDUST_OK) status = stardust_expr_evaluate(program, input, &explain, &output);
stardust_result_release(output);
stardust_expr_program_release(program);
stardust_document_release(input);
stardust_document_release(expression);
if (status != STARDUST_OK) return status;
stardust_expr_eval_explain result = explain;

Result / 632 B

{:value {:prepare_variables {:cpu         2
                             :wall        2
                             :arena_bytes 0}
         :evaluate          {:cpu         23
                             :wall        23
                             :arena_bytes 0}
         :adopt_document    {:cpu         2
                             :wall        2
                             :arena_bytes 0}
         :total             {:cpu         30
                             :wall        30
                             :arena_bytes 0}
         :arena_capacity    65536
         :document_bytes    42
         :stages_completed  3}}

Explain query execution

query_text holds this document:

edn

66 B

{:find    [?name]
 :where   [[?e "name" ?name]]
 :orderBy [?name]}

670 B

stardust_document *definition = NULL;
stardust_db_query *query = NULL;
stardust_db_rows *rows = NULL;
stardust_db_query_explain explain = {0};
uint32_t status = stardust_parse(query_text, STARDUST_EDN, &definition);
stardust_db_source source = {.kind = STARDUST_SOURCE_DOCUMENT, .document = definition};
if (status == STARDUST_OK) status = stardust_db_query_open(db, &source, &query);
if (status == STARDUST_OK) status = stardust_db_query_run(query, NULL, NULL, &explain, &rows);
stardust_db_rows_release(rows);
stardust_db_query_release(query);
stardust_document_release(definition);
if (status != STARDUST_OK) return status;
stardust_db_query_explain result = explain;

Result / 1.1 kB

{:value {:load_definition   {:cpu         1326
                             :wall        1370
                             :arena_bytes 1680}
         :parameters        {:cpu         7
                             :wall        9
                             :arena_bytes 0}
         :run               {:cpu         9090
                             :wall        9090
                             :arena_bytes 7424}
         :total             {:cpu         10449
                             :wall        10495
                             :arena_bytes 9104}
         :arena_capacity    4194304
         :work              6082
         :rows              3
         :compile           301
         :plan              459
         :execute           0
         :group             0
         :order             0
         :memory_bytes      5995
         :memory_refusals   0
         :budget_bytes      10489856512
         :budget_used_bytes 10194
         :budget_peak_bytes 14267
         :budget_refusals   0
         :retry             0
         :stages_completed  3
         :retries           0}}