Explain and cost

Explain output shows the work that a call completed. The example executes the names query with an explain record and reads its completed stages.

Example

Explain a query

query_text holds this document:

json

134 B

{
  "find": [
    "?name"
  ],
  "where": [
    [
      "?e",
      "name",
      "?name"
    ]
  ],
  "orderBy": [
    "?name"
  ]
}

671 B

stardust_document *definition = NULL;
stardust_db_query *query = NULL;
stardust_db_rows *rows = NULL;
stardust_db_query_explain explain = {0};
uint32_t status = stardust_parse(query_text, STARDUST_JSON, &definition);
stardust_db_source source = {.kind = STARDUST_SOURCE_DOCUMENT, .document = definition};
if (status == STARDUST_OK) status = stardust_db_query_open(db, &source, &query);
if (status == STARDUST_OK) status = stardust_db_query_run(query, NULL, NULL, &explain, &rows);
stardust_db_rows_release(rows);
stardust_db_query_release(query);
stardust_document_release(definition);
if (status != STARDUST_OK) return status;
stardust_db_query_explain result = explain;

Result / 671 B

{
  "load_definition": {
    "cpu": 872,
    "wall": 883,
    "arena_bytes": 1680
  },
  "parameters": {
    "cpu": 13,
    "wall": 22,
    "arena_bytes": 0
  },
  "run": {
    "cpu": 12719,
    "wall": 12804,
    "arena_bytes": 7424
  },
  "total": {
    "cpu": 13640,
    "wall": 13745,
    "arena_bytes": 9104
  },
  "arena_capacity": 4194304,
  "work": 8603,
  "rows": 3,
  "compile": 362,
  "plan": 673,
  "execute": 0,
  "group": 0,
  "order": 0,
  "memory_bytes": 5995,
  "memory_refusals": 0,
  "budget_bytes": 10489856512,
  "budget_used_bytes": 10194,
  "budget_peak_bytes": 14267,
  "budget_refusals": 0,
  "retry": 0,
  "stages_completed": 3,
  "retries": 0
}

Failure output

A failed call can still supply explain output for the stages that completed.

Read stages_completed.
Then read the fields of the stages that completed.

The record tables below give the fields, absent values, and time units of each operation.

A call's cost record describes that execution. It does not load stored plan history. Use the plan operation in Definitions with the explain action to read accepted plans and their history. Each revision reports its version, source and reason. Trial decisions include candidate proposals and measured verdicts.

Set page_size and max_bytes to bound one diagnostic response. The response includes the current accepted transaction, the complete plan state and a history page from one snapshot. Pass a nonzero next_after as the next page's argument.

The adaptive query planner can retry a capped plan with the statistics plan. Query explain reports those attempts in retry and retries.

stardust_cost

This record measures one explain stage. The cpu and wall fields are durations. The ABI gives 100-ns ticks. Thus, a time less than one tick is zero.

Field	C type	Unit	Range	Meaning
`cpu_ticks`	`int64_t`	100 ns ticks	Signed i64	Thread CPU time. The call measures this value. Independent calls can differ.
`wall_ticks`	`int64_t`	100 ns ticks	Signed i64	Wall time. The call measures this value. Independent calls can differ.
`arena_bytes`	`int64_t`	Bytes	Signed i64	Bytes the stage allocated from the scratch arena of the call. Heap-owned outputs and the LuaJIT heap are not included. The call measures this value. Independent calls can differ.

stardust_expr_build_explain

This record contains explain output for build.

Field	C type	Unit	Range	Meaning
`prepare_expression`	`stardust_cost`	None	C type range	—
`compile`	`stardust_cost`	None	C type range	—
`total`	`stardust_cost`	None	C type range	The cost of all stages. `arena_bytes` records the peak scratch-arena allocation.
`arena_capacity`	`int64_t`	Bytes	Signed i64	Bytes the scratch arena reserved. The call measures this value. Independent calls can differ.
`stages_completed`	`uint32_t`	Count	Unsigned u32	Stages that ran.
`cache_hit`	`uint32_t`	None	Unsigned u32	The leased runtime holds a compiled copy of the expression. The call measures this value. Independent calls can differ.

stardust_expr_eval_explain

This record contains explain output for evaluate.

Field	C type	Unit	Range	Default	Meaning
`prepare_variables`	`stardust_cost`	None	C type range	The call sets it	—
`evaluate`	`stardust_cost`	None	C type range	The call sets it	—
`adopt_document`	`stardust_cost`	None	C type range	The call sets it	—
`total`	`stardust_cost`	None	C type range	The call sets it	The cost of all stages. `arena_bytes` records the peak scratch-arena allocation.
`arena_capacity`	`int64_t`	Bytes	Signed i64	The call sets it	Bytes the scratch arena reserved. The call measures this value. Independent calls can differ.
`document_bytes`	`int64_t`	Bytes	Signed i64	The call sets it	Used bytes of the native value, string and source buffers of the result. The call measures this value. Independent calls can differ.
`stages_completed`	`uint32_t`	Count	Unsigned u32	The call sets it	Stages that ran.
`reserved`	`uint32_t`	None	Unsigned u32	0	Must be zero.

stardust_db_query_explain

This record contains explain output for query run. The compile, plan, execute, group and order fields divide the wall time of run into stages.

Field	C type	Unit	Range	Meaning
`load_definition`	`stardust_cost`	None	C type range	—
`parameters`	`stardust_cost`	None	C type range	—
`run`	`stardust_cost`	None	C type range	Planning, execution and aggregation. Pages render later, in next.
`total`	`stardust_cost`	None	C type range	The cost of all stages. `arena_bytes` records the peak scratch-arena allocation.
`arena_capacity`	`int64_t`	Bytes	Signed i64	Bytes the scratch arena reserved. The call measures this value. Independent calls can differ.
`work_ticks`	`int64_t`	100 ns ticks	Signed i64	Thread CPU duration consumed by the query. The call measures this value. Independent calls can differ.
`rows`	`int64_t`	Count	Signed i64	Result rows.
`compile_ticks`	`int64_t`	100 ns ticks	Signed i64	The call measures this value. Independent calls can differ.
`plan_ticks`	`int64_t`	100 ns ticks	Signed i64	The call measures this value. Independent calls can differ.
`execute_ticks`	`int64_t`	100 ns ticks	Signed i64	Production of solutions. A streamed result produces solutions while pages render. Thus, execute measures only the solutions that run reads. The call measures this value. Independent calls can differ.
`group_ticks`	`int64_t`	100 ns ticks	Signed i64	Aggregates, windows and computed columns. The call measures this value. Independent calls can differ.
`order_ticks`	`int64_t`	100 ns ticks	Signed i64	The call measures this value. Independent calls can differ.
`memory_bytes`	`int64_t`	Bytes	Signed i64	The most working memory the run held reserved in the database's memory budget at once. The call measures this value. Independent calls can differ.
`memory_refusals`	`int64_t`	Count	Signed i64	Reservations of the run that the memory budget granted less than asked. The call measures this value. Independent calls can differ.
`budget_bytes`	`int64_t`	Bytes	Signed i64	The database's memory budget (`memory_budget_bytes`).
`budget_used_bytes`	`int64_t`	Bytes	Signed i64	Bytes every query of the database held reserved when run returned. The call measures this value. Independent calls can differ.
`budget_peak_bytes`	`int64_t`	Bytes	Signed i64	The most bytes the database's queries held reserved at once since it opened. The call measures this value. Independent calls can differ.
`budget_refusals`	`int64_t`	Count	Signed i64	Reservations the database's budget granted less than asked since it opened. The call measures this value. Independent calls can differ.
`retry_ticks`	`int64_t`	100 ns ticks	Signed i64	Thread CPU duration of the attempts given up because the planner's plan passed its cap. work includes it. The call measures this value. Independent calls can differ.
`stages_completed`	`uint32_t`	Count	Unsigned u32	Stages that ran.
`retries`	`uint32_t`	Count	Unsigned u32	Attempts given up because the planner's plan passed its cap; the query then ran the statistics' plan within the rest of its budget.

stardust_db_mutation_explain

This record contains explain output for mutation run.

Field	C type	Unit	Range	Meaning
`load_definition`	`stardust_cost`	None	C type range	—
`parameters`	`stardust_cost`	None	C type range	—
`run`	`stardust_cost`	None	C type range	—
`total`	`stardust_cost`	None	C type range	The cost of all stages. `arena_bytes` records the peak scratch-arena allocation.
`arena_capacity`	`int64_t`	Bytes	Signed i64	Bytes the scratch arena reserved. The call measures this value. Independent calls can differ.
`stages_completed`	`uint32_t`	Count	Unsigned u32	Stages that ran.
`attempts`	`uint32_t`	Count	Unsigned u32	Evaluations. More than one when a concurrent write forced a retry.

stardust_db_patch_explain

This record contains explain output for merge_patch.

Field	C type	Unit	Range	Default	Meaning
`copy_patch`	`stardust_cost`	None	C type range	The call sets it	—
`commit`	`stardust_cost`	None	C type range	The call sets it	Validation and the commit.
`total`	`stardust_cost`	None	C type range	The call sets it	The cost of all stages. `arena_bytes` records the peak scratch-arena allocation.
`arena_capacity`	`int64_t`	Bytes	Signed i64	The call sets it	Bytes the scratch arena reserved. The call measures this value. Independent calls can differ.
`stages_completed`	`uint32_t`	Count	Unsigned u32	The call sets it	Stages that ran.
`reserved`	`uint32_t`	None	Unsigned u32	0	Must be zero.

Explain examples

The generator captured each result from the native library.

Inspect one measured cost stage

expression_text holds this document:

json

23 B

[
  "+",
  "?x",
  1
]

446 B

stardust_document *expression = NULL;
stardust_expr_build_explain explain = {0};
stardust_expr_program *program = NULL;
uint32_t status = stardust_parse(expression_text, STARDUST_JSON, &expression);
if (status == STARDUST_OK) status = stardust_expr_build(expression, &explain, &program);
stardust_expr_program_release(program);
stardust_document_release(expression);
if (status != STARDUST_OK) return status;
stardust_cost result = explain.total;

Result / 49 B

{
  "cpu": 13,
  "wall": 14,
  "arena_bytes": 0
}

Explain expression compilation

expression_text holds this document:

json

23 B

[
  "+",
  "?x",
  1
]

454 B

stardust_document *expression = NULL;
stardust_expr_build_explain explain = {0};
stardust_expr_program *program = NULL;
uint32_t status = stardust_parse(expression_text, STARDUST_JSON, &expression);
if (status == STARDUST_OK) status = stardust_expr_build(expression, &explain, &program);
stardust_expr_program_release(program);
stardust_document_release(expression);
if (status != STARDUST_OK) return status;
stardust_expr_build_explain result = explain;

Result / 296 B

{
  "prepare_expression": {
    "cpu": 4,
    "wall": 4,
    "arena_bytes": 0
  },
  "compile": {
    "cpu": 8,
    "wall": 8,
    "arena_bytes": 0
  },
  "total": {
    "cpu": 15,
    "wall": 15,
    "arena_bytes": 0
  },
  "arena_capacity": 65536,
  "stages_completed": 2,
  "cache_hit": true
}

Explain patch validation and commit

seed_text holds this document:

json

333 B

{
  "#_ada": {
    "name": "Ada",
    "role": "engineer",
    "age": 36
  },
  "#_bob": {
    "name": "Bob",
    "role": "engineer",
    "age": 41,
    "mentor": {
      "#link": "#_ada"
    }
  },
  "#_cy": {
    "name": "Cy",
    "role": "designer",
    "age": 29
  },
  "#_tx": {
    "author": "smoke",
    "reason": "seed"
  }
}

426 B

stardust_document *patch = NULL;
stardust_db_patch_explain explain = {0};
stardust_result *written = NULL;
uint32_t status = stardust_parse(seed_text, STARDUST_JSON, &patch);
if (status == STARDUST_OK) status = stardust_db_merge_patch(db, patch, NULL, &explain, &written);
stardust_result_release(written);
stardust_document_release(patch);
if (status != STARDUST_OK) return status;
stardust_db_patch_explain result = explain;

Result / 298 B

{
  "copy_patch": {
    "cpu": 11049,
    "wall": 11091,
    "arena_bytes": 3096
  },
  "commit": {
    "cpu": 315,
    "wall": 31173,
    "arena_bytes": 8008
  },
  "total": {
    "cpu": 11512,
    "wall": 42419,
    "arena_bytes": 11118
  },
  "arena_capacity": 4194304,
  "stages_completed": 2
}

Explain mutation execution

birthday_text holds this document:

json

186 B

{
  "query": {
    "find": [
      "?e"
    ],
    "where": [
      [
        "?e",
        "name",
        "Ada"
      ]
    ]
  },
  "patch": {
    "?e": {
      "age": 37
    }
  }
}

711 B

stardust_document *definition = NULL;
stardust_db_mutation *mutation = NULL;
stardust_result *written = NULL;
stardust_db_mutation_explain explain = {0};
uint32_t status = stardust_parse(birthday_text, STARDUST_JSON, &definition);
stardust_db_source source = {.kind = STARDUST_SOURCE_DOCUMENT, .document = definition};
if (status == STARDUST_OK) status = stardust_db_mutation_open(db, &source, &mutation);
if (status == STARDUST_OK) status = stardust_db_mutation_run(mutation, NULL, NULL, &explain, &written);
stardust_result_release(written);
stardust_db_mutation_release(mutation);
stardust_document_release(definition);
if (status != STARDUST_OK) return status;
stardust_db_mutation_explain result = explain;

Result / 389 B

{
  "load_definition": {
    "cpu": 1202,
    "wall": 1209,
    "arena_bytes": 2320
  },
  "parameters": {
    "cpu": 6,
    "wall": 8,
    "arena_bytes": 0
  },
  "run": {
    "cpu": 7245,
    "wall": 47836,
    "arena_bytes": 43856
  },
  "total": {
    "cpu": 8537,
    "wall": 49139,
    "arena_bytes": 46176
  },
  "arena_capacity": 4194304,
  "stages_completed": 3,
  "attempts": 1
}

Explain expression evaluation

expression_text holds this document:

json

23 B

[
  "+",
  "?x",
  1
]

input_text holds this document:

json

14 B

{
  "x": 41
}

762 B

stardust_document *expression = NULL;
stardust_document *input = NULL;
stardust_expr_program *program = NULL;
stardust_result *output = NULL;
stardust_expr_eval_explain explain = {0};
uint32_t status = stardust_parse(expression_text, STARDUST_JSON, &expression);
if (status == STARDUST_OK) status = stardust_parse(input_text, STARDUST_JSON, &input);
if (status == STARDUST_OK) status = stardust_expr_build(expression, NULL, &program);
if (status == STARDUST_OK) status = stardust_expr_evaluate(program, input, &explain, &output);
stardust_result_release(output);
stardust_expr_program_release(program);
stardust_document_release(input);
stardust_document_release(expression);
if (status != STARDUST_OK) return status;
stardust_expr_eval_explain result = explain;

Result / 378 B

{
  "prepare_variables": {
    "cpu": 6,
    "wall": 7,
    "arena_bytes": 0
  },
  "evaluate": {
    "cpu": 24,
    "wall": 24,
    "arena_bytes": 0
  },
  "adopt_document": {
    "cpu": 2,
    "wall": 2,
    "arena_bytes": 0
  },
  "total": {
    "cpu": 36,
    "wall": 36,
    "arena_bytes": 0
  },
  "arena_capacity": 65536,
  "document_bytes": 42,
  "stages_completed": 3
}

Explain query execution

query_text holds this document:

json

134 B

{
  "find": [
    "?name"
  ],
  "where": [
    [
      "?e",
      "name",
      "?name"
    ]
  ],
  "orderBy": [
    "?name"
  ]
}

671 B

stardust_document *definition = NULL;
stardust_db_query *query = NULL;
stardust_db_rows *rows = NULL;
stardust_db_query_explain explain = {0};
uint32_t status = stardust_parse(query_text, STARDUST_JSON, &definition);
stardust_db_source source = {.kind = STARDUST_SOURCE_DOCUMENT, .document = definition};
if (status == STARDUST_OK) status = stardust_db_query_open(db, &source, &query);
if (status == STARDUST_OK) status = stardust_db_query_run(query, NULL, NULL, &explain, &rows);
stardust_db_rows_release(rows);
stardust_db_query_release(query);
stardust_document_release(definition);
if (status != STARDUST_OK) return status;
stardust_db_query_explain result = explain;

Result / 670 B

{
  "load_definition": {
    "cpu": 1870,
    "wall": 1878,
    "arena_bytes": 1680
  },
  "parameters": {
    "cpu": 8,
    "wall": 10,
    "arena_bytes": 0
  },
  "run": {
    "cpu": 8937,
    "wall": 8968,
    "arena_bytes": 7424
  },
  "total": {
    "cpu": 10842,
    "wall": 10883,
    "arena_bytes": 9104
  },
  "arena_capacity": 4194304,
  "work": 5761,
  "rows": 3,
  "compile": 372,
  "plan": 517,
  "execute": 0,
  "group": 0,
  "order": 0,
  "memory_bytes": 5995,
  "memory_refusals": 0,
  "budget_bytes": 10489856512,
  "budget_used_bytes": 10194,
  "budget_peak_bytes": 14267,
  "budget_refusals": 0,
  "retry": 0,
  "stages_completed": 3,
  "retries": 0
}