Explain and cost

Explain output shows the work that a call completed. The example executes the names query with an explain record and reads its completed stages.

Example

Explain a query

query-text holds this document:

json

134 B

{
  "find": [
    "?name"
  ],
  "where": [
    [
      "?e",
      "name",
      "?name"
    ]
  ],
  "orderBy": [
    "?name"
  ]
}

clojure

187 B

(with-open [definition (sd/parse query-text :json)
            query (sd/query db definition)
            ^java.io.Closeable rows (sd/run query nil {:explain? true})]
  (sd/explain rows))

Result / 669 B

{
  "load_definition": {
    "cpu": 993,
    "wall": 1018,
    "arena_bytes": 1680
  },
  "parameters": {
    "cpu": 8,
    "wall": 10,
    "arena_bytes": 0
  },
  "run": {
    "cpu": 9002,
    "wall": 9084,
    "arena_bytes": 7424
  },
  "total": {
    "cpu": 10035,
    "wall": 10144,
    "arena_bytes": 9104
  },
  "arena_capacity": 4194304,
  "work": 5862,
  "rows": 3,
  "compile": 271,
  "plan": 501,
  "execute": 0,
  "group": 0,
  "order": 0,
  "memory_bytes": 5995,
  "memory_refusals": 0,
  "budget_bytes": 10489856512,
  "budget_used_bytes": 10194,
  "budget_peak_bytes": 14267,
  "budget_refusals": 0,
  "retry": 0,
  "stages_completed": 3,
  "retries": 0
}

Failure output

A failed call can still supply explain output for the stages that completed.

Read stages_completed.
Then read the fields of the stages that completed.

The record tables below give the fields, absent values, and time units of each operation.

A call's cost record describes that execution. It does not load stored plan history. Use the plan operation in Definitions with the explain action to read accepted plans and their history. Each revision reports its version, source and reason. Trial decisions include candidate proposals and measured verdicts.

Set page_size and max_bytes to bound one diagnostic response. The response includes the current accepted transaction, the complete plan state and a history page from one snapshot. Pass a nonzero next_after as the next page's argument.

The adaptive query planner can retry a capped plan with the statistics plan. Query explain reports those attempts in retry and retries.

cost

This record measures one explain stage. The cpu and wall fields are durations. The ABI gives 100-ns ticks. Thus, a time less than one tick is zero.

Field	Type	Unit	Range	Absent	Meaning
`cpu`	`Duration`	Host duration (100 ns ticks at the ABI)	Tick range	nil	Thread CPU time. The call measures this value. Independent calls can differ.
`wall`	`Duration`	Host duration (100 ns ticks at the ABI)	Tick range	—	Wall time. The call measures this value. Independent calls can differ.
`arena_bytes`	`Integer`	Bytes	Host type range	—	Bytes the stage allocated from the scratch arena of the call. Heap-owned outputs and the LuaJIT heap are not included. The call measures this value. Independent calls can differ.

build_explain

This record contains explain output for build.

Field	Type	Unit	Meaning
`prepare_expression`	`Cost`	None	—
`compile`	`Cost`	None	—
`total`	`Cost`	None	The cost of all stages. `arena_bytes` records the peak scratch-arena allocation.
`arena_capacity`	`Integer`	Bytes	Bytes the scratch arena reserved. The call measures this value. Independent calls can differ.
`stages_completed`	`Count`	Count	Stages that ran.
`cache_hit`	`Bool`	None	The leased runtime holds a compiled copy of the expression. The call measures this value. Independent calls can differ.

eval_explain

This record contains explain output for evaluate.

Field	Type	Unit	Meaning
`prepare_variables`	`Cost`	None	—
`evaluate`	`Cost`	None	—
`adopt_document`	`Cost`	None	—
`total`	`Cost`	None	The cost of all stages. `arena_bytes` records the peak scratch-arena allocation.
`arena_capacity`	`Integer`	Bytes	Bytes the scratch arena reserved. The call measures this value. Independent calls can differ.
`document_bytes`	`Integer`	Bytes	Used bytes of the native value, string and source buffers of the result. The call measures this value. Independent calls can differ.
`stages_completed`	`Count`	Count	Stages that ran.

query_explain

This record contains explain output for query run. The compile, plan, execute, group and order fields divide the wall time of run into stages.

Field	Type	Unit	Range	Meaning
`load_definition`	`Cost`	None	Host type range	—
`parameters`	`Cost`	None	Host type range	—
`run`	`Cost`	None	Host type range	Planning, execution and aggregation. Pages render later, in next.
`total`	`Cost`	None	Host type range	The cost of all stages. `arena_bytes` records the peak scratch-arena allocation.
`arena_capacity`	`Integer`	Bytes	Host type range	Bytes the scratch arena reserved. The call measures this value. Independent calls can differ.
`work`	`Duration`	Host duration (100 ns ticks at the ABI)	Tick range	Thread CPU duration consumed by the query. The call measures this value. Independent calls can differ.
`rows`	`Integer`	Count	Host type range	Result rows.
`compile`	`Duration`	Host duration (100 ns ticks at the ABI)	Tick range	The call measures this value. Independent calls can differ.
`plan`	`Duration`	Host duration (100 ns ticks at the ABI)	Tick range	The call measures this value. Independent calls can differ.
`execute`	`Duration`	Host duration (100 ns ticks at the ABI)	Tick range	Production of solutions. A streamed result produces solutions while pages render. Thus, execute measures only the solutions that run reads. The call measures this value. Independent calls can differ.
`group`	`Duration`	Host duration (100 ns ticks at the ABI)	Tick range	Aggregates, windows and computed columns. The call measures this value. Independent calls can differ.
`order`	`Duration`	Host duration (100 ns ticks at the ABI)	Tick range	The call measures this value. Independent calls can differ.
`memory_bytes`	`Integer`	Bytes	Host type range	The most working memory the run held reserved in the database's memory budget at once. The call measures this value. Independent calls can differ.
`memory_refusals`	`Integer`	Count	Host type range	Reservations of the run that the memory budget granted less than asked. The call measures this value. Independent calls can differ.
`budget_bytes`	`Integer`	Bytes	Host type range	The database's memory budget (`memory_budget_bytes`).
`budget_used_bytes`	`Integer`	Bytes	Host type range	Bytes every query of the database held reserved when run returned. The call measures this value. Independent calls can differ.
`budget_peak_bytes`	`Integer`	Bytes	Host type range	The most bytes the database's queries held reserved at once since it opened. The call measures this value. Independent calls can differ.
`budget_refusals`	`Integer`	Count	Host type range	Reservations the database's budget granted less than asked since it opened. The call measures this value. Independent calls can differ.
`retry`	`Duration`	Host duration (100 ns ticks at the ABI)	Tick range	Thread CPU duration of the attempts given up because the planner's plan passed its cap. work includes it. The call measures this value. Independent calls can differ.
`stages_completed`	`Count`	Count	Host type range	Stages that ran.
`retries`	`Count`	Count	Host type range	Attempts given up because the planner's plan passed its cap; the query then ran the statistics' plan within the rest of its budget.

mutation_explain

This record contains explain output for mutation run.

Field	Type	Unit	Meaning
`load_definition`	`Cost`	None	—
`parameters`	`Cost`	None	—
`run`	`Cost`	None	—
`total`	`Cost`	None	The cost of all stages. `arena_bytes` records the peak scratch-arena allocation.
`arena_capacity`	`Integer`	Bytes	Bytes the scratch arena reserved. The call measures this value. Independent calls can differ.
`stages_completed`	`Count`	Count	Stages that ran.
`attempts`	`Count`	Count	Evaluations. More than one when a concurrent write forced a retry.

patch_explain

This record contains explain output for merge_patch.

Field	Type	Unit	Meaning
`copy_patch`	`Cost`	None	—
`commit`	`Cost`	None	Validation and the commit.
`total`	`Cost`	None	The cost of all stages. `arena_bytes` records the peak scratch-arena allocation.
`arena_capacity`	`Integer`	Bytes	Bytes the scratch arena reserved. The call measures this value. Independent calls can differ.
`stages_completed`	`Count`	Count	Stages that ran.

Explain examples

The generator captured each result from the native library.

Inspect one measured cost stage

expression-text holds this document:

json

23 B

[
  "+",
  "?x",
  1
]

clojure

148 B

(with-open [expression (sd/parse expression-text :json)
            program (sd/build expression {:explain? true})]
  (:total (sd/explain program)))

Result / 49 B

{
  "cpu": 52,
  "wall": 60,
  "arena_bytes": 0
}

Explain expression compilation

expression-text holds this document:

json

23 B

[
  "+",
  "?x",
  1
]

clojure

139 B

(with-open [expression (sd/parse expression-text :json)
            program (sd/build expression {:explain? true})]
  (sd/explain program))

Result / 299 B

{
  "prepare_expression": {
    "cpu": 7,
    "wall": 14,
    "arena_bytes": 0
  },
  "compile": {
    "cpu": 38,
    "wall": 38,
    "arena_bytes": 0
  },
  "total": {
    "cpu": 51,
    "wall": 58,
    "arena_bytes": 0
  },
  "arena_capacity": 65536,
  "stages_completed": 2,
  "cache_hit": true
}

Explain patch validation and commit

seed-text holds this document:

json

333 B

{
  "#_ada": {
    "name": "Ada",
    "role": "engineer",
    "age": 36
  },
  "#_bob": {
    "name": "Bob",
    "role": "engineer",
    "age": 41,
    "mentor": {
      "#link": "#_ada"
    }
  },
  "#_cy": {
    "name": "Cy",
    "role": "designer",
    "age": 29
  },
  "#_tx": {
    "author": "smoke",
    "reason": "seed"
  }
}

clojure

104 B

(with-open [patch (sd/parse seed-text :json)]
  (sd/explain (sd/merge-patch db patch {:explain? true})))

Result / 298 B

{
  "copy_patch": {
    "cpu": 12321,
    "wall": 12502,
    "arena_bytes": 3096
  },
  "commit": {
    "cpu": 766,
    "wall": 33672,
    "arena_bytes": 8008
  },
  "total": {
    "cpu": 13252,
    "wall": 46346,
    "arena_bytes": 11118
  },
  "arena_capacity": 4194304,
  "stages_completed": 2
}

Explain mutation execution

birthday-text holds this document:

json

186 B

{
  "query": {
    "find": [
      "?e"
    ],
    "where": [
      [
        "?e",
        "name",
        "Ada"
      ]
    ]
  },
  "patch": {
    "?e": {
      "age": 37
    }
  }
}

clojure

150 B

(with-open [source (sd/parse birthday-text :json)
            mutation (sd/mutation db source)]
  (sd/explain (sd/run mutation nil {:explain? true})))

Result / 389 B

{
  "load_definition": {
    "cpu": 785,
    "wall": 790,
    "arena_bytes": 2320
  },
  "parameters": {
    "cpu": 10,
    "wall": 12,
    "arena_bytes": 0
  },
  "run": {
    "cpu": 9039,
    "wall": 40514,
    "arena_bytes": 43856
  },
  "total": {
    "cpu": 9937,
    "wall": 41433,
    "arena_bytes": 46176
  },
  "arena_capacity": 4194304,
  "stages_completed": 3,
  "attempts": 1
}

Explain expression evaluation

expression-text holds this document:

json

23 B

[
  "+",
  "?x",
  1
]

input-text holds this document:

json

14 B

{
  "x": 41
}

clojure

231 B

(with-open [expression (sd/parse expression-text :json)
            program (sd/build expression)
            input (sd/parse input-text :json)
            output (sd/evaluate program input {:explain? true})]
  (sd/explain output))

Result / 383 B

{
  "prepare_variables": {
    "cpu": 9,
    "wall": 18,
    "arena_bytes": 0
  },
  "evaluate": {
    "cpu": 173,
    "wall": 174,
    "arena_bytes": 0
  },
  "adopt_document": {
    "cpu": 7,
    "wall": 7,
    "arena_bytes": 0
  },
  "total": {
    "cpu": 196,
    "wall": 205,
    "arena_bytes": 0
  },
  "arena_capacity": 65536,
  "document_bytes": 42,
  "stages_completed": 3
}

Explain query execution

query-text holds this document:

json

134 B

{
  "find": [
    "?name"
  ],
  "where": [
    [
      "?e",
      "name",
      "?name"
    ]
  ],
  "orderBy": [
    "?name"
  ]
}

clojure

187 B

(with-open [definition (sd/parse query-text :json)
            query (sd/query db definition)
            ^java.io.Closeable rows (sd/run query nil {:explain? true})]
  (sd/explain rows))

Result / 670 B

{
  "load_definition": {
    "cpu": 836,
    "wall": 892,
    "arena_bytes": 1680
  },
  "parameters": {
    "cpu": 7,
    "wall": 12,
    "arena_bytes": 0
  },
  "run": {
    "cpu": 12141,
    "wall": 12239,
    "arena_bytes": 7424
  },
  "total": {
    "cpu": 13019,
    "wall": 13177,
    "arena_bytes": 9104
  },
  "arena_capacity": 4194304,
  "work": 8593,
  "rows": 3,
  "compile": 413,
  "plan": 697,
  "execute": 0,
  "group": 0,
  "order": 0,
  "memory_bytes": 5995,
  "memory_refusals": 0,
  "budget_bytes": 10489856512,
  "budget_used_bytes": 10194,
  "budget_peak_bytes": 14267,
  "budget_refusals": 0,
  "retry": 0,
  "stages_completed": 3,
  "retries": 0
}