History retention#

The history plugin purges old entity history. The stardust/history setting of an entity states how much of its history to keep. Enable it at open with plugins {history {}} to use the default schedule. See plugin selection for the open options.

The history and backups plugins cannot share a database. History retention rewrites the transaction chain that backups must preserve.

Setting#

Write stardust/history on an entity with a merge patch or a mutation, like any other field:

Offset (hex)	Bytes (hex)	Meaning
00	54 52 4F 4E	magic TRON
04	4C 23647572	txt "#dur"
09	4C 50333044	txt "P30D"
0E	0F 0A 04000000 0900 0000	map leaf, size 10: @4 → @9
18	6C 72657461696E	txt "retain"
1F	0F 0A 18000000 0E00 0000	map leaf, size 10: @18 → @E
29	7C 63617363616465	txt "cascade"
31	09	bit true
32	0F 0A 29000000 3100 0000	map leaf, size 10: @29 → @31
3C	07 0E 80010000 1F00 0000 32000000	map branch, size 14, depth 0 (root), slots 7, 8: @1F → @32
4A	14 10 737461726475 73742F686973746F 7279	txt "stardust/history"
5C	0F 0A 4A000000 3C00 0000	map leaf, size 10: @4A → @3C
66	4C 6E616D65	txt "name"
6B	3C 4C6F67	txt "Log"
6F	0F 0A 66000000 6B00 0000	map leaf, size 10: @66 → @6B
79	07 0E 00820000 5C00 0000 6F000000	map branch, size 14, depth 0 (root), slots 9, 15: @5C → @6F
87	5C 235F6C6F67	txt "#_log"
8D	0F 0A 87000000 7900 0000	map leaf, size 10: @87 → @79
97	8D000000 00000000	root → @8D; previous: none

The value must be a map with retain and an optional cascade, and no other member. Other values fail with the rejected status and the code invalid. Write null to remove the setting. Deleting the entity also removes its setting.

Descendants are the entities below it in the path tree, through stardust/path/parent. A descendant with its own setting uses that setting. A descendant without one uses the nearest ancestor setting that has cascade true. An own setting without cascade applies to that entity only: its children still use the cascading setting above it. Data entities that a merge patch creates have no children, so cascade has no effect on them.

Parent cycles#

If stardust/path/parent links form a cycle, each cascade walk visits an entity at most once. The plugin skips repeated entities and does not report the cycle as an error. Purging continues with the resolved retention settings. An entity's own setting still overrides an inherited setting.

Purge timing#

retain selects the history to remove during a later purge. Until that purge finishes, older facts or more than N transactions can remain. The plugin purges after commits settle, in bounded steps.

Synchronization#

Retention runs automatically according to the schedule. This plugin provides no sync operation. A plugin_call of sync on "history" reports invalid_argument with code unknown_operation.

What a purge removes#

A purge removes facts completely. Purged facts cannot be recovered.

• Each field of the entity keeps its state at the purge bound. A read as of an earlier transaction sees that oldest kept state. No read fails.
• History at or after the bound does not change.
• The newest transaction of the database is never purged. A setting never removes the current state.
• Definitions are never purged.
• Purged space is reused for later writes. The database file does not shrink.

Schedule#

Supply schedule settings in the history load settings when opening the database:

Offset (hex)	Bytes (hex)	Meaning
00	54 52 4F 4E	magic TRON
04	4C 23647572	txt "#dur"
09	4C 50543253	txt "PT2S"
0E	0F 0A 04000000 0900 0000	map leaf, size 10: @4 → @9
18	4C 23647572	txt "#dur"
1D	5C 5054333053	txt "PT30S"
23	0F 0A 18000000 1D00 0000	map leaf, size 10: @18 → @1D
2D	4C 23647572	txt "#dur"
32	4C 5054314D	txt "PT1M"
37	0F 0A 2D000000 3200 0000	map leaf, size 10: @2D → @32
41	8C 6D61785F776169 74	txt "max_wait"
4A	0F 0A 41000000 2300 0000	map leaf, size 10: @41 → @23
54	8C 6465626F756E63 65	txt "debounce"
5D	0F 0A 54000000 0E00 0000	map leaf, size 10: @54 → @E
67	7C 7265636865636B	txt "recheck"
6F	0F 0A 67000000 3700 0000	map leaf, size 10: @67 → @37
79	07 0E 01010000 5D00 0000 6F000000	map branch, size 14, depth 1, slots 0, 8: @5D → @6F
87	07 0E 05000000 4A00 0000 79000000	map branch, size 14, depth 0 (root), slots 0, 2: @4A → @79
95	7C 686973746F7279	txt "history"
9D	0F 0A 95000000 8700 0000	map leaf, size 10: @95 → @87
A7	9D000000 00000000	root → @9D; previous: none

These settings are durations. Negative values and unknown setting names fail before the database opens. An empty settings object uses all defaults. Settings apply to this open database and are not stored on entities.

A commit that changes a retention setting, or touches an entity that a setting covers, starts the schedule. The next purge is due at the earlier deadline: debounce after the last such commit, or max_wait after the first. The committer runs it when admission and queue capacity allow. The plugin checks duration settings again after recheck, also without commits. Open starts the same schedule, so purging resumes after a restart.

Live reactors#

The reactors plugin evaluates each live mutation as of the transaction it reacts to. While a live mutation has not processed a transaction, the purge keeps every fact from that transaction on. The rest of the purge waits until the reactor cursor moves past it. The plugin then checks again after recheck.

Without the plugin#

On a database that does not enable history, a write that sets stardust/history fails with the unavailable status and the code plugin_required. Removing a setting is always accepted. A setting stored while the plugin was enabled stays, but nothing is purged until the plugin is enabled again.