History retention#

A cache of processing status is useful because it tells an application what is happening now. Recent changes may help explain a delay, while last month's intermediate states add little to today's work. The value of that information falls quickly as newer information arrives. Keeping every intermediate state adds information the cache no longer needs.

Stardust normally serves as the source of truth, where earlier facts and definitions help explain decisions. An instance is also cheap enough to use for a temporary cache with a different purpose. The authoritative record can remain elsewhere while this instance supports work that needs a recent view. That role calls for a different decision about history.

A sliding window#

History retention lets the cache keep current state and a recent window of changes. For example, an application may need the current processing status and the changes from the last few hours. As time passes, older changes lose their value to that task. A sliding window lets the useful period move forward with the work.

The window should follow the information's useful lifetime. A short-lived processing cache and a long-running investigation need different evidence, even if both use Stardust. Choosing a temporary role for one instance avoids treating every intermediate update as a permanent business record. The application can use the same data concepts while giving that instance a narrower responsibility.

Authority and auditing#

This is a different goal from preserving an authoritative history for auditing and recovery. A backup protects the ability to recover earlier knowledge, while a temporary cache deliberately limits how much of that knowledge remains useful locally. The history retention and backups plugins therefore serve different roles and cannot be enabled together for one database. The application should decide which role an instance has before relying on its history.