Installation#

Plugins add behavior to the embedded library. Install the core library before selecting plugins.

Included plugins#

The library's plugins operation lists the plugins it can use and their requirements. The coming soon list describes prototype features that are not available in this library.

Release packages ship the adaptive planner, Reactors, SDK generation, the object store and backups as plugin libraries. History retention is compiled into the core library. The static archive in the same package contains all these plugins. In all three forms, a database enables plugins when it opens, as described in Enable plugins.

Plugin libraries#

A release package keeps each platform's shared core library and plugin libraries together in that platform's directory, lib/<os>_<arch>/:

Keep these files together, in the directory of the core library, and use them only with the release they came with. The core library looks for plugin libraries only in its own directory, the first time a call needs the plugin list. Opening a database, listing plugins and checking a license can trigger this scan. The bindings need no separate discovery setting.

Verification#

The release build records each finished plugin library's SHA-256 in the core library. The core library accepts only the plugin libraries it was built with:

1. It carries a SHA-256 for the plugin named by the file.
2. The file's SHA-256 equals that value.
3. The plugin library is from the same build and exports the plugin entry under its own name.

The core library reads the file once, checks its SHA-256, and loads the bytes that it checked. On Linux it loads the file through its open descriptor. On Windows it keeps the file open without write sharing until it loads. On macOS it keeps the file open and relies on the library's code signature.

Check the release download with minisign, as described in Verify the download. Each binding checks the core library's SHA-256. The core then checks its plugin libraries. See Load the core for the binding check. These checks detect a mismatched, corrupted, or replaced plugin library.

Available and unavailable plugins#

A plugin library is available only after it passes every check. Missing libraries and failed checks make the plugin unavailable. Selecting an unavailable plugin causes database open to fail.

The plugins operation lists each available plugin with its name, description, version, requirements, willNotWorkWith, definition types, hooks, index providers, and database hooks. After them, it lists each unavailable plugin library as an object with name, file, and unavailable, which states the reason:

Development builds#

Contributors who build the library from source can compile a development core library that loads plugin libraries without checking their SHA-256. This switch exists only at compile time, for development. Release packaging refuses a library built with it. Release libraries always check their plugin libraries.

Plugins compiled into the core#

A plugin compiled into the core library is part of that library file. It needs no plugin library file or SHA-256 check, and it is always available to databases. The plugins that a core library contains are chosen when it is built, so changing them requires rebuilding the core library.

Plugins in a static link#

Go, Rust, Odin and C programs can link the static archive (libstardust_full.a, or stardust_full.lib on Windows) instead of the shared core library. Linking gives the commands. The archive contains the core and every plugin of its release, compiled in. The plugin set is fixed when you link your program. The backups plugin in the archive carries its own Zstandard copy under stardust_-prefixed symbol names, so your program can also link its own libzstd.

• Ship only your executable. A static program uses no core library, no plugin libraries and no SHA-256 checks.
• A static program does not look for plugin libraries. Plugin library files beside the executable have no effect.
• To use the plugins of another release, link your program again with that release's static archive.
• Each database still enables plugins when it opens, as described in Enable plugins. The license plugins permission applies as it does for the shared library.

Check the download with minisign before you link the archive, as described in Verify the download. The bindings do not check the SHA-256 of a statically linked core: it is part of your executable.

Enable plugins#

Use the plugins database option when opening a database. It works the same way for plugin libraries, plugins compiled into the core, and static links. Selection applies until the database closes. Only named plugins are enabled.

Each object key is a plugin name. Its value is a settings object. An empty settings object uses the plugin's defaults. Select only available plugins. Include each selected plugin's required dependencies in the object. The willNotWorkWith list names plugins that cannot be enabled together. If either plugin lists the other, opening the database fails with invalid_argument before configuration or storage changes. A plugin in this list can be absent from the library or disabled. Unknown settings and invalid values fail before the database opens. The adaptive planner accepts memoryShare. The history plugin accepts scheduling settings, backups accepts its targets, and SDK generation accepts language output settings. The reactors and objects plugins accept only empty settings. A disabled plugin does not install its database hooks, definition types, or index providers. A write that sets stardust/history reports unavailable when the database does not enable history. A write that sets stardust/object/digest reports unavailable when the database does not enable objects.

Plugin operations#

A plugin serves named operations through two calls:

Bindings provide typed wrappers for the supported operations. Each wrapper builds a request and calls the same underlying operation. A disabled plugin reports unavailable. An unknown operation reports invalid_argument with code unknown_operation. Plugins name their runtime description status and their wait for committed work sync, when they offer them.

Plugin boundary#

All plugins are first-party plugins from the Stardust project. The plugin entry that a plugin library exports is internal to Stardust: it is not part of the C ABI, and the bindings do not call it. There is no interface for third-party plugins.

Operations#

Database.plugin_call#

def plugin_call(self, plugin: str, operation: str, request: Document | None = None) -> Document

Parameters#

Errors#

Notes#

The plugin decodes the request and builds the result. A result is null when the operation returns nothing. Returned documents omit empty object and array members recursively; empty roots, array elements and scalar values remain. Each plugin documents its operations: object store, reactors and SDK generation.

Examples#

Wait for live mutations, then describe them#

async with await stardust.parse("{maxWait {#dur PT5S}}", "dust") as options:
    # sync returns a null document; db.reactors().sync wraps the same call.
    await (await db.plugin_call("reactors", "sync", options)).close()
# No request reads as an empty object.
result = await db.plugin_call("reactors", "status")

{:value {:enabled true
         :cursor  4
         :head    4
         :pending false}}

C exports: stardust_db_plugin_call, stardust_result_take_document, stardust_result_error, stardust_result_release

Database.plugin_stream#

def plugin_stream(self, plugin: str, operation: str, request: Document | None = None) -> PluginStream

Parameters#

Errors#

Notes#

A stream supports the steps its operation documents. Each plugin documents its stream operations: object store.

Examples#

Upload a payload and publish it#

# Close abandons the upload unless it finished.
async with await db.plugin_stream("objects", "upload") as upload:
    await upload.write(b"Hello, ")
    await upload.write(b"objects")
    result = await upload.finish()

{:value {:digest       {:#sha256 "95e84da38cbefdf3f9f6ef02201074c73124f7cf2025d3e1d69e591a410856fb"}
         :size         14
         :deduplicated false}}

C exports: stardust_db_plugin_stream_open, stardust_result_error, stardust_result_release

PluginStream.write#

def write(self, data: bytes) -> int

Requires exclusive use of the receiver handle.

Parameters#

Errors#

Notes#

A failure also reports the count accepted before it, returned together with the error. Those bytes stay part of the stream; continue with the remaining bytes.

Examples#

Append bytes to an upload#

async with await db.objects().upload() as upload:
    result = await upload.write(b"Hello, objects")

{:value 14}

C exports: stardust_db_plugin_stream_write

PluginStream.read#

def read(self, size: int = -1) -> bytes

Requires exclusive use of the receiver handle.

Parameters#

Errors#

Notes#

A read is shorter than length only at the end, and empty at or past it.

Examples#

Read the start of a payload#

async with await db.objects().upload() as upload:
    await upload.write(b"Hello, objects")
    await (await upload.finish()).close()
async with await db.objects().open("95e84da38cbefdf3f9f6ef02201074c73124f7cf2025d3e1d69e591a410856fb") as reader:
    # read() without a size reads to the end.
    data = await reader.read(5)
# data holds b"Hello".
result = len(data)

{:value 5}

C exports: stardust_db_plugin_stream_read

PluginStream.seek#

def seek(self, offset: int, whence: int = 0) -> int

Requires exclusive use of the receiver handle.

Parameters#

Errors#

Notes#

whence 0 counts offset from the start, 1 from the current position and 2 from the end. A refused seek keeps the position.

Examples#

Seek from the end of a payload#

async with await db.objects().upload() as upload:
    await upload.write(b"Hello, objects")
    await (await upload.finish()).close()
async with await db.objects().open("95e84da38cbefdf3f9f6ef02201074c73124f7cf2025d3e1d69e591a410856fb") as reader:
    # whence 2 counts from the end, so -7 is the start of "objects".
    result = await reader.seek(-7, 2)

{:value 7}

C exports: stardust_db_plugin_stream_seek

PluginStream.finish#

def finish(self) -> Document

Requires exclusive use of the receiver handle.

Parameters#

Errors#

Notes#

A failed finish keeps the stream, so finish can be retried. Close the stream separately.

Examples#

Publish the same payload twice#

# The second finish reports deduplicated true and stores nothing new.
result = None
for _ in range(2):
    async with await db.objects().upload() as upload:
        await upload.write(b"Hello, objects")
        if result is not None:
            await result.close()
        result = await upload.finish()

{:value {:digest       {:#sha256 "95e84da38cbefdf3f9f6ef02201074c73124f7cf2025d3e1d69e591a410856fb"}
         :size         14
         :deduplicated true}}

C exports: stardust_db_plugin_stream_finish, stardust_result_take_document, stardust_result_error, stardust_result_release

PluginStream.close#

def close(self) -> None

Consumes the receiver handle.

Parameters#

Notes#

Close waits only for a call already in progress on the stream; start no other call once close begins. It is valid after the database closes. A plugin documents what close abandons; the object store reclaims an upload stage at the next open when close could not abandon it.

Examples#

Abandon an unfinished upload#

upload = await db.objects().upload()
await upload.write(b"draft")
# Closing before finish publishes nothing.
await upload.close()

C exports: stardust_db_plugin_stream_release