Installation#

Plugins add behavior to the embedded library. Install the core library before selecting plugins.

Included plugins#

The library's plugins operation lists the plugins it can use and their requirements. The coming soon list describes prototype features that are not available in this library.

Release packages ship the adaptive planner, Reactors, SDK generation, the object store and backups as plugin libraries. History retention is compiled into the core library. The static archive in the same package contains all these plugins. In all three forms, a database enables plugins when it opens, as described in Enable plugins.

Plugin libraries#

A release package keeps each platform's shared core library and plugin libraries together in that platform's directory, lib/<os>_<arch>/:

Keep these files together, in the directory of the core library, and use them only with the release they came with. The core library looks for plugin libraries only in its own directory, the first time a call needs the plugin list. Opening a database, listing plugins and checking a license can trigger this scan. The bindings need no separate discovery setting.

Verification#

The release build records each finished plugin library's SHA-256 in the core library. The core library accepts only the plugin libraries it was built with:

1. It carries a SHA-256 for the plugin named by the file.
2. The file's SHA-256 equals that value.
3. The plugin library is from the same build and exports the plugin entry under its own name.

The core library reads the file once, checks its SHA-256, and loads the bytes that it checked. On Linux it loads the file through its open descriptor. On Windows it keeps the file open without write sharing until it loads. On macOS it keeps the file open and relies on the library's code signature.

Check the release download with minisign, as described in Verify the download. Each binding checks the core library's SHA-256. The core then checks its plugin libraries. See Load the core for the binding check. These checks detect a mismatched, corrupted, or replaced plugin library.

Available and unavailable plugins#

A plugin library is available only after it passes every check. Missing libraries and failed checks make the plugin unavailable. Selecting an unavailable plugin causes database open to fail.

The plugins operation lists each available plugin with its name, description, version, requirements, willNotWorkWith, definition types, hooks, index providers, and database hooks. After them, it lists each unavailable plugin library as an object with name, file, and unavailable, which states the reason:

Development builds#

Contributors who build the library from source can compile a development core library that loads plugin libraries without checking their SHA-256. This switch exists only at compile time, for development. Release packaging refuses a library built with it. Release libraries always check their plugin libraries.

Plugins compiled into the core#

A plugin compiled into the core library is part of that library file. It needs no plugin library file or SHA-256 check, and it is always available to databases. The plugins that a core library contains are chosen when it is built, so changing them requires rebuilding the core library.

Plugins in a static link#

Go, Rust, Odin and C programs can link the static archive (libstardust_full.a, or stardust_full.lib on Windows) instead of the shared core library. Linking gives the commands. The archive contains the core and every plugin of its release, compiled in. The plugin set is fixed when you link your program. The backups plugin in the archive carries its own Zstandard copy under stardust_-prefixed symbol names, so your program can also link its own libzstd.

• Ship only your executable. A static program uses no core library, no plugin libraries and no SHA-256 checks.
• A static program does not look for plugin libraries. Plugin library files beside the executable have no effect.
• To use the plugins of another release, link your program again with that release's static archive.
• Each database still enables plugins when it opens, as described in Enable plugins. The license plugins permission applies as it does for the shared library.

Check the download with minisign before you link the archive, as described in Verify the download. The bindings do not check the SHA-256 of a statically linked core: it is part of your executable.

Enable plugins#

Use the plugins database option when opening a database. It works the same way for plugin libraries, plugins compiled into the core, and static links. Selection applies until the database closes. Only named plugins are enabled.

Each object key is a plugin name. Its value is a settings object. An empty settings object uses the plugin's defaults. Select only available plugins. Include each selected plugin's required dependencies in the object. The willNotWorkWith list names plugins that cannot be enabled together. If either plugin lists the other, opening the database fails with invalid_argument before configuration or storage changes. A plugin in this list can be absent from the library or disabled. Unknown settings and invalid values fail before the database opens. The adaptive planner accepts memoryShare. The history plugin accepts scheduling settings, backups accepts its targets, and SDK generation accepts language output settings. The reactors and objects plugins accept only empty settings. A disabled plugin does not install its database hooks, definition types, or index providers. A write that sets stardust/history reports unavailable when the database does not enable history. A write that sets stardust/object/digest reports unavailable when the database does not enable objects.

Plugin operations#

A plugin serves named operations through two calls:

Bindings provide typed wrappers for the supported operations. Each wrapper builds a request and calls the same underlying operation. A disabled plugin reports unavailable. An unknown operation reports invalid_argument with code unknown_operation. Plugins name their runtime description status and their wait for committed work sync, when they offer them.

Plugin boundary#

All plugins are first-party plugins from the Stardust project. The plugin entry that a plugin library exports is internal to Stardust: it is not part of the C ABI, and the bindings do not call it. There is no interface for third-party plugins.

Operations#

Database.PluginCall#

func (d *Database) PluginCall(plugin, operation string, request *Document) (*Document, error)

Parameters#

Errors#

Notes#

The plugin decodes the request and builds the result. A result is null when the operation returns nothing. Returned documents omit empty object and array members recursively; empty roots, array elements and scalar values remain. Each plugin documents its operations: object store, reactors and SDK generation.

Examples#

Wait for live mutations, then describe them#

options, err := stardust.Parse([]byte(`{maxWait {#dur PT5S}}`), stardust.DUST)
if err != nil {
	return err
}
defer options.Close()
// sync returns a null document; db.Reactors().Sync wraps the same call.
synced, err := db.PluginCall("reactors", "sync", options)
if err != nil {
	return err
}
synced.Close()

// A nil request reads as an empty object.
result, err := db.PluginCall("reactors", "status", nil)
if err != nil {
	return err
}
defer result.Close()

C exports: stardust_db_plugin_call, stardust_result_take_document, stardust_result_error, stardust_result_release

Database.PluginStream#

func (d *Database) PluginStream(plugin, operation string, request *Document) (*PluginStream, error)

Parameters#

Errors#

Notes#

A stream supports the steps its operation documents. Each plugin documents its stream operations: object store.

Examples#

Upload a payload and publish it#

upload, err := db.PluginStream("objects", "upload", nil)
if err != nil {
	return err
}
// Close ends the stream; the upload is abandoned unless it finished.
defer upload.Close()

if _, err := upload.Write([]byte("Hello, ")); err != nil {
	return err
}
if _, err := upload.Write([]byte("objects")); err != nil {
	return err
}

result, err := upload.Finish()
if err != nil {
	return err
}
defer result.Close()

C exports: stardust_db_plugin_stream_open, stardust_result_error, stardust_result_release

PluginStream.Write#

func (s *PluginStream) Write(data []byte) (int, error)

Requires exclusive use of the receiver handle.

Parameters#

Errors#

Notes#

A failure also reports the count accepted before it, returned together with the error. Those bytes stay part of the stream; continue with the remaining bytes.

Examples#

Append bytes to an upload#

upload, err := db.Objects().Upload()
if err != nil {
	return err
}
defer upload.Close()

written, err := upload.Write([]byte("Hello, objects"))
if err != nil {
	return err
}
result := int64(written)

C exports: stardust_db_plugin_stream_write

PluginStream.Read#

func (s *PluginStream) Read(p []byte) (int, error)

Requires exclusive use of the receiver handle.

Parameters#

Errors#

Notes#

A read is shorter than length only at the end, and empty at or past it.

Examples#

Read the start of a payload#

upload, err := db.Objects().Upload()
if err != nil {
	return err
}
defer upload.Close()
if _, err := upload.Write([]byte("Hello, objects")); err != nil {
	return err
}
published, err := upload.Finish()
if err != nil {
	return err
}
defer published.Close()

reader, err := db.Objects().Open("95e84da38cbefdf3f9f6ef02201074c73124f7cf2025d3e1d69e591a410856fb")
if err != nil {
	return err
}
defer reader.Close()
// Read implements io.Reader: it reports io.EOF at the end.
data := make([]byte, 5)
read, err := reader.Read(data)
if err != nil {
	return err
}
// data holds "Hello".
result := int64(read)

C exports: stardust_db_plugin_stream_read

PluginStream.Seek#

func (s *PluginStream) Seek(offset int64, whence int) (int64, error)

Requires exclusive use of the receiver handle.

Parameters#

Errors#

Notes#

whence 0 counts offset from the start, 1 from the current position and 2 from the end. A refused seek keeps the position.

Examples#

Seek from the end of a payload#

upload, err := db.Objects().Upload()
if err != nil {
	return err
}
defer upload.Close()
if _, err := upload.Write([]byte("Hello, objects")); err != nil {
	return err
}
published, err := upload.Finish()
if err != nil {
	return err
}
defer published.Close()

reader, err := db.Objects().Open("95e84da38cbefdf3f9f6ef02201074c73124f7cf2025d3e1d69e591a410856fb")
if err != nil {
	return err
}
defer reader.Close()
// io.SeekEnd counts from the end, so -7 is the start of "objects".
result, err := reader.Seek(-7, io.SeekEnd)
if err != nil {
	return err
}

C exports: stardust_db_plugin_stream_seek

PluginStream.Finish#

func (s *PluginStream) Finish() (*Document, error)

Requires exclusive use of the receiver handle.

Errors#

Notes#

A failed finish keeps the stream, so finish can be retried. Close the stream separately.

Examples#

Publish the same payload twice#

// The second finish reports deduplicated true and stores nothing new.
var result *stardust.Document
for range 2 {
	upload, err := db.Objects().Upload()
	if err != nil {
		return err
	}
	if _, err := upload.Write([]byte("Hello, objects")); err != nil {
		upload.Close()
		return err
	}
	if result != nil {
		result.Close()
	}
	result, err = upload.Finish()
	upload.Close()
	if err != nil {
		return err
	}
}
defer result.Close()

C exports: stardust_db_plugin_stream_finish, stardust_result_take_document, stardust_result_error, stardust_result_release

PluginStream.Close#

func (s *PluginStream) Close()

Consumes the receiver handle.

Notes#

Close waits only for a call already in progress on the stream; start no other call once close begins. It is valid after the database closes. A plugin documents what close abandons; the object store reclaims an upload stage at the next open when close could not abandon it.

Examples#

Abandon an unfinished upload#

upload, err := db.Objects().Upload()
if err != nil {
	return err
}
if _, err := upload.Write([]byte("draft")); err != nil {
	upload.Close()
	return err
}
// Closing before finish publishes nothing.
upload.Close()

C exports: stardust_db_plugin_stream_release