Golem Host functions
Golem provides two WIT packages of host functions for accessing Golem-specific functionality on top of the standard WebAssembly system interfaces.
These two packages are:
golem:api@1.1.0
- provides access to various aspects of Golem for the workersgolem:rpc@0.1.0
- defines the types and functions serving the Worker to Worker communication in Golem
Please check the language specific guidelines to learn the best way to work with these APIs in your language of choice:
This page provides an overview of all the exported functions and types in a language-agnostic way.
Types
An oplog index points to a specific instruction in the worker's history:
/// An index into the persistent log storing all performed operations of a worker
type oplog-index = u64;
The worker id uniquely identifies a worker in the Golem system:
/// Represents a Golem worker
record worker-id {
component-id: component-id,
worker-name: string
}
Components are identified by a component id which is in fact a UUID:
/// Represents a Golem component
record component-id {
uuid: uuid,
}
/// UUID
record uuid {
high-bits: u64,
low-bits: u64
}
Component versions are unsigned integers:
/// Represents a Golem component's version
type component-version = u64;
Promises
Types
A promise is identified by a promise-id
type consisting of the owner worker's identifier and a number that points to
the create promise function in the worker's history:
/// A promise ID is a value that can be passed to an external Golem API to complete that promise
/// from an arbitrary external source, while Golem workers can await for this completion.
record promise-id {
worker-id: worker-id,
oplog-idx: oplog-index,
}
Functions
The following functions define the promise API:
Function name | Definition | Description |
---|---|---|
create-promise | func() -> promise-id | Creates a new promise and returns its ID |
await-promise | func(promise-id) -> list<u8> | Suspends execution until the given promise gets completed, and returns the payload passed to the promise completion. |
complete-promise | func(promise-id, list<u8>) | Completes the given promise with the given payload. Returns true if the promise was completed, false if the promise was already completed. The payload is passed to the worker that is awaiting the promise. |
delete-promise | func(promise-id) -> () | Deletes the given promise. |
Worker metadata
Types
Worker metadata is described by the following WIT types:
record worker-metadata {
worker-id: worker-id,
args: list<string>,
env: list<tuple<string, string>>,
status: worker-status,
component-version: u64,
retry-count: u64
}
enum worker-status {
/// The worker is running an invoked function
running,
/// The worker is ready to run an invoked function
idle,
/// An invocation is active but waiting for something (sleeping, waiting for a promise)
suspended,
/// The last invocation was interrupted but will be resumed
interrupted,
/// The last invocation failed and a retry was scheduled
retrying,
/// The last invocation failed and the worker can no longer be used
failed,
/// The worker exited after a successful invocation and can no longer be invoked
exited,
}
Functions
The following functions define the worker metadata API:
Function name | Definition | Description |
---|---|---|
get-worker-metadata | func(worker-id) -> option<worker-metadata> | Returns the metadata of a worker |
get-self-metadata | func() -> worker-metadata | Returns the metadata of the worker that calls this function |
Worker enumeration
The worker enumeration API allows listing all the workers belonging to a specific component. This is a slow operation and should be used for maintenance tasks and not for the core business logic.
Worker enumeration is a WIT resource, providing a method that returns a page of workers each time it is called:
resource get-workers {
constructor(component-id: component-id, filter: option<worker-any-filter>, precise: bool);
get-next: func() -> option<list<worker-metadata>>;
}
Once get-next
returns none
, the enumeration is done.
There are two parameters for customizing the worker enumeration:
Filters
An optional filter
parameter can be passed to the worker enumeration, with the following definition:
record worker-any-filter {
filters: list<worker-all-filter>
}
The worker-any-filter
matches workers that satisfy any of the listed filters.
record worker-all-filter {
filters: list<worker-property-filter>
}
A worker-all-filter
matches workers only that satisfy all of the listed filters.
variant worker-property-filter {
name(worker-name-filter),
status(worker-status-filter),
version(worker-version-filter),
created-at(worker-created-at-filter),
env(worker-env-filter)
}
The worker-name-filter
matches workers by one of their properties, such as the worker name, status, version, creation date or environment variables.
Each of these variants take a filter record holding a value and a comparator:
record worker-name-filter {
comparator: string-filter-comparator,
value: string
}
record worker-status-filter {
comparator: filter-comparator,
value: worker-status
}
record worker-version-filter {
comparator: filter-comparator,
value: u64
}
record worker-created-at-filter {
comparator: filter-comparator,
value: u64
}
record worker-env-filter {
name: string,
comparator: string-filter-comparator,
value: string
}
Where filter-comparator
and string-filter-comparator
are defined as:
enum filter-comparator {
equal,
not-equal,
greater-equal,
greater,
less-equal,
less
}
enum string-filter-comparator {
equal,
not-equal,
like,
not-like
}
Precise enumeration
The precise
flag switches the worker enumeration into a more costly variant which guarantees that every worker metadata returned is the latest one available at the time the result was generated.
When precise
is set to false
, the enumeration returns the last cached worker metadata available for each worker, which may be lagging behind the actual state of the workers.
Transactions and persistence control
Golem's transaction API allows customizing the execution engine's durability and transactional behaviors. These are low level functions, which are wrapped by SDKs providing a higher level transaction API for each supported language.
Types
Retry policy is defined by the following record:
/// Configures how the executor retries failures
record retry-policy {
/// The maximum number of retries before the worker becomes permanently failed
max-attempts: u32,
/// The minimum delay between retries (applied to the first retry)
min-delay: duration,
/// The maximum delay between retries
max-delay: duration,
/// Multiplier applied to the delay on each retry to implement exponential backoff
multiplier: f64,
/// The maximum amount of jitter to add to the delay
max-jitter-factor: option<f64>
}
It is possible to switch between persistence modes:
/// Configurable persistence level for workers
variant persistence-level {
persist-nothing,
persist-remote-side-effects,
smart
}
Functions
The following functions define the transaction API:
Function name | Definition | Description |
---|---|---|
get-oplog-index | func() -> oplog-index | Returns the current position in the persistent op log |
set-oplog-index | func(oplog-idx: oplog-index) -> () | Makes the current worker travel back in time and continue execution from the given position in the persistent op log. |
oplog-commit | func(replicas: u8) -> () | Blocks the execution until the oplog has been written to at least the specified number of replicas, or the maximum number of replicas if the requested number is higher. |
mark-begin-operation | func() -> oplog-index | Marks the beginning of an atomic operation. In case of a failure within the region selected by mark-begin-operation and mark-end-operation , the whole region will be reexecuted on retry. The end of the region is when mark-end-operation is called with the returned oplog-index. |
mark-end-operation | func(begin: oplog-index) -> () | Commits this atomic operation. After mark-end-operation is called for a given index, further calls with the same parameter will do nothing. |
get-retry-policy | func() -> retry-policy | Gets the current retry policy associated with the worker |
set-retry-policy | func(policy: retry-policy) -> () | Overrides the current retry policy associated with the worker. Following this call, get-retry-policy will return the new retry policy. |
get-oplog-persistence-level | func() -> persistence-level | Gets the worker's current persistence level. |
set-oplog-persistence-level | func(new-persistence-level: persistence-level) -> () | Sets the worker's current persistence level. This can increase the performance of execution in cases where durable execution is not required. |
get-idempotence-mode | func() -> bool | Gets the worker's current idempotence mode. |
set-idempotence-mode | func(idempotent: bool) -> () | Sets the current idempotence mode. The default is true. True means side-effects are treated idempotent and Golem guarantees at-least-once semantics. In case of false the executor provides at-most-once semantics, failing the worker in case it is not known if the side effect was already executed. |
generate-idempotency-key | func() -> uuid | Generates a new idempotency key. |
Update
The update API enables workers to trigger automatic or manual update of other workers.
Types
Automatic and manual updates are distinguished by the following enum:
/// Describes how to update a worker to a different component version
enum update-mode {
/// Automatic update tries to recover the worker using the new component version
/// and may fail if there is a divergence.
automatic,
/// Manual, snapshot-based update uses a user-defined implementation of the `save-snapshot` interface
/// to store the worker's state, and a user-defined implementation of the `load-snapshot` interface to
/// load it into the new version.
snapshot-based
}
Functions
The following function defines the update API:
Function name | Definition | Description |
---|---|---|
update-worker | func(worker-id: worker-id, target-version: component-version, mode: update-mode) -> () | Initiates an update attempt for the given worker. The function returns immediately once the request has been processed, not waiting for the worker to get updated. |
Oplog search and query
The oplog
interface in golem:api
provides functions to search and query the worker's persisted oplog.
The interface defines a big variant
data type called oplog-entry
:
variant oplog-entry {
/// The initial worker oplog entry
create(create-parameters),
/// The worker invoked a host function
imported-function-invoked(imported-function-invoked-parameters),
/// The worker has been invoked
exported-function-invoked(exported-function-invoked-parameters),
/// The worker has completed an invocation
exported-function-completed(exported-function-completed-parameters),
/// Worker suspended
suspend(datetime),
/// Worker failed
error(error-parameters),
/// Marker entry added when get-oplog-index is called from the worker, to make the jumping behavior
/// more predictable.
no-op(datetime),
/// The worker needs to recover up to the given target oplog index and continue running from
/// the source oplog index from there
/// `jump` is an oplog region representing that from the end of that region we want to go back to the start and
/// ignore all recorded operations in between.
jump(jump-parameters),
/// Indicates that the worker has been interrupted at this point.
/// Only used to recompute the worker's (cached) status, has no effect on execution.
interrupted(datetime),
/// Indicates that the worker has been exited using WASI's exit function.
exited(datetime),
/// Overrides the worker's retry policy
change-retry-policy(change-retry-policy-parameters),
/// Begins an atomic region. All oplog entries after `BeginAtomicRegion` are to be ignored during
/// recovery except if there is a corresponding `EndAtomicRegion` entry.
begin-atomic-region(datetime),
/// Ends an atomic region. All oplog entries between the corresponding `BeginAtomicRegion` and this
/// entry are to be considered during recovery, and the begin/end markers can be removed during oplog
/// compaction.
end-atomic-region(end-atomic-region-parameters),
/// Begins a remote write operation. Only used when idempotence mode is off. In this case each
/// remote write must be surrounded by a `BeginRemoteWrite` and `EndRemoteWrite` log pair and
/// unfinished remote writes cannot be recovered.
begin-remote-write(datetime),
/// Marks the end of a remote write operation. Only used when idempotence mode is off.
end-remote-write(end-remote-write-parameters),
/// An invocation request arrived while the worker was busy
pending-worker-invocation(pending-worker-invocation-parameters),
/// An update request arrived and will be applied as soon the worker restarts
pending-update(pending-update-parameters),
/// An update was successfully applied
successful-update(successful-update-parameters),
/// An update failed to be applied
failed-update(failed-update-parameters),
/// Increased total linear memory size
grow-memory(grow-memory-parameters),
/// Created a resource instance
create-resource(create-resource-parameters),
/// Dropped a resource instance
drop-resource(drop-resource-parameters),
/// Adds additional information for a created resource instance
describe-resource(describe-resource-parameters),
/// The worker emitted a log message
log(log-parameters),
/// The worker's has been restarted, forgetting all its history
restart(datetime),
/// Activates a plugin
activate-plugin(activate-plugin-parameters),
/// Deactivates a plugin
deactivate-plugin(deactivate-plugin-parameters)
}
and two resources for querying a worker's oplog.
- the
get-oplog
resource enumerates through all entries of the oplog - the
search-oplog
resource accepts a search expression and only returns the matching entries
Both resources, once constructed, provide a get-next
function that returns a chunk of oplog entries. Repeatedly calling this function goes through the whole data set, and eventually returns none
.