Welcome to the new Golem Cloud Docs! 👋
Golem Host Functions

Golem Host functions

Golem provides three WIT packages of host functions for accessing Golem-specific functionality on top of the standard WebAssembly system interfaces.

These three packages are:

  • golem:api@1.1.6 - provides access to various aspects of Golem for the workers
  • golem:rpc@0.2.0 - defines the types and functions serving the Worker to Worker communication in Golem
  • golem:durability@1.2.0 - provides an API to implement custom durability in libraries

Please check the language specific guidelines to learn the best way to work with these APIs in your language of choice:

This page provides an overview of all the exported functions and types in a language-agnostic way.

Types

An oplog index points to a specific instruction in the worker's history:

package golem:api@1.1.6;
 
interface host {
    /// An index into the persistent log storing all performed operations of a worker
    type oplog-index = u64;
}

The worker id uniquely identifies a worker in the Golem system:

package golem:rpc@0.2.0;
 
interface types {
    /// Represents a Golem worker
    record worker-id {
        component-id: component-id,
        worker-name: string
    }
}

Components are identified by a component id which is in fact a UUID:

package golem:rpc@0.2.0;
 
interface types {
    /// Represents a Golem component
    record component-id {
        uuid: uuid,
    }
 
    /// UUID
    record uuid {
    high-bits: u64,
    low-bits: u64
    }
}

Component versions are unsigned integers:

package golem:api@1.1.6;
 
interface host {
    /// Represents a Golem component's version
    type component-version = u64;
}

Promises

Types

A promise is identified by a promise-id type consisting of the owner worker's identifier and a number that points to the create promise function in the worker's history:

package golem:api@1.1.6;
 
interface host {
    /// A promise ID is a value that can be passed to an external Golem API to complete that promise
    /// from an arbitrary external source, while Golem workers can await for this completion.
    record promise-id {
        worker-id: worker-id,
        oplog-idx: oplog-index,
    }
}

Functions

The following functions define the promise API:

Function nameDefinitionDescription
create-promisefunc() -> promise-idCreates a new promise and returns its ID
await-promisefunc(promise-id) -> list<u8>Suspends execution until the given promise gets completed, and returns the payload passed to the promise completion.
complete-promisefunc(promise-id, list<u8>)Completes the given promise with the given payload. Returns true if the promise was completed, false if the promise was already completed. The payload is passed to the worker that is awaiting the promise.
delete-promisefunc(promise-id) -> ()Deletes the given promise.

Worker metadata

Types

Worker metadata is described by the following WIT types:

package golem:api@1.1.6;
 
interface host {
    record worker-metadata {
        worker-id: worker-id,
        args: list<string>,
        env: list<tuple<string, string>>,
        status: worker-status,
        component-version: u64,
        retry-count: u64
    }
 
    enum worker-status {
        /// The worker is running an invoked function
        running,
        /// The worker is ready to run an invoked function
        idle,
        /// An invocation is active but waiting for something (sleeping, waiting for a promise)
        suspended,
        /// The last invocation was interrupted but will be resumed
        interrupted,
        /// The last invocation failed and a retry was scheduled
        retrying,
        /// The last invocation failed and the worker can no longer be used
        failed,
        /// The worker exited after a successful invocation and can no longer be invoked
        exited,
    }
}

Functions

The following functions define the worker metadata API:

Function nameDefinitionDescription
get-worker-metadatafunc(worker-id) -> option<worker-metadata>Returns the metadata of a worker
get-self-metadatafunc() -> worker-metadataReturns the metadata of the worker that calls this function

Worker enumeration

The worker enumeration API allows listing all the workers belonging to a specific component. This is a slow operation and should be used for maintenance tasks and not for the core business logic.

Worker enumeration is a WIT resource, providing a method that returns a page of workers each time it is called:

package golem:api@1.1.6;
 
interface host {
    resource get-workers {
        constructor(component-id: component-id, filter: option<worker-any-filter>, precise: bool);
 
        get-next: func() -> option<list<worker-metadata>>;
    }
}

Once get-next returns none, the enumeration is done.

There are two parameters for customizing the worker enumeration:

Filters

An optional filter parameter can be passed to the worker enumeration, with the following definition:

package golem:api@1.1.6;
 
interface host {
    record worker-any-filter {
        filters: list<worker-all-filter>
    }
}

The worker-any-filter matches workers that satisfy any of the listed filters.

package golem:api@1.1.6;
 
interface host {
    record worker-all-filter {
        filters: list<worker-property-filter>
    }
}

A worker-all-filter matches workers only that satisfy all of the listed filters.

package golem:api@1.1.6;
 
interface host {
    variant worker-property-filter {
        name(worker-name-filter),
        status(worker-status-filter),
        version(worker-version-filter),
        created-at(worker-created-at-filter),
        env(worker-env-filter)
    }
}

The worker-name-filter matches workers by one of their properties, such as the worker name, status, version, creation date or environment variables.

Each of these variants take a filter record holding a value and a comparator:

package golem:api@1.1.6;
 
interface host {
    record worker-name-filter {
        comparator: string-filter-comparator,
        value: string
    }
 
    record worker-status-filter {
        comparator: filter-comparator,
        value: worker-status
    }
 
    record worker-version-filter {
        comparator: filter-comparator,
        value: u64
    }
 
    record worker-created-at-filter {
        comparator: filter-comparator,
        value: u64
    }
 
    record worker-env-filter {
        name: string,
        comparator: string-filter-comparator,
        value: string
    }
}

Where filter-comparator and string-filter-comparator are defined as:

package golem:api@1.1.6;
 
interface host {
    enum filter-comparator {
        equal,
        not-equal,
        greater-equal,
        greater,
        less-equal,
        less
    }
 
    enum string-filter-comparator {
        equal,
        not-equal,
        like,
        not-like
    }
}

Precise enumeration

The precise flag switches the worker enumeration into a more costly variant which guarantees that every worker metadata returned is the latest one available at the time the result was generated.

When precise is set to false, the enumeration returns the last cached worker metadata available for each worker, which may be lagging behind the actual state of the workers.

Transactions and persistence control

Golem's transaction API allows customizing the execution engine's durability and transactional behaviors. These are low level functions, which are wrapped by SDKs providing a higher level transaction API for each supported language.

Types

Retry policy is defined by the following record:

package golem:api@1.1.6;
 
interface host {
    /// Configures how the executor retries failures
    record retry-policy {
        /// The maximum number of retries before the worker becomes permanently failed
        max-attempts: u32,
        /// The minimum delay between retries (applied to the first retry)
        min-delay: duration,
        /// The maximum delay between retries
        max-delay: duration,
        /// Multiplier applied to the delay on each retry to implement exponential backoff
        multiplier: f64,
        /// The maximum amount of jitter to add to the delay
        max-jitter-factor: option<f64>
    }
}

It is possible to switch between persistence modes:

package golem:api@1.1.6;
 
interface host {
    /// Configurable persistence level for workers
    variant persistence-level {
        persist-nothing,
        persist-remote-side-effects,
        smart
    }
}

Functions

The following functions define the transaction API:

Function nameDefinitionDescription
get-oplog-indexfunc() -> oplog-indexReturns the current position in the persistent op log
set-oplog-indexfunc(oplog-idx: oplog-index) -> ()Makes the current worker travel back in time and continue execution from the given position in the persistent op log.
oplog-commitfunc(replicas: u8) -> ()Blocks the execution until the oplog has been written to at least the specified number of replicas, or the maximum number of replicas if the requested number is higher.
mark-begin-operationfunc() -> oplog-indexMarks the beginning of an atomic operation. In case of a failure within the region selected by mark-begin-operation and mark-end-operation, the whole region will be reexecuted on retry. The end of the region is when mark-end-operation is called with the returned oplog-index.
mark-end-operationfunc(begin: oplog-index) -> ()Commits this atomic operation. After mark-end-operation is called for a given index, further calls with the same parameter will do nothing.
get-retry-policyfunc() -> retry-policyGets the current retry policy associated with the worker
set-retry-policyfunc(policy: retry-policy) -> ()Overrides the current retry policy associated with the worker. Following this call, get-retry-policy will return the new retry policy.
get-oplog-persistence-level func() -> persistence-levelGets the worker's current persistence level.
set-oplog-persistence-levelfunc(new-persistence-level: persistence-level) -> ()Sets the worker's current persistence level. This can increase the performance of execution in cases where durable execution is not required.
get-idempotence-modefunc() -> boolGets the worker's current idempotence mode.
set-idempotence-modefunc(idempotent: bool) -> ()Sets the current idempotence mode. The default is true. True means side-effects are treated idempotent and Golem guarantees at-least-once semantics. In case of false the executor provides at-most-once semantics, failing the worker in case it is not known if the side effect was already executed.
generate-idempotency-keyfunc() -> uuidGenerates a new idempotency key.

Update

The update API enables workers to trigger automatic or manual update of other workers.

Types

Automatic and manual updates are distinguished by the following enum:

package golem:api@1.1.6;
 
interface host {
    /// Describes how to update a worker to a different component version
    enum update-mode {
        /// Automatic update tries to recover the worker using the new component version
        /// and may fail if there is a divergence.
        automatic,
 
        /// Manual, snapshot-based update uses a user-defined implementation of the `save-snapshot` interface
        /// to store the worker's state, and a user-defined implementation of the `load-snapshot` interface to
        /// load it into the new version.
        snapshot-based
    }
}

Functions

The following function defines the update API:

Function nameDefinitionDescription
update-workerfunc(worker-id: worker-id, target-version: component-version, mode: update-mode) -> ()Initiates an update attempt for the given worker. The function returns immediately once the request has been processed, not waiting for the worker to get updated.

Oplog search and query

The oplog interface in golem:api provides functions to search and query the worker's persisted oplog.

The interface defines a big variant data type called oplog-entry:

package golem:api@1.1.6;
 
interface host {
    variant oplog-entry {
        /// The initial worker oplog entry
        create(create-parameters),
        /// The worker invoked a host function
        imported-function-invoked(imported-function-invoked-parameters),
        /// The worker has been invoked
        exported-function-invoked(exported-function-invoked-parameters),
        /// The worker has completed an invocation
        exported-function-completed(exported-function-completed-parameters),
        /// Worker suspended
        suspend(datetime),
        /// Worker failed
        error(error-parameters),
        /// Marker entry added when get-oplog-index is called from the worker, to make the jumping behavior
        /// more predictable.
        no-op(datetime),
        /// The worker needs to recover up to the given target oplog index and continue running from
        /// the source oplog index from there
        /// `jump` is an oplog region representing that from the end of that region we want to go back to the start and
        /// ignore all recorded operations in between.
        jump(jump-parameters),
        /// Indicates that the worker has been interrupted at this point.
        /// Only used to recompute the worker's (cached) status, has no effect on execution.
        interrupted(datetime),
        /// Indicates that the worker has been exited using WASI's exit function.
        exited(datetime),
        /// Overrides the worker's retry policy
        change-retry-policy(change-retry-policy-parameters),
        /// Begins an atomic region. All oplog entries after `BeginAtomicRegion` are to be ignored during
        /// recovery except if there is a corresponding `EndAtomicRegion` entry.
        begin-atomic-region(datetime),
        /// Ends an atomic region. All oplog entries between the corresponding `BeginAtomicRegion` and this
        /// entry are to be considered during recovery, and the begin/end markers can be removed during oplog
        /// compaction.
        end-atomic-region(end-atomic-region-parameters),
        /// Begins a remote write operation. Only used when idempotence mode is off. In this case each
        /// remote write must be surrounded by a `BeginRemoteWrite` and `EndRemoteWrite` log pair and
        /// unfinished remote writes cannot be recovered.
        begin-remote-write(datetime),
        /// Marks the end of a remote write operation. Only used when idempotence mode is off.
        end-remote-write(end-remote-write-parameters),
        /// An invocation request arrived while the worker was busy
        pending-worker-invocation(pending-worker-invocation-parameters),
        /// An update request arrived and will be applied as soon the worker restarts
        pending-update(pending-update-parameters),
        /// An update was successfully applied
        successful-update(successful-update-parameters),
        /// An update failed to be applied
        failed-update(failed-update-parameters),
        /// Increased total linear memory size
        grow-memory(grow-memory-parameters),
        /// Created a resource instance
        create-resource(create-resource-parameters),
        /// Dropped a resource instance
        drop-resource(drop-resource-parameters),
        /// Adds additional information for a created resource instance
        describe-resource(describe-resource-parameters),
        /// The worker emitted a log message
        log(log-parameters),
        /// The worker's has been restarted, forgetting all its history
        restart(datetime),
        /// Activates a plugin
        activate-plugin(activate-plugin-parameters),
        /// Deactivates a plugin
        deactivate-plugin(deactivate-plugin-parameters),
        /// Revert a worker to a previous state
        revert(revert-parameters),
        /// Cancel a pending invocation
        cancel-invocation(cancel-invocation-parameters),
        /// Start a new span in the invocation context
        start-span(start-span-parameters),
        /// Finish an open span in the invocation context
        finish-span(finish-span-parameters),
        /// Set an attribute on an open span in the invocation context
        set-span-attribute(set-span-attribute-parameters),
    }
}

and two resources for querying a worker's oplog.

  • the get-oplog resource enumerates through all entries of the oplog
  • the search-oplog resource accepts a search expression and only returns the matching entries

Both resources, once constructed, provide a get-next function that returns a chunk of oplog entries. Repeatedly calling this function goes through the whole data set, and eventually returns none.

Durability

The golem:durability package contains an API that can be leveraged by libraries to provide a custom durability implementation for their own API. This is the same interface that Golem uses under the hood to make the WASI interfaces durable. Golem applications are not supposed to use this package directly.

Types

The durable-function-type is a variant that categorizes a durable function in the following way:

package golem:api@1.1.6;
 
interface host {
    variant durable-function-type {
        /// The side-effect reads from the worker's local state (for example local file system,
        /// random generator, etc.)
        read-local,
        /// The side-effect writes to the worker's local state (for example local file system)
        write-local,
        /// The side-effect reads from external state (for example a key-value store)
        read-remote,
        /// The side-effect manipulates external state (for example an RPC call)
        write-remote,
        /// The side-effect manipulates external state through multiple invoked functions (for example
        /// a HTTP request where reading the response involves multiple host function calls)
        ///
        /// On the first invocation of the batch, the parameter should be `None` - this triggers
        /// writing a `BeginRemoteWrite` entry in the oplog. Followup invocations should contain
        /// this entry's index as the parameter. In batched remote writes it is the caller's responsibility
        /// to manually write an `EndRemoteWrite` entry (using `end_function`) when the operation is completed.
        write-remote-batched(option<oplog-index>)
    }
}

The durable-execution-state record provides information about the current execution state, and can be queried using the current-durable-execution-state function:

package golem:api@1.1.6;
 
interface host {
    record durable-execution-state {
        is-live: bool,
        persistence-level: persistence-level,
    }
}

Here the is-live field indicates whether the executor is currently replaying a worker's previously persisted state or side effects should be executed. The persistence-level is a user-configurable setting that can turn off persistence for certain sections of the code.

The persisted-durable-function-invocation is a record holding all the information about one persisted durable function. This should be used during replay to simulate the side effect instead of actually running it.

Functions

The durability API consists of a couple of low-level functions that must be called in a correct way to make it work.

The logic to be implemented is the following, in pseudocode:

observe-function-call("interface", "function")
state = current-durable-execution-state()
if state.is-live {
  result = perform-side-effect(input)
  persist-typed-durable-function-invocation("function", encode(input), encode(result), durable-function-type)
} else {
  // Execute the side effect
  persisted = read-persisted-durable-function-invocation()
  result = decode(persisted.response)
}

The input and result values must be encoded into value-and-type, the dynamic value representation from the golem:rpc package.

In cases when a durable function's execution interleaves with other calls, the begin-durable-function and end-durable-function calls can be used to mark the beginning and end of the operation.

Invocation context

Golem associates an invocation context with each invocation, which contains various information depending on how the exported function was called. This context gets inherited when making further invocations via worker-to-worker communication, and it is also possible to define custom spans and associate custom attributes to it.

The spans are not automatically sent to any tracing system but they can be reconstructed from the oplog, for example using oplog processor plugins, to provide real-time tracing information.

To get the current invocation context, use the current-context host function:

package golem:api@1.1.6;
 
/// Invocation context support
interface context {
    current-context: func() -> invocation-context;
}

The invocation-context itself is a resource with various methods for querying attributes of the invocation context:

methoddescription
trace-idReturns the trace ID associated with the context, coming from either an external trace header or generated at the edge of Golem
span-idReturns the span ID associated with the context
parentReturns the parent invocation context, if any
get-attributeGets an attribute from the context by key
get-attributesGets all attributes from the context
get-attribute-chainGets all values of a given attribute from the current and parent contexts
get-attribute-chainsGet all attributes and their previous values
trace-context-headersGets the W3C Trace Context headers associated with the current invocation context

Custom attributes can only be set on custom spans. First start a new span using start-span

package golem:api@1.1.6;
 
/// Invocation context support
interface context {
    /// Starts a new `span` with the given name, as a child of the current invocation context
    start-span: func(name: string) -> span;
}

and then use the span resource's methods:

methoddescription
started-atReturns the timestamp when the span was started
set-attributeSets an attribute on the span
set-attributesSets multiple attributes on the span
finishEnds the current span

Dropping the resource is equivalent to calling finish on the span.

The custom spans are pushed onto the invocation context stack, so whenever an RPC call or HTTP call is made, their parent span(s) will include the user-defined custom spans as well as the rest of the invocation context.