Control the retry policy from Python

Using Golem's retry mechanism

Golem applies a retry mechanism to all workers. In case of a failure, Golem will automatically recover the worker to the point before the failure and retry the operation. An exponential backoff and an upper limit on the number of retries are applied.

If the maximum number of retries is reached, the worker will be marked as failed and no further invocations will be possible on it.

This mechanism is automatic and applied to all kind of failures. To rely on it, just let the Python code fail (raise an exception).

Customizing the retry policy

The retry policy which controls the maximum number of retries and the exponential backoff is a global configuration of the Golem servers, but it can be customized for each worker.

Once the golem:api/host@1.1.0 interface is imported in the WIT file (see the previous page for more information), the retry policy can be controlled with the get_retry_policy and set_retry_policy functions.

import py_example
import py_example.imports
import py_example.imports.types
import py_example.imports.host
import py_example.exports
import py_example.exports.api
from py_example.imports.host import get_retry_policy, set_retry_policy, RetryPolicy
 
old = get_retry_policy()
try:
    set_retry_policy(RetryPolicy(max_attempts=10, min_delay=1_000_000_000, max_delay=1_000_000_000, multiplier=1))
    # ...
finally:
    set_retry_policy(old)
// ...
 
golem_api_host_get_retry_policy(&previous);

The RetryPolicy type itself is originated from Golem's WIT definition in the following way:

/// Configures how the executor retries failures
record retry-policy {
    /// The maximum number of retries before the worker becomes permanently failed
    max-attempts: u32,
    /// The minimum delay between retries (applied to the first retry)
    min-delay: duration,
    /// The maximum delay between retries
    max-delay: duration,
    /// Multiplier applied to the delay on each retry to implement exponential backoff
    multiplier: f64,
    /// The maximum amount of jitter to add to the delay
    max-jitter-factor: option<f64>
}

In python each field is represented by an integer.

Durability Promises