Exposing Durable Streams over HTTP

Golem exposes an HTTP-annotated agent method in Durable Streams mode whenever the method has a top-level stream input or a stream-bearing result. No Durable Streams annotation is required. The method’s HTTP path becomes the base of a protocol-compatible URL family; the method schema fixes the available slots and their representations.

This is useful when a browser, service, or an unmodified Durable Streams client needs to drive a long-running agent invocation without holding one HTTP request open.

URL family

For a method mounted at {base}, Golem provides:

Session and fork IDs contain 1–128 ASCII letters, digits, ., _, or -. A session identifies one invocation and all of its input, output, and result slots. A direct stream<T> result is $result; when a result record contains streams, each field is a separate output slot and there is no aggregate $result. A non-stream result is a single JSON value in $result. Unit results have no result slot.

Top-level stream<u8> slots are opaque byte streams. They use application/octet-stream by default and offsets advance through bytes. All other slots are typed JSON streams using application/json; each array member in a read response is one logical item. A custom opaque MIME type is available only for direct byte streams. Golem has no native text/plain mode for stream<string>.

See Customizing Durable Stream Routes for public slot aliases, byte-stream MIME types, operation switches, and per-route admission limits. There is no fork opt-out.

Start and drive a session

This example assumes an echo method with a JSON input slot named input and an output slot named $result.

# Create the invocation. Non-stream arguments use the endpoint's normal
# path, query, header, and body bindings.
curl -i -X PUT \
  https://api.example.com/echo/invocations/demo-1 \
  -H 'Authorization: Bearer …' \
  -H 'Stream-TTL: 300'
 
# Append two items and close the input atomically.
curl -i -X POST \
  https://api.example.com/echo/invocations/demo-1/streams/input \
  -H 'Authorization: Bearer …' \
  -H 'Content-Type: application/json' \
  -H 'Producer-Id: upload-1' \
  -H 'Producer-Epoch: 0' \
  -H 'Producer-Seq: 0' \
  -H 'Stream-Closed: true' \
  --data '["first", "second"]'
 
# Read everything produced so far. Keep the returned Stream-Next-Offset
# opaque and pass it back unchanged when resuming.
curl -i \
  'https://api.example.com/echo/invocations/demo-1/streams/%24result?offset=-1'

Slot PUT and input POST can create a missing session only when every non-stream method argument is bound in the route path or query. Otherwise create the session first. Base and session creation consume the endpoint’s normal request body; do not assume it is always a JSON argument object.

Reading

GET accepts these query parameters:

Offsets are opaque StreamOffsetV1 values. Their hexadecimal representation is a transport token, not an integer or item count. Store and echo it exactly. A read returns Stream-Next-Offset, Stream-Up-To-Date, and terminal metadata as applicable. JSON catch-up and long-poll bodies are arrays; byte bodies are exact bytes. SSE control events carry streamNextOffset, upToDate, and either streamClosed: true or a string streamCursor.

HEAD returns the same committed stream metadata without a body. Closed pages can use ETag and conditional GET; other responses are normally Cache-Control: no-store.

Appending and idempotent producers

Only input slots accept POST. A JSON body may be one value or an array batch; arrays flatten exactly one level, so an array-valued item needs an outer array. A byte append is one opaque byte batch. A non-empty body requires the slot’s content type. Send Stream-Closed: true with data to append and close atomically, or with an empty body for close-only.

For retry-safe writes, supply Producer-Id, Producer-Epoch, and Producer-Seq together. Sequence starts at zero and advances per request, not per item. Retrying the exact tuple is a no-op after the first commit. A newer epoch fences older writers and starts at sequence zero; gaps are rejected. Persist the tuple and body before sending, and never replace an uncertain request with different data.

Expiry

Creation PUT, lazy slot PUT, lazy input POST, and fork creation support one session-wide expiry policy:

• Stream-TTL: <seconds> is a sliding idle timeout in whole seconds.
• Stream-Expires-At: <RFC3339 timestamp> is an absolute future deadline.

The headers are mutually exclusive; malformed, duplicated, past, or conflicting policies return 400. The policy belongs to the entire session, not one slot. HEAD reports the configured value. A new origin GET and an accepted or duplicate POST refresh a sliding TTL; refresh records are coalesced until they extend the deadline by at least 10% of the TTL. HEAD, replayed PUT, subsequent activity inside an already-open live response, and agent output do not refresh it.

Expiry cooperatively cancels open streams, retires the whole URL family to 404, and—on the normal durable-agent path—allows an explicit session PUT or eligible slot PUT to create a new invocation with the same public session ID and fresh stream identities. Lazy POST does not recreate an expired session. Ephemeral agents cannot release their key-derived identity and require a new session ID; expired fork targets also cannot be recreated.

Authentication and browsers

Every generated operation inherits the endpoint’s authentication policy. Send the same bearer or configured API credentials to base, session, stream, and fork URLs. Session URLs are not bearer capabilities by themselves.

CORS also inherits the route policy. Preflight supports the Durable Streams request headers, including closure, producer, fork, expiry, cache, and content headers. Successful browser responses expose the relevant Location, offset, terminal, producer, expiry, retry, ETag, cursor, and SSE-encoding headers. Configure the endpoint’s allowed origins and credentials as described in Making Custom APIs.

Operation and status reference

The deployment-specific OpenAPI document is authoritative for the concrete slots, schemas, media types, security, and operations enabled on a route. The table below summarizes the complete runtime family.

Common response metadata is Stream-Next-Offset, Stream-Closed, Stream-Cancelled, Stream-Up-To-Date, Cache-Control, and ETag. Stream-Cancelled: true distinguishes cooperative cancellation from normal EOF; buffered history may still be returned first. An already-terminal stream keeps its original terminal state.

Producer outcomes use Producer-Epoch and Producer-Seq. A stale epoch is 403 with the current Producer-Epoch; a sequence gap is 409 with Producer-Expected-Seq and Producer-Received-Seq. Body and schema errors may use application/problem+json, but callers must accept bodyless errors too. Disabled operations return 405 with an exact Allow; overload returns 429 or 503 with Retry-After.