Adding LLM and AI Capabilities (MoonBit)

Call the provider’s REST API with the SDK’s Preview 3 HTTP wrapper. Load golem-make-http-request-moonbit for HTTP retry and recovery behavior. Do not use Preview 2 OutgoingRequest, handle, pollables, or blocking body reads.

Setup

Add these imports to the component’s moon.pkg (keep its existing imports):

import {
  "golemcloud/golem_sdk/http" @http,
  "golemcloud/golem_sdk/config" @config,
  "moonbitlang/core/encoding/utf8" @utf8,
}

Declare the provider model and credential as typed configuration. Select a model available to your provider account; this example does not pin a model’s lifetime.

#derive.config
pub(all) struct ChatConfig {
  model : String
  api_key : @config.Secret[String]
}

Set agents.ChatAgent.config.model in golem.yaml, and create the secret at the normalized apiKey path with golem secret create apiKey --type String. Enter the key at the hidden prompt. Load golem-add-secret-moonbit for secret provisioning. Never return or log the credential.

Request and Response Lifecycle

Use the installed SDK’s @http.Request constructor and body accessors. They are generated from the current WIT; inspect them rather than translating Preview 2 resource methods. The relevant contract is:

async fn send_request(request : @http.Request) -> Result[@http.Response, @http.ErrorCode] {
  @http.send(request)
}

To implement a provider adapter:

1. Serialize a structured JSON object, not interpolated JSON text. For OpenAI Chat Completions use {"model": model.to_json(), "messages": [{"role": "user", "content": prompt.to_json()}]} as a MoonBit Json value and call .stringify(). String literals have a JSON literal overload; String variables require .to_json().
2. Encode the JSON with @utf8.encode(body[:]). Do not use String::to_bytes(); it produces UTF-16LE, not HTTP UTF-8.
3. Build a Preview 3 Request with method POST, scheme HTTPS, authority api.openai.com, path /v1/chat/completions, Content-Type: application/json and the provider’s authorization header. Supply the bytes through its async body stream and complete the body/trailers according to the generated API.
4. Call @http.send from an async function and handle its Result.
5. Inspect the response status. A 4xx/5xx response is not a transport error.
6. Consume all response body chunks until the stream ends, then await and inspect its completion future/trailers. Release the stream, completion future and other owned resources. A single read is not a full-response guarantee.
7. Decode the collected bytes as UTF-8 and parse the JSON. Check the provider’s error shape before extracting choices[0].message.content.

The adapter below returns the full provider JSON response as UTF-8 text. Decode its provider-specific schema separately to extract an assistant message. Keep the helper and every caller async:

async fn provider_chat(model : String, credential : String, prompt : String) -> Result[String, String] {
  let body : Json = {
    "model": model.to_json(),
    "messages": [{ "role": "user", "content": prompt.to_json() }],
  }
  let encoded = Bytes::to_fixedarray(@utf8.encode(body.stringify()[:]))
  let headers = @http.Fields::from_list([
    ("Content-Type", Bytes::to_fixedarray(@utf8.encode("application/json"))),
    ("Authorization", Bytes::to_fixedarray(@utf8.encode("Bearer " + credential))),
  ]).unwrap()
  let contents = @http.Stream::produce(async fn(sink) {
    if sink.write_all(encoded[:]) {
      sink.close()
    }
  })
  let (request, sent) = @http.Request::new(
    headers, Some(contents), @http.Future::ready(Ok(None)), None,
  )
  defer sent.drop()
  request.set_method(@http.Method::Post).unwrap()
  request.set_scheme(Some(@http.Scheme::Https)).unwrap()
  request.set_authority(Some("api.openai.com")).unwrap()
  request.set_path_with_query(Some("/v1/chat/completions")).unwrap()
  let response = match @http.send(request) {
    Ok(response) => response
    Err(_) => return Err("HTTP transport failed")
  }
  let status = response.get_status_code()
  let (incoming, completed) = @http.Response::consume_body(
    response, @http.Future::ready(Ok(())),
  )
  defer incoming.drop()
  defer completed.drop()
  let bytes : Array[Byte] = []
  for ;; {
    match incoming.read(65536) {
      Some(chunk) => for byte in chunk { bytes.push(byte) }
      None => break
    }
  }
  match completed.get() {
    Err(_) => return Err("HTTP response body failed")
    Ok(Some(trailers)) => trailers.drop()
    Ok(None) => ()
  }
  match sent.get() {
    Err(_) => return Err("HTTP request body failed")
    Ok(_) => ()
  }
  if status < 200 || status >= 300 {
    return Err("Provider returned HTTP status \{status}")
  }
  Ok(@utf8.decode_lossy(Bytes::from_array(bytes)[:]))
}
 
#derive.agent
struct ChatAgent {
  chat_name : String
  config : @config.Config[ChatConfig]
}
 
fn ChatAgent::new(chat_name : String, config : @config.Config[ChatConfig]) -> ChatAgent {
  { chat_name, config }
}
 
pub async fn ChatAgent::ask(self : Self, question : String) -> Result[String, String] {
  let cfg = self.config.value
  let credential = try cfg.api_key.get() catch {
    _ => return Err("Provider credential unavailable")
  }
  provider_chat(cfg.model, credential, question)
}
 
fn main {}

This example exposes RPC only. If HTTP access is required, first add #derive.mount("/chats/{chat_name}") to ChatAgent, then #derive.endpoint(post="/ask") to ask, and deploy the agent through httpApi in golem.yaml. Config is injected and has no mount variable. Load golem-add-http-endpoint-moonbit for deployment and authentication setup.

Other Providers

Adapt authority, path, headers and JSON schema to the provider documentation:

Completed HTTP outcomes replay from the oplog. A lost response does not prove a provider did not process or bill the request. Host retries can repeat remote writes; use provider-supported idempotency when available.