Adding LLM and AI Capabilities (TypeScript)

Overview

Use a provider’s REST API with fetch, or evaluate its third-party npm SDK. Golem provides fetch via WASI HTTP, but that alone does not establish package compatibility: SDKs can also depend on unsupported runtime APIs. Build and exercise the exact package version and API path in Golem before relying on it. The package examples below are integration starting points, not tested runtime-compatibility guarantees.

Recommended Libraries

OpenAI

The official OpenAI npm package is openai:


npm install openai


import OpenAI from 'openai';
 
const client = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY,
});
 
const response = await client.chat.completions.create({
  model: 'gpt-4o',
  messages: [{ role: 'user', content: 'Hello!' }],
});
 
const text = response.choices[0]?.message?.content ?? '';

Anthropic

The official Anthropic npm package is @anthropic-ai/sdk:


npm install @anthropic-ai/sdk


import Anthropic from '@anthropic-ai/sdk';
 
const client = new Anthropic({
  apiKey: process.env.ANTHROPIC_API_KEY,
});
 
const response = await client.messages.create({
  model: 'claude-sonnet-5-5',
  max_tokens: 1024,
  messages: [{ role: 'user', content: 'Hello!' }],
});

Other Providers

Provider SDK candidates include:

Google AI (@google/genai) — Gemini models; Google’s maintained replacement for the deprecated @google/generative-ai
Cohere (cohere-ai) — chat, embeddings, reranking
Mistral (@mistralai/mistralai) — Mistral models
Groq (groq-sdk) — fast inference

Anthropic’s model lifecycle page lists claude-sonnet-5-5 as active and as the replacement for the retired claude-sonnet-4-20250514. Google’s library guide recommends @google/genai; use its migration guide rather than only renaming imports in legacy code. These provider facts do not establish Golem runtime compatibility. Confirm model availability for your account and test the selected SDK in a deployed agent.

Calling Any LLM API Directly

You can also call any LLM provider’s REST API directly using fetch:


const response = await fetch('https://api.openai.com/v1/chat/completions', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'Authorization': `Bearer ${process.env.OPENAI_API_KEY}`,
  },
  body: JSON.stringify({
    model: 'gpt-4o',
    messages: [{ role: 'user', content: 'Hello!' }],
  }),
});
 
const data = await response.json();
const text = data.choices[0]?.message?.content ?? '';

See the golem-make-http-request-ts skill for more details on making HTTP requests.

Setting API Keys

Store provider API keys as secrets using Golem’s typed config system. See the golem-add-secret-ts skill for full details. In brief, declare the key as a config field marked with s.secret(...):


import { z } from 'zod';
import { defineAgent, s } from '@golemcloud/golem-ts-sdk';
 
export const MyAgent = defineAgent({
  name: 'MyAgent',
  id: { name: z.string() },
  config: {
    apiKey: s.secret(z.string()),
  },
  methods: { /* ... */ },
});

Then manage it via the CLI:


golem secret create apiKey --type string --value "sk-..."

Access it inside a handler with this.config.apiKey.get() — a secret field surfaces as a lazy Secret<string> handle; call .get() to reveal the current value.

Complete Agent Example


import { z } from 'zod';
import { defineAgent, method, http, s } from '@golemcloud/golem-ts-sdk';
import OpenAI from 'openai';
 
export const ChatAgent = defineAgent({
  name: 'ChatAgent',
  id: { chatName: z.string() },
  http: http.mount('/chats/{chatName}'),
  config: {
    apiKey: s.secret(z.string()),
  },
  methods: {
    ask: method({ input: { question: z.string() }, returns: z.string(), http: http.post('/ask') }),
  },
});
 
export const ChatAgentImpl = ChatAgent.implement({
  // `init` receives a context with `id`, `config`, `principal`, `phantomId`.
  init: ({ id, config }) => {
    const client = new OpenAI({ apiKey: config.apiKey.get() });
    const messages: OpenAI.ChatCompletionMessageParam[] = [
      { role: 'system', content: `You are a helpful assistant for chat '${id.chatName}'` },
    ];
    return { client, messages };
  },
  methods: {
    async ask({ question }) {
      this.messages.push({ role: 'user', content: question });
 
      const response = await this.client.chat.completions.create({
        model: 'gpt-4o',
        messages: this.messages,
      });
 
      const reply = response.choices[0]?.message?.content ?? '';
      this.messages.push({ role: 'assistant', content: reply });
      return reply;
    },
  },
});

Note: Inside a method handler, this is bound to the state returned by init plus SDK helpers (this.config, this.getId(), this.getPrincipal()). Inside init, read config/id from the context argument instead: init: ({ id, config }) => ....

Key Constraints

Prefer REST with fetch, or validate the selected npm SDK version in Golem; using fetch or node:http internally is not by itself a compatibility guarantee
Libraries that depend on native C/C++ bindings (e.g., onnxruntime-node) will not work
API keys should be stored as secrets using Golem’s typed config system (See the golem-add-secret-ts guide)
All HTTP requests made from agent code are automatically durably persisted by Golem