Making custom APIs

The Worker Gateway service not only exposes the low-level REST API for invocations but also provides a way to define fully customizable HTTP APIs. Future versions will support other protocols as well such as gRPC or GraphQL.

Example

Take the shopping cart example (available via golem-cli new --example rust-example-shopping-cart):

$ golem-cli component get --component-name cart

Component with URN urn:component:c57de1ee-fbe5-4068-a67d-908addc8aa44. Version: 0. Component size is 2900607 bytes.
Component name: cart.
Exports:
	golem:component/api.{initialize-cart}(user-id: string)
	golem:component/api.{add-item}(item: record { product-id: string, name: string, price: float32, quantity: u32 })
	golem:component/api.{remove-item}(product-id: string)
	golem:component/api.{update-item-quantity}(product-id: string, quantity: u32)
	golem:component/api.{checkout}() -> variant { error(string), success(record { order-id: string }) }
	golem:component/api.{get-cart-contents}() -> list<record { product-id: string, name: string, price: float32, quantity: u32 }>

The following API definition exposes the get-cart-contents function as a custom http endpoint.

{
  "id": "my-shopping-cart-v1",
  "draft": true,
  "version": "0.0.3",
  "routes": [
    {
      "method": "Get",
      "path": "/{user-id}/get-cart-contents",
      "binding": {
        "type": "wit-worker",
        "componentId": {
          "componentId": "94b657f6-238f-4737-9fc3-2b25ff2f60da",
          "version": 0
        },
        "workerName": "${let user: u64 = request.body.user-id; \"my-worker-${user}\" }",
        "response": "${let result = golem:it/api.{get-cart-contents}(); {status: 200u64, body: result}}"
      }
    }
  ]
}

The API definition's fields have the following meaning:

Field	Description
`id`	This field represents the unique identifier for the API definition. In this case, it is set to "shopping-cart-v1".
`version`	This field indicates the version of the API definition. Here, it is set to "0.0.3".
`routes`	This field contains an array of route objects, each representing a specific endpoint definition.
`method`	Indicates the HTTP method associated with the route. In this example, it is set to "Get", indicating that this route handles GET requests.
`path`	Specifies the URL path pattern for the route. It may include path parameters enclosed in curly braces. Here, the path is `/{user-id}/get-cart-contents`, indicating that this route handles requests to retrieve the contents of a shopping cart for a specific user.
`binding`	This object contains information about how the request should be handled by the Golem worker.

Golem Worker Binding

Let's break down the binding object.

Field	Description
`type`	Specifies the type of binding. Here, it is set to "wit-worker", indicating that the binding involves a Golem worker. Currently, the only supported type is "wit-worker".
`componentId`	Provides the component ID associated with the worker binding. As you can see the the golem-worker
`workerName`	Specifies the Name of the Golem worker responsible for handling the request. Here it is `my-worker-${request.path.user-id}`. The value that is wrapped in code block (starting with `${` and ending with `}`) will be a Rib expression. Here `request.path.user-id` is a valid Rib expression where it selects the field `path` from the `request` block, and further selects `user-id` from it.
`response`	The `response` field here is a Rib expression (therefore wrapped in a code block) that allows you to call any worker function and manipulates its output.

Note that anything wrapped in this block is an Rib Expression which gets evaluated at runtime and in this case, it gets evaluated to actual user-id which is then concatenated with the text "my-worker-". If you pass complex Rib expressions to be concatenated with a worker-id (for example: It is evaluated to a WASM record) then you will get a response back saying invalid worker-name. If your worker name is a constant, you can avoid interpolations ${} and directly write the worker name as a text. See the Rib reference for the full reference of the binding language.

In this example, the following Rib expression defines the request:

let result = golem:it/api.{get-cart-contents}();
{status: 200u64, body: result}

The first line is about calling the worker-function and assigning its reuslt to the variable result. The last line is a WASM record where you are mapping to the response. Here body is result itself. That means, we are not manipulating the result returned by the function, and simply forward it as response body. Status is 200u64. u64 is the type of the number as Rib is typed and it needs to the full type of the WASM record. 200u64 is equivalent to let x: u64 = 200; x; in Rib.

There are various possibilities here to manipulate the data using Rib language, such as selecting only a particular field from the result of the function invocation. Say for example, the result is a WASM result which can be ok or err. In this case you can have a complex Rib expression as below

let result = golem:it/api.{get-cart-contents}();

let response_status = match result {
  ok(_) => 200u32,
  err(_) => 400u32
};

let response_body = match result {
  ok(result) => result,
  err(msg) => msg
}

{status: response_status, body: response_body}

The result of the Rib expression is whatever the result of the last line/expression in the above Rib program, similar to many programming languages.

Upload API definition

The API definitions can be uploaded to Golem using one of the following methods:

Use the Golem CLI tool
Use the Rest APIs
Use Golem Console (opens in a new tab) (Only available in Golem Cloud)

The following example uses golem-cli to add a new API definition to the system.

Terminal

golem-cli api-definition add api-definition.json

This returns:

API Definition created with ID my-shopping-cart-v1 and version 0.0.2.
Routes:
+--------+------------------------------+---------------+-----------------------------------+
| Method | Path                         | Component URN | Worker Name                       |
+--------+------------------------------+---------------+-----------------------------------+
| Get    | /{user-id}/get-cart-contents | *908addc8aa44 | my-worker-${request.path.user-id} |
+--------+------------------------------+---------------+-----------------------------------+

This API definition is still a draft since it is still not deployed and have the endpoint of get-cart-contents exposed. Once it is deployed you cannot further update this definition unless you change the version, to avoid breaking your public API.

The API definition can also be uploaded directly using REST APIs.

Terminal

curl -X POST http://localhost:9881/v1/api/definitions -H "Content-Type: application/json" -d @api-definition.json

Here localhost:9881 is where worker-service (that implements the Worker Gateway) is running, ready to accept these API management requests. The docker-compose example (explained in quick-start) currently exposes the port 9881 for worker-service.

Deploy the API Definition

Using golem-cli, deploying the API definition is straight forward.

Terminal

golem-cli api-deployment deploy --definition my-shopping-cart-v1/0.0.2 --host localhost:9006

The above command returns the following:

API deployment on localhost:9006 with definition my-shopping-cart-v1/0.0.2

Here we deploy the definition and now the endpoints should be available at host localhost:9006. If you were following docker examples to spin up Golem, then the port has to be 9006. Refer to .env in docker-examples folder in Golem OSS.

You can test the deployment now, by trying to call the endpoint defined in the API definition.

curl -H "Accept: application/json" -X GET http://localhost:9006/afsal/get-cart-contents
[]

Alternatively you can use REST APIs to deploy the API definition.

Let's create a deployment.json as given below. Based on the example above, the apiDefinitionId is shopping-cart-v1 and version is 0.0.2.

If you were having a domain registered such as my-site.com, you would replace localhost:9006 with my-site.com, that in turn redirects to localhost:9006, or wherever the worker-service is running (perhaps AWS).

{
  "apiDefinitionId": "shopping-cart-v1",
  "version": "0.0.3",
  "site": "localhost:9006"
}

Terminal

 
curl -X POST http://localhost:9881/v1/api/deployments/deploy -H "Content-Type: application/json"  -d @deployment.json

Start using the API

As you have seen before, after the deployment, we call the endpoint.

Terminal

 
curl -X GET http://localhost:9006/123/get-cart-contents

Now the Worker Gateway identifies the user to be 123 and evaluates the worker-name expr to be worker-123. It then forwards the request to the worker with id worker-123 and invokes the function golem:it/api.{get-cart-contents} with empty parameters, and simply get the worker-response (which is a WASM value) and converts it to Json and sends it back to the client.

Response Mapping

Response Mapping is under the response field under each binding. This is written using Rib to call the worker function and manipulate it's results and return a value that's understandable for each protocol we use.

Here is a more complex example of an API definition that gives you more idea on what can be achieved with Rib expression under response field.

{
  "id": "my-shopping-cart-v2",
  "draft": true,
  "version": "2.0.11",
  "routes": [
    {
      "method": "Get",
      "path": "/{user-id}/get-cart-contents",
      "binding": {
        "type": "wit-worker",
        "componentId": "c57de1ee-fbe5-4068-a67d-908addc8aa44",
        "workerName": "my-worker",
        "response": "${let result = golem:it/api.{get-cart-contents}(); {status: 200u64, body: result}}"
      }
    },
    {
      "method": "Post",
      "path": "/{user-id}/initialize-cart",
      "binding": {
        "type": "wit-worker",
        "componentId": "c57de1ee-fbe5-4068-a67d-908addc8aa44",
        "workerName": "my-worker",
        "response": "let empty: list<u16> = []; ${golem:it/api.{initialize-cart}(request.path.user-id); {status: 200u64, body: empty}}"
      }
    },
    {
      "method": "Post",
      "path": "/{user-id}/add-item",
      "binding": {
        "type": "wit-worker",
        "componentId": "c57de1ee-fbe5-4068-a67d-908addc8aa44",
        "workerName": "my-worker",
        "response": "${let quantity: f64 = request.body.quantity; let product_id: u64 = request.body.product-id; let name: str = request.body.name; let price: f64 = request.body.price; let input = {product-id: product_id, name: name, price: price, quantity: quantity}; golem:it/api.{add-item}(input); {status: 200u64, body: \"success\"}}"
      }
    },
    {
      "method": "Post",
      "path": "/{user-id}/checkout",
      "binding": {
        "type": "wit-worker",
        "componentId": "c57de1ee-fbe5-4068-a67d-908addc8aa44",
        "workerName": "my-worker",
        "response": "${let result = golem:it/api.{checkout}(); {status: 200u64, body: match result { success(resp) => resp.order-id, error(msg) => msg }}}"
      }
    }
  ]
}

This example exposes multiple functions of the shopping-cart component through a custom HTTP API. Some of these examples include longer Rib scripts under response field. While this is not user friendly, with golem cloud console exposing an editor, you can write in a far more better way. Improvements in this space are in progress.

Note that each line in Rib code block is separated by ; and the last line in the Rib script is the return value which shouldn't have ;.

Most of the above examples are self explanatory, but worth pointing out a few details.

The examples show how you can pass parameters to the function. This is more or less same as many programming languages - a comma separated list of arguments.


golem:it/api.{initialize-cart}(request.path.user-id)

Here Rib compiler knows the requirement of initialize-cart function and infers that request.path.user-id should be U64. This is a hint, that Rib is aware of the types in the code. In places where it find hard to infer these typs, you may face some compile time error (when uploading API definition), and the best solution is to be explicit about types. For example

let user_id: u64 = request.path.user-id;
golem:it/api.{initialize-cart}(user_id)

The most interesting example out of all the endpoints is the checkout endpoint.

Here the response mapping in Rib is this:

let result = golem:it/api.{checkout}();

{status: 200u64, body: match result { success(resp) => resp.order-id, error(msg) => msg }}

This can also be written as the following for more readability:

let result = golem:it/api.{checkout}();
let status: u64 = 200;
let body =  match result { success(resp) => resp.order-id, error(msg) => msg };
{status: status, body: body}

Here we manipulate the result from worker function using a match expression. More details on match expression is explained under Rib documentation. It matches on success and error variants because the result type of golem:it/api.{checkout} is a variant which can be either success or error.

Looking up data from the request body and path

In the above example, we have an endpoint to add products into the cart.

{
  "method": "Post",
  "path": "/{user-id}/add-item",
  "binding": {
    "type": "wit-worker",
    "componentId": "c57de1ee-fbe5-4068-a67d-908addc8aa44",
    "workerName": "${let user_id: u64 = request.path.user-id; \"my-worker-${user_id}\"",
    "response": "${let product_id: u64 = request.body.product-id; golem:it/api.{add-item}({product-id: product_id}); let empty_body: list<u16> = []; {status: 200u64, body: empty_body}}"
  }
}

This implies, the http request should be method POST with request body having the following fields: product-id, name, price and quantity. Similarly, yoiu can select field from the request path in Rib expression. We already used this to fetch user-id to form the worker name.

Test the API endpoints

Once you upload and deploy the API definition above, you can try the following:

With the following request body saved in a file request_body.json

{
  "product-id": "foo",
  "name": "iphone",
  "price": 2500,
  "quantity": 1
}

curl -H "Accept: application/json" -X POST http://localhost:9006/afsal/initialize-cart
#[]


curl -H "Accept: application/json" -X POST http://localhost:9006/afsal/add-item -d @request_body.json

[]%

curl -H "Accept: application/json" -X GET http://localhost:9006/afsal/get-cart-contents
[{"name":"foo","price": 2500,"product-id":"foo","quantity":1}]%

curl -H "Accept: application/json" -X POST http://localhost:9006/afsal/checkout
#238738674%

Note that there is currently no way to express unit in Rib as Rib is mostly WASM-WAVE syntax. This is why for certain endpoints we simply returned []. You can also prefer to return an empty string "".

Support to import OpenAPI spec

Golem allows you to import API definition in OpenAPI spec format. This is because, users may have already written an OpenAPI spec for their endpoints for various purposes. By adding extra details of into the same spec, we can use it as an API definition. Internally it gets converted to the Worker Gateway's native format of API definition, discussed in the beginning of this documentation.

The main advantage of this feature is the re-usability of the same endpoint definitions across your system. For example, you can use the same file now to register with Golem's Worker Gateway and register with another external API gateway. More on this below. However once deployed, it returns native format then onwards, and updates needs to be done in the native format.

{
  "openapi": "3.0.0",
  "x-golem-api-definition-version": "0.0.3",
  "x-golem-api-definition-id": "shopping-cart-v1",
  "info": {
    "title": "Sample API",
    "version": "1.0.2"
  },
  "paths": {
    "/{user-id}/get-cart-contents": {
      "x-golem-worker-bridge": {
        "worker-name": "my-worker",
        "component-id": "2696abdc-df3a-4771-8215-d6af7aa4c408",
        "response": "${ { headers: { ContentType: \"json\", userid: \"foo\"}, body: golem:it/api.{get-cart-contents}(), status: 200u64 } }"
      },
      "get": {
        "summary": "Get Cart Contents",
        "description": "Get the contents of a user's cart",
        "parameters": [
          {
            "name": "user-id",
            "in": "path",
            "required": true,
            "schema": {
              "type": "string"
            }
          }
        ],
        "responses": {
          "200": {
            "description": "OK",
            "content": {
              "application/json": {
                "schema": {
                  "$ref": "#/components/schemas/CartItem"
                }
              }
            }
          },
          "404": {
            "description": "Contents not found"
          }
        }
      }
    }
  },
  "components": {
    "schemas": {
      "CartItem": {
        "type": "object",
        "properties": {
          "id": {
            "type": "string"
          },
          "name": {
            "type": "string"
          },
          "price": {
            "type": "number"
          }
        }
      }
    }
  }
}

To import OpenAPI spec using Golem CLI,

Terminal

 
golem-cli api-definition import @open_api_spec.json

Alternatively, you can use REST API endpoint

Terminal

curl -X POST http://localhost:9881/v1/api/definitions/oas -H "Content-Type: application/json"  -d @openapi_spec.json

Integrating Golem with existing API Gateways

Note that, while the Golem's Worker Gateway is a powerful tool for defining and managing API endpoints, it is not a full-fledged API Gateway. However, it can be used conjunction with existing API Gateways, allowing you to leverage the capabilities of both systems. For example, you can use Golem's Worker Gateway to define and manage the worker bindings for your API endpoints, while using an API Gateway to handle other aspects of API management, such as authentication, rate limiting, and monitoring. In this scenario, the third party API Gateway would route incoming requests to Golem based on the defined endpoints, allowing Golem's Worker Gateway to handle the request processing and response generation.

Tyk

This section shows how to integrate with Tyk API gateway (opens in a new tab).

Note that Tyk allows users to upload OpenAPI spec similar to Golem. You can upload the same OpenAPI spec with worker-bridge info to Tyk, with 1 more extra information which is servers block with the value of the URL of the Golem Worker Gateway, that tells the API gateway to route the request to worker-bridge. Obviously, it depends on how you installed Tyk. If you installed Tyk in the same network as worker-bridge, you can use the localhost as servers. If you are using a separate docker network with Tyk, you will need to give the machine IP address to reach the Worker Gateway URL.

{
  "openapi": "3.0.0",
  "x-golem-api-definition-version": "0.0.3",
  "x-golem-api-definition-id": "shopping-cart-v1",
  "info": {
    "title": "Sample API",
    "version": "1.0.2"
  },
  "servers": [
    {
      "url": "http://ip-address-of-your-local-machine:9881"
    }
  ],
  "paths": {
    "/{user-id}/get-cart-contents": {
      "x-golem-worker-bridge": {
        "worker-name": "worker-${request.path.user-id}",
        "component-id": "2696abdc-df3a-4771-8215-d6af7aa4c408",
        "response": "${ { headers: { ContentType: \"json\", userid: \"foo\"}, body: golem:it/api.{get-cart-contents}(), status: 200u64 } }"
      },
      "get": {
        "summary": "Get Cart Contents",
        "description": "Get the contents of a user's cart",
        "parameters": [
          {
            "name": "user-id",
            "in": "path",
            "required": true,
            "schema": {
              "type": "string"
            }
          }
        ],
        "responses": {
          "200": {
            "description": "OK",
            "content": {
              "application/json": {
                "schema": {
                  "$ref": "#/components/schemas/CartItem"
                }
              }
            }
          },
          "404": {
            "description": "Contents not found"
          }
        }
      }
    }
  },
  "components": {
    "schemas": {
      "CartItem": {
        "type": "object",
        "properties": {
          "id": {
            "type": "string"
          },
          "name": {
            "type": "string"
          },
          "price": {
            "type": "number"
          }
        }
      }
    }
  }
}

The difference here is we added the server's block. You can include this servers block from the beginning so that it's exactly the same OpenAPI spec which you used to upload to Golem as well as Tyk.

Install Tyk

Terminal

git clone https://github.com/TykTechnologies/tyk-gateway-docker
cd tyk-gateway-docker
docker-compose up

Registration with Tyk

Let's say we saved the above json as open_api.json

Terminal

curl -X POST http://localhost:8080/tyk/apis/oas/import --header 'x-tyk-authorization: foo' --header 'Content-Type: text/plain' -d @open_api.json

Reload the Tyk API Gateway, otherwise the API is not deployed with Tyk yet, so this is an important step. Note that, if you are encountering issues following these steps, please refer to Tyk documentations.

curl -H "x-tyk-authorization: foo" -s http://localhost:8080/tyk/reload/group

Note that Tyk is now running at 8080, and now requests has to go into 8080 and not the Golem Worker Gateway.

Terminal

curl -X GET http://localhost:9006/adam/get-cart-contents

FAQ

How does worker service know which API definition to pick for a given endpoint?

When a request comes in, worker-service looks at the host in the request and matches it with the site in the deployment. If there is a deployment corresponding to the site, it picks the API definition ID and version from the deployment and gets the API definition, to further process the request

CLI Persistence