HTTP servers

The HTTP engine runs a server you already have. Surfra starts your server command inside the replica, waits until it is ready, and forwards each queued request to it over localhost. Your server never faces the internet. Callers go through Surfra with an API key.

Build the image

Install the Surfra supervisor in your image and make it the entrypoint. It is a small static binary for Linux on amd64 and arm64, published on GitHub. The install script picks the right one and checks its checksum.

Dockerfile

FROM vllm/vllm-openai:latest
RUN curl -fsSL https://surfra.io/install.sh | sh
ENTRYPOINT ["surfra", "worker"]

Use wget -qO- https://surfra.io/install.sh | sh in images that have wget but not curl. In images with neither, such as python:3.12-slim, download the binary directly:

ARG TARGETARCH
ADD --chmod=755 https://github.com/surfra-io/surfra-worker/releases/latest/download/surfra-linux-${TARGETARCH} /usr/local/bin/surfra

Set SURFRA_VERSION=v0.2.1 before sh to pin a release.

Each release also has .deb, .rpm, and .apk packages for both architectures, such as surfra_0.2.1_amd64.deb, if you would rather install it with the system package manager.

The supervisor reads the app’s settings when the replica starts, so the same image works with different models and ports.

Configure the app

import surfra

app = surfra.App("qwen")
app.deploy(
    image="ghcr.io/me/vllm-surfra:1",
    entrypoint="vllm serve Qwen/Qwen3-0.6B --host 127.0.0.1 --port 8000",
    runtime={
        "engine": "http",
        "port": 8000,
        "path": "/v1/chat/completions",
        "ready_path": "/health",
        "ready_timeout": 1800,
    },
    env={"HF_TOKEN": "hf_..."},
    autoscale={
        "max_replicas": 3,
        "execution_timeout_sec": 600,
        "providers": [{"provider": "runpod", "gpu": "rtx4090", "priority": 1}],
    },
)

Runtime field	Meaning	Default
engine	"http"
port	The port your server listens on	8000
path	Where a queued request is sent	/
ready_path	Polled until it answers 200 before work is sent	/health
ready_timeout	Seconds the server has to become ready, including model downloads	600

entrypoint is the command that starts your server. Bind it to 127.0.0.1. If the command exits, the replica stops and Surfra starts a new one. The last lines the server printed before a failed start are shown under Logs.

execution_timeout_sec is how long one request may take. A gateway call waits that long before it returns 504.

Call it

An HTTP app answers three ways. All of them need Authorization: Bearer $SURFRA_API_KEY.

OpenAI-compatible. Point any OpenAI client at /openai/{app}/v1:

import os
from openai import OpenAI

client = OpenAI(base_url="https://api.surfra.io/openai/qwen/v1", api_key=os.environ["SURFRA_API_KEY"])
reply = client.chat.completions.create(
    model="Qwen/Qwen3-0.6B",
    messages=[{"role": "user", "content": "Say hi"}],
)
print(reply.choices[0].message.content)

Any path. /http/{app}/{path} forwards to that path on your server:

curl https://api.surfra.io/http/qwen/v1/models \
  -H "Authorization: Bearer $SURFRA_API_KEY"

The queue. app.run(payload) posts the payload as JSON to path.

Every gateway call is a queued request, so it scales replicas, retries on failure, and shows up in Metrics like any other call. Bodies must be JSON and at most 8 MB. Streaming responses are not supported yet. Send "stream": false.

Exposed ports

http_ports publishes container ports directly on providers that support it, and /v1/apps/{app}/http/{path} proxies to a ready replica round-robin, without the queue. With ports set, Surfra adds a replica for every max_concurrent requests in flight. Use this for servers that need low latency and handle their own concurrency.

autoscale={"http_ports": [8000], "max_concurrent": 32, "providers": [...]}

Only some providers can publish ports. See Machines and providers.

Ready-made images

The SDK repository has Dockerfiles for vLLM, SGLang, TGI, Ollama, and a plain Python echo server under examples/docker/. The dashboard’s templates use them.