First endpoint

1. Write the handler. Warmup runs once when the worker starts. The handler runs once per request.

   import surfra
   
   model = None
   
   @surfra.warmup
   def load():
       global model
       model = load_model()
   
   @surfra.handler
   def predict(request: surfra.Request):
       prompt = request.input["prompt"]
       surfra.progress(0.2, "generating")
       return {"text": model.generate(prompt)}

2. Build an image that includes this file and the SDK. The container command is the Surfra worker.

   CMD ["python", "-m", "surfra.worker"]

   Set SURFRA_ENTRYPOINT to the path of the handler file. The autoscaler sets that variable, along with the replica token, when it starts the container.

3. Publish the app. The name is the stable id. Deploying the same name again updates it.

   import surfra
   
   app = surfra.App("hello-model")
   app.deploy(
       image="ghcr.io/example/hello-model:latest",
       entrypoint="python -m surfra.worker",
       autoscale={
           "min_replicas": 0,
           "max_replicas": 1,
           "gpu": "rtx4090",
           "providers": [
               {"provider": "runpod", "gpu": "rtx4090", "priority": 1, "num_workers": 1},
           ],
       },
   )

request.input is the JSON object the caller sent. Return a JSON-serializable value. surfra.progress(fraction, message) reports a fraction from 0 to 1 while the handler is running.