Progress

A handler can report a fraction from 0 to 1 while it runs. The latest value is stored on the request and shown on the app’s Requests tab.

From the handler

import surfra

@surfra.handler
def predict(request: surfra.Request):
    surfra.progress(0.1, "downloading")
    data = download(request.input["url"])
    surfra.progress(0.5, "transcribing")
    text = model.transcribe(data)
    surfra.progress(1.0, "done")
    return {"text": text}

surfra.progress(fraction, message) reports for the request that is running now. request.progress(fraction, message) does the same on a specific request. Calling surfra.progress outside a handler raises RuntimeError. A fraction outside 0 to 1 raises ValueError.

From inside the model

Many libraries take a callback that they call during generation. Decorate that callback with @surfra.progress and every call records its first argument.

@surfra.progress
def on_step(fraction: float):
    return fraction

@surfra.handler
def predict(request: surfra.Request):
    return pipeline(request.input["prompt"], callback=on_step)

pipeline calls on_step(i / total), and Surfra records each value on the current request.