Core concepts
An app is a container image plus the settings to run it: the entrypoint, the engine, environment variables, and autoscale settings. Each app has a name that is unique in your account and an id that starts with sfa_. The API and SDK accept either.
Engine
Section titled “Engine”The engine decides how a replica runs your code.
- Python starts
surfra worker, imports your entrypoint file, and calls its@surfra.handlerwith each request. - HTTP starts your server command, waits until its ready path answers, and forwards each request to it. See HTTP servers.
Replica
Section titled “Replica”A replica is one running copy of your image on one machine, such as one RTX 4090 on Salad. Surfra starts replicas when work is queued, keeps min_replicas running, and stops idle ones after idle_timeout_sec. You are billed per second while a replica runs.
Worker
Section titled “Worker”A worker is a loop inside a replica that pulls requests and runs them. A replica runs concurrent_workers of them. In isolated mode each worker is its own process with its own copy of the model. In joint mode they are threads that share one copy. See Batching and concurrency.
Request
Section titled “Request”A request is one call to an app. It waits in the queue until a worker leases it. If the worker fails or stops heartbeating, the request goes back to the queue and another worker retries it, up to three attempts.
Statuses: queued, leased, succeeded, failed. A request inside a workflow can also be blocked until the stages it depends on finish, or canceled when one of them fails.
Transaction
Section titled “Transaction”A transaction groups the requests from one call. app.run() creates a transaction with one stage. A workflow run creates one stage per step. The transaction succeeds when every stage succeeds.
Priority list
Section titled “Priority list”Each app ranks the machines it may use, such as Salad RTX 3090 first and RunPod RTX 3090 second. A new replica tries the rows in order and takes the first one that can place it. If a provider is out of capacity, the next row is tried. See Machines and providers.
Runner
Section titled “Runner”A runner is a computer you already own, registered with surfra runner register. It can appear in an app’s priority list like any provider. See Your machines.