Monitoring
Open an app from Serverless to see its tabs.
Overview
Section titled “Overview”Status, hourly rate, running workers, and copyable snippets for calling the app. The facts list shows the app id, image, runtime, GPUs, and replica range. Below that are the last 24 hours of requests and the current worker counts: ready, running, unhealthy, and queued.
Metrics
Section titled “Metrics”Charts for a time range you choose, in buckets from 5 minutes to 6 hours. Turn on Live to refresh every 15 seconds.
| Chart | What it shows |
|---|---|
| Requests | Completed, failed, and in-progress requests per bucket |
| Execution time | p50, p70, and p90 time from a worker taking a request to finishing it |
| Delay time | p50, p70, and p90 time a request waited in the queue |
| Cold start count | Replicas that started in each bucket |
| Cold start time | How long those replicas took to become ready, including image pull and warmup |
Long delay times with short execution times mean the app needs more replicas or workers. Raise max_replicas, concurrent_workers, or min_replicas.
Requests
Section titled “Requests”Every request with its status, when it arrived, how long it waited in the queue, how long it ran, and which worker ran it. Click a row to see its input and output.
Everything your app writes to stdout and stderr, from every replica. Each line shows its time to the millisecond, its stream, the replica, and the request it belongs to when Surfra can tell. sys lines come from Surfra itself, such as the server starting or becoming ready.
- Search matches lines that contain every word you type.
- Request ID shows one request’s lines. Click a request id in the table to filter by it.
- Stream shows only
out,err, orsyslines. - Live refreshes every 5 seconds.
Workers send lines about once a minute, so the newest lines can take up to 90 seconds to appear. Logs are kept for 24 hours. See Worker process for how lines are tagged with requests.
Failures, below the logs, lists errors from failed requests and from replicas that could not start, such as an image that would not pull or a server that never became ready.
Workers
Section titled “Workers”Two tables. Workers lists each worker with its GPU, provider, last heartbeat, missed heartbeats, failures in a row, and the request it holds. A worker that misses three heartbeats is marked unhealthy and its request returns to the queue.
Replicas lists each machine with its provider, GPU, hourly rate, the amount it has accrued, and the last error it reported.
Releases
Section titled “Releases”Every save of the app with what changed: image, entrypoint, runtime, replica limits, timeouts, and the priority list.