Monitoring

Français · Type: how-to + reference · Audience: admin · Context: hosted Studio · Status: ready · Applies from: 4.2.3

The Studio Server (hosted team Studio) · contract: experimental · verified by CI (support matrix of 4.4.0, 2026-10-23): Postgres, S3 and Keycloak in CI; Chromium and Firefox (job studio-hosted); WebKit on macOS, by hand only · automated checks: jobs studio-hosted, studio-install-smoke; CI plan studioHosted; 3 test files · people's passes: studio-hosted not run (#149), studio-collab not run (#150), studio-preview not run (#166), studio-assets-hosted not run (#167), studio-mcp-remote not run (#196), studio-oidc-generic not run (#197) · known limits: none listed on this page

What to watch on a hosted Studio, and how: the probes, the Prometheus metrics behind a token, the logs, and the alerts that should reach a person.

What you get

A scraper reading /metrics with its token, a prober on /readyz, Clever Cloud's deployment notifications, and six alerts sent to a person: no instance serving writes, the journal not shipping, the worker piling up, jobs failing, members waiting on each other, the daily backup missing.

Context and versions

• A hosted Studio on Clever Cloud. /metrics and STUDIO_METRICS_TOKEN exist from 4.2.3; the per-phase preview metrics and log fields from 4.4.0.
• Verified on: 19 October 2026, by the reference deployment, for the route only (engine commit c66fbf56, package.json 4.2.2 before the 4.2.3 tag): /metrics answered 401 without a token or with a wrong one, 200 with it (text/plain; version=0.0.4), and 401 on the game's origin. Not verified: no collector and no alert were set on the test organisation, and no alert was triggered on purpose.

Prerequisites

• STUDIO_METRICS_TOKEN set on the studio app, 32 characters or more (Configuration).
• A scraper you run (Prometheus with its Alertmanager, or any service that reads the Prometheus text format), or at least an HTTP prober. Clever Cloud does not read /metrics.

Where to run the commands

clever notify-email, clever logs: on your machine, in the engine repository where the apps are linked. The scraper: on a machine of yours that reaches https://studio.example.fr.

Steps

1. Check the route answers with the token, without printing it:

   # on your machine, the token in a 0600 file
   curl -s -o /dev/null -w '%{http_code}\n' -H "Authorization: Bearer $(cat ~/secrets/my-game/metrics-token)" https://studio.example.fr/metrics
   

   200 with the token; 401 with none or another (compared in constant time); 404 when the app has no STUDIO_METRICS_TOKEN (the route does not exist). It is never behind the session, and never on the game's origin, where any request without the game's session is refused before its path is read (401, "open the game from the Studio").

Guarantee · metrics-token-required · referenced · executed on aff6caab7514 · scope self-checked by the author (AI), not reviewed /metrics answers only to Authorization: Bearer <STUDIO_METRICS_TOKEN>: without the token or with another, a 401, the token compared in constant time.

2. Point your scraper at it, with the token as a bearer credential. For Prometheus:

   # prometheus.yml, on the machine that runs Prometheus
   scrape_configs:
     - job_name: my-game-studio
       scheme: https
       metrics_path: /metrics
       authorization:
         type: Bearer
         credentials_file: /etc/prometheus/my-game-metrics-token
       static_configs:
         - targets: ['studio.example.fr']
   

3. Have Clever Cloud tell you about failed deployments (webhooks too: clever webhooks add):

   # on your machine, in the engine repository
   clever notify-email add --notify you@example.fr --event DEPLOYMENT_FAIL studio-deploys
   

   Clever's console also shows each app's CPU, memory and logs.

4. Set the alerts, to a person:
   • /readyz is not 200 for 2 min: no instance serves writes. A redeploy keeps it at 503 for 12 to 20 s, the first deploy onto 4.2.3 or later for about 2.5 min: expect the alert then (Redeploys).
   • studio_journal_rpo_seconds above 60 for 5 min (the target is 1 min, D42), or ship-failed keeps rising.
   • studio_worker_jobs_in_flight above 3 for 10 min: the worker runs one job at a time per lane, two lanes, so work is piling up.
   • Jobs fail: studio_worker_job_seconds_count{outcome!="ok"} rises, or health-failed does.
   • studio_lease_conflicts_total jumps: people waiting on each other's spans. Not a fault, a sign to look at.
   • The daily backup's line ("event":"backup") is missing from the logs one morning.

Expected result, and how to check it

The scraper shows the target up, studio_service_lease_held at 1 and studio_journal_rpo_seconds at 0 when nobody writes. To test the path to a person, trigger one alert on purpose (stop the studio app a few minutes off hours, or lower a threshold) and time how long it takes to reach you: that has not been measured by this project.

Common errors

Files or data changed

Your scraper's configuration and the token's file on its machine; Clever Cloud's notification settings. Nothing in the Studio.

Recovery or going back

Remove the token (clever env rm STUDIO_METRICS_TOKEN --alias my-game-studio, then redeploy): the route is gone (404). Changing it is Rotating keys.

Reference: the metrics

They are counts and durations: no path, no name, no author, no content. They live in the instance's memory, so a restart starts them again from zero. The worker's own queue is not measured from the worker: the Studio's side of it is.

OpenTelemetry is not supported (it was left for later in 4.2.3, and 4.4.0 did not add it).

Reference: the probes and the logs

• GET /livez: 200 while the process answers. GET /startz: 200 once the server has started (Clever's health check). GET /readyz: 200 when this instance holds the service lease and serves writes, 503 ("writes":"waiting") otherwise, with rpoMs. None needs a session or gives data.
• The logs are one JSON line per event, on standard output (clever logs): request (method, path, status, ms), lease-taken, lease-lost, backup, ship-failed, health-failed, preview-failed, and since 4.4.0 a "event":"preview" line with queuedMs, snapMs, jobMs, storeMs and sinceWriteMs, a "event":"worker-call" line per job (the Studio's side) with putMs, postMs, getMs and delMs, and the worker's own worker-job line with queueMs (from its POST received to its run's start), fetchMs, unpackMs, childMs and putMs. The worker logs a refused job as worker-job-refused with its reason.
• clever logs follows until it is stopped, even with --until: run it in the background and stop it by its process id.

References

• studio-server/src/server/metrics.ts, studio-server/src/server/http.ts; the test: tests/studio-metrics.test.ts.
• STUDIO_METRICS_TOKEN; for members: Health and metrics routes.
• docs/dev/QUALIF-4.2.3-REFERENCE.md §2.

Edit the source of this page · Report an error