Docs · Fleet

Health and alerts

Five commands answer five different questions. Pick the right one and a problem usually names itself.

Which command answers which question#

doctor is the one to start with. It is read-only, it names problems with typed codes, and every result that is not a pass comes with something to do about it.

nodeau doctor#

nodeau doctor
nodeau doctor --json          # every result, code, explanation and remedy
nodeau doctor --host-only     # this machine only, without contacting the cluster
nodeau doctor --node nodeau-c # one machine, plus the installation-wide checks

Every check is read-only. Doctor looks and reports; it never installs, configures, mounts, starts or stops anything.

It covers two halves:

• The host: is this machine able to run Nodeau? Operating system, processor, memory, whether it is bare metal, storage safety, the state directory, Secure Boot, the GPU on the PCI bus, the driver, VRAM and network.
• The installation: is Nodeau itself working? Its control plane, the agent on each machine, whether every GPU report is fresh enough to schedule against, whether the models on disk still verify, and what your services and batch jobs are doing.

Exit 0: everything passed, or only warnings. Exit 1: something failed. Exit 2: the command was called wrongly.

Doctor codes#

Stable, machine-readable, and identical in --json and in the human report.

Reaching the cluster

Hardware

Models

Workloads

Local endpoints

Account and plan

On a Mac

Fixes for each are in troubleshooting.

nodeau health#

nodeau health
nodeau health --json
nodeau health history --machine nodeforge --since 6h

The current state of every machine, processor, memory, storage and network, next to its GPU readings, together with anything that needs attention.

nodeau-c  (worker, ready, reported 6s ago)
    CPU              0 %            16 threads, 0 % stalled
    memory           12 %           1.9 GiB of 15.5 GiB in use
    storage used     21 %           709.7 GiB free on /dev/nvme0n1p2
    network          7.5 KiB/s in   2.9 KiB/s out, counting enp6s0, wlp5s0
    GPU              NVIDIA GeForce RTX 3080, 44 MiB of 9,877 MiB in use, healthy

Every reading is taken by the machine it describes, and stays on your own hardware. The fleet console shows your machines and their state; these readings stay local.

Missing is not zero#

A value shown as "not collected" is missing, which is a different answer from zero. A card reporting 0% utilisation is idle; a card Nodeau could not read is unknown, and the two never look the same.

health history follows the same rule: an interval with no observations is absent rather than zero, so a gap in the output is a gap in what was observed. Nodeau leaves it as a gap.

history flags#

Retention#

24 hours, the same on every plan.

History lives in the fleet's datastore, which is the control-plane machine's disk. It stays on that one machine, so if that disk is lost, the history goes with it.

What health reports, and where it stays#

• Storage is how full each filesystem is. Drive health data such as SMART is outside what Nodeau reads, so keep your usual disk tools for that.
• Figures are per machine and per card. Per-process figures, process lists, command lines and connection tables are outside what it collects.
• Readings stay on your hardware. Your account shows your machines and their state; the readings stay local.
• Alerts appear where you look: in nodeau health and the local dashboard. Delivering them by email or webhook is on the roadmap.

Alerts#

The controller watches a small set of conditions and reports what needs attention:

An alert has three states:

Only Active counts anywhere. No badge, tile or nodeau health line moves for a Pending alert, so a brief blip never reads as a problem.

nodeau status#

nodeau status

Everything here is read fresh. The workload state comes from the cluster, the endpoint state from the service manager, and "answering" from an actual connection to the port, so a stale record can never make it report something untrue.

It also lists anything on the machine that makes a model reachable from your network. Nodeau binds only 127.0.0.1, so if something shows up here, it came from somewhere else and is worth a look.

nodeau ps#

nodeau ps            # everything holding a GPU
nodeau ps --all      # including finished batch jobs
nodeau ps -A         # across all namespaces

Services and batch jobs alike, with where each one is and what it is doing.

NAME        TYPE     MODEL              STATE    GPU                         MACHINE    PROGRESS  AGE  ENDPOINT
qwen-local  service  qwen3.5-4b-q4km    serving  NVIDIA GeForce RTX 5060 Ti  nodeforge  1/1       4d   http://127.0.0.1:8080/v1

nodeau logs#

nodeau logs qwen-local -f
nodeau logs my-batch
nodeau logs --system nodeau-controller

Your workloads are looked up by name. On Linux, Nodeau's own components are under --system.

Prompts, completions and batch records are never written to a log, so they can never appear here.

The local dashboard#

nodeau dashboard

The same picture in a browser: hardware, models, running services, alerts and plan. It binds loopback, asks for a token, is compiled into the binary, and works with no internet connection at all.

It is read-only apart from one action: it can stop a batch job.

These docs describe the current release, the beta channel. Spotted something wrong or missing? Tell us. A report from a machine we have never seen is the most useful thing we get.