Docs · Getting started

Your first model

One command takes a set-up machine to a local OpenAI-compatible endpoint. Here's every step it performs, and what each one tells you.

Install first: Linux or a Mac.

Linux / NVIDIA

Terminal

nodeau quickstart

Apple Silicon

On a Mac the same job is one run:

Terminal

nodeau run qwen3.5-4b-q4km

It resolves the model, asks before downloading, checks the fit, starts the model on the Mac's GPU through Metal and brings up the loopback endpoint. A Mac runs standalone and serves chat, embedding, reranking and vision models, several at once on ports of their own, and nodeau model list shows what you can run.

What you'll see#

Quickstart starts by telling you exactly what it wants to download, and asks:

Output

Recommended starter model

  Qwen3.5-4B Q4_K_M

  Download
    ~2.55 GiB

  License
    Apache-2.0

  Downloaded from
    huggingface.co/unsloth/Qwen3.5-4B-GGUF

  Nodeau checked your graphics card and expects this to fit.
  It will refuse and explain rather than crash if it turns out not to.

Download and start it? [Y/n]

Then it gets on with it, one step at a time:

Output

==> downloading Qwen3.5-4B-Q4_K_M.gguf
==> model ready: /var/lib/nodeau/models/Qwen3.5-4B-Q4_K_M.gguf
==> waiting for Nodeau to admit and start it (up to 10m0s)
    ...
==> starting the local endpoint
    ...

──────────────────────────────────────
Nodeau is ready.
──────────────────────────────────────

Model
  qwen3.5-4b-q4km

Local API
  http://127.0.0.1:8080/v1

  Only this computer can reach it. It is not on your network.

Below the banner it prints a curl command you can paste exactly as it is, with your API key pulled in by nodeau auth show rather than pasted into the text. The first reply can take a few seconds while the model warms up.

Then have a look for yourself:

Terminal

nodeau ps        # what's using a GPU, and where
nodeau status    # workloads, endpoints, and whether the port really answers

nodeau status reads nothing from a cache. The workload state comes from the cluster, the endpoint state from the service manager, and "answering" from a real connection to the port.

What quickstart does#

Every step is one of the other commands, so nothing here is magic and you can run each part on its own.

Picks the starter model from the curated catalog: a compact general model of about 2.6 GB that suits an 8 GB card.
Shows you the size, the licence and the source, and asks. Several gigabytes over your connection should always be your decision. --consent or --yes answers in advance, for a script.
Downloads the weights from the publisher over HTTPS, from a URL pinned to an immutable revision. An interrupted download resumes.
Checks the SHA-256 before the file is allowed to be used. A file that fails is moved aside, so the evidence survives and the next run can't mistake it for a good one.
Creates a GPUService, the declarative workload object.
Checks the fit. Nodeau predicts peak GPU memory from the model, the quantisation, the context window and the concurrency, and compares it with what your card can really offer. See how Nodeau decides.
Waits for the model to load into GPU memory and start serving.
Brings up the local endpoint, a loopback reverse proxy on 127.0.0.1:8080.
Prints a curl command that works, with your API key already wired in.

Running it again is safe: a model you already downloaded isn't fetched again, and an existing service is reused rather than recreated.

Options#

Flag	Default	What it does
`--model <id>`	`qwen3.5-4b-q4km`	Which model to install and serve
`--name <name>`	`qwen-local`	Name for the workload
`--namespace`, `-n`	`nodeau-dev`	Namespace to create it in
`--context-size <n>`	`4096`	Context window
`--parallel <n>`	`1`	Concurrent sequences
`--port <n>`	`8080`	Loopback port for the endpoint
`--timeout <d>`	`10m`	How long to wait for it to become ready
`--consent`	off	Agree to the download without being asked
`--yes`, `-y`	off	Answer yes to everything

Talk to it#

Terminal

export NODEAU_API_KEY="$(nodeau auth show --quiet)"

curl http://127.0.0.1:8080/v1/chat/completions \
  -H "Authorization: Bearer $NODEAU_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
        "messages": [{"role": "user", "content": "Say hello in one sentence."}],
        "max_tokens": 2048
      }'

If text comes back, it worked, and your own graphics card wrote it. Python, streaming and the rest are in the OpenAI-compatible API.

If quickstart stops#

Quickstart exits 0 when your endpoint is up. Otherwise it prints what happened and exits non-zero, and nothing it downloaded is thrown away. There are two common reasons.

Nodeau declined the model#

Quickstart prints the whole decision on the spot. To see it again at any time:

Terminal

nodeau service explain qwen-local

That shows the arithmetic the controller used when it decided: what the card can address, what else is on it, the safety reserve and what the model needs, plus a typed reason code and what to try next. It reads the recorded decision rather than recalculating it, so it always agrees with what the platform actually did.

The usual answers:

GPU_TOO_SMALL: try a shorter context (--context-size 2048) or a smaller model.
MODEL_UNVERIFIED or MODEL_VERIFICATION_PENDING: the checksum hasn't been confirmed yet. The second one clears by itself in a moment.
GPU_ALREADY_ALLOCATED: another workload holds the card. nodeau ps shows which.

Every code is listed in reason codes.

It was still starting when the wait ran out#

Nothing is rolled back. The model is probably still loading, and quickstart tells you what it last saw. nodeau status shows when it's ready, and nodeau run qwen-local brings the endpoint up once it is.

Stopping and starting again#

Terminal

nodeau stop qwen-local              # stop the endpoint, keep the model loaded
nodeau stop qwen-local --workload   # stop the model too, freeing the GPU
nodeau run qwen-local               # bring it back

Your downloaded model files stay put through all of these.

Next#

The OpenAI-compatible API: chat, streaming and Python
Embeddings, reranking, tools and vision: the other tasks
The model catalog: what else you can run
Add and run machines: bring in a second machine
CLI reference: every command

These docs describe the current release, the beta channel. Spotted something wrong or missing? Tell us. A report from a machine we have never seen is the most useful thing we get.