Docs · Getting started

The API and your first request

Nodeau gives you an OpenAI-compatible HTTP endpoint on 127.0.0.1. Any client that can talk to OpenAI can talk to it, with a base URL and a key.

The endpoint#

http://127.0.0.1:8080/v1

The port is whatever nodeau run or nodeau quickstart used — 8080 unless you chose otherwise or that port was busy. nodeau ps and nodeau status print the real one.

It binds 127.0.0.1 and nothing else. Not 0.0.0.0, not a LAN address, and the bind address is not a setting. Installing Nodeau must never put an inference endpoint, or a GPU, on somebody's home network by accident.

Authentication#

Every request needs a bearer token. The endpoint forwards the Authorization header to the model server, which is what enforces it — so that any process on the machine cannot use your GPU just by knowing the port.

export NODEAU_API_KEY="$(nodeau auth show --quiet)"

nodeau auth show prints your local key, creating one on first use. It is generated with a CSPRNG, stored 0600 in your Nodeau config directory, and never leaves the machine.

That is not automatically the key a particular service enforces — a service can reference a credential Nodeau did not publish. To get the token that will actually be accepted:

export NODEAU_API_KEY="$(nodeau auth token qwen-local)"

That one checks which credential the service enforces, and refuses rather than printing one that would be rejected. See the two kinds of credential.

Chat completions#

curl http://127.0.0.1:8080/v1/chat/completions \
  -H "Authorization: Bearer $NODEAU_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
        "messages": [
          {"role": "system", "content": "You are concise."},
          {"role": "user",   "content": "What is a GGUF file?"}
        ],
        "max_tokens": 2048
      }'

The model field#

One Nodeau endpoint serves exactly one model, so model is optional. If you do send it, it is checked: a request naming a different model is refused with 404 invalid_request_error / model_not_found rather than answered by the model that happens to be there.

{"error":{"message":"This Nodeau endpoint serves \"qwen3.5-4b-q4km\", not \"gpt-4\". One endpoint serves exactly one model, so the request was refused rather than answered by a different one.","type":"invalid_request_error","param":"model","code":"model_not_found"}}

Run a second model to get a second endpoint on a second port.

Token budgets#

max_tokens is the number of tokens the model may generate. On a reasoning model the internal monologue is drawn from that same budget before any answer exists, so asking for a short answer by asking for few tokens is exactly backwards: a short answer from a reasoning model needs a large budget, because the answer is what is left.

• Nodeau's own default, when Nodeau is the one choosing, is 2048.
• Below 512 it will tell you the budget is too small to be an answer.
• Structured output has its own floor of 1024, because the constrained part cannot begin until the thinking is done.

Nodeau never rewrites a budget you set. A client that asked for 160 asked for 160, and silently raising it would make the usage block a lie.

Streaming#

curl -N http://127.0.0.1:8080/v1/chat/completions \
  -H "Authorization: Bearer $NODEAU_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"messages":[{"role":"user","content":"Count to five."}],
       "max_tokens":2048,"stream":true}'

Server-sent events, in the ordinary OpenAI shape, flushed as they arrive.

Python, with the OpenAI SDK#

from openai import OpenAI

client = OpenAI(
    base_url="http://127.0.0.1:8080/v1",
    api_key="<paste the output of: nodeau auth show --quiet>",
)

reply = client.chat.completions.create(
    model="qwen3.5-4b-q4km",          # must match what this endpoint serves
    messages=[{"role": "user", "content": "What is a GGUF file?"}],
    max_tokens=2048,
)
print(reply.choices[0].message.content)

Any OpenAI-compatible client works the same way: point base_url at http://127.0.0.1:8080/v1 and give it the key.

Which routes an endpoint answers#

A workload is started for a task, and it answers that task's routes and refuses the others. This is not a security boundary — it is there because the model server underneath does not refuse: an embedding model asked for a chat completion returns HTTP 200 and fluent-looking nonsense.

A request to another task's route gets 404 with invalid_request_error / model_not_found, and a message saying what this endpoint does answer. A route Nodeau does not know about is forwarded unchanged.

nodeau run <embedding-model> --task embed --port 8081

Embeddings#

curl http://127.0.0.1:8081/v1/embeddings \
  -H "Authorization: Bearer $NODEAU_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"input": ["a sentence to embed", "and another"]}'

The curated catalog has a dedicated embedding model on the 8 GB rung.

Reranking#

curl http://127.0.0.1:8082/v1/rerank \
  -H "Authorization: Bearer $NODEAU_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"query": "how do I add a machine",
       "documents": ["nodeau fleet invite prints a code",
                     "GGUF is a model file format"]}'

Tool calling#

Ordinary OpenAI tool calling: you pass tools, the model may answer with tool_calls, and you execute the tool and send the result back. Nodeau does not execute anything.

curl http://127.0.0.1:8080/v1/chat/completions \
  -H "Authorization: Bearer $NODEAU_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [{"role":"user","content":"What is the weather in Oslo?"}],
    "max_tokens": 2048,
    "tools": [{
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Current weather for a city",
        "parameters": {
          "type": "object",
          "properties": {"city": {"type": "string"}},
          "required": ["city"]
        }
      }
    }]
  }'

Not every model can do this. A curated model's capabilities are listed by nodeau model info <model>; an imported model claims a capability only after it has proved it.

Structured output#

curl http://127.0.0.1:8080/v1/chat/completions \
  -H "Authorization: Bearer $NODEAU_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [{"role":"user","content":"Describe a GPU as JSON."}],
    "max_tokens": 2048,
    "response_format": {
      "type": "json_schema",
      "json_schema": {
        "name": "gpu",
        "schema": {
          "type": "object",
          "properties": {"name": {"type":"string"}, "vram_gb": {"type":"integer"}},
          "required": ["name", "vram_gb"]
        }
      }
    }
  }'

Image input#

Multimodal models take an image in the ordinary OpenAI content-parts shape:

{
  "messages": [{
    "role": "user",
    "content": [
      {"type": "text", "text": "What is in this picture?"},
      {"type": "image_url", "image_url": {"url": "data:image/png;base64,iVBORw0KG..."}}
    ]
  }],
  "max_tokens": 2048
}

The curated catalog has multimodal models on the 8 GB and 12 GB rungs.

What is not here#

There is no supported speech, transcription, audio, text-to-speech or image-generation capability, and no /v1/batches. Nodeau's own batch inference is a different thing with its own CLI and is not OpenAI Batch API compatible.

Errors#

An empty content with "finish_reason": "length" is not an error — it is the token budget running out before an answer existed. Raise max_tokens.

Changing the port#

nodeau run qwen-local --port 8123

Reaching it from another machine#

Nodeau does not do this for you, and will not put a GPU on your network by accident. If you want a model reachable from elsewhere on a network you control, that is your own reverse proxy in front of the loopback endpoint, with your own TLS and your own authentication — and the security page is worth reading first.

Documentation for the current published build, the beta channel. Something here wrong or missing? Tell us — a report from a machine we have never seen is the most useful thing we get.