Install

Run your first local AI model with Nodeau.

Turn a Linux/NVIDIA PC or an Apple Silicon Mac into a private, OpenAI-compatible local AI endpoint. Three commands, then a request.

free · no account needed

Full documentation · Linux install in detail · macOS install in detail

Nodeau runs on Linux with an NVIDIA GPU, and on Apple Silicon Macs through Metal. Both are qualified. A Mac runs standalone, and batch inference runs on Linux. On Linux it expects the graphics driver to be working before it starts. Everything else on this page it does for you. If something does go wrong, tell us. A report from a machine we have never seen is the most useful thing we get.

Before you start

What you need

Linux: Ubuntu 24.04 LTS on x86_64, installed normally (not in a virtual machine without GPU passthrough), with an NVIDIA GPU and a working driver already installed
macOS: an Apple Silicon Mac (M-series). No driver to install, and no password needed. Tested on macOS 26.5, and earlier versions are untested rather than unsupported, so the installer leaves the choice to you
About 20 GB free disk, since Nodeau is under 100 MB and the starter model is 2.6 GB. On Linux the container images add about 3 GB
An internet connection during installation and the model download
sudo on Linux for the setup step only. A Mac install needs no password at all

Current limits

Apple Silicon Macs run standalone, working happily on their own
Batch inference is Linux-only, and on a Mac it says so clearly
On a Mac, whether a model fits depends on what else is running, so the same request can be admitted today and correctly refused tomorrow
AMD and Intel GPUs are unsupported
Several GPUs per machine on Linux/NVIDIA, and a single model can be split across them when one card needs a hand. A Mac uses its one integrated GPU
The installer sets up one machine
llama.cpp runtime
No Windows, and no WSL yet

Nodeau has been run on an RTX 5070 Ti, an RTX 3080, an RTX 5060 Ti and an RTX 2080 under Ubuntu 24.04, and on an M3 Pro under macOS. On a card it has not measured directly it works out a cautious estimate and tells you that is what it did, instead of guessing quietly. Other NVIDIA cards are expected to work well. They simply have not been measured yet. See the roadmap.

What runs where

Four levels, and we use them literally. Recommended is what we run every day and measure against. Qualified means a real machine has been through the whole customer path and the exclusions below are known and listed. Experimental would mean it runs but we would rather you did not rely on it. Everything we ship has moved past that. Unsupported means Nodeau refuses rather than half-works.

Platform support. A level is a claim about evidence, not about intent.
Platform	Level	What that rests on
Linux · NVIDIA Ubuntu 24.04, x86_64	Recommended	Several machines and four NVIDIA cards, through reboots, a power cut and a GPU swap.
Apple Silicon M-series, macOS	Qualified	One machine, an M3 Pro on macOS 26.5, installed from the public path and serving on Metal. Earlier macOS versions are untested rather than unsupported.
One model across several GPUs Linux · NVIDIA, same machine	Qualified	One heterogeneous pair, an RTX 3080 and an RTX 5060 Ti, running a model whose weights fit on neither card alone. Other combinations are untested. It buys capacity, not speed. Not available on Apple Silicon, which uses its one integrated GPU.
Independent batch workers across several GPUs Linux · NVIDIA, same machine	Qualified	The same heterogeneous pair, with both cards observed running a model server at once and every record accounted for exactly once. A worker count is a request: workers that cannot get a card wait, and the job still completes. Not available on Apple Silicon, where batch inference is absent rather than untested.
Windows via WSL	Unsupported	The GPU path through WSL is genuinely different, and is refused rather than half-supported. Native Windows is not planned.
AMD and Intel GPUs	Unsupported	A different runtime and memory story, not a configuration flag.

What each qualified platform does. A dash is a capability that is not there, not one that is slower.
Capability	Linux · NVIDIA	Apple Silicon
The same on both
Local inference on your own GPU	Included	Included
Model catalog, download and verified cache	Included	Included
Will it fit? Admission runs before anything starts	Included	Included
OpenAI-compatible API	Included	Included
`run` · `ps` · `stop` · `logs` · `doctor`	Included	Included
Local dashboard	Included	Included
Accounts, plans and offline entitlements	Included	Included
Works with no internet once installed	Included	Included
Clean uninstall	Included	Included
Where the two differ
Several GPUs in one machine	Included	Not applicable one integrated GPU
Choosing where work runs Efficiency, Balanced or Performance, and why it chose	Included	Not available one GPU, nothing to choose
Power budgets and per-GPU limits a ceiling Nodeau will not schedule past, and a cap a card will not draw past	Included	Not available no equivalent control
Several machines as one fleet	Included	Not available runs standalone
Batch inference chat or embeddings, a file of requests at a time	Included	Not available Linux only, and says so clearly
Embeddings vectors for search and retrieval	Included	Not available not yet measured on a Mac
Reranking scoring documents against a query	Included	Not available not yet measured on a Mac
Structured output and tool calling JSON constrained to your schema	Included	Not available not yet measured on a Mac
Image understanding send a picture with your prompt	Included	Not available not yet measured on a Mac
Speech to text	Not available Linux only	Not available Linux only

On Apple Silicon the GPU shares one pool of memory with everything else, so whether a model fits depends on what else is running. The same request can be admitted today and correctly refused tomorrow. Nodeau shows you the arithmetic either way rather than leaving you to guess.

Check your graphics driver

Open a terminal and run this. It is the one thing Nodeau needs you to have working already.

bash

nvidia-smi

If you see a table with your graphics card in it, you are ready.

If it prints an error, stop here. Nodeau expects a working NVIDIA driver and will not install, replace or repair one. That decision affects whether your computer boots to a desktop, so it stays yours. Install your distribution’s driver first, or ask us.

Install the Nodeau CLI

This downloads one small program, verifies its checksum, and puts it in a directory you own. It does not use sudo and changes nothing else.

bash

curl -fsSL https://get.nodeau.ai/install.sh | bash

Want to read the installer first?

A good habit, and the script is short enough to make it practical.

bash

curl -fsSL https://get.nodeau.ai/install.sh -o install-nodeau.sh
less install-nodeau.sh
bash install-nodeau.sh

If your shell then says nodeau: command not found, close the terminal, open a new one, and carry on.

Prepare the machine

Nodeau checks the computer and sets up the local infrastructure it needs to run models.

bash

nodeau install

It shows you what it found and what it plans to do, lists every command that needs your password, and waits for you to agree. Ubuntu will ask for your password once or twice. This part can take several minutes.

Want to see the plan without doing anything? Run nodeau install --dry-run.

Nodeau will not replace your NVIDIA driver, disable Secure Boot, change how your computer starts up, touch your partitions or any Windows files, or delete a Kubernetes cluster it did not create. If it cannot proceed safely, it stops and says why.

Download a model and start it

bash

nodeau quickstart

Nodeau will:

offer the starter model, Qwen3.5-4B, Apache-2.0, about a 2.6 GB download
tell you the size and licence, and ask before downloading anything
verify the file against a known SHA-256
check the model fits safely in your GPU’s memory
start it, and bring up the local API

On a slow connection, the download is the long part.

Talk to it

Nodeau prints a working command when it finishes. It looks like this, and the key is unique to your machine.

bash

export NODEAU_API_KEY="$(nodeau auth show --quiet)"

bash

curl http://127.0.0.1:8080/v1/chat/completions \
  -H "Authorization: Bearer $NODEAU_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen3.5-4b-q4km",
    "messages": [{"role": "user", "content": "Say hello from my local GPU."}],
    "max_tokens": 2048
  }'

If text comes back, it worked. Your own graphics card generated it. The first reply can take a few seconds while the model warms up. Streaming works too: add "stream": true and use curl -N.

Use it from Python

It is an ordinary OpenAI-compatible endpoint, so the official client works by changing the base URL.

python

import os
from openai import OpenAI

client = OpenAI(
    base_url="http://127.0.0.1:8080/v1",
    api_key=os.environ["NODEAU_API_KEY"],
)

reply = client.chat.completions.create(
    model="qwen3.5-4b-q4km",
    messages=[{"role": "user", "content": "Say hello from my local GPU."}],
    max_tokens=2048,
)
print(reply.choices[0].message.content)

Set NODEAU_API_KEY first, as in the step above.

Afterwards

Day-to-day commands

`nodeau status`

What is running, and whether the endpoint is answering.

`nodeau doctor`

Read-only check of the machine and the installation. Changes nothing.

`nodeau stop qwen-local`

Stop the endpoint. Your downloaded model stays. Add --workload to free the GPU as well.

`nodeau run qwen-local`

Start it again.

`nodeau model list`

Which models Nodeau can serve, and what you have downloaded.

`nodeau uninstall`

Shows an uninstall plan and asks before changing anything. Keeps your models, your driver, and what was on the machine before Nodeau.

Every command takes --json if you are scripting it, and the exit code tells you which kind of answer you got: a refusal is not the same as a failure, and neither is “not yet”. Every command, with its flags, and the exit-code contract.

More than one machine

This guide sets up one machine, on purpose.

Nodeau’s control plane spans more than one GPU machine, on mixed NVIDIA hardware. Each machine reports what it has, checks its own copy of the model, and gets considered on its own merits when Nodeau decides where something should run.

This guide covers one machine because that is what the installer sets up today, and because it is the right place to start even if you have three. Several machines is a Home Pro and Business capability, and the single command that adds one is what we are working on now.

If you already have the hardware, we would like to talk to you while that is being finished.

What works across machines Talk to us about a fleet

Across several machines

More than one GPU machine under one control plane
Mixed NVIDIA generations, judged individually
Every machine considered, and the reasoning kept
Each machine verifies its own copy of the model

One thing Nodeau does not do anywhere: fail over on its own. It decides where a workload runs and leaves it there. See the roadmap.

If something goes wrong

Send us one file. Please don’t repair it by hand.

If you have to fix something manually, that is a bug we need to know about, and patching it up by hand hides it from us. We would much rather have the report.

If the CLI works

bash

nodeau support bundle

Writes one .tar.gz into the current folder. It contains no API keys, no credentials, no prompts and no model output, and it is sent nowhere, so you attach it yourself. It is an ordinary archive, so you are very welcome to open it and look first.

If it failed before that

path

~/.local/state/nodeau/install.log

Everything nodeau install prints is also written there. Send that file, or simply copy and paste whatever the terminal showed. Anything is better than nothing.

Before that, it is worth a look at troubleshooting, which is organised by what you are seeing, and what a support bundle contains.

Either way: founders@nodeau.ai, or use the contact form.

Ready?

Start at the driver check, then work down. It should take about ten minutes plus the download.

Back to step one Read the FAQ