Good hardware deserves better than sitting idle.

From a graphics card to AI infrastructure

Between a GPU and a model you can call sit drivers, runtime libraries, model formats, quantisation, containers, a model server, an API, authentication and something to keep it all running. None of that is why anybody wanted to run a model.

We built Nodeau so that whole stack becomes a couple of commands. And then, because most people who own one good GPU end up owning two, so that a handful of machines behaves like one private AI cloud.

Decide first, then start

Two copies of the same model can need very different amounts of memory depending on how they were compressed, how much context you give them and how many requests they handle at once. So before Nodeau starts anything, it works out what this exact configuration needs and compares that with what the GPU can really give.

Where the configuration has been measured on hardware like yours, Nodeau uses the measurement. Where it hasn't, Nodeau works out a deliberately cautious figure from the model's own shape, keeps extra room in reserve, and marks the decision as an estimate. That's what lets Nodeau run well on cards we've never touched, and you can always see which kind of number you got.

The memory a model can really use

A card sold as 10 GB gives a workload somewhat less. Part of it sits outside what the compute API can reach, your desktop uses some for every window on screen, and a model that fits with nothing to spare fails the moment anything else asks for memory.

So Nodeau plans with the memory a model can actually use, keeps a small reserve, and shows you the arithmetic. It's the difference between a model that runs all day and one that stops halfway through loading.

One GPU needs a fit check. Several need a choice.

With one machine the question is simple: does this model fit here? Add a second and a new one appears: which of these should run it?

Knowing that a machine "has a GPU" is enough to start something and nowhere near enough to know it'll keep running. Two machines can both report one GPU and be nothing alike: different usable memory, different amounts already taken by the desktop, a different copy of the model on disk. Nodeau asks each machine what it really has, checks its copy of the model by hash, runs the same fit check on every candidate, and then picks by the scheduling mode you chose. Every candidate keeps the reason it won or lost.

And anything already running happily stays put. A better score somewhere else is no reason to interrupt something that works.

Why owned hardware

Running models on hardware you control keeps them close to your data and your applications, gives you capacity you can plan around, and puts machines you already own to work. Quite often it also means using a card you bought for something else entirely.

The economics follow utilisation. A GPU that's busy most of the day makes the strongest case for owning it, and that's exactly the kind of machine Nodeau helps you keep busy.

The promises we keep

• Every number says where it came from. Measured or estimated, the decision tells you, and nothing is dressed up as more certain than it is.
• Running work stays where it was placed. Nodeau holds a workload in place rather than moving it on a guess. If a machine goes offline, its work stops with it, the way it would on any single computer, and you choose what happens next.
• Your machines stay yours. Your driver, your boot setup and your disks are left exactly as you set them up, and uninstalling removes only what Nodeau added.
• Built for people who trust each other. Nodeau is designed for machines you and your team control, where everyone with access to a machine is trusted.

Measured on our own machine

A few numbers from our development machine, an RTX 3080 running an 8B model. They describe that machine and nothing else, and yours will differ.

That last one is why Nodeau keeps its control plane out of the request path: restarting Nodeau's own pieces left every request answered.