Julian KramerA GPU farm in the polder.
Data-centre GPUs on my own home fleet. When someone on Vast.ai wants them, they rent them by the hour. When nobody does, the same cards run and train our own models: the local LLM behind Carddex's listing classifier, and the small student models distilled from it.
Operational numbers from October 2026. The farm's books stay off this page.
Renters first, our own models in the gaps.
A card earns its keep by the hour, and spends every hour nobody pays for on our own work. Nothing sits idle, and a paying renter never waits.
Rented by the hour
The cards are listed on Vast.ai, a marketplace where anyone can rent a GPU by the hour. Renters get a sandboxed container behind a dedicated firewall, and my own network never sees them. A pricer sets the asks against a market view I collect myself, with bandwidth and disk priced on their own, because renters choose on the GPU and disk price.
Our own LLM when it's idle
The farm bids for its own spare cards at a floor price: a paying renter always outbids it, and whatever is left serves an open-weights Qwen model on vLLM. That model reads Carddex's marketplace listings and classifies them — language, condition, what's really for sale — with no per-token bill. When every card is rented, the work moves to Apple silicon, and after that it simply waits.
Teacher and student
The big model labels; a 0.6-billion-parameter student learns from its consensus, so the everyday work can one day run on far smaller hardware. Every candidate is scored on hand-checked gold sets and tracked in MLflow, and a model is only licensed when a statistical lower bound on its accuracy, over at least 150 checked samples, clears 90%. The current student runs in shadow next to the teacher until it earns that.
Uptime is the whole game
Vast ranks hosts on reliability, so a minute dark costs days of score. Renter traffic lands on a static IP at a small cloud edge and comes home over WireGuard; when the fibre drops, LTE takes over in about 15 seconds. Both rigs sit on a UPS, and probe tunnels on other paths tell a home-network fault from a tunnel fault.
Who gets a card, in order.
- Renters — on demand, by the hour, and always first in line.
- The production LLM — Carddex's listing classifier, on whichever cards are free, with Apple silicon as the fallback.
- Training and sweeps — a serial job queue that waits for a free card, survives a reboot, and records every run in MLflow.
The farm runs on the habits I bring to clients: everything as code in Proxmox, Ansible and Terraform, deployed through pipelines with a manual gate, monitored, and measured before anything changes, prices included. It's also where I learn, the hard way, what a GPU platform really needs: a power supply that survives a load spike, a tunnel that survives a flaky line, and a queue that survives a reboot.