In this Update: Fika Works Day, LocalAI-ZTA
Fika Works Day#
On Friday I co-presented “Automatic Provisioning of Bare Metal Edge Systems for AI Inference” with William Rizzo at Fika Works Day. I had a killer demo lined up: bring the Radxa X4, boot Kairos, and show a zero-touch appliance on stage. Then, during the final tweaks, I fried the board. I stayed up way too late trying to revive it and eventually surrendered. The next morning I yanked a node from my homelab, took it on the train, and reprovisioned it on the fly. The other passengers definitely noticed. We still pulled off the demo, but between the stress and the lack of sleep it didn’t land as cleanly as I hoped. Maybe I should stick to VM demos… but the tactile vibe of real hardware is hard to give up.
LocalAI-ZTA#
The star of the show was the LocalAI-ZTA appliance, powered by LocalAI. Drop the LiveCD on a USB drive, boot once, and the installer takes over. On the second boot it self-configures LocalAI, announces itself via mDNS, and suddenly http://localai.local gets you a ChatGPT-style interface with whatever models you preloaded. If the box has network access you can pull more models on demand. LocalAI also mirrors the OpenAI API, so you can point your apps at either endpoint depending on how much privacy you want. I still get a kick out of seeing that workflow come to life on a tiny box.