Sovereign AI: Running Useful, Self-Hosted Agents on Modest Hardware

Slides from Brady Dibble's talk at SeaGL 2026, Friday 2026-10-23. The demo clips on slides 10 and 16 play in place.

Companion page: every command and numberDownload PowerPoint (18 MB)

Arrow keys move between slides. Each slide number links to that slide.

Slide 1: Sovereign AI
1
Slide 2: Brady Dibble
2
Slide 3: “
3
Slide 4: Privacy
4
Slide 5: Today, October 23, OpenAI retires 12 API models.
5
Slide 6: Sovereign AI is AI you run and control.
6
Slide 7: Models come closed, open weight or fully open source.
7
Slide 8: Every option trades one set of problems for another.
8
Slide 9: On shell commands a small local model nearly tied Claude. On homelab questions it did not.
9
Slide 10: A 2018 mini PC fixed real desktop problems, and the checks caught its miss.
10
Slide 11: Ten layers sit between a chip and a working agent.
11
Slide 12: Runtimes come in two families.
12
Slide 13: On its own, a model is inert: a file behind an API.
13
Slide 14: The same model passed or failed depending on the setup around it.
14
Slide 15: We fixed our own harness and gateway before open models worked well.
15
Slide 16: Qwen3.5-4B in pi fixed a bug from one sentence.
16
Slide 17: A good harness covers for what a small model lacks.
17
Slide 18: What makes an agent useful is what makes it dangerous.
18
Slide 19: On small machines, reading a long prompt is most of the wait.
19
Slide 20: Smaller quants ran faster here but lose quality.
20
Slide 21: Models bigger than your RAM still run, slowly.
21
Slide 22: Reported, not measured: a gaming GPU runs 27B to 35B models fast.
22
Slide 23: One step up: a 128 GB Strix Halo box built our company wiki.
23
Slide 24: Anyone can use HPC techniques to join cheap machines and run bigger models, if the network is fast.
24
Slide 25: Our three test mini PCs boot from one image over the network.
25
Slide 26: The network decides: the same four Macs ran 16 tok/s over TCP and 28 with RDMA.
26
Slide 27: Start tonight with three installs.
27
Slide 28: Run your own and you keep four things.
28
Slide 29: Questions
29