A Show HN post is getting traction. Developer ThinkOffApp stuffed a Qwen model into a Raspberry Pi, wired it to the car's OBD port and the manufacturer's cloud API, and built a local Agent that can answer "what type of oil should I use" completely offline.
"I have it running an impressively intelligent and stable 35B Qwen model on a Raspberry Pi, hooked up to the car's OBD-II port for internal state data and to the manufacturer's cloud for remote control of things like AC and door locks. It's running a car manual knowledge base so it can answer questions like when should I add oil and what kind of oil even offline."
Project link: ThinkOffApp/CarWatch.
Three things worth unpacking: (1) a 35B model on a Pi is counterintuitive; (2) the local-plus-cloud Agent collaboration pattern; (3) cars might be the first large-scale proving ground for edge Agents.
Let's break each down.
What the project actually does
The build:
- Local LLM: Raspberry Pi runs Qwen 35B for conversation and tool calling.
- OBD-II hookup: an ELM327-compatible adapter reads coolant temp, fuel level, RPM, and diagnostic trouble codes.
- Manufacturer cloud: official APIs to control AC, door locks, and windows remotely.
- Knowledge base: the full vehicle manual (hundreds of PDF pages) indexed for RAG retrieval.
- Agent collaboration: real-time and privacy-sensitive tasks stay local; heavy reasoning gets routed to the cloud when connectivity is available.
Offline scenario: ask "what type of oil should I add," and the system pulls the answer straight from the manual without hitting the network.
How a 35B model fits on a Pi
A full-precision 35B in fp16 weighs around 70 GB. The Pi 5 maxes out at 8 GB of RAM. On paper, this should not work.
Possible approaches:
- Aggressive quantization: Qwen2.5 ships in GGUF formats like Q4_K_M, Q3_K_S, and IQ2_M. A 35B Q4_K_M sits around 20 GB; IQ2_M can squeeze it to ~10 GB, but with noticeable accuracy loss.
- Layer offload to NVMe: inactive layers get swapped to SSD, at the cost of latency spikes.
- MoE variant: activated parameters are far smaller than 35B.
- Marketing spin: the actual run might be a distilled version or an extreme quantization tier.
My read: most likely Q4_K_M (or something even more aggressive) with layer offload, not genuine full-precision in-memory inference.
llama.cpp supports layer-splitting across multiple devices, so in theory layers can be distributed between Pi RAM and SSD. But expect token generation around 1–2 tokens/s, which is painful.
The biggest open question: the author hasn't shared inference speed or quantization settings, so the realism discount applies. If anyone has benchmarked this, drop a comment.
The edge-plus-cloud Agent collaboration pattern
The author's design:
- Local Agent (small model, real-time, private): reads OBD, runs local retrieval, handles emergency decisions.
- Cloud Agent family (large models, heavy reasoning): trip planning, long-term maintenance, cross-vehicle analytics.
Routing logic roughly looks like:
if query == realtime_or_private:
→ local_agent()
→ OBD data + local manual
elif connectivity and query == heavy_reasoning:
→ cloud_agent()
else:
→ local fallback
This "small model in front, large model behind" architecture is fundamentally different from a cloud-only setup (Mercedes MBUX, Tesla Grok integration):
| Dimension | Cloud-only | Edge + cloud |
|---|---|---|
| Privacy | All data uploaded | Sensitive data stays in the car |
| Latency | 200–500 ms | <50 ms |
| Reliability | Network-dependent | Works offline |
| Cost | Pay per token | One-time hardware |
| Capability ceiling | Bound by cloud model | Capped by local model |
In the car context, latency and reliability matter more than raw IQ. If a driver yells "Hey, coolant temp is spiking," a 3-second cloud round-trip means the engine is already cooked. A 7B local model that decides "pull over now" in 200 ms beats a 70B cloud model every time.
Where a car Agent actually lands
Cars are a much harsher Agent environment than most:
- Privacy: location history, driving habits, home address, on par with medical data in sensitivity.
- Real-time: braking, steering, throttle decisions operate on millisecond timescales.
- Reliability: can't shrug like Siri when it doesn't understand.
- Regulation: UN-R157, ISO 26262.
- Environment: −20 °C to 70 °C, electromagnetic noise, vibration, unstable 12 V power.
OEMs building full-stack Agents is inevitable. But the problems are obvious:
- Closed APIs, no cross-vendor protocol
- Fragmented OBD extension PIDs
- OTA cycles of 6–12 months
- Weak in-vehicle compute, still last-gen ARM SoCs
The value of a third-party project: sidestepping the closed ecosystem with open hardware and open models as middleware.
If it works out, the biggest disruption lands on the aftermarket. A 2018 sedan suddenly gets a "conversational maintenance assistant" that rivals what new EVs ship with, and it wasn't installed by the manufacturer.
Pitfalls when replicating this build
Ordered by how likely you are to hit them:
Hardware
- Pi 5 8 GB is the minimum; Pi 4 lacks memory bandwidth.
- Add an NVMe HAT (Pimoroni NVMe Base) with a 1 TB SSD.
- Power: Pi 5 needs 27 W USB-C PD; use a stable 12 V → 5 V converter.
Model (the critical part)
- Qwen2.5-32B-Instruct Q4_K_M (note: 32B, the author's "35B" might be loose labeling).
- Or Qwen2.5-14B-Instruct Q5_K_M, the best accuracy-vs-speed compromise.
- 72B and up, even at IQ2, is basically unusable.
OBD adapter
- ELM327 clones are rampant; go with OBDLink MX+ or Vgate iCar Pro.
- Post-2008 gasoline cars mostly use CAN ISO 15765; diesels may use ISO 14230.
- Use the
python-OBDorobdiilibrary.
Manufacturer cloud integration (the deepest rabbit hole)
Per-vendor APIs:
- Toyota: Connected Services, OAuth2 + VIN binding
- Tesla: Fleet API, requires a developer account
- BMW: Connected Drive, sparse documentation
- VW/Audi: We Connect, VW Car-Net API
99% of projects die at the OAuth callback or VIN binding step. Start with Toyota, its docs are the most complete.
Software stack
llama.cppfor GGUF inferencelanggraphfor Agent orchestration, state graphs to manage online/offline brancheschromadbas a local vector storeobdPython library- MQTT or WebSocket to sync with cloud
Where you'll probably get stuck
- Pi 5 still can't fit 35B in RAM, so you enable swap on SSD and latency jumps from 50 ms to 2 s.
- Manufacturer API rate limits (Toyota caps at ~1000 calls/day).
- Noisy OBD data, coolant temp sensor jumps, you need filtering.
- Pi thermals under sustained load, you need active cooling to avoid throttling.
Budget:
| Item | Price |
|---|---|
| Pi 5 8 GB | $80 |
| NVMe HAT + 1 TB SSD | $100 |
| OBDLink MX+ | $100 |
| 12 V → 5 V automotive power supply | $20 |
| Cooling case + fan | $30 |
| Total | ~$330 |
Not cheap, but a lot cheaper than a new head unit.
Editor's take
On the surface this looks like a Show HN flex, "I crammed 35B into a Pi." Stopping at that layer misses what's actually interesting.
First, the feasibility frontier for edge LLMs just moved forward. Even with glacial speed and quantization damage, this proves "ARM running large models" is no longer pure theory. But the author hasn't published token/s, and "impressive" might refer to intelligence rather than throughput. My read: for the next 1–2 years, production-grade edge deployments will live in the 7B–14B high-quality-quantized sweet spot; 35B stays in demo territory.
Second, "small model in front, large model behind" might be the right shape for Agent deployment, not the other way around. This inverts the cloud-first thinking that dominates phone assistants, smart speakers, and infotainment systems. The old assumption was "cloud is stronger, so send everything there." Under privacy, latency, and reliability constraints, local small models aren't a transitional hack, they're the endgame. Cloud Agents become optional augmentation, not the main path.
Third, cars may be the first edge-Agent category to scale, ahead of phones and ahead of IoT. Why: stable power (12 V plus alternator), enough physical space (Pi + SSD fit), a contained context (one car, one independent Agent), and strong user willingness to pay. Phones are actually harder: limited battery, compute fights the OS and apps, and users are far less latency-tolerant.
What remains uncertain: real-world 35B inference speed on Pi, stability of manufacturer APIs, whether Pi 5 thermals can handle sustained inference. These wait on more benchmarks.
But one thing is already settled: the second half of in-car Agent development belongs to third-party open-source ecosystems, not OEMs. Whether CarWatch itself survives or not, it opened a door.
References:
- ThinkOffApp/CarWatch on GitHub
- Original Show HN thread
- Qwen2.5 technical report