#端侧推理

共 23 篇带此标签

23 articles Newest 2026-11-21
Archive · #端侧推理 11 articles
Products

Edge0-145B: Local Models Are Not Yet Edge Agents

Fitting a 145B-parameter model on a personal computer does not make it a practical AI Agent. Hardware requirements, sustained throughput, tool-calling reliability, and reproducibility determine whether Edge0-145B can handle real workloads.

2026-11-14 Source 2026-11-12 8 min 68 views
Tools

Will Apple Make Local LLMs Easier on Mac?

Apple is reportedly developing a Mac app designed to simplify local AI model deployment. Its success will depend on whether it unifies model distribution, hardware configuration, debugging, and local-to-cloud routing rather than adding another interface for existing tools.

2026-11-11 Source 2026-11-10 7 min 68 views
Products

Can DeepX’s Second-Generation PIM Chip Run 100B Models Under 5W?

DeepX says its second-generation LPDDR5X PIM chip can support models as large as 100B parameters while consuming less than 5W during inference, with a claimed 6.7× performance gain. Capacity, bandwidth, latency, power boundaries, and software support still require independent validation.

2026-11-10 Source 2026-11-09 6 min 68 views
Tools

How Maple Fits Gemma 4 Into 6GB of VRAM

Maple applies Sub-2-BPW Moe Pruning to Gemma 4 26B, with claims of a 6GB GPU footprint and 45 tokens per second. The headline figures are intriguing, but independent replication and pruning-quality data remain limited.

2026-11-08 Source 2026-11-08 6 min 68 views
Case Studies

What $20,000 Really Buys for an Open-Weight Image Model

A two-person lab trained the PJIT-21K text-to-image model from scratch for roughly $20,000 and released its weights under Apache 2.0. This analysis breaks down training, inference, licensing, and deployment costs for teams evaluating open models.

2026-11-06 Source 2026-11-04 7 min 68 views
Products

Cloud AI 100 Ultra: A 128GB Inference Card Built for Scale

Cloud AI 100 Ultra targets cloud inference with 128GB of memory, 548GB/s of bandwidth, and a 150W power envelope. Its hardware strategy is clear, but real purchasing decisions still require data on throughput, latency, compatibility, and migration costs.

2026-11-05 Source 2026-11-04 7 min 68 views
Case Studies

Qwen 3.8 27B Inspects Model Weights Locally

Simon Willison ran Qwen 3.8 27B on his laptop to inspect another model’s weights. The experiment offers a practical workflow for connecting a local LLM to reproducible statistics, hypothesis generation, and verification without treating generated explanations as causal proof.

2026-11-04 Source 2026-11-03 8 min 68 views
Tools

EPFL's Neural Chip Shifts Focus to Brain Interactions

EPFL's specialized neural chip decodes interactions between brain regions rather than isolated neurons. Its approach offers a design lesson for AI Agents: module interfaces should preserve interaction context and timing, not just final results.

2026-10-25 Source 2026-10-24 7 min 68 views