SignalTech/AI
Show HN: Which Mac runs which LLM, and how fast
High-priority alert based on source significance and keyword relevance.
Context
What happened: A Show HN calculator estimates which LLMs fit on which Apple Silicon Macs and how fast they generate, using two formulas: memory for weights plus context and overhead, and memory bandwidth divided by weight size plus a fixed per-token cost.
Key numbers: Llama 3.3 70B at Q4_K_M on an M4 Pro with 64 GB is estimated at about 5.7 tokens/sec, with 42.8 GB of
Why it matters: The author states the overhead term is held constant across chips, so Max and Ultra figures are more likely low than high, and that every row outside the M4 and M5 Pro is calculated rather than measured.
What changed
A Show HN calculator estimates which LLMs fit on which Apple Silicon Macs and how fast they generate, using two formulas: memory for weights plus context and overhead, and memory bandwidth divided by weight size plus a fixed per-token cost.
Why it matters
The author states the overhead term is held constant across chips, so Max and Ultra figures are more likely low than high, and that every row outside the M4 and M5 Pro is calculated rather than measured.