iPhone runs 35B multimodal model offline! New domestic open-source model takes only 3GB of memory

What happened: ModelBest and OpenBMB open sourced MiniCPM-V 4.7, a multimodal model with a 35B-A3B MoE version and a 1B version. The 35B-A3B model runs offline on iPhone and Android using about 3GB of memory. Key numbers: 35B total parameters, 3B activated per inference. Why it matters: The release targets on device deployment where memory cost and capacity constrain phone makers, since 12GB is described as the baseline for AI phones. ModelBest says it is the first open source MoE model to run smoothly on device without sacrificing performance. Coverage: 3 outlets in 24h

ModelBest and OpenBMB open sourced MiniCPM-V 4.7, a multimodal model with a 35B-A3B MoE version and a 1B version. The 35B-A3B model runs offline on iPhone and Android using about 3GB of memory.

The release targets on device deployment where memory cost and capacity constrain phone makers, since 12GB is described as the baseline for AI phones. ModelBest says it is the first open source MoE model to run smoothly on device without sacrificing performance.