StockDuty ← Dashboard Oct 08, 2026 06:37 PM ET
SignalTech/AI

Sub-1-Bit LLM Compression via Latent Factorization

High-priority alert based on source significance and keyword relevance.

What happened: Researchers released the official implementation of LittleBit (NeurIPS 2025) and LittleBit-2 (ICML 2026), methods that compress large language models into the sub-1-bit regime via latent factorization. Key numbers: Designed for 1.0 to 0.1 bits per weight. Why it matters: LittleBit-2 adds an opt-in Joint-ITQ initialization (--use_itq) that aligns SVD-derived latent factors with the binary hypercube before QAT, with no additional inference overhead. The codebase supports OPT, Llama and Llama 2/3, Phi-4, Qwen2.5, QwQ, Gemma 2 and Gemma 3, and Qwen3.
Researchers released the official implementation of LittleBit (NeurIPS 2025) and LittleBit-2 (ICML 2026), methods that compress large language models into the sub-1-bit regime via latent factorization.
LittleBit-2 adds an opt-in Joint-ITQ initialization (--use_itq) that aligns SVD-derived latent factors with the binary hypercube before QAT, with no additional inference overhead. The codebase supports OPT, Llama and Llama 2/3, Phi-4, Qwen2.5, QwQ, Gemma 2 and Gemma 3, and Qwen3.
Sources