According to reports, NVIDIA and MIT researchers released Lightning OPD (Offline On-Policy Distillation), a new post-training framework for large language models that eliminates the need to keep a teacher model running during training. By precomputing the teacher model’s log-probabilities offline, the framework improves training efficiency by 4x while freeing all GPU resources for student model training.
In testing on 8 NVIDIA H100 GPUs, Lightning OPD successfully distilled Qwen3-30B-A3B-Base (a 30-billion parameter MoE model) and achieved 71.0 on the AIME 2024 benchmark, whereas standard OPD ran out of memory on the same hardware. For the smaller Qwen3-8B model, the framework required only 30 GPU hours to reach 69.9 points.
Disclaimer: The information on this page may come from third parties and does not represent the views or opinions of Gate. The content displayed on this page is for reference only and does not constitute any financial, investment, or legal advice. Gate does not guarantee the accuracy or completeness of the information and shall not be liable for any losses arising from the use of this information. Virtual asset investments carry high risks and are subject to significant price volatility. You may lose all of your invested principal. Please fully understand the relevant risks and make prudent decisions based on your own financial situation and risk tolerance. For details, please refer to
Disclaimer.
Related Articles
Republican Lawmakers Seek Investigation into Altman's $3.75B Personal Investments and OpenAI Conflicts
According to ChainCatcher, U.S. House Oversight Committee Chair James Comer recently demanded that OpenAI CEO Sam Altman disclose information about his personal investments and potential conflicts of interest with OpenAI. Comer's inquiry focuses on Altman's extensive investment portfolio, which
GateNews56m ago
TT Chain Integrates AegisAI Security System for RWA Protection on May 11
According to TT Chain's announcement on May 11, the Layer-2 blockchain network integrated AegisAI's AI-driven security infrastructure into its real-world asset ecosystem. The collaboration enhances protection for tokenized asset transactions and cross-chain operations through advanced threat
GateNews1h ago
AI Startup Wispr in Funding Talks at $2B Valuation on May 12
According to PANews, AI startup Wispr is in fundraising negotiations on May 12, with a valuation of $2 billion.
GateNews1h ago
Sam Altman to Testify Today in Musk's $38M OpenAI Lawsuit
According to BlockBeats, Sam Altman is scheduled to testify today (May 12) in federal court in Oakland, California regarding Elon Musk's lawsuit against OpenAI. Altman will take the stand after OpenAI board chair Bret Taylor completes his testimony.
Musk filed the lawsuit in 2024 alleging that Open
GateNews2h ago
Depthfirst Finds Critical Internet Flaws at One-Tenth Cost of Anthropic's Mythos
According to BlockBeats, on May 12, AI cybersecurity startup Depthfirst announced that its self-developed AI vulnerability discovery model found multiple critical security flaws missed by Anthropic's Mythos, claiming the total cost was just one-tenth of the latter's. Depthfirst CEO Qasim Mithani sta
GateNews2h ago
Cerebras IPOs This Week at $55 Billion Valuation, Giving OpenAI Potential 11% Stake
According to Odaily, artificial intelligence chip manufacturer Cerebras will go public this week at a $55 billion IPO valuation. OpenAI's $200 billion investment commitment could grant it an 11% stake in the
GateNews3h ago
