Frontier MoE models keep changing their load-balancing method because of three constraints: narrower balancing weakens expert specialization, expert parallelism needs balance on each GPU at each step, and at low activation ratios current fixes can fail in shallow layers.前沿 MoE 模型一直在换均衡方法，原因可归为三个约束：均衡的范围越窄，专家分工越弱；专家并行要求每张卡每一步都均衡；激活比例很低时，现有方法可能在浅层失效。
Energy and efficiency are first principles of intelligence; that is why my research moved from world models to pretraining and infrastructure.能量和效率是智能的第一性原理，所以我的研究从世界模型转向了预训练和基础设施。
At one scale, mostly yes: a law fitted on two or three short runs predicts loss under unseen schedules.同一规模下基本可以：用两三条短训练拟合的定律，就能预测没见过的调度下的 loss。
In constant-rate GPT-2-small runs with weight decay, an update–weight cosine of only −0.02 to −0.06 supplies 94–97% of weight-norm growth.在恒定学习率、带权重衰减的 GPT-2 small 训练里，更新与权重的余弦只有 −0.02 到 −0.06，却贡献了 94–97% 的权重范数增长。
do_bench clears L2 by writing zeros, so the next kernel first writes dirty lines back to HBM. That can flip a benchmark ranking.do_bench 用写零的方式清 L2，下一个 kernel 要先把脏行写回 HBM。多出的时间足以让性能排名翻转。
At inference the cost is not computing attention but keeping its past. MQA, GQA and MLA all shrink the KV cache.推理时，贵的不是计算 attention，而是保存历史 token 的 KV cache。MQA、GQA 和 MLA 都在压缩 KV cache。
On MMLU, across Olmo 3 7B post-training checkpoints, SFT and DPO cause almost all of the rise in calibration error. RLVR adds little.在 MMLU 上，Olmo 3 7B 各后训练 checkpoint 的校准误差上升几乎都来自 SFT 和 DPO，RLVR 影响很小。
The original Transformer adds a fixed sine–cosine vector to each word embedding. A shift by k is a rotation.原始 Transformer 给每个词向量加上一个固定的正弦余弦向量。平移 k 个位置等于一次旋转。
TD-MPC does not reconstruct every visible detail. It predicts only what matters for action: latent dynamics, reward and value.TD-MPC 不还原每个可见细节，只预测对动作有用的部分：潜空间动力学、奖励和价值。
A planner calls the transition function again and again, so the world model must keep contact, sliding and landing as separate regimes.planner 会反复调用转移函数，所以世界模型必须把接触、滑动、落地分成不同的区段，不能平均成一条平滑曲线。
VLWM changes the time scale of the dynamics interface; PRISM-WM changes its regime structure. Both reject one smooth transition function.VLWM 改变了动力学接口的时间尺度，PRISM-WM 改变了它的区段结构。两者都否定了一个平滑的整体转移函数。
IDM-, VLA- and MoT-style world models approximate the same policy distribution. They differ only in the structure in between.IDM、VLA、MoT 三类世界模型近似的是同一个策略分布，区别只在中间结构。
In IDM-, VLA- and MoT-style world models, action is the prediction. In TD-MPC, it is the condition for prediction and planning.在 IDM、VLA、MoT 三类世界模型里，动作是被预测的对象。在 TD-MPC 里，动作是潜空间预测和规划的条件。
A policy can output pose targets, force commands or motor commands. The layer decides what the model learns and where safety lives.策略输出的可以是位姿目标、力与柔顺指令，或电机指令。输出在哪一层，决定了模型学什么、安全由谁负责。
A diffusion model learns small denoising corrections along a path, not one jump from noise to data.Diffusion 模型不学从噪声到数据的一步跳跃，而是学一条路径上每一小步的去噪修正。
Cosmos Policy keeps the video model's tensor shapes and changes what some latent slots mean: action, future state and value.Cosmos Policy 保留视频模型的张量形状，只改变部分潜变量槽位的含义：动作、未来状态和价值。
Puppeteer splits visual humanoid control into two TD-MPC2-style world models: a MoCap-trained tracker and a vision-driven agent.Puppeteer 把视觉人形机器人控制拆成两层 TD-MPC2 式世界模型。低层用动捕数据学跟踪，高层从视觉生成跟踪目标。
Qwen-AgentWorld predicts what a digital environment returns after an agent acts, so a text model meets the world-model interface.Qwen-AgentWorld 预测 agent 执行动作后数字环境返回什么，所以文本模型也满足世界模型的接口。
GRASP turns the intermediate states of a rollout into virtual states, optimized with the actions and checked by dynamics consistency.GRASP 把 rollout 的中间状态变成虚拟状态，与动作一起优化，再用动力学一致性检验。
One multi-view latent world model for manipulation serves policy evaluation, policy improvement and test-time planning on a real robot.一个用于机器人操作的多视角潜空间世界模型，在真机上同时用于策略评估、策略改进和测试时规划。
WAV splits action-conditioned prediction into two checks, state plausibility and action reachability, so the world model can find its own errors.WAV 把动作条件预测拆成两个检验：状态是否合理，动作能否到达。世界模型由此能发现自己的错误。
The verifier decodes each imagined future into task predicates and checks whether the event happened, not feature distance to the goal.verifier 把每个想象的未来解码成任务谓词，检查事件是否发生，而不是算特征到目标的距离。
Offline model-based RL without explicit conservatism: a posterior over world models sets how long to roll out.离线 model-based RL，不加显式保守项：由世界模型的后验决定 rollout 多长。
“Long horizon” means reliability over time for language-model agents and fidelity over a rollout for world models.“长程”在语言模型 agent 里指随时间的可靠性，在世界模型里指 rollout 的保真度。
Coding agents, robot world models and contact-rich control all fail at long horizons, yet rarely cite each other's fixes.代码 agent、机器人世界模型和接触丰富的控制都在长程上失败，却很少引用彼此的解法。
Long-horizon failures happen where parts of the closed loop meet, not inside one prediction.长程失败发生在闭环各部分的接口上，而不在单步预测里。
Nearly fifty recent world-model papers, each placed on the one part of the robot's plan–act loop it changes.近 50 篇近期世界模型论文，逐篇对应到它改动的那一个闭环环节。
In a closed loop, the planner turns a small prediction error into a wrong action. Six coupled failures follow.在闭环里，planner 会把小的预测误差放大成错误的动作，由此产生六类互相耦合的失败。
A long-horizon claim counts only when it names a task, a metric and a failure it reduces.一个长程主张，只有写明任务、指标和它减少的失败，才算数。