Notes笔记

Standalone notes that reorganize papers, technical reports, and implementations around the questions I want to remember.

把论文、技术报告和实现细节按我想记住的问题重新整理。每个主题单独成页。

Chinese note · Derivation & experiments推导与实验 · 奖励噪声

GRPO group std: target shift and outlier suppressionGRPO 的组内标准差:目标偏移与离群值压制

With noisy rewards, group-std normalization can change the objective: shared labels shift the target toward the median, and independent noise suppresses outliers in small groups. Includes propositions, a closed form, seven reproducible experiments, and LLM judge rewards.

奖励带噪时,除以组内 std 可能改变优化目标:共享标签使目标偏向中位数,独立噪声使小组压制离群值。含命题、闭式解、七组可复现实验,并讨论 LLM judge 奖励。

GRPOCalibrationMedianPseudo-Huber
Chinese note · Evolution & research演化调研 · 机制与研究方向

TabPFN

From prior-data fitting to TabICL and TabPFN-3: worked examples of context-based prediction, row representations, and label retrieval, followed by three testable research proposals.

从 PFN 到 TabICL 与 TabPFN-3:手算上下文预测,图解行列注意力与标签检索,再讨论校准、压缩和关系数据上的三个研究问题。

PFNTabICLTabPFN-3Research
Chinese note · Evolution & research演化调研 · 机制与研究方向

Loop LLM

From adaptive computation and Universal Transformers to Huginn, Ouro, and LoopFormer: shared depth, latent-state updates, training, halting, and the real cost of recurrent caches.

从 ACT、Universal Transformer 到 Huginn、Ouro 与 LoopFormer:解释共享深度、隐藏状态迭代、训练与退出,并拆解循环缓存的真实代价。

HuginnOuroLatent ReasoningKV Cache
Illustrated guide · 7 modules图解入门 · 7 个模块

Qwen3.8-Flash-Next

For readers familiar with Transformer and LLaMA: compare architectural changes across seven modules, with worked examples, diagrams, and supporting implementation details.

面向熟悉 Transformer 与 LLaMA 的读者,对照七个模块的具体改动,用逐步算例与图解解释新增机制、发展脉络和实现代价。

QSAGated ResidualN-gram EmbeddingMuon