[{"data":1,"prerenderedAt":1982},["ShallowReactive",2],{"post-\u002Fposts\u002Fcmu-15-779-profiler-operator-batching":3,"archive-public-posts":1479},{"id":4,"title":5,"body":6,"categories":1458,"cover":1460,"coverAlt":1461,"date":1462,"description":1463,"draft":1464,"extension":1465,"haloPublished":1464,"haloSync":1464,"hidden":1464,"legacy":1464,"meta":1466,"navigation":182,"path":1467,"published":1464,"readingTime":1468,"seo":1469,"series":1470,"seriesOrder":1471,"stem":1472,"tags":1473,"__hash__":1478},"posts\u002Fposts\u002Fcmu-15-779-profiler-operator-batching.md","CMU 15-779 学习记录 05：Profiler、算子调用与批量推理",{"type":7,"value":8,"toc":1449},"minimark",[9,16,19,47,50,62,66,69,79,82,88,99,106,110,115,291,306,379,389,409,412,416,421,537,547,550,576,583,654,668,671,675,681,687,690,772,787,868,1052,1066,1084,1087,1101,1108,1235,1242,1253,1256,1262,1265,1268,1271,1274,1281,1285,1291,1346,1349,1373,1376,1417,1425,1445],[10,11,12],"blockquote",{},[13,14,15],"p",{},"本篇对照课程官网的 Introduction to ML Systems 整理，是个人配套 CPU 实验，不是课程官方作业。脚本编号是练习顺序，不对应课程讲次。",[13,17,18],{},"前几篇一直在补 PyTorch 基础。到了这一步，我在学习聊天里问：现在是不是偏离 15-779 的主线了？随后把重点接回了系统问题：同一个网络执行了哪些操作，时间花在哪里，代码的组织方式又怎样影响执行开销。",[13,20,21,28,29,34,35,39,40,39,43,46],{},[22,23,27],"a",{"href":24,"rel":25},"https:\u002F\u002Fwww.cs.cmu.edu\u002F~zhihaoj2\u002F15-779\u002Fschedule.html",[26],"nofollow","课程安排","中的 Introduction to ML Systems 配有",[22,30,33],{"href":31,"rel":32},"https:\u002F\u002Fwww.cs.cmu.edu\u002F~zhihaoj2\u002F15-779\u002Fslides\u002F02-MLSys-basics.pdf",[26],"第 2 讲课件","，阅读材料包括 TensorFlow 和 PyTorch 两篇系统论文。本篇先围绕已经熟悉的两层网络做三个小实验：",[36,37,38],"code",{},"22_profile_forward.py","、",[36,41,42],{},"23_profile_batching.py",[36,44,45],{},"24_batch_size_sweep.py","。",[13,48,49],{},"这些脚本在本机执行，并在整理时独立复跑。本文的 batch size 数据来自附件中保存的原始 JSON，对应本机固定环境下的实测快照。",[13,51,52,57,58],{},[22,53,56],{"href":54,"rel":55},"https:\u002F\u002Fgithub.com\u002Fliangqianxing\u002Fcmu-15-779-labs\u002Ftree\u002Fcecf5110d8a95161bb5763eadfca94cd7a0cc50f",[26],"查看本篇代码"," · ",[22,59,61],{"href":60},"\u002Fmaterials\u002Fcmu-15-779\u002Fexperiments\u002Fpytorch-profiler-batching-20261015.zip","下载代码与结果 ZIP",[63,64,65],"h2",{"id":65},"先固定模型和输入",[13,67,68],{},"这次不训练网络，直接观察前向计算。模型结构仍然是：",[70,71,77],"pre",{"className":72,"code":74,"language":75,"meta":76},[73],"language-text","输入 → Linear(1, 8) → ReLU → Linear(8, 1) → 输出\n","text","",[36,78,74],{"__ignoreMap":76},[13,80,81],{},"对于一批 1000 个样本，张量形状依次为：",[70,83,86],{"className":84,"code":85,"language":75,"meta":76},[73],"[1000, 1] → [1000, 8] → [1000, 8] → [1000, 1]\n",[36,87,85],{"__ignoreMap":76},[13,89,90,91,94,95,98],{},"第一层有 ",[36,92,93],{},"1 × 8 + 8 = 16"," 个参数，第二层有 ",[36,96,97],{},"8 × 1 + 1 = 9"," 个参数，总共只有 25 个。这是一个刻意保留的小模型，便于观察调用开销，不代表真实大模型的计算规模。",[13,100,101,102,105],{},"脚本设置了 ",[36,103,104],{},"torch.manual_seed(0)","，每种推理方式复用相同的模型参数和输入。参数没有经过训练，所以这里不讨论预测是否接近某个目标函数；要检查的是，不同执行方式是否计算了相同的结果。",[63,107,109],{"id":108},"用-profiler-看一次前向计算","用 Profiler 看一次前向计算",[13,111,112,114],{},[36,113,38],{}," 先预热 10 次，再记录一次完整的前向计算：",[70,116,120],{"className":117,"code":118,"language":119,"meta":76,"style":76},"language-python shiki shiki-themes github-dark-dimmed github-light houston","with torch.no_grad():\n    for _ in range(10):\n        model(x)\n\n    with profile(activities=[ProfilerActivity.CPU]) as prof:\n        prediction = model(x)\n\nprint(prof.key_averages().table(\n    sort_by=\"self_cpu_time_total\",\n    row_limit=20,\n))\n","python",[36,121,122,142,168,177,184,218,231,236,257,272,285],{"__ignoreMap":76},[123,124,127,131,135,139],"span",{"class":125,"line":126},"line",1,[123,128,130],{"class":129},"slEfO","with",[123,132,134],{"class":133},"sWURj"," torch.",[123,136,138],{"class":137},"sq3c-","no_grad",[123,140,141],{"class":133},"():\n",[123,143,145,148,151,154,158,161,165],{"class":125,"line":144},2,[123,146,147],{"class":129},"    for",[123,149,150],{"class":133}," _ ",[123,152,153],{"class":129},"in",[123,155,157],{"class":156},"svkvy"," range",[123,159,160],{"class":133},"(",[123,162,164],{"class":163},"sgENW","10",[123,166,167],{"class":133},"):\n",[123,169,171,174],{"class":125,"line":170},3,[123,172,173],{"class":137},"        model",[123,175,176],{"class":133},"(x)\n",[123,178,180],{"class":125,"line":179},4,[123,181,183],{"emptyLinePlaceholder":182},true,"\n",[123,185,187,190,193,195,199,203,206,209,212,215],{"class":125,"line":186},5,[123,188,189],{"class":129},"    with",[123,191,192],{"class":137}," profile",[123,194,160],{"class":133},[123,196,198],{"class":197},"sA9aF","activities",[123,200,202],{"class":201},"sf2cM","=",[123,204,205],{"class":133},"[ProfilerActivity.",[123,207,208],{"class":163},"CPU",[123,210,211],{"class":133},"]) ",[123,213,214],{"class":129},"as",[123,216,217],{"class":133}," prof:\n",[123,219,221,224,226,229],{"class":125,"line":220},6,[123,222,223],{"class":133},"        prediction ",[123,225,202],{"class":201},[123,227,228],{"class":137}," model",[123,230,176],{"class":133},[123,232,234],{"class":125,"line":233},7,[123,235,183],{"emptyLinePlaceholder":182},[123,237,239,242,245,248,251,254],{"class":125,"line":238},8,[123,240,241],{"class":156},"print",[123,243,244],{"class":133},"(prof.",[123,246,247],{"class":137},"key_averages",[123,249,250],{"class":133},"().",[123,252,253],{"class":137},"table",[123,255,256],{"class":133},"(\n",[123,258,260,263,265,269],{"class":125,"line":259},9,[123,261,262],{"class":197},"    sort_by",[123,264,202],{"class":201},[123,266,268],{"class":267},"sWqY2","\"self_cpu_time_total\"",[123,270,271],{"class":133},",\n",[123,273,275,278,280,283],{"class":125,"line":274},10,[123,276,277],{"class":197},"    row_limit",[123,279,202],{"class":201},[123,281,282],{"class":163},"20",[123,284,271],{"class":133},[123,286,288],{"class":125,"line":287},11,[123,289,290],{"class":133},"))\n",[13,292,293,294,297,298,301,302,305],{},"输出除了 ",[36,295,296],{},"linear"," 和 ",[36,299,300],{},"relu","，还有 ",[36,303,304],{},"addmm","、转置、广播、复制等操作。本次环境中，几个主要事件的调用次数是：",[253,307,308,325],{},[309,310,311],"thead",{},[312,313,314,318,322],"tr",{},[315,316,317],"th",{},"Profiler 事件",[315,319,321],{"align":320},"right","次数",[315,323,324],{},"对应的计算",[326,327,328,342,354,367],"tbody",{},[312,329,330,336,339],{},[331,332,333],"td",{},[36,334,335],{},"aten::linear",[331,337,338],{"align":320},"2",[331,340,341],{},"两层线性层",[312,343,344,349,351],{},[331,345,346],{},[36,347,348],{},"aten::addmm",[331,350,338],{"align":320},[331,352,353],{},"当前线性层执行中的矩阵乘法与相加",[312,355,356,361,364],{},[331,357,358],{},[36,359,360],{},"aten::relu",[331,362,363],{"align":320},"1",[331,365,366],{},"中间的激活函数",[312,368,369,374,376],{},[331,370,371],{},[36,372,373],{},"aten::clamp_min",[331,375,363],{"align":320},[331,377,378],{},"当前 ReLU 执行中的下界截断",[13,380,381,382,385,386,388],{},"表里的行有嵌套关系。一层 ",[36,383,384],{},"Linear"," 内部可以调用 ",[36,387,304],{},"，这两个名字不代表网络又多了一层。",[13,390,391,392,395,396,399,400,402,403,408],{},"读时间列时也要区分：",[36,393,394],{},"Self CPU"," 不含子算子的时间，",[36,397,398],{},"CPU total"," 包含子算子的时间。把所有行的 ",[36,401,398],{}," 直接相加，会重复计算嵌套调用。这一点在 ",[22,404,407],{"href":405,"rel":406},"https:\u002F\u002Fdocs.pytorch.org\u002Ftutorials\u002Frecipes\u002Frecipes\u002Fprofiler_recipe.html",[26],"PyTorch 官方 Profiler 教程","中也有解释。",[13,410,411],{},"这次用 Profiler 确认操作和调用次数；速度比较放到 Profiler 外进行，避免将记录开销混入基准测量。",[63,413,415],{"id":414},"同样的-1000-个样本为什么调用次数差这么多","同样的 1000 个样本，为什么调用次数差这么多",[13,417,418,420],{},[36,419,42],{}," 比较两种写法：",[70,422,424],{"className":117,"code":423,"language":119,"meta":76,"style":76},"def predict_one_by_one():\n    outputs = []\n    for i in range(x.shape[0]):\n        outputs.append(model(x[i:i + 1]))\n    return torch.cat(outputs, dim=0)\n\ndef predict_batch():\n    return model(x)\n",[36,425,426,438,448,468,493,516,520,529],{"__ignoreMap":76},[123,427,428,432,436],{"class":125,"line":126},[123,429,431],{"class":430},"sST-V","def",[123,433,435],{"class":434},"sS_0i"," predict_one_by_one",[123,437,141],{"class":133},[123,439,440,443,445],{"class":125,"line":144},[123,441,442],{"class":133},"    outputs ",[123,444,202],{"class":201},[123,446,447],{"class":133}," []\n",[123,449,450,452,455,457,459,462,465],{"class":125,"line":170},[123,451,147],{"class":129},[123,453,454],{"class":133}," i ",[123,456,153],{"class":129},[123,458,157],{"class":156},[123,460,461],{"class":133},"(x.shape[",[123,463,464],{"class":163},"0",[123,466,467],{"class":133},"]):\n",[123,469,470,473,476,478,481,484,487,490],{"class":125,"line":179},[123,471,472],{"class":133},"        outputs.",[123,474,475],{"class":137},"append",[123,477,160],{"class":133},[123,479,480],{"class":137},"model",[123,482,483],{"class":133},"(x[i:i ",[123,485,486],{"class":201},"+",[123,488,489],{"class":163}," 1",[123,491,492],{"class":133},"]))\n",[123,494,495,498,500,503,506,509,511,513],{"class":125,"line":186},[123,496,497],{"class":129},"    return",[123,499,134],{"class":133},[123,501,502],{"class":137},"cat",[123,504,505],{"class":133},"(outputs, ",[123,507,508],{"class":197},"dim",[123,510,202],{"class":201},[123,512,464],{"class":163},[123,514,515],{"class":133},")\n",[123,517,518],{"class":125,"line":220},[123,519,183],{"emptyLinePlaceholder":182},[123,521,522,524,527],{"class":125,"line":233},[123,523,431],{"class":430},[123,525,526],{"class":434}," predict_batch",[123,528,141],{"class":133},[123,530,531,533,535],{"class":125,"line":238},[123,532,497],{"class":129},[123,534,228],{"class":137},[123,536,176],{"class":133},[13,538,539,542,543,546],{},[36,540,541],{},"x[i:i + 1]"," 保留了 ",[36,544,545],{},"[1, 1]"," 的批次形状。逐个预测调用模型 1000 次，整批预测调用模型 1 次；两种方式都完整处理 1000 个样本。",[13,548,549],{},"先比较结果，再看性能：",[70,551,553],{"className":117,"code":552,"language":119,"meta":76,"style":76},"torch.allclose(single_output, batch_output, atol=1e-6)\n",[36,554,555],{"__ignoreMap":76},[123,556,557,560,563,566,569,571,574],{"class":125,"line":126},[123,558,559],{"class":133},"torch.",[123,561,562],{"class":137},"allclose",[123,564,565],{"class":133},"(single_output, batch_output, ",[123,567,568],{"class":197},"atol",[123,570,202],{"class":201},[123,572,573],{"class":163},"1e-6",[123,575,515],{"class":133},[13,577,578,579,582],{},"本次输出近似一致，最大绝对误差为 ",[36,580,581],{},"2.9802322387695312e-08","。浮点运算的组织方式变化后，结果可以有很小的末位差异。",[253,584,585,598],{},[309,586,587],{},[312,588,589,592,595],{},[315,590,591],{},"操作",[315,593,594],{"align":320},"逐个预测",[315,596,597],{"align":320},"整批预测",[326,599,600,611,621,632,643],{},[312,601,602,606,609],{},[331,603,604],{},[36,605,335],{},[331,607,608],{"align":320},"2000",[331,610,338],{"align":320},[312,612,613,617,619],{},[331,614,615],{},[36,616,348],{},[331,618,608],{"align":320},[331,620,338],{"align":320},[312,622,623,627,630],{},[331,624,625],{},[36,626,360],{},[331,628,629],{"align":320},"1000",[331,631,363],{"align":320},[312,633,634,639,641],{},[331,635,636],{},[36,637,638],{},"aten::slice",[331,640,629],{"align":320},[331,642,464],{"align":320},[312,644,645,650,652],{},[331,646,647],{},[36,648,649],{},"aten::cat",[331,651,363],{"align":320},[331,653,464],{"align":320},[13,655,656,657,659,660,663,664,667],{},"模型有两层 ",[36,658,384],{},"，所以逐个执行时是 ",[36,661,662],{},"1000 × 2 = 2000"," 次，整批执行时是 ",[36,665,666],{},"1 × 2 = 2"," 次。整批运算每次处理更多数据，样本数并没有减少，同时省掉了大部分 Python 循环、分发和切片开销。",[13,669,670],{},"这个脚本的计时在 Profiler 外单独进行：每组完整执行 10 遍，先求每遍的平均耗时，再取 7 组结果的中位数。调用次数只对一次完整处理进行记录，不把预热和计时重复次数混在里面。",[63,672,674],{"id":673},"再把-batch-size-拆成多个档位","再把 batch size 拆成多个档位",[13,676,677,678,680],{},"只比较“1 个”和“1000 个”还看不到中间变化。",[36,679,45],{}," 固定总样本数为 1000，扫描：",[70,682,685],{"className":683,"code":684,"language":75,"meta":76},[73],"1、4、16、64、128、256、1000\n",[36,686,684],{"__ignoreMap":76},[13,688,689],{},"每一档都使用同一个函数：",[70,691,693],{"className":117,"code":692,"language":119,"meta":76,"style":76},"def predict_in_batches(batch_size):\n    outputs = []\n    for start in range(0, sample_count, batch_size):\n        outputs.append(model(x[start:start + batch_size]))\n    return torch.cat(outputs, dim=0)\n",[36,694,695,710,718,736,754],{"__ignoreMap":76},[123,696,697,699,702,704,708],{"class":125,"line":126},[123,698,431],{"class":430},[123,700,701],{"class":434}," predict_in_batches",[123,703,160],{"class":133},[123,705,707],{"class":706},"s9NhK","batch_size",[123,709,167],{"class":133},[123,711,712,714,716],{"class":125,"line":144},[123,713,442],{"class":133},[123,715,202],{"class":201},[123,717,447],{"class":133},[123,719,720,722,725,727,729,731,733],{"class":125,"line":170},[123,721,147],{"class":129},[123,723,724],{"class":133}," start ",[123,726,153],{"class":129},[123,728,157],{"class":156},[123,730,160],{"class":133},[123,732,464],{"class":163},[123,734,735],{"class":133},", sample_count, batch_size):\n",[123,737,738,740,742,744,746,749,751],{"class":125,"line":179},[123,739,472],{"class":133},[123,741,475],{"class":137},[123,743,160],{"class":133},[123,745,480],{"class":137},[123,747,748],{"class":133},"(x[start:start ",[123,750,486],{"class":201},[123,752,753],{"class":133}," batch_size]))\n",[123,755,756,758,760,762,764,766,768,770],{"class":125,"line":186},[123,757,497],{"class":129},[123,759,134],{"class":133},[123,761,502],{"class":137},[123,763,505],{"class":133},[123,765,508],{"class":197},[123,767,202],{"class":201},[123,769,464],{"class":163},[123,771,515],{"class":133},[13,773,774,775,778,779,782,783,786],{},"这里是便于阅读的写法，完整脚本从命令行参数读取 ",[36,776,777],{},"sample_count","。包括 ",[36,780,781],{},"batch_size=1000"," 在内，每一档都有切片和一次最终拼接，因此比第 23 个脚本的直接 ",[36,784,785],{},"model(x)"," 多了一点工作。",[13,788,789,790,836,837,867],{},"总样本数为 ",[123,791,794,816],{"className":792},[793],"katex",[123,795,798],{"className":796},[797],"katex-mathml",[799,800,802],"math",{"xmlns":801},"http:\u002F\u002Fwww.w3.org\u002F1998\u002FMath\u002FMathML",[803,804,805,812],"semantics",{},[806,807,808],"mrow",{},[809,810,811],"mi",{},"N",[813,814,811],"annotation",{"encoding":815},"application\u002Fx-tex",[123,817,821],{"className":818,"ariaHidden":820},[819],"katex-html","true",[123,822,825,830],{"className":823},[824],"base",[123,826],{"className":827,"style":829},[828],"strut","height:0.6833em;",[123,831,811],{"className":832,"style":835},[833,834],"mord","mathnormal","margin-right:0.109em;","、batch size 为 ",[123,838,840,854],{"className":839},[793],[123,841,843],{"className":842},[797],[799,844,845],{"xmlns":801},[803,846,847,852],{},[806,848,849],{},[809,850,851],{},"B",[813,853,851],{"encoding":815},[123,855,857],{"className":856,"ariaHidden":820},[819],[123,858,860,863],{"className":859},[824],[123,861],{"className":862,"style":829},[828],[123,864,851],{"className":865,"style":866},[833,834],"margin-right:0.0502em;"," 时，批次数是：",[123,869,872],{"className":870},[871],"katex-display",[123,873,875,913],{"className":874},[793],[123,876,878],{"className":877},[797],[799,879,881],{"xmlns":801,"display":880},"block",[803,882,883,910],{},[806,884,885,888,891,906],{},[809,886,887],{},"K",[889,890,202],"mo",{},[806,892,893,896,903],{},[889,894,895],{"fence":820},"⌈",[897,898,899,901],"mfrac",{},[809,900,811],{},[809,902,851],{},[889,904,905],{"fence":820},"⌉",[809,907,909],{"mathvariant":908},"normal",".",[813,911,912],{"encoding":815},"K = \\left\\lceil \\frac{N}{B} \\right\\rceil.",[123,914,916,938],{"className":915,"ariaHidden":820},[819],[123,917,919,922,926,931,935],{"className":918},[824],[123,920],{"className":921,"style":829},[828],[123,923,887],{"className":924,"style":925},[833,834],"margin-right:0.0715em;",[123,927],{"className":928,"style":930},[929],"mspace","margin-right:0.2778em;",[123,932,202],{"className":933},[934],"mrel",[123,936],{"className":937,"style":930},[929],[123,939,941,945,1045,1049],{"className":940},[824],[123,942],{"className":943,"style":944},[828],"height:2.4em;vertical-align:-0.95em;",[123,946,949,960,1039],{"className":947},[948],"minner",[123,950,955],{"className":951,"style":954},[952,953],"mopen","delimcenter","top:0em;",[123,956,895],{"className":957},[958,959],"delimsizing","size3",[123,961,963,967,1035],{"className":962},[833],[123,964],{"className":965},[952,966],"nulldelimiter",[123,968,970],{"className":969},[897],[123,971,975,1026],{"className":972},[973,974],"vlist-t","vlist-t2",[123,976,979,1021],{"className":977},[978],"vlist-r",[123,980,984,998,1009],{"className":981,"style":983},[982],"vlist","height:1.3603em;",[123,985,987,992],{"style":986},"top:-2.314em;",[123,988],{"className":989,"style":991},[990],"pstrut","height:3em;",[123,993,995],{"className":994},[833],[123,996,851],{"className":997,"style":866},[833,834],[123,999,1001,1004],{"style":1000},"top:-3.23em;",[123,1002],{"className":1003,"style":991},[990],[123,1005],{"className":1006,"style":1008},[1007],"frac-line","border-bottom-width:0.04em;",[123,1010,1012,1015],{"style":1011},"top:-3.677em;",[123,1013],{"className":1014,"style":991},[990],[123,1016,1018],{"className":1017},[833],[123,1019,811],{"className":1020,"style":835},[833,834],[123,1022,1025],{"className":1023},[1024],"vlist-s","​",[123,1027,1029],{"className":1028},[978],[123,1030,1033],{"className":1031,"style":1032},[982],"height:0.686em;",[123,1034],{},[123,1036],{"className":1037},[1038,966],"mclose",[123,1040,1042],{"className":1041,"style":954},[1038,953],[123,1043,905],{"className":1044},[958,959],[123,1046],{"className":1047,"style":1048},[929],"margin-right:0.1667em;",[123,1050,909],{"className":1051},[833],[13,1053,1054,1055,1058,1059,39,1061,1058,1063,1065],{},"这个两层网络应当执行 ",[36,1056,1057],{},"2K"," 次 ",[36,1060,296],{},[36,1062,887],{},[36,1064,300],{},"。完整脚本对这些次数做了断言，也检查了最后一批和输出总样本数。",[13,1067,1068,1069,1072,1073,1075,1076,1079,1080,1083],{},"例如 ",[36,1070,1071],{},"B=16"," 时，前 62 批各有 16 个样本，最后还剩 8 个，所以共有 63 批，",[36,1074,296],{}," 调用 ",[36,1077,1078],{},"126"," 次。不能直接用 ",[36,1081,1082],{},"1000 \u002F\u002F 16"," 丢掉尾批。",[63,1085,1086],{"id":1086},"原始测量结果",[13,1088,1089,1090,1093,1094,39,1097,1100],{},"本次扫描使用 macOS arm64、Python 3.11.15、PyTorch 2.14.1、CPU、",[36,1091,1092],{},"float32","，固定一个 CPU 计算线程。每档均关闭梯度记录，输出与整批参考值通过 ",[36,1095,1096],{},"rtol=1e-5",[36,1098,1099],{},"atol=1e-6"," 的近似一致性检查。",[13,1102,1103,1104,1107],{},"计时使用 ",[36,1105,1106],{},"torch.utils.benchmark.Timer(..., num_threads=1).blocked_autorange(min_run_time=0.2)","。它通过分块重复运行获得测量值，并摊薄计时器调用的开销。表中的时间是完整处理全部 1000 个样本的中位耗时。",[253,1109,1110,1129],{},[309,1111,1112],{},[312,1113,1114,1117,1120,1123,1126],{},[315,1115,1116],{"align":320},"Batch size",[315,1118,1119],{"align":320},"批次数",[315,1121,1122],{"align":320},"最后一批",[315,1124,1125],{"align":320},"Linear 调用",[315,1127,1128],{"align":320},"全部样本耗时 \u002F ms",[326,1130,1131,1144,1160,1176,1192,1207,1222],{},[312,1132,1133,1135,1137,1139,1141],{},[331,1134,363],{"align":320},[331,1136,629],{"align":320},[331,1138,363],{"align":320},[331,1140,608],{"align":320},[331,1142,1143],{"align":320},"8.1494",[312,1145,1146,1149,1152,1154,1157],{},[331,1147,1148],{"align":320},"4",[331,1150,1151],{"align":320},"250",[331,1153,1148],{"align":320},[331,1155,1156],{"align":320},"500",[331,1158,1159],{"align":320},"2.1457",[312,1161,1162,1165,1168,1171,1173],{},[331,1163,1164],{"align":320},"16",[331,1166,1167],{"align":320},"63",[331,1169,1170],{"align":320},"8",[331,1172,1078],{"align":320},[331,1174,1175],{"align":320},"0.5378",[312,1177,1178,1181,1183,1186,1189],{},[331,1179,1180],{"align":320},"64",[331,1182,1164],{"align":320},[331,1184,1185],{"align":320},"40",[331,1187,1188],{"align":320},"32",[331,1190,1191],{"align":320},"0.1390",[312,1193,1194,1197,1199,1202,1204],{},[331,1195,1196],{"align":320},"128",[331,1198,1170],{"align":320},[331,1200,1201],{"align":320},"104",[331,1203,1164],{"align":320},[331,1205,1206],{"align":320},"0.0724",[312,1208,1209,1212,1214,1217,1219],{},[331,1210,1211],{"align":320},"256",[331,1213,1148],{"align":320},[331,1215,1216],{"align":320},"232",[331,1218,1170],{"align":320},[331,1220,1221],{"align":320},"0.0385",[312,1223,1224,1226,1228,1230,1232],{},[331,1225,629],{"align":320},[331,1227,363],{"align":320},[331,1229,629],{"align":320},[331,1231,338],{"align":320},[331,1233,1234],{"align":320},"0.0130",[13,1236,1237],{},[1238,1239],"img",{"alt":1240,"src":1241},"固定 1000 个样本时，batch size 从 1 增至 1000，CPU 完整处理耗时从约 8.1494 ms 降至 0.0130 ms，Linear 调用次数从 2000 降至 2；两幅图均采用对数坐标","\u002Fimages\u002Fposts\u002Fcmu-15-779-profiler-operator-batching\u002Fcmu-batch-size-sweep-20261015.webp",[13,1243,1244,1245,1248,1249,1252],{},"图中的点直接读取 ",[36,1246,1247],{},"results\u002F24_batch_size_sweep.json","，可",[22,1250,1251],{"href":1241},"打开大图查看坐标","。连线只连接已测量的档位，不表示中间每个 batch size 都做过测量。原始文件还保留了中位数、四分位距、重复测量信息、环境和脚本 SHA-256。",[13,1254,1255],{},"这一组数据里，batch size 增大后，模型调用次数和总耗时都明显下降。这个网络只有 25 个参数，每次计算很小，调用和数据组织开销值得关注。",[13,1257,1258,1259,1261],{},"计时范围包含 Python 分批循环、切片、前向计算和一次 ",[36,1260,502],{},"；不包含创建模型和输入、检查结果、Profiler 或写文件。因此它描述的是“这 1000 个已准备好的样本怎样更快算完”，没有测请求到达、等待凑批或在线服务的响应延迟。",[63,1263,1264],{"id":1264},"这次结果能说明什么",[13,1266,1267],{},"批量计算仍然处理相同的样本，只是把许多小调用合成了较少的大调用。本次算子计数和耗时测量支持这一点。",[13,1269,1270],{},"但不能据此断言 batch size 总是越大越好。这里没有改变模型大小，没有测 GPU、显存、并发请求或延迟约束。后续换成更大的网络时，需要重新测量计算量、内存占用和吞吐之间的关系。",[13,1272,1273],{},"第 23 个脚本与第 24 个脚本的计时协议也不同：前者整批分支直接调用模型，后者所有档位统一切片和拼接，并固定一个线程。两份脚本适合回答各自的对照问题，不应拿其中某两个时间直接计算跨实验的加速比。",[13,1275,1276,1277,1280],{},"整理时复跑了三份脚本，输出一致性与调用次数全部通过；另外用 7 个样本检查了 batch size 为 ",[36,1278,1279],{},"1、3、10"," 的情况，确认尾批和“大于总样本数的一批”也能完整处理输入。耗时会随系统负载变化，文章保留原始测量值。",[63,1282,1284],{"id":1283},"代码数据和复现","代码、数据和复现",[13,1286,1287,1288,1290],{},"ZIP 包含三份脚本、依赖文件、运行说明，以及原始的 ",[36,1289,1247],{},"。运行前两个脚本：",[70,1292,1296],{"className":1293,"code":1294,"language":1295,"meta":76,"style":76},"language-bash shiki shiki-themes github-dark-dimmed github-light houston","python3.11 -m venv .venv\n.venv\u002Fbin\u002Fpython -m pip install -r requirements.txt\n.venv\u002Fbin\u002Fpython 22_profile_forward.py\n.venv\u002Fbin\u002Fpython 23_profile_batching.py\n","bash",[36,1297,1298,1313,1332,1339],{"__ignoreMap":76},[123,1299,1300,1304,1307,1310],{"class":125,"line":126},[123,1301,1303],{"class":1302},"sLsJe","python3.11",[123,1305,1306],{"class":163}," -m",[123,1308,1309],{"class":267}," venv",[123,1311,1312],{"class":267}," .venv\n",[123,1314,1315,1318,1320,1323,1326,1329],{"class":125,"line":144},[123,1316,1317],{"class":1302},".venv\u002Fbin\u002Fpython",[123,1319,1306],{"class":163},[123,1321,1322],{"class":267}," pip",[123,1324,1325],{"class":267}," install",[123,1327,1328],{"class":163}," -r",[123,1330,1331],{"class":267}," requirements.txt\n",[123,1333,1334,1336],{"class":125,"line":170},[123,1335,1317],{"class":1302},[123,1337,1338],{"class":267}," 22_profile_forward.py\n",[123,1340,1341,1343],{"class":125,"line":179},[123,1342,1317],{"class":1302},[123,1344,1345],{"class":267}," 23_profile_batching.py\n",[13,1347,1348],{},"重新扫描时，把结果写到一个新文件，便于与随附的原始记录比较：",[70,1350,1352],{"className":1293,"code":1351,"language":1295,"meta":76,"style":76},".venv\u002Fbin\u002Fpython 24_batch_size_sweep.py \\\n  --output results\u002F24_batch_size_sweep-rerun.json\n",[36,1353,1354,1365],{"__ignoreMap":76},[123,1355,1356,1358,1361],{"class":125,"line":126},[123,1357,1317],{"class":1302},[123,1359,1360],{"class":267}," 24_batch_size_sweep.py",[123,1362,1364],{"class":1363},"sxpgZ"," \\\n",[123,1366,1367,1370],{"class":125,"line":144},[123,1368,1369],{"class":163},"  --output",[123,1371,1372],{"class":267}," results\u002F24_batch_size_sweep-rerun.json\n",[13,1374,1375],{},"也可以明确指定样本数和档位：",[70,1377,1379],{"className":1293,"code":1378,"language":1295,"meta":76,"style":76},".venv\u002Fbin\u002Fpython 24_batch_size_sweep.py \\\n  --samples 7 --batch-sizes 1 3 10 \\\n  --output results\u002F24_batch_size_sweep-remainder.json\n",[36,1380,1381,1389,1410],{"__ignoreMap":76},[123,1382,1383,1385,1387],{"class":125,"line":126},[123,1384,1317],{"class":1302},[123,1386,1360],{"class":267},[123,1388,1364],{"class":1363},[123,1390,1391,1394,1397,1400,1402,1405,1408],{"class":125,"line":144},[123,1392,1393],{"class":163},"  --samples",[123,1395,1396],{"class":163}," 7",[123,1398,1399],{"class":163}," --batch-sizes",[123,1401,489],{"class":163},[123,1403,1404],{"class":163}," 3",[123,1406,1407],{"class":163}," 10",[123,1409,1364],{"class":1363},[123,1411,1412,1414],{"class":125,"line":170},[123,1413,1369],{"class":163},[123,1415,1416],{"class":267}," results\u002F24_batch_size_sweep-remainder.json\n",[13,1418,1419,57,1423],{},[22,1420,1422],{"href":54,"rel":1421},[26],"本篇 GitHub 代码",[22,1424,61],{"href":60},[13,1426,1427,1428,39,1432,39,1436,39,1440,46],{},"参考资料：",[22,1429,1431],{"href":24,"rel":1430},[26],"CMU 课程安排",[22,1433,1435],{"href":31,"rel":1434},[26],"Introduction to ML Systems 课件",[22,1437,1439],{"href":405,"rel":1438},[26],"PyTorch Profiler 教程",[22,1441,1444],{"href":1442,"rel":1443},"https:\u002F\u002Fdocs.pytorch.org\u002Fdocs\u002F2.14\u002Fbenchmark_utils.html",[26],"PyTorch Benchmark API",[1446,1447,1448],"style",{},"html pre.shiki code .slEfO, html code.shiki .slEfO{--shiki-default:#F47067;--shiki-default-font-style:inherit;--shiki-light:#D73A49;--shiki-light-font-style:inherit;--shiki-cyber:#54B9FF;--shiki-cyber-font-style:italic}html pre.shiki code .sWURj, html code.shiki .sWURj{--shiki-default:#ADBAC7;--shiki-light:#24292E;--shiki-cyber:#EEF0F9}html pre.shiki code .sq3c-, html code.shiki .sq3c-{--shiki-default:#ADBAC7;--shiki-light:#24292E;--shiki-cyber:#00DAEF}html pre.shiki code .svkvy, html code.shiki .svkvy{--shiki-default:#6CB6FF;--shiki-light:#005CC5;--shiki-cyber:#EEF0F9}html pre.shiki code .sgENW, html code.shiki .sgENW{--shiki-default:#6CB6FF;--shiki-light:#005CC5;--shiki-cyber:#FFD493}html pre.shiki code .sA9aF, html code.shiki .sA9aF{--shiki-default:#F69D50;--shiki-default-font-style:inherit;--shiki-light:#E36209;--shiki-light-font-style:inherit;--shiki-cyber:#4BF3C8;--shiki-cyber-font-style:italic}html pre.shiki code .sf2cM, html code.shiki .sf2cM{--shiki-default:#F47067;--shiki-light:#D73A49;--shiki-cyber:#EEF0F9}html pre.shiki code .sWqY2, html code.shiki .sWqY2{--shiki-default:#96D0FF;--shiki-light:#032F62;--shiki-cyber:#FFD493}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html.light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html .cyber .shiki span {color: var(--shiki-cyber);background: var(--shiki-cyber-bg);font-style: var(--shiki-cyber-font-style);font-weight: var(--shiki-cyber-font-weight);text-decoration: var(--shiki-cyber-text-decoration);}html.cyber .shiki span {color: var(--shiki-cyber);background: var(--shiki-cyber-bg);font-style: var(--shiki-cyber-font-style);font-weight: var(--shiki-cyber-font-weight);text-decoration: var(--shiki-cyber-text-decoration);}html pre.shiki code .sST-V, html code.shiki .sST-V{--shiki-default:#F47067;--shiki-light:#D73A49;--shiki-cyber:#54B9FF}html pre.shiki code .sS_0i, html code.shiki .sS_0i{--shiki-default:#DCBDFB;--shiki-light:#6F42C1;--shiki-cyber:#00DAEF}html pre.shiki code .s9NhK, html code.shiki .s9NhK{--shiki-default:#ADBAC7;--shiki-default-font-style:inherit;--shiki-light:#24292E;--shiki-light-font-style:inherit;--shiki-cyber:#FFD493;--shiki-cyber-font-style:italic}html pre.shiki code .sLsJe, html code.shiki .sLsJe{--shiki-default:#F69D50;--shiki-light:#6F42C1;--shiki-cyber:#00DAEF}html pre.shiki code .sxpgZ, html code.shiki .sxpgZ{--shiki-default:#F47067;--shiki-light:#005CC5;--shiki-cyber:#EEF0F9}",{"title":76,"searchDepth":144,"depth":170,"links":1450},[1451,1452,1453,1454,1455,1456,1457],{"id":65,"depth":144,"text":65},{"id":108,"depth":144,"text":109},{"id":414,"depth":144,"text":415},{"id":673,"depth":144,"text":674},{"id":1086,"depth":144,"text":1086},{"id":1264,"depth":144,"text":1264},{"id":1283,"depth":144,"text":1284},[1459],"技术","\u002Fimages\u002Fposts\u002Fcmu-15-779-profiler-operator-batching\u002Fcmu-pytorch-profiling-cover-20261015.webp","放大镜观察线性层与 ReLU 模块，旁边面板呈现执行片段的浅色技术教具插画","2026-09-15","对照 ML Systems 基础，用 Profiler 拆开两层网络的执行过程，再固定 1000 个样本扫描 batch size，观察算子调用次数、尾批和 CPU 耗时。",false,"md",{"codeUrl":54,"attachmentUrl":60},"\u002Fposts\u002Fcmu-15-779-profiler-operator-batching",null,{"title":5,"description":1463},"CMU 15-779","5","posts\u002Fcmu-15-779-profiler-operator-batching",[1470,1474,1475,1476,1477],"PyTorch","Profiler","批量推理","性能分析","dhjgSZbh1pZvFLIAxFP7wcKRymT-t05j0pKtQfjaToI",[1480,1497,1513,1527,1541,1555,1570,1585,1599,1611,1626,1638,1651,1663,1676,1688,1704,1719,1730,1742,1757,1770,1784,1796,1810,1822,1826,1840,1854,1865,1878,1895,1910,1923,1935,1947,1960,1973],{"slug":1481,"path":1482,"title":1483,"date":1484,"tags":1485,"description":1490,"cover":1491,"coverAlt":1492,"excerpt":1493,"draft":1464,"hidden":1464,"published":182,"readingTime":238,"series":1494,"seriesOrder":238,"codeUrl":1495,"attachmentUrl":1496},"cs336-cross-entropy-adamw-training-loop","\u002Fposts\u002Fcs336-cross-entropy-adamw-training-loop","Stanford CS336 学习记录 08：数值稳定交叉熵、AdamW、余弦调度与完整训练闭环","2026-10-27",[1486,1474,1487,1488,1489],"CS336","AdamW","CrossEntropy","训练闭环","用 LogSumExp 消去交叉熵中的指数溢出，手写解耦权重衰减的 AdamW 优化器、全局 L2 梯度裁剪、Warmup 余弦退火调度、memmap 数据采样、Checkpoint 序列化与 Top-p 解码，完成 CS336 Assignment 1 全部 48 项测试。","\u002Fimages\u002Fposts\u002Fcs336-cross-entropy-adamw-training-loop\u002Fcs336-cross-entropy-adamw-training-loop-cover.webp","滑动窗口采样、余弦退火曲线、双动量优化轨迹与检查点归档构成的训练闭环 AI 辅助概念插画","作为 Stanford CS336 Assignment 1 系列的收官篇，我们已经拥有了分词器（bpe.py、tokenizer.py）和完整的语言模型架构（transformer.py）。要让模型在语料上真正训练并生成文本，还需","Stanford CS336","https:\u002F\u002Fgithub.com\u002Fliangqianxing\u002Fcs336-labs\u002Ftree\u002Fb5c93ad0cc73178b2b2a62e518a0900da47a0fb7","\u002Fmaterials\u002Fcs336\u002Fexperiments\u002Fcs336-08-adamw-training-loop-20261027.zip",{"slug":1498,"path":1499,"title":1500,"date":1501,"tags":1502,"description":1509,"cover":1510,"coverAlt":1511,"excerpt":1512,"draft":1464,"hidden":1464,"published":182,"readingTime":238},"editcache-video-dit-inference-acceleration","\u002Fposts\u002Feditcache-video-dit-inference-acceleration","EditCache：视频 Diffusion Transformer (DiT) 局部编辑的跨步特征复用与区域重算加速","2026-10-25",[1503,1504,1505,1506,1507,1508],"扩散模型","视频生成","AI Infra","推理加速","GPU","论文解读","拆解视频扩散模型（Wan2.1-VACE）局部编辑中的三重算力冗余，通过可重入 Latent、跨步 Block 隐状态缓存复用、带 Halo 边界的局部区域重算与自适应推理起点，在单卡 H100 上实现 2.46× 模型计算加速与 1.49× 严格同步端到端加速。","\u002Fimages\u002Fposts\u002Feditcache-video-dit-inference-acceleration\u002Feditcache-dit-cover-20261025.webp","视频扩散 Transformer 时空三维体素网格中冻结背景缓存与暖橙色局部编辑重算通道的等距技术插画","在基于 Diffusion Transformer（DiT）的视频编辑与局部重绘（Video Inpainting \u002F Local Editing）任务中，用户的典型工作流并不是一次性生成最终成片，而是针对同一段源视频反复调整掩码区",{"slug":1514,"path":1515,"title":1516,"date":1501,"tags":1517,"description":1522,"cover":1523,"coverAlt":1524,"excerpt":1525,"draft":1464,"hidden":1464,"published":182,"readingTime":238,"series":1494,"seriesOrder":233,"codeUrl":1495,"attachmentUrl":1526},"cs336-bpe-tokenizer-training-encoding","\u002Fposts\u002Fcs336-bpe-tokenizer-training-encoding","Stanford CS336 学习记录 07：Byte-Level BPE 分词器训练、倒排索引加速与流式编码",[1486,1518,1519,1520,1521],"Tokenizer","BPE","Tiktoken","算法优化","实现 GPT-2 正则预分词、特殊 token 硬边界切分、基于 pair-to-word 倒排索引的增量 BPE 合并算法（\u003C 0.25s 通过速度基准），以及与 tiktoken 完全对齐的 Tokenizer。","\u002Fimages\u002Fposts\u002Fcs336-bpe-tokenizer-training-encoding\u002Fcs336-bpe-tokenizer-training-encoding-cover.webp","UTF-8 字节块经过正则切分闸口后沿合并树聚合成子词单元的 AI 辅助概念插画","前面六篇完成了 Transformer 语言模型本身的全部神经网络组件。要让模型真正处理自然语言文本，还需要打通入口处的 ByteLevel BPE（BytePair Encoding）分词器。这一篇记录 cs336basics\u002Fb","\u002Fmaterials\u002Fcs336\u002Fexperiments\u002Fcs336-07-bpe-tokenizer-20261025.zip",{"slug":1528,"path":1529,"title":1530,"date":1531,"tags":1532,"description":1537,"cover":1538,"coverAlt":1539,"excerpt":1540,"draft":1464,"hidden":1464,"published":182,"readingTime":238},"in-situ-autoguidance-editable-scientific-illustration","\u002Fposts\u002Fin-situ-autoguidance-editable-scientific-illustration","从原位自校正引导（In-situ Autoguidance）到可编辑科研插图生成（AutoFigure-Edit）","2026-10-24",[1503,1533,1534,1535,1536,1508],"多模态","Agent","ICML","ACL","串联两项生成式模型与多模态智能体研究：如何通过推理期局部随机微扰在单模型内构造免训练扩散自引导向量（ICML 2025 Workshop 第一作者，arXiv:2510.17136），以及如何通过五阶段视觉-符号混合流水线生成原生可编辑 SVG 科研插图（ACL 2026 Demo，FigureBench 83.0% 胜率）。","\u002Fimages\u002Fposts\u002Fin-situ-autoguidance-editable-scientific-illustration\u002Fautoguidance-autofigure-cover-20261024.webp","左侧扩散流形自校正轨迹矢量场与右侧五阶段可编辑 SVG 科研插图分层网格相衔接的等距技术插画","生成式视觉模型正在经历从“生成高保真像素（Pixel Synthesis）”向“可控结构化生成（Controllable & Editable Generation）”的演进。在这一进程中，我作为第一作者或核心贡献者参与了两项分别处",{"slug":1542,"path":1543,"title":1544,"date":1545,"tags":1546,"description":1550,"cover":1551,"coverAlt":1552,"excerpt":1553,"draft":1464,"hidden":1464,"published":182,"readingTime":238,"series":1494,"seriesOrder":220,"codeUrl":1495,"attachmentUrl":1554},"cs336-pre-norm-transformer-block-lm","\u002Fposts\u002Fcs336-pre-norm-transformer-block-lm","Stanford CS336 学习记录 06：Pre-Norm Transformer Block 与完整语言模型装配","2026-10-23",[1486,1474,1547,1548,1549],"Transformer","Pre-Norm","FLOPs","将 RMSNorm、带 RoPE 的因果多头自注意力和 SwiGLU 组装成 Pre-Norm Transformer Block，搭建完整 TransformerLM，并核算 GPT-2 XL 的参数量与前向 FLOPs。","\u002Fimages\u002Fposts\u002Fcs336-pre-norm-transformer-block-lm\u002Fcs336-pre-norm-transformer-block-lm-cover.webp","干净残差流主干穿过多个 Pre-Norm 子层并汇聚到词表投影层的 AI 辅助概念插画","前五篇把 Linear、Embedding、RMSNorm、SwiGLU、RotaryPositionalEmbedding 和因果 MultiHeadSelfAttention 全部单独打通了。这一篇进入 Assignment 1","\u002Fmaterials\u002Fcs336\u002Fexperiments\u002Fcs336-06-transformer-block-lm-20261023.zip",{"slug":1556,"path":1557,"title":1558,"date":1545,"tags":1559,"description":1565,"cover":1566,"coverAlt":1567,"excerpt":1568,"draft":1464,"hidden":1464,"published":182,"readingTime":1569},"xgrammar-concurrent-cache-generation-fix","\u002Fposts\u002Fxgrammar-concurrent-cache-generation-fix","XGrammar 结构化解码缓存剖析：Unlimited 模式并发 Clear 字节记账竞态与代际修复（PR #1011）",[1505,1560,1561,1562,1563,1564],"C++","XGrammar","结构化解码","并发编程","源码分析","拆解 XGrammar GrammarCompiler 的双层缓存架构与 ThreadSafeLRUCache 锁分离实现，定位 Unlimited 模式下 Clear() 字节残留与在途编译（In-flight Computation）跨代污染根因，给出代际号 + 读写锁原子修复与零 sleep 确定性并发 GoogleTest 验证。","\u002Fimages\u002Fposts\u002Fxgrammar-concurrent-cache-generation-fix\u002Fxgrammar-cache-fix-cover-20261022.webp","双层磨砂玻璃流道经暖橙色代际闸门过滤后汇入晶格缓存矩阵的等距技术插画","在大语言模型服务（如 SGLang、vLLM）的结构化输出场景中，JSON Schema 或 EBNF 语法需要先编译为词表级掩码缓存（CompiledGrammar），才能在每个 Decode 步以微秒级开销生成合法 token ",14,{"slug":1571,"path":1572,"title":1573,"date":1574,"tags":1575,"description":1581,"cover":1582,"coverAlt":1583,"excerpt":1584,"draft":1464,"hidden":1464,"published":182,"readingTime":1569},"multi-constraint-agent-routing-dag-eval","\u002Fposts\u002Fmulti-constraint-agent-routing-dag-eval","多约束生产级 Agent 系统实践：两阶段工具路由、DAG 局部重规划与轨迹级评测闭环","2026-10-22",[1534,1576,1577,1578,1579,1580],"LLM","工具调用","上下文工程","DPO","强化学习","围绕跨境酒旅等强耦合多约束业务场景，拆解向量 Top-K 召回 + LLM 精排 + 跨语言实体链接与 Schema 门控（96%+ 调用合法率）、基于状态机的 DAG 局部子图重规划、Observation 冗余裁剪（节省 40% Token），以及从线上失败恢复轨迹构造 SFT\u002FDPO 偏好对的评测与训练闭环。","\u002Fimages\u002Fposts\u002Fmulti-constraint-agent-routing-dag-eval\u002Fagent-dag-routing-cover-20261020.webp","候选模块经漏斗滤网进入双分支有向无环图轨道并在受阻节点触发暖橙色局部旁路重连的等距技术插画","在玩具级 Demo 里，几个工具加上一段 ReAct 循环就能跑通大多数单步问答。但当 Agent 进入跨境酒旅、复杂交易或研发沙箱等真实业务生产环境时，单靠线性 ReAct 循环很快会撞上四堵墙： 1. 工具空间与参数耦合爆炸：候",{"slug":1586,"path":1587,"title":1588,"date":1589,"tags":1590,"description":1594,"cover":1595,"coverAlt":1596,"excerpt":1597,"draft":1464,"hidden":1464,"published":182,"readingTime":1598},"llm-mrd-distillation-auditable-review-agents","\u002Fposts\u002Fllm-mrd-distillation-auditable-review-agents","多视角推理蒸馏（LLM-MRD）与可审计生成式评审系统（DeepReviewer 2.0）深度拆解","2026-10-21",[1591,1533,1592,1534,1593,1508],"知识蒸馏","Qwen2.5-VL","Post-training","对比大模型落地的两条互补路径：如何用 Qwen2.5-VL 离线多视角分析结合 MLP 残差校准与温度缩放 KL 蒸馏，将推理能力压入 358M 学生模型实现 17.2 ms 独立推理（LLM-MRD，DASFAA 2026 Oral）；以及如何通过主张-证据-风险台账、PASA 可比性门控与导出预算校验构建可审计的同行评审 Agent（DeepReviewer 2.0）。","\u002Fimages\u002Fposts\u002Fllm-mrd-distillation-auditable-review-agents\u002Fmrd-review-distill-cover-20261016.webp","三块半透明几何棱镜发出平行光束穿过中央校准环汇聚至右侧高密度晶体模块的等距技术插画","把大语言模型或视觉语言模型（VLM）的深度推理能力用到实际系统中，通常会面临一个两难选择： 如果场景受限于毫秒级在线延迟（如高吞吐多模态内容审核与虚假信息检测）：直接在线调用 7B～72B 的多模态大模型做长链条思维链（CoT）生成",18,{"slug":1600,"path":1601,"title":1602,"date":1589,"tags":1603,"description":1606,"cover":1607,"coverAlt":1608,"excerpt":1609,"draft":1464,"hidden":1464,"published":182,"readingTime":233,"series":1494,"seriesOrder":186,"codeUrl":1495,"attachmentUrl":1610},"cs336-rope-rotary-position-embedding","\u002Fposts\u002Fcs336-rope-rotary-position-embedding","Stanford CS336 学习记录 05：RoPE 旋转位置编码与带位置的多头注意力",[1486,1474,1547,1604,1605],"RoPE","Attention","从二维平面正交旋转推导 RoPE 的奇偶特征配对公式，理解为什么内积只依赖相对位置差 (i - j)，并用 register_buffer 预计算 cos\u002Fsin 缓存接入因果多头自注意力。","\u002Fimages\u002Fposts\u002Fcs336-rope-rotary-position-embedding\u002Fcs336-rope-rotary-position-embedding-cover.webp","二维特征平面中的旋转向量对与预计算余弦正弦网格的 AI 辅助概念插画","上一篇完成了不含位置编码的因果多头自注意力。如果只看那一套矩阵乘法，把输入序列前两个 token 交换顺序（并在因果 mask 允许的范围内看同一个后缀 query 对它们的点积），点积本身并不知道哪个 key 离当前 query ","\u002Fmaterials\u002Fcs336\u002Fexperiments\u002Fcs336-05-rope-mha-20261021.zip",{"slug":1612,"path":1613,"title":1614,"date":1615,"tags":1616,"description":1622,"cover":1623,"coverAlt":1624,"excerpt":1625,"draft":1464,"hidden":1464,"published":182,"readingTime":238},"llm-gateway-semantic-cache-dynamic-batching","\u002Fposts\u002Fllm-gateway-semantic-cache-dynamic-batching","高并发 LLM 推理网关架构实践：两级精确\u002F语义缓存、50ms 有界动态批处理与尾部延迟治理","2026-10-19",[1505,1617,1618,1619,1620,1621],"LLM 推理","分布式系统","Redis","FastAPI","后端","深度拆解生产级大模型推理网关 LLM-Gateway 的核心系统设计：基于 SHA-256 与 Redis 向量余弦相似度的两级精确\u002F语义缓存、50ms 有界等待动态批处理调度器（Dynamic Batching）、令牌桶多租户限流与 Prometheus 全链路可观测性，将命中请求 P99 延迟从 5,307 ms 压降至 17 ms。","\u002Fimages\u002Fposts\u002Fllm-gateway-semantic-cache-dynamic-batching\u002Fllm-gateway-cover-20261019.webp","高并发请求流依次穿过精确哈希滤网、向量球面语义缓存层与 50ms 动态批处理时间窗拼装器的等距系统架构插画","在将大语言模型接入高并发业务系统时，如果让上游微服务直接裸连底层推理引擎（vLLM \u002F SGLang 或外部模型 API），系统很快会暴露出三个典型的生产痛点： 1. 高频重复与近义查询消耗巨量算力：大量用户请求在语义上高度等价（例",{"slug":1627,"path":1628,"title":1629,"date":1615,"tags":1630,"description":1632,"cover":1633,"coverAlt":1634,"excerpt":1635,"draft":1464,"hidden":1464,"published":182,"readingTime":1569,"series":1494,"seriesOrder":179,"codeUrl":1636,"attachmentUrl":1637},"cs336-multihead-attention-shapes","\u002Fposts\u002Fcs336-multihead-attention-shapes","Stanford CS336 学习记录 04：多头注意力怎样拆头、计算和合头",[1486,1474,1547,1605,1631],"张量形状","沿着 (2, 5, 64) 的输入追踪 QKV 投影、reshape、transpose、共享因果 mask 和合头，理解每个头仍处理完整序列，以及最后为什么还需要输出 Linear。","\u002Fimages\u002Fposts\u002Fcs336-multihead-attention-shapes\u002Fcs336-multihead-attention-shapes-cover.webp","宽向量束分成多条并行特征轨道后重新合并的 AI 辅助概念插画","上一篇完成了缩放点积注意力。那个函数接收已经准备好的 Q、K、V，计算匹配分数，应用 mask，再用权重汇总 V。这一篇继续把它放进一个带参数的多头自注意力模块。 这次最需要看清的是“拆头”。代码里只是几行 reshape 和 tr","https:\u002F\u002Fgithub.com\u002Fliangqianxing\u002Fcs336-labs\u002Ftree\u002F31363223498fafd1528c550a51efbe0aed231736","\u002Fmaterials\u002Fcs336\u002Fexperiments\u002Fcs336-04-multihead-attention-20261019.zip",{"slug":1639,"path":1640,"title":1641,"date":1642,"tags":1643,"description":1647,"cover":1648,"coverAlt":1649,"excerpt":1650,"draft":1464,"hidden":1464,"published":182,"readingTime":259},"qwen25-sft-dpo-alignment-tax-ablation","\u002Fposts\u002Fqwen25-sft-dpo-alignment-tax-ablation","大模型后训练偏好对齐实录：Qwen2.5 SFT 多维消融、动态参考 DPO 与对齐税（Alignment Tax）量化分析","2026-10-18",[1593,1579,1644,1645,1580,1646],"SFT","LoRA","Qwen2.5","基于 Qwen2.5 完整复现 Instruct SFT → Bradley-Terry DPO 两阶段后训练管线：通过 3 随机种子消融 LoRA Rank、学习率与数据规模（GSM8K +12.73pp），实现免双倍权重的动态 Adapter 切换 DPO 训练，并定量揭示偏好对齐对确定性数学推理的对齐税效应。","\u002Fimages\u002Fposts\u002Fqwen25-sft-dpo-alignment-tax-ablation\u002Fqwen25-sft-dpo-cover-20261018.webp","左侧监督微调轨迹流经中央 Bradley-Terry 偏好分流天平分离出正负样本通道并连接右侧对齐税监控仪表盘的等距技术插画","在大语言模型后训练（PostTraining）管线中，监督微调（SFT）负责教会模型任务格式与领域推理模式，而直接偏好优化（DPO, Direct Preference Optimization）则利用成对偏好数据 $(x, yw,",{"slug":1652,"path":1653,"title":1654,"date":1642,"tags":1655,"description":1657,"cover":1658,"coverAlt":1659,"excerpt":1660,"draft":1464,"hidden":1464,"published":182,"readingTime":287,"series":1494,"seriesOrder":170,"codeUrl":1661,"attachmentUrl":1662},"cs336-softmax-attention-mask","\u002Fposts\u002Fcs336-softmax-attention-mask","Stanford CS336 学习记录 03：Softmax 的数值稳定与 Attention Mask",[1486,1474,1547,1656,1605],"Softmax","从减去最大值为何不改变 Softmax 开始，理清函数与有参数层的区别，再沿着 Q、K、V 的形状理解缩放点积注意力，以及 mask 为什么必须填负无穷。","\u002Fimages\u002Fposts\u002Fcs336-softmax-attention-mask\u002Fcs336-softmax-attention-mask-cover.webp","注意力分数、权重与因果遮罩的 AI 辅助概念插画","上一篇写到 SwiGLU，接下来进入 Attention。真正开始相乘之前，先做了 Softmax：它把一组分数变成总和为 1 的权重，后面用这些权重汇总不同 token 提供的信息。 这段学习里，有两个很具体的问题值得记下来。一个","https:\u002F\u002Fgithub.com\u002Fliangqianxing\u002Fcs336-labs\u002Ftree\u002Fa39eb684710817f7f389626423fcb26ca0f6d629","\u002Fmaterials\u002Fcs336\u002Fexperiments\u002Fcs336-03-softmax-sdpa-20261018.zip",{"slug":1664,"path":1665,"title":1666,"date":1667,"tags":1668,"description":1671,"cover":1672,"coverAlt":1673,"excerpt":1674,"draft":1464,"hidden":1464,"published":182,"readingTime":259,"series":1494,"seriesOrder":144,"codeUrl":1661,"attachmentUrl":1675},"cs336-rmsnorm-swiglu","\u002Fposts\u002Fcs336-rmsnorm-swiglu","Stanford CS336 学习记录 02：RMSNorm 的广播与 SwiGLU 的门控","2026-10-16",[1486,1474,1547,1669,1670],"RMSNorm","SwiGLU","用一个 token 向量理解均方根归一化，拆开 dim=-1 与 keepdim=True 的作用，再沿着两条分支读懂 SiLU、逐元素乘法和 SwiGLU。","\u002Fimages\u002Fposts\u002Fcs336-rmsnorm-swiglu\u002Fcs336-rmsnorm-swiglu-cover.webp","特征尺度整理和双分支门控的 AI 辅助概念插画","上一篇把 Linear 和 Embedding 的参数形状理顺了。继续往下，最先遇到的是 RMSNorm：公式不算长，但把它落到一批 token 上时，dim=1、keepdim=True 和广播必须一起理解。 然后是 SiLU 和","\u002Fmaterials\u002Fcs336\u002Fexperiments\u002Fcs336-02-rmsnorm-swiglu-20261016.zip",{"slug":1677,"path":1678,"title":1679,"date":1680,"tags":1681,"description":1683,"cover":1684,"coverAlt":1685,"excerpt":1686,"draft":1464,"hidden":1464,"published":182,"readingTime":274,"series":1494,"seriesOrder":126,"codeUrl":1661,"attachmentUrl":1687},"cs336-linear-embedding-shapes","\u002Fposts\u002Fcs336-linear-embedding-shapes","Stanford CS336 学习记录 01：从 Linear、Embedding 搞清张量形状","2026-10-14",[1486,1474,1547,384,1682],"Embedding","从 PyTorch 类、参数与 forward 的关系开始，理解 Linear 为什么保存两行三列的权重、计算时为什么要转置，再区分 Embedding 查表与线性变换。","\u002Fimages\u002Fposts\u002Fcs336-linear-embedding-shapes\u002Fcs336-linear-embedding-shapes-cover.webp","token 方块、向量表与矩阵变换的 AI 辅助概念插画","开始沿着 Stanford CS336 的 Assignment 1 一步步实现。BPE 之前已经接触过，这轮先跳过分词器，从 Transformer 的基础层开始。这里记录的是当时写代码时真正卡住的问题，以及对应测试能确认到的范围","\u002Fmaterials\u002Fcs336\u002Fexperiments\u002Fcs336-01-linear-embedding-20261014.zip",{"slug":1689,"path":1690,"title":1691,"date":1692,"tags":1693,"description":1697,"cover":1698,"coverAlt":1699,"excerpt":1700,"draft":1464,"hidden":1464,"published":182,"readingTime":1701,"series":1470,"seriesOrder":1569,"codeUrl":1702,"attachmentUrl":1703},"cmu-15-779-kv-cache-granularity-study","\u002Fposts\u002Fcmu-15-779-kv-cache-granularity-study","CMU 15-779 学习记录 14：KV 页大小、共享边界与 CPU 实现成本","2026-10-12",[1470,1694,1695,1696],"KV cache","PagedAttention","系统实验","用 210 组 CPU 配置比较连续与分页 KV cache，观察 31\u002F32-token 共享前缀的页边界，分别记录活跃槽位、池容量、复制计数和完整调用耗时。","\u002Fimages\u002Fposts\u002Fcmu-15-779-kv-serving-speculative-sampling\u002Fcmu-serving-cover-20261019.webp","浅色手绘分页存储抽屉与共享前缀卡片，旁边连接不同长度的请求队列","个人课程项目中的 CPU 实验记录，延续记录 11 的分页 KV 实现。已完成 210 组配置与 1470 次计时，输出均通过连续存储参考对照；本篇不代表正式作业提交、整门课程或项目全部交付完成。 同样是共享 31 个 token，",12,"https:\u002F\u002Fgithub.com\u002Fliangqianxing\u002Fcmu-15-779-labs\u002Ftree\u002F200faf569eccf94fb125e65110bb5d14094d454a","\u002Fmaterials\u002Fcmu-15-779\u002Fexperiments\u002Fkv-cache-granularity-20261019.zip",{"slug":1705,"path":1706,"title":1707,"date":1708,"tags":1709,"description":1712,"cover":1713,"coverAlt":1714,"excerpt":1715,"draft":1464,"hidden":1464,"published":182,"readingTime":186,"series":1470,"seriesOrder":1716,"codeUrl":1717,"attachmentUrl":1718},"cmu-15-779-moe-routing-capacity","\u002Fposts\u002Fcmu-15-779-moe-routing-capacity","CMU 15-779 学习记录 13：MoE 路由、稀疏分发与容量溢出","2026-10-09",[1470,1710,1711],"MoE","Routing","用 CPU top-2 MoE 检查稀疏分发的输出与梯度，统计 expert-token 行数，并保留容量丢弃会改变输出的反例。","\u002Fimages\u002Fposts\u002Fcmu-15-779-moe-routing-capacity\u002Fcmu-moe-cover-20261019.webp","浅色手绘分拣器将小珠分配到四个专家工作屋，队列呈现负载不均","第 18 讲的个人配套实验。36 号脚本串行运行 CPU 专家，检查 topk 路由后的分发与合并；没有训练 router，也没有执行 GPU grouped GEMM 或跨设备 alltoall。 上一节记录低秩适配，这次转向每个",13,"https:\u002F\u002Fgithub.com\u002Fliangqianxing\u002Fcmu-15-779-labs\u002Ftree\u002Fc4851e0cedadf13af7c421867e83c8ff57244fa0","\u002Fmaterials\u002Fcmu-15-779\u002Fexperiments\u002Fcpu-parallel-serving-lora-moe-20261019.zip",{"slug":1720,"path":1721,"title":1722,"date":1723,"tags":1724,"description":1726,"cover":1727,"coverAlt":1728,"excerpt":1729,"draft":1464,"hidden":1464,"published":182,"readingTime":186,"series":1470,"seriesOrder":1701,"codeUrl":1717,"attachmentUrl":1718},"cmu-15-779-lora-uniform-quantization","\u002Fposts\u002Fcmu-15-779-lora-uniform-quantization","CMU 15-779 学习记录 12：LoRA 首步梯度、权重合并与均匀 4-bit 打包","2026-10-06",[1470,1645,1725],"Quantization","用一次 CPU SGD 更新检查 LoRA 初始化、冻结基座与合并等价，再分别记录均匀 4-bit 存储字节数和量化误差。","\u002Fimages\u002Fposts\u002Fcmu-15-779-lora-uniform-quantization\u002Fcmu-lora-cover-20261019.webp","浅色手绘的大计算板、两块细长适配板和成对打包的小方块","第 17 讲的个人配套实验，35 号脚本实际执行一次 CPU LoRA 更新，并实现均匀 4bit 打包。实验检查机制与数值误差，不评估微调任务效果，也没有实现完整 QLoRA。 这次把两个问题分开记录：低秩分支是否能合并进原权重，",{"slug":1731,"path":1732,"title":1733,"date":1734,"tags":1735,"description":1739,"cover":1698,"coverAlt":1740,"excerpt":1741,"draft":1464,"hidden":1464,"published":182,"readingTime":233,"series":1470,"seriesOrder":287,"codeUrl":1717,"attachmentUrl":1718},"cmu-15-779-kv-serving-speculative-sampling","\u002Fposts\u002Fcmu-15-779-kv-serving-speculative-sampling","CMU 15-779 学习记录 11：分页 KV、连续批处理与推测采样","2026-10-03",[1470,1736,1737,1738],"LLM Serving","KV Cache","Speculative Decoding","在 CPU 上验证 KV 页共享和写时复制，用单位成本模型检查连续批处理，再用条件概率小模型核对推测采样的接受与补采样。","浅色手绘的共享卡片抽屉与不断补入小珠子的请求托盘","第 15、16 讲的个人配套实验。33 号实现 CPU 分页 KV 存储，37 号模拟请求准入，34 号运行小概率模型的推测采样。它们分别验证存储、调度和分布问题，没有测量真实 LLM 服务吞吐或 GPU 延迟。 记录 08已经对照",{"slug":1743,"path":1744,"title":1745,"date":1746,"tags":1747,"description":1752,"cover":1753,"coverAlt":1754,"excerpt":1755,"draft":1464,"hidden":1464,"published":182,"readingTime":1756},"qwen3-mid-post-train-mlx","\u002Fposts\u002Fqwen3-mid-post-train-mlx","在 M2 Pro 上完成 Qwen3 两阶段训练：Mid Train、SFT 与可复核评估","2026-09-30 11:19:10",[1576,1748,1749,1645,1750,1644,1751],"Qwen3","MLX","持续预训练","Apple Silicon","用 Qwen3 0.6B Base 和 MLX 完成领域持续训练与监督微调，记录数据划分、聊天模板、损失掩码、数值稳定性修复和独立测试结果。","\u002Fimages\u002Fposts\u002Fqwen3-mid-post-train-mlx\u002Fqwen3-mlx-cover-20261018.webp","Apple Silicon 统一内存芯片底座支撑两阶段持续预训练与带精确 Loss 掩码滤网的监督微调桥接插画","在一台 M2 Pro、16 GB 内存的 Mac 上，能否把一个 Base 语言模型接续训练成输出 JSON 的运维事件分级助手？这次项目完成了领域持续训练、监督微调、检查点选择、重新加载和独立测试，并保留了权重、日志和数据哈希。 ",16,{"slug":1758,"path":1759,"title":1760,"date":1761,"tags":1762,"description":1766,"cover":1767,"coverAlt":1768,"excerpt":1769,"draft":1464,"hidden":1464,"published":182,"readingTime":220,"series":1470,"seriesOrder":274,"codeUrl":1717,"attachmentUrl":1718},"cmu-15-779-cpu-parallelism-pipeline","\u002Fposts\u002Fcmu-15-779-cpu-parallelism-pipeline","CMU 15-779 学习记录 10：加权梯度、状态分片与同步流水线","2026-09-30",[1470,1763,1764,1765],"Parallel Training","Gloo","Pipeline","用双 CPU 进程 Gloo 检查不等数据分片的梯度，推导持久状态字节数，再用依赖模型比较 GPipe 与同步 1F1B 的激活存活。","\u002Fimages\u002Fposts\u002Fcmu-15-779-cpu-parallelism-pipeline\u002Fcmu-parallel-cover-20261019.webp","浅色手绘的四阶段桌面流水线与按数量汇总的托盘","第 12 至 14 讲的个人配套实验，脚本编号 31、32 不对应官方作业。Gloo 实际运行两个 CPU 进程；张量切分、状态字节数和流水线分别按下文的数学或成本模型解释。真实 GPU 实测按学习安排跳过。 上一节检查单个计算图的",{"slug":1771,"path":1772,"title":1773,"date":1774,"tags":1775,"description":1778,"cover":1779,"coverAlt":1780,"excerpt":1781,"draft":1464,"hidden":1464,"published":182,"readingTime":233,"series":1470,"seriesOrder":259,"codeUrl":1782,"attachmentUrl":1783},"cmu-15-779-cpu-compiler-optimizations","\u002Fposts\u002Fcmu-15-779-cpu-compiler-optimizations","CMU 15-779 学习记录 09：CPU 编译优化、合法重写与候选搜索","2026-09-27",[1470,1776,1777,1474],"ML Compiler","TorchInductor","用 CPU 实验验证 BatchNorm 折叠、RMSNorm 重排与非线性反例，测量分块候选，并运行 TorchInductor，分别记录首调用与预热后的时间。","\u002Fimages\u002Fposts\u002Fcmu-15-779-cpu-compiler-optimizations\u002Fcmu-cpu-compiler-cover-20261019.webp","淡紫计算积木经过薄荷绿折叠模具变成浅蓝积木，前方摆放四种候选托盘的浅色手绘插画","编译章节的个人配套实验，编号 30 不对应官方作业。图重写、分块候选和 TorchInductor 均在 CPU 上运行；真实 GPU 实验按学习安排跳过。 上一节记录了分块 Attention 和 KV cache。这次转到编译优","https:\u002F\u002Fgithub.com\u002Fliangqianxing\u002Fcmu-15-779-labs\u002Ftree\u002F9261bea6aedf31cdccbbab2541ca693fc97b3743","\u002Fmaterials\u002Fcmu-15-779\u002Fexperiments\u002Fcpu-compiler-optimizations-20261019.zip",{"slug":1785,"path":1786,"title":1787,"date":1788,"tags":1789,"description":1791,"cover":1792,"coverAlt":1793,"excerpt":1794,"draft":1464,"hidden":1464,"published":182,"readingTime":1701,"series":1470,"seriesOrder":238,"codeUrl":1702,"attachmentUrl":1795},"cmu-15-779-attention-online-softmax","\u002Fposts\u002Fcmu-15-779-attention-online-softmax","CMU 15-779 学习记录 08：分块 Attention、在线 Softmax 与 KV cache","2026-09-24",[1470,1605,1656,1790,1474],"FlashAttention","在 CPU 上对照完整 Attention、在线 Softmax 分块实现与 PyTorch SDPA，检查因果掩码、浮点误差和分数块大小，理解 FlashAttention 的一部分算法基础。","\u002Fimages\u002Fposts\u002Fcmu-15-779-attention-online-softmax\u002Fcmu-online-attention-cover-20261016.webp","浅紫色卡片分批进入绿色计算托盘，旁边三个状态容器保存累积量的浅色手绘插画","第 5 讲 Transformers & Attention 的个人配套实验。28\u002F29 号是 CPU 单头前向与 KV cache；后续补入 38 号 CUDA 多头合成张量的显式 SDPA MATH\u002FFLASH 对照。没有手写 ","\u002Fmaterials\u002Fcmu-15-779\u002Fexperiments\u002Fcuda-triton-sdpa-20261019.zip",{"slug":1797,"path":1798,"title":1799,"date":1800,"tags":1801,"description":1806,"cover":1807,"coverAlt":1808,"excerpt":1809,"draft":1464,"hidden":1464,"published":182,"readingTime":274,"series":1470,"seriesOrder":233,"codeUrl":1702,"attachmentUrl":1795},"cmu-15-779-matmul-tiling-tree-reduction","\u002Fposts\u002Fcmu-15-779-matmul-tiling-tree-reduction","CMU 15-779 学习记录 07：矩阵分块、数据复用与树形归约","2026-09-21",[1470,1802,1803,1804,1805],"CUDA","矩阵乘法","数据复用","归约","用 CPU 模型统计矩阵分块的输入读取次数，再比较串行求和和树形归约的依赖深度，记录补零、同步与浮点加法顺序的边界。","\u002Fimages\u002Fposts\u002Fcmu-15-779-matmul-tiling-tree-reduction\u002Fcmu-matmul-reduction-cover-20261016.webp","矩阵积木在共享托盘中复用，旁边的小珠沿树形管道逐层合并的浅色手绘插画","GPU Architecture & CUDA Programming 的个人配套实验。首次记录用 CPU 模型检查数据复用与依赖深度；后续补入单卡 GPU 原生 CUDA 矩阵验证，以及 Triton 两阶段归约和矩阵乘法实测。 ",{"slug":1811,"path":1812,"title":1813,"date":1814,"tags":1815,"description":1818,"cover":1819,"coverAlt":1820,"excerpt":1821,"draft":1464,"hidden":1464,"published":182,"readingTime":274,"series":1470,"seriesOrder":220,"codeUrl":1702,"attachmentUrl":1795},"cmu-15-779-cuda-thread-indexing","\u002Fposts\u002Fcmu-15-779-cuda-thread-indexing","CMU 15-779 学习记录 06：CUDA 线程索引与边界检查","2026-09-18",[1470,1802,1507,1816,1817],"线程索引","边界检查","用 CPU 演示 CUDA 的 block、thread 与全局下标：10 个元素如何分给 3 个线程块，为什么要跳过越界线程，以及这些验证能说明什么。","\u002Fimages\u002Fposts\u002Fcmu-15-779-cuda-thread-indexing\u002Fcmu-cuda-indexing-cover-20261015.webp","分组的浅紫色计算方块与末端边界挡板组成线程分配教具的浅色插画","GPU Architecture & CUDA Programming 的个人配套练习。首次记录为 CPU 索引演示；后续在单卡 GPU 环境补入原生 CUDA 编译、运行与 memcheck，以及 Triton 向量加法实测。 上",{"slug":1823,"path":1467,"title":5,"date":1462,"tags":1824,"description":1463,"cover":1460,"coverAlt":1461,"excerpt":1825,"draft":1464,"hidden":1464,"published":182,"readingTime":274,"series":1470,"seriesOrder":186,"codeUrl":54,"attachmentUrl":60},"cmu-15-779-profiler-operator-batching",[1470,1474,1475,1476,1477],"本篇对照课程官网的 Introduction to ML Systems 整理，是个人配套 CPU 实验，不是课程官方作业。脚本编号是练习顺序，不对应课程讲次。 前几篇一直在补 PyTorch 基础。到了这一步，我在学习聊天里问：现",{"slug":1827,"path":1828,"title":1829,"date":1830,"tags":1831,"description":1834,"cover":1835,"coverAlt":1836,"excerpt":1837,"draft":1464,"hidden":1464,"published":182,"readingTime":259,"series":1470,"seriesOrder":179,"codeUrl":1838,"attachmentUrl":1839},"cmu-15-779-pytorch-linear-relu-absolute-value","\u002Fposts\u002Fcmu-15-779-pytorch-linear-relu-absolute-value","CMU 15-779 学习记录 04：多输出线性层、ReLU 与绝对值网络","2026-09-12",[1470,1474,1803,1832,1833],"ReLU","神经网络","把 nn.Linear 对应到矩阵乘法，弄清多个输出怎样计算，再逐层拆开一个绝对值网络，比较手动设置参数与训练得到的结果。","\u002Fimages\u002Fposts\u002Fcmu-15-779-pytorch-linear-relu-absolute-value\u002Fcmu-pytorch-absolute-cover-20261015.webp","双分支经过折角模块后汇入紫色 V 形曲线面板的浅色技术教具插画","本篇整理的是学习 CMU 15779 时补充的 PyTorch 预备练习。脚本编号不对应课程讲次，也不是课程官方作业。 接着前一篇的批量推理与矩阵乘法，我继续把 model(x) 拆开看：一层里有多个输出时，每个数是怎么算出来的？两","https:\u002F\u002Fgithub.com\u002Fliangqianxing\u002Fcmu-15-779-labs\u002Ftree\u002F4cc63904a3a583da4a2122c246a14b43aac38528","\u002Fmaterials\u002Fcmu-15-779\u002Fexperiments\u002Fpytorch-linear-relu-abs-20261015.zip",{"slug":1841,"path":1842,"title":1843,"date":1844,"tags":1845,"description":1848,"cover":1849,"coverAlt":1850,"excerpt":1851,"draft":1464,"hidden":1464,"published":182,"readingTime":259,"series":1470,"seriesOrder":170,"codeUrl":1852,"attachmentUrl":1853},"cmu-15-779-pytorch-dataloader-checkpoints-batching","\u002Fposts\u002Fcmu-15-779-pytorch-dataloader-checkpoints-batching","CMU 15-779 学习记录 03：DataLoader、模型保存与批量推理","2026-09-09",[1470,1474,1846,1476,1847],"DataLoader","模型保存","用 DataLoader 分批训练平方函数，保存并加载模型参数，再比较 1000 个样本与 1 个样本的推理耗时，最后把批量计算对应到矩阵乘法。","\u002Fimages\u002Fposts\u002Fcmu-15-779-pytorch-dataloader-checkpoints-batching\u002Fcmu-pytorch-batching-cover-20261015.webp","样本卡片按批次进入计算模块，旁边配有参数存储盒和输出矩阵面板的技术插画","本篇整理的是学习 CMU 15779 时补充的 PyTorch 预备练习。脚本编号不对应课程讲次，也不是课程官方作业。 接着前一篇的平方函数实验，这一节继续练习分批训练、保存参数和独立预测。最后做了一次计时：同一个小网络，逐个处理 ","https:\u002F\u002Fgithub.com\u002Fliangqianxing\u002Fcmu-15-779-labs","\u002Fmaterials\u002Fcmu-15-779\u002Fexperiments\u002Fpytorch-batching-20261015.zip",{"slug":1855,"path":1856,"title":1857,"date":1858,"tags":1859,"description":1860,"cover":1861,"coverAlt":1862,"excerpt":1863,"draft":1464,"hidden":1464,"published":182,"readingTime":259,"series":1470,"seriesOrder":144,"codeUrl":1852,"attachmentUrl":1864},"cmu-15-779-pytorch-linear-module-relu","\u002Fposts\u002Fcmu-15-779-pytorch-linear-module-relu","CMU 15-779 学习记录 02：nn.Linear、张量形状与 ReLU","2026-09-06",[1470,1474,1833,1832],"从带偏置的线性回归开始，弄清 nn.Linear 和 nn.Module 的分工，再用平方函数观察线性模型的限制、ReLU 的作用和测试点的选择。","\u002Fimages\u002Fposts\u002Fcmu-15-779-pytorch-linear-module-relu\u002Fcmu-pytorch-relu-cover-20261015.webp","输入经过两层神经网络和 ReLU 折角模块，输出曲线拟合结果的技术插画","本篇整理的是学习 CMU 15779 时补充的 PyTorch 预备练习。脚本编号不对应课程讲次，也不是课程官方作业。 上一篇里，我用一个参数拟合了 $y=2x$。这一篇继续顺着往下写，先加上偏置，再把手写参数换成 nn.Linea","\u002Fmaterials\u002Fcmu-15-779\u002Fexperiments\u002Fpytorch-models-20261015.zip",{"slug":1866,"path":1867,"title":1868,"date":1869,"tags":1870,"description":1873,"cover":1874,"coverAlt":1875,"excerpt":1876,"draft":1464,"hidden":1464,"published":182,"readingTime":274,"series":1470,"seriesOrder":126,"codeUrl":1852,"attachmentUrl":1877},"cmu-15-779-pytorch-autograd-linear-regression","\u002Fposts\u002Fcmu-15-779-pytorch-autograd-linear-regression","CMU 15-779 学习记录 01：PyTorch 自动求导与线性回归","2026-09-03",[1470,1474,1871,1872],"自动求导","线性回归","从自动求导和手动梯度下降开始，比较学习率，再用 SGD 训练线性回归。记录 w * 4 写错后停在 1.25 的原因和修正结果。","\u002Fimages\u002Fposts\u002Fcmu-15-779-pytorch-autograd-linear-regression\u002Fcmu-pytorch-autograd-cover-20261015.webp","梯度曲面上的小球沿路径下降，前侧面板展示散点与拟合直线的技术插画","本篇整理的是学习 CMU 15779 时补充的 PyTorch 预备练习。脚本编号不对应课程讲次，也不是课程官方作业。 开始学习 CMU 15779 时，我会 Python，但还没用过 PyTorch。这次先在 Mac 上用 CPU","\u002Fmaterials\u002Fcmu-15-779\u002Fexperiments\u002Fpytorch-basics-20261013.zip",{"slug":1879,"path":1880,"title":1881,"date":1882,"tags":1883,"description":1889,"cover":1890,"coverAlt":1891,"excerpt":1892,"draft":1464,"hidden":1464,"published":182,"readingTime":1716,"codeUrl":1893,"attachmentUrl":1894},"embodied-pickplace-lab-robustness","\u002Fposts\u002Fembodied-pickplace-lab-robustness","Pickplace Lab 项目复盘：抓取成功率之外的时延与失败分析","2026-08-29",[1884,1885,1886,1474,1887,1888],"具身智能","模仿学习","MuJoCo","机器人","鲁棒性","从 MuJoCo 抓取搬运闭环出发，记录 22,100 个评测回合。动作平滑在自建任务的 50 ms 延迟下有效，换到 Panda Lift 却没有支持同样的收益。","\u002Fimages\u002Fposts\u002Fembodied-pickplace-lab-robustness\u002Fpickplace-grasp-cover-20261016.webp","MuJoCo 实际渲染的笛卡尔夹爪抓取方块场景，桌面上标出放置目标","Pickplace Lab 从一个简单任务开始：让夹爪抓起方块，搬到指定位置，放下，再退开。先把仿真、示范采集、训练和闭环评测接起来，随后再问：干净条件下成功的策略，换一种执行条件，还能不能完成任务？ 最值得讨论的结果出现在指令延迟","https:\u002F\u002Fgithub.com\u002Fliangqianxing\u002Fembodied-pickplace-lab\u002Ftree\u002Fv0.1.0","https:\u002F\u002Fgithub.com\u002Fliangqianxing\u002Fembodied-pickplace-lab\u002Freleases\u002Fdownload\u002Fv0.1.0\u002Fpickplace-lab-research-v1.zip",{"slug":1896,"path":1897,"title":1898,"date":1899,"tags":1900,"description":1905,"cover":1906,"coverAlt":1907,"excerpt":1908,"draft":1464,"hidden":1464,"published":182,"readingTime":1909},"qwen25-vl-sglang-verl-single-gpu-lab","\u002Fposts\u002Fqwen25-vl-sglang-verl-single-gpu-lab","从 Mac 到单卡 GPU：Qwen2.5-VL 部署、GRPO 与计数 LoRA 实践","2026-08-19",[1592,1533,1901,1902,1903,1904,1749],"SGLang","verl","GRPO","推理优化","在 M2 Pro 和云端单卡上部署 Qwen2.5-VL-3B，实测连续批处理、前缀缓存与 SGLang 参数调优，再用 verl 完成多模态 GRPO 更新和同条件前后评测，记录有效优化、兼容性问题与未观察到提升的结果。","\u002Fimages\u002Fposts\u002Fqwen25-vl-sglang-verl-single-gpu-lab\u002Fqvl-lab-cover-20261010.webp","笔记本与单卡计算模块通过图像和 token 光轨连接的实验台插画","这次实验从一个很具体的问题开始：M2 Pro、16 GB 统一内存，能不能在本机跑 Qwen2.5VL？跑起来之后，我又想把图文问答、推理性能测量和强化学习训练接成一条完整流程。 本机的 MLX 4bit 服务和云端的 SGLang",29,{"slug":1911,"path":1912,"title":1913,"date":1914,"tags":1915,"description":1919,"cover":1920,"coverAlt":1921,"excerpt":1922,"draft":1464,"hidden":1464,"published":182,"readingTime":1716},"cycleresearcher-post-training","\u002Fposts\u002Fcycleresearcher-post-training","从 DeepReviewer 2.0 到 CycleResearcher：科研 Agent 如何真正做 Post-training","2026-08-07 17:33:05",[1576,1593,1534,1916,1917,1918],"SimPO","Reward Model","ICLR","区分推理时 Agent 编排与模型后训练，并拆解 CycleResearcher 的领域 SFT、生成式 Reward Model、Iterative SimPO 和评测风险。","\u002Fimages\u002Fposts\u002Fcycleresearcher-post-training\u002Fcycleresearcher-cover-20261018.webp","策略生成模块产出多份候选文稿经多维评审棱镜排序后通过偏好梯度反馈弧闭环优化的等距技术插画","看到一个系统会检索、规划、反思和调用工具，不能据此判断它做过 Posttraining。Agent 描述的是模型在运行时如何完成任务；Posttraining 描述的是预训练之后，是否继续用监督数据、偏好数据或奖励信号更新模型参数。",{"slug":1924,"path":1925,"title":1926,"date":1927,"tags":1928,"description":1930,"cover":1931,"coverAlt":1932,"excerpt":1933,"draft":1464,"hidden":1464,"published":182,"readingTime":1569,"series":1934,"seriesOrder":126},"cuda-fused-rmsnorm-kernel","\u002Fposts\u002Fcuda-fused-rmsnorm-kernel","从 PyTorch 到 CUDA Kernel：融合 RMSNorm 在 RTX 4090D 上加速 4.27 倍","2026-07-22 18:16:50",[1802,1505,1507,1474,1617,1929],"Kernel 优化","从零实现 PyTorch CUDA 扩展，拆解 RMSNorm 的访存瓶颈、warp 归约、向量化加载、算子融合与冷热缓存 benchmark。","\u002Fimages\u002Fposts\u002Fcuda-fused-rmsnorm-kernel\u002Fcuda-rmsnorm-cover-20261018.webp","融合残差加法与向量化 float4 访存经 Warp Shuffle 蝴蝶树归约输出归一化张量的等距硬件架构插画","为了准备 AI Infra 岗位，我做了一个尽量小而完整的 CUDA 项目：不追求堆很多算子，而是选一个真实的 LLM 推理路径，把实现、正确性、benchmark 和性能解释全部做扎实。 项目实现了两个 PyTorch CUDA ","AI Infra 实战",{"slug":1936,"path":1937,"title":1938,"date":1939,"tags":1940,"description":1942,"cover":1943,"coverAlt":1944,"excerpt":1945,"draft":1464,"hidden":1464,"published":182,"readingTime":238,"series":1946,"seriesOrder":126},"agent-memory","\u002Fposts\u002Fagent-memory","Agent 上下文工程与分层记忆架构：从 Observation 剪枝、滚动摘要到向量召回闭环","2026-04-24",[1576,1534,1941,1505],"RAG","结合生产级多约束 Agent 与超 100K Token 科研智能体实践，系统拆解长程会话中的上下文膨胀与注意力稀释问题：涵盖 Observation 结构化剪枝（降低约 40% Token 开销）、滚动工作摘要、向量检索长期记忆及 Prefix Cache 友好的上下文组装。","\u002Fimages\u002Fposts\u002Fagent-memory\u002Fagent-memory-cover-20261018.webp","底层高噪数据经剪枝滤网进入中期滑动窗口与金色状态台账并由顶层向量星座按需召回的三层记忆架构插画","在构建多步工具调用（ToolUse）与长周期自主规划（Autonomous Planning）的 Agent 系统时，上下文管理（Context Engineering）往往比单纯改写 System Prompt 更决定线上稳定性。","Agent 上下文工程",{"slug":1948,"path":1949,"title":1950,"date":1951,"tags":1952,"description":1955,"cover":1956,"coverAlt":1957,"excerpt":1958,"draft":1464,"hidden":1464,"published":182,"readingTime":186,"series":1959,"seriesOrder":144},"backend-stack-deep-dive","\u002Fposts\u002Fbackend-stack-deep-dive","高并发 AI 平台后端基础设施实践：FastAPI 异步网关、SQLAlchemy 2.0 Async、PostgreSQL 与 Docker 沙箱编排","2026-04-07",[1621,1618,1620,1953,1954,1505],"PostgreSQL","Docker","结合 DeepScientist 生产级 AI 科研平台实践，拆解 FastAPI ASGI 异步事件循环、SQLAlchemy 2.0 + AsyncPG 连接池调优、PostgreSQL + MinIO 存储分层与 Docker SDK 动态沙箱容器编排的职责边界与核心设计。","\u002Fimages\u002Fposts\u002Fbackend-stack-deep-dive\u002Fbackend-stack-cover-20261018.webp","ASGI 异步事件循环涡轮将非阻塞协程流分发至 PostgreSQL 连接池、MinIO 存储矩阵与动态 Docker 沙箱的等距架构插画","结合 DeepScientist 生产级 AI 科研平台（服务 7,000+ 用户）的实际工程落地，本文系统拆解支撑高并发流式推理与容器沙箱调度的后端基础设施栈——FastAPI（ASGI 异步网关 + SSE 流式推送）、SQLA","DeepScientist 工程复盘",{"slug":1961,"path":1962,"title":1963,"date":1964,"tags":1965,"description":1968,"cover":1969,"coverAlt":1970,"excerpt":1971,"draft":1464,"hidden":1464,"published":182,"readingTime":287,"series":1972,"seriesOrder":126},"minicode-source-analysis","\u002Fposts\u002Fminicode-source-analysis","MiniCode 架构与工具评测深度拆解：开源终端 Agentic Coding 引擎（1k+ Stars）的 ReAct、MCP 与 30 项基准验证","2026-04-06",[1534,1966,1967,1564,1505],"TypeScript","MCP","深度剖析开源终端 AI 编程智能体 MiniCode（1k+ GitHub Stars）的核心架构：从零重型依赖的 ReAct 规划循环、声明式工具注册表与权限拦截层、MCP 协议帧自动协商，到 30 项可执行 Tool-Use 评测基准（86.7% 完成率、100% 错误恢复率）的失败分布归因与闭环优化。","\u002Fimages\u002Fposts\u002Fminicode-source-analysis\u002Fminicode-cover-20261018.webp","终端编程智能体控制台被 ReAct 状态环、Unified Diff 补丁预览门与 MCP 协议插槽环绕的等距技术插画","MiniCode 是我作为核心开发者参与构建的开源终端 Agentic Coding 引擎（目前已在 GitHub 获得 1,000+ Stars）。项目的核心设计目标是：在不引入庞大运行时框架的前提下，用约 5,000 行高内聚的","AI 项目源码导读",{"slug":1974,"path":1975,"title":1976,"date":1964,"tags":1977,"description":1978,"cover":1979,"coverAlt":1980,"excerpt":1981,"draft":1464,"hidden":1464,"published":182,"readingTime":274,"series":1959,"seriesOrder":126},"deepscientist-tech-stack","\u002Fposts\u002Fdeepscientist-tech-stack","DeepScientist 架构与运行时全解析：超 100K 上下文科研智能体、MCP 沙箱与高并发流式系统",[1534,1505,1967,1954,1620,1618],"深度拆解服务 7,000+ 用户的科研自主智能体平台 DeepScientist：从一题一仓库（Quest）持久化状态机、MCP 工具栈（memory \u002F artifact \u002F bash_exec）、Docker SDK 动态容器编排，到 SSE + WebSocket 双流通信与 PostgreSQL \u002F Redis \u002F Prometheus 生产级基础设施。","\u002Fimages\u002Fposts\u002Fdeepscientist-tech-stack\u002Fdeepscientist-cover-20261018.webp","科研智能体编排核心通过 memory、artifact 与隔离容器沙箱三座协议桥连接双流工作台的等距架构插画","DeepScientist（deepscientist.cc）是我们在西湖大学自然语言处理实验室研发的自主科学发现与科研管理平台，目前累计服务 7,000+ 科研用户。与单轮对话或纯文本文献问答工具不同，科研智能体需要在数小时乃至数",1795088945945]