[{"data":1,"prerenderedAt":1581},["ShallowReactive",2],{"post-\u002Fposts\u002Fdistributed-systems-week3":3,"all-posts-nav":1300},{"id":4,"title":5,"body":6,"categories":1283,"date":1285,"description":1286,"draft":1287,"extension":1288,"hidden":1287,"meta":1289,"navigation":198,"path":1290,"published":1287,"seo":1291,"stem":1292,"tags":1293,"__hash__":1299},"posts\u002Fposts\u002Fdistributed-systems-week3.md","Week 3：分布式系统速成——MapReduce、Raft、容错与 Distributed KV Store",{"type":7,"value":8,"toc":1246},"minimark",[9,18,21,28,33,36,47,50,53,75,78,82,85,88,112,115,119,130,133,139,142,229,232,255,261,265,268,328,331,335,338,418,421,441,444,450,453,457,464,467,550,553,556,562,565,568,582,586,589,595,598,601,607,610,614,620,623,626,632,639,642,646,652,655,660,663,683,687,690,694,697,701,704,710,712,718,721,725,728,731,734,738,741,747,750,756,759,763,766,772,775,778,789,793,796,802,805,809,812,818,821,825,828,834,837,841,844,850,853,857,860,864,867,870,874,877,880,884,887,890,894,897,903,906,923,926,930,933,939,942,962,968,972,975,978,984,987,1010,1013,1017,1020,1023,1046,1049,1053,1056,1062,1065,1151,1154,1158,1161,1193,1196,1200,1203,1232,1236,1239,1242],[10,11,12,13,17],"p",{},"Week 1 我们用 mini autograd 理解了深度学习框架的本质；Week 2 我们从 GPU、Kernel、KV cache 和 batching 理解了推理系统的性能瓶颈。Week 3 要补的是另一块底层能力：",[14,15,16],"strong",{},"分布式系统思想","。",[10,19,20],{},"这部分不需要做 MIT 6.824 的 lab，也不需要陷入每个 RPC 细节。只要抓住四个主题：MapReduce 思想、Raft 共识、Fault tolerance、Distributed KV store。学完之后，你会发现 Ray、分布式推理、多 agent 系统、workflow engine 的底层心智模型高度相似。甚至 DeepScientist 本质上也可以看成一个 mini distributed system。",[10,22,23],{},[24,25],"img",{"alt":26,"src":27},"MapReduce pipeline","\u002Fimages\u002Fposts\u002Fdistributed-systems\u002Fmapreduce.svg",[29,30,32],"h2",{"id":31},"_1-为什么-ai-系统也要学分布式","1. 为什么 AI 系统也要学分布式",[10,34,35],{},"很多 AI 项目表面上是模型应用，实际运行起来都是分布式系统：",[37,38,44],"pre",{"className":39,"code":41,"language":42,"meta":43},[40],"language-text","用户请求\n  -> API gateway\n  -> planner\n  -> retriever\n  -> LLM worker\n  -> tool worker\n  -> memory store\n  -> evaluator\n  -> result aggregator\n","text","",[45,46,41],"code",{"__ignoreMap":43},[10,48,49],{},"只要系统里有多个 worker、多个服务、异步任务、失败重试、状态保存、并发请求，就已经进入分布式系统范畴。",[10,51,52],{},"分布式系统最核心的问题不是“怎么让很多机器一起干活”，而是：",[54,55,56,60,63,66,69,72],"ul",{},[57,58,59],"li",{},"任务怎么拆；",[57,61,62],{},"状态放哪里；",[57,64,65],{},"节点挂了怎么办；",[57,67,68],{},"重试会不会产生副作用；",[57,70,71],{},"多个副本如何保持一致；",[57,73,74],{},"如何在吞吐、延迟、可靠性之间取舍。",[10,76,77],{},"这些问题在 Ray、LLM serving、multi-agent、workflow engine 里都会反复出现。",[29,79,81],{"id":80},"_2-分布式系统的基本矛盾","2. 分布式系统的基本矛盾",[10,83,84],{},"单机程序默认有一个很强的假设：函数调用会返回，内存状态是本地的，失败通常是进程级别的。但分布式系统里这些假设都不成立。",[10,86,87],{},"你必须接受几个事实：",[89,90,91,94,97,100,103,106,109],"ol",{},[57,92,93],{},"网络请求可能丢、可能重复、可能乱序、可能超时；",[57,95,96],{},"节点可能 crash，也可能只是变慢；",[57,98,99],{},"远程调用不等于本地函数调用；",[57,101,102],{},"“没收到回复”不代表对方没执行；",[57,104,105],{},"多个副本之间天然会出现状态差异；",[57,107,108],{},"想要高可用，就必须设计重试和恢复；",[57,110,111],{},"想要强一致，就必须付出通信和延迟成本。",[10,113,114],{},"这就是为什么分布式系统的关键词永远绕不开：partition、replication、consensus、idempotency、retry、timeout、lease、log、snapshot。",[29,116,118],{"id":117},"_3-mapreduce-思想把大任务拆成可重试的小任务","3. MapReduce 思想：把大任务拆成可重试的小任务",[10,120,121,122,125,126,129],{},"MapReduce 最重要的不是 ",[45,123,124],{},"map"," 和 ",[45,127,128],{},"reduce"," 两个函数，而是它背后的任务调度与容错思想。",[10,131,132],{},"一个大任务被拆成很多小任务：",[37,134,137],{"className":135,"code":136,"language":42,"meta":43},[40],"input files\n  -> split\n  -> map tasks\n  -> shuffle by key\n  -> reduce tasks\n  -> output files\n",[45,138,136],{"__ignoreMap":43},[10,140,141],{},"MapReduce 的经典例子是词频统计：",[37,143,147],{"className":144,"code":145,"language":146,"meta":43,"style":43},"language-python shiki shiki-themes github-dark-dimmed github-light","def map(doc):\n    for word in doc.split():\n        emit(word, 1)\n\n\ndef reduce(word, counts):\n    emit(word, sum(counts))\n","python",[45,148,149,166,181,193,200,205,217],{"__ignoreMap":43},[150,151,154,158,162],"span",{"class":152,"line":153},"line",1,[150,155,157],{"class":156},"s6PUj","def",[150,159,161],{"class":160},"swcJU"," map",[150,163,165],{"class":164},"ssh_m","(doc):\n",[150,167,169,172,175,178],{"class":152,"line":168},2,[150,170,171],{"class":156},"    for",[150,173,174],{"class":164}," word ",[150,176,177],{"class":156},"in",[150,179,180],{"class":164}," doc.split():\n",[150,182,184,187,190],{"class":152,"line":183},3,[150,185,186],{"class":164},"        emit(word, ",[150,188,189],{"class":160},"1",[150,191,192],{"class":164},")\n",[150,194,196],{"class":152,"line":195},4,[150,197,199],{"emptyLinePlaceholder":198},true,"\n",[150,201,203],{"class":152,"line":202},5,[150,204,199],{"emptyLinePlaceholder":198},[150,206,208,210,214],{"class":152,"line":207},6,[150,209,157],{"class":156},[150,211,213],{"class":212},"sNjOc"," reduce",[150,215,216],{"class":164},"(word, counts):\n",[150,218,220,223,226],{"class":152,"line":219},7,[150,221,222],{"class":164},"    emit(word, ",[150,224,225],{"class":160},"sum",[150,227,228],{"class":164},"(counts))\n",[10,230,231],{},"但系统层面真正关键的是：",[54,233,234,237,240,243,246,249,252],{},[57,235,236],{},"输入数据被切成多个 split；",[57,238,239],{},"master 负责给 worker 分配任务；",[57,241,242],{},"worker 执行 map 或 reduce；",[57,244,245],{},"map 输出中间文件；",[57,247,248],{},"shuffle 把相同 key 的数据送到同一个 reduce；",[57,250,251],{},"worker 挂了，master 重新调度任务；",[57,253,254],{},"慢 worker 会拖尾，master 可以 speculative execution。",[10,256,257,258,17],{},"MapReduce 的心法是：",[14,259,260],{},"让任务变成确定性的、可重试的、可重新调度的单元",[29,262,264],{"id":263},"_4-mapreduce-为什么影响后来的系统","4. MapReduce 为什么影响后来的系统",[10,266,267],{},"很多现代系统都继承了 MapReduce 的思想，只是形态变了。",[269,270,271,284],"table",{},[272,273,274],"thead",{},[275,276,277,281],"tr",{},[278,279,280],"th",{},"系统",[278,282,283],{},"MapReduce 式思想",[285,286,287,296,304,312,320],"tbody",{},[275,288,289,293],{},[290,291,292],"td",{},"Spark",[290,294,295],{},"把计算图拆成 stage 和 task，失败后按 lineage 重算",[275,297,298,301],{},[290,299,300],{},"Ray",[290,302,303],{},"把 Python 函数变成远程 task，把对象放入 object store",[275,305,306,309],{},[290,307,308],{},"Workflow engine",[290,310,311],{},"把流程拆成 step，每步可重试、可恢复",[275,313,314,317],{},[290,315,316],{},"多 agent 系统",[290,318,319],{},"把复杂任务拆给不同 agent，再聚合结果",[275,321,322,325],{},[290,323,324],{},"分布式推理",[290,326,327],{},"把请求、batch、layer、token 分给不同 worker",[10,329,330],{},"如果你理解了 MapReduce，就会自然理解为什么系统要有 driver\u002Fmaster、task queue、worker heartbeat、retry、checkpoint、shuffle、object store。",[29,332,334],{"id":333},"_5-ray把-mapreduce-思想推广到通用任务图","5. Ray：把 MapReduce 思想推广到通用任务图",[10,336,337],{},"Ray 可以理解成一个通用分布式执行框架。它不像 MapReduce 只适合 map 和 reduce，而是允许你把任意 Python 函数变成远程任务。",[37,339,341],{"className":144,"code":340,"language":146,"meta":43,"style":43},"@ray.remote\ndef f(x):\n    return x * x\n\nrefs = [f.remote(i) for i in range(10)]\nresults = ray.get(refs)\n",[45,342,343,349,359,373,377,408],{"__ignoreMap":43},[150,344,345],{"class":152,"line":153},[150,346,348],{"class":347},"saVmf","@ray.remote\n",[150,350,351,353,356],{"class":152,"line":168},[150,352,157],{"class":156},[150,354,355],{"class":347}," f",[150,357,358],{"class":164},"(x):\n",[150,360,361,364,367,370],{"class":152,"line":183},[150,362,363],{"class":156},"    return",[150,365,366],{"class":164}," x ",[150,368,369],{"class":156},"*",[150,371,372],{"class":164}," x\n",[150,374,375],{"class":152,"line":195},[150,376,199],{"emptyLinePlaceholder":198},[150,378,379,382,385,388,391,394,396,399,402,405],{"class":152,"line":202},[150,380,381],{"class":164},"refs ",[150,383,384],{"class":156},"=",[150,386,387],{"class":164}," [f.remote(i) ",[150,389,390],{"class":156},"for",[150,392,393],{"class":164}," i ",[150,395,177],{"class":156},[150,397,398],{"class":160}," range",[150,400,401],{"class":164},"(",[150,403,404],{"class":160},"10",[150,406,407],{"class":164},")]\n",[150,409,410,413,415],{"class":152,"line":207},[150,411,412],{"class":164},"results ",[150,414,384],{"class":156},[150,416,417],{"class":164}," ray.get(refs)\n",[10,419,420],{},"这里的核心概念是：",[54,422,423,426,429,432,435,438],{},[57,424,425],{},"remote function：远程执行的任务；",[57,427,428],{},"object ref：远程对象引用；",[57,430,431],{},"scheduler：决定任务在哪个 worker 上跑；",[57,433,434],{},"object store：保存任务输出，供下游任务读取；",[57,436,437],{},"actor：有状态的远程 worker；",[57,439,440],{},"fault tolerance：worker 挂了后重建或重跑任务。",[10,442,443],{},"Ray 的心智模型可以写成：",[37,445,448],{"className":446,"code":447,"language":42,"meta":43},[40],"driver 构建任务图\n  -> scheduler 分配 task\u002Factor\n  -> worker 执行\n  -> object store 保存中间结果\n  -> 下游任务继续消费\n",[45,449,447],{"__ignoreMap":43},[10,451,452],{},"这和 MapReduce 一脉相承，只是任务图更灵活。",[29,454,456],{"id":455},"_6-fault-tolerance容错不是异常处理","6. Fault Tolerance：容错不是异常处理",[10,458,459,460,463],{},"容错不是写一个 ",[45,461,462],{},"try\u002Fexcept","。分布式容错关心的是：系统某些部分失败时，整体还能不能给出正确或可接受的结果。",[10,465,466],{},"常见失败类型：",[269,468,469,482],{},[272,470,471],{},[275,472,473,476,479],{},[278,474,475],{},"失败",[278,477,478],{},"例子",[278,480,481],{},"处理思路",[285,483,484,495,506,517,528,539],{},[275,485,486,489,492],{},[290,487,488],{},"Crash failure",[290,490,491],{},"worker 进程挂了",[290,493,494],{},"heartbeat + retry",[275,496,497,500,503],{},[290,498,499],{},"Omission failure",[290,501,502],{},"请求或响应丢了",[290,504,505],{},"timeout + retry",[275,507,508,511,514],{},[290,509,510],{},"Slow node",[290,512,513],{},"某个 worker 很慢",[290,515,516],{},"speculative execution \u002F 剔除",[275,518,519,522,525],{},[290,520,521],{},"Network partition",[290,523,524],{},"节点之间网络断开",[290,526,527],{},"quorum \u002F leader election",[275,529,530,533,536],{},[290,531,532],{},"Data loss",[290,534,535],{},"本地状态丢失",[290,537,538],{},"replication \u002F checkpoint",[275,540,541,544,547],{},[290,542,543],{},"Duplicate execution",[290,545,546],{},"重试导致执行两次",[290,548,549],{},"idempotency \u002F exactly-once 设计",[10,551,552],{},"最容易踩坑的是重试。重试看似简单，但如果操作有副作用，就可能出错。",[10,554,555],{},"例如：",[37,557,560],{"className":558,"code":559,"language":42,"meta":43},[40],"用户支付请求超时\n  -> client 重试\n  -> server 实际执行了两次扣款\n",[45,561,559],{"__ignoreMap":43},[10,563,564],{},"所以很多系统要求操作具备幂等性：同一个请求执行多次，结果和执行一次一样。",[10,566,567],{},"常见做法：",[54,569,570,573,576,579],{},[57,571,572],{},"给请求分配唯一 request id；",[57,574,575],{},"服务端记录已处理请求；",[57,577,578],{},"重复请求直接返回之前结果；",[57,580,581],{},"对外部副作用做事务或补偿。",[29,583,585],{"id":584},"_7-timeout超时不是失败证明","7. Timeout：超时不是失败证明",[10,587,588],{},"分布式系统里，timeout 只能说明“我在规定时间内没收到回复”，不能说明对方没有执行。",[37,590,593],{"className":591,"code":592,"language":42,"meta":43},[40],"client -> server: put(x=1)\nserver 执行成功\nresponse 在网络中丢失\nclient timeout\n",[45,594,592],{"__ignoreMap":43},[10,596,597],{},"此时 client 如果重试，server 可能再次执行。因此所有可重试操作都要考虑幂等性。",[10,599,600],{},"这也是 workflow engine 经常保存 step 状态的原因：",[37,602,605],{"className":603,"code":604,"language":42,"meta":43},[40],"PENDING -> RUNNING -> SUCCEEDED \u002F FAILED\n",[45,606,604],{"__ignoreMap":43},[10,608,609],{},"状态持久化以后，系统重启或重试时才能知道某一步到底执行到哪里。",[29,611,613],{"id":612},"_8-raft为什么需要共识","8. Raft：为什么需要共识",[10,615,616],{},[24,617],{"alt":618,"src":619},"Raft consensus","\u002Fimages\u002Fposts\u002Fdistributed-systems\u002Fraft.svg",[10,621,622],{},"如果一个服务只有单副本，机器挂了就不可用。为了高可用，我们会复制多个副本。但复制带来一个问题：多个副本如何保持一致？",[10,624,625],{},"比如一个 KV store 有三个副本：",[37,627,630],{"className":628,"code":629,"language":42,"meta":43},[40],"node1: x = 1\nnode2: x = 1\nnode3: x = ?\n",[45,631,629],{"__ignoreMap":43},[10,633,634,635,638],{},"如果客户端写入 ",[45,636,637],{},"x = 2"," 时 node1 成功、node2 成功、node3 网络断了，那么系统应该认为写成功吗？之后 node3 恢复时怎么追上？如果两个节点同时认为自己是 leader 怎么办？",[10,640,641],{},"Raft 解决的是：在不可靠网络和节点故障下，让多个节点对同一串日志达成一致。",[29,643,645],{"id":644},"_9-raft-的三个核心leaderlogmajority","9. Raft 的三个核心：Leader、Log、Majority",[10,647,648,649,17],{},"Raft 可以用一句话概括：",[14,650,651],{},"Leader 接收客户端写请求，把命令作为日志复制给 follower；当日志被多数派保存后，leader commit，并让所有节点按日志顺序应用到状态机",[10,653,654],{},"关键组件：",[656,657,659],"h3",{"id":658},"_91-leader","9.1 Leader",[10,661,662],{},"Raft 中任意时刻最多应该只有一个有效 leader。客户端写请求通常发给 leader。Leader 负责：",[54,664,665,668,671,674,677,680],{},[57,666,667],{},"接收写请求；",[57,669,670],{},"追加本地 log；",[57,672,673],{},"发送 AppendEntries 给 followers；",[57,675,676],{},"等待多数派确认；",[57,678,679],{},"推进 commit index；",[57,681,682],{},"通知状态机 apply。",[656,684,686],{"id":685},"_92-follower","9.2 Follower",[10,688,689],{},"Follower 被动接收 leader 的日志复制和心跳。如果长时间收不到 leader 心跳，就会发起选举。",[656,691,693],{"id":692},"_93-candidate","9.3 Candidate",[10,695,696],{},"Candidate 是选举过程中的临时角色。节点超时后变成 candidate，增加 term，向其他节点请求投票。如果拿到多数票，就成为 leader。",[656,698,700],{"id":699},"_94-log","9.4 Log",[10,702,703],{},"Log 是 Raft 的核心数据结构。每条 log 包含：",[37,705,708],{"className":706,"code":707,"language":42,"meta":43},[40],"index, term, command\n",[45,709,707],{"__ignoreMap":43},[10,711,555],{},[37,713,716],{"className":714,"code":715,"language":42,"meta":43},[40],"1: term=1, put x=1\n2: term=1, put y=2\n3: term=2, delete x\n",[45,717,715],{"__ignoreMap":43},[10,719,720],{},"状态机只按 committed log 的顺序执行命令。只要所有节点执行同一串 committed log，最终状态就一致。",[656,722,724],{"id":723},"_95-majority","9.5 Majority",[10,726,727],{},"Raft 不要求所有节点都成功，只要求多数派成功。",[10,729,730],{},"对于 3 个节点，多数派是 2；对于 5 个节点，多数派是 3。多数派的关键性质是：任意两个多数派集合一定有交集。",[10,732,733],{},"这个交集保证了 committed log 不会在下一任 leader 中消失。",[29,735,737],{"id":736},"_10-raft-写入流程","10. Raft 写入流程",[10,739,740],{},"一次写入大概是：",[37,742,745],{"className":743,"code":744,"language":42,"meta":43},[40],"client -> leader: put(x=1)\nleader: append log locally\nleader -> followers: AppendEntries\nfollowers: append log and ack\nleader: receive majority ack\nleader: commit log\nleader: apply put(x=1) to state machine\nleader -> client: success\nfollowers: eventually commit and apply\n",[45,746,744],{"__ignoreMap":43},[10,748,749],{},"重点是：写入不是直接改状态，而是先写日志，再提交，再 apply。",[37,751,754],{"className":752,"code":753,"language":42,"meta":43},[40],"command -> replicated log -> committed log -> state machine\n",[45,755,753],{"__ignoreMap":43},[10,757,758],{},"这个结构非常重要。很多系统的可靠性都建立在 log 上：数据库 WAL、Kafka log、Raft log、workflow event history，本质上都是先记录事实，再从事实恢复状态。",[29,760,762],{"id":761},"_11-raft-选举流程","11. Raft 选举流程",[10,764,765],{},"如果 follower 一段时间没收到 leader 心跳，会认为 leader 可能挂了，然后发起选举：",[37,767,770],{"className":768,"code":769,"language":42,"meta":43},[40],"follower timeout\n  -> become candidate\n  -> term += 1\n  -> vote for self\n  -> request votes from others\n  -> majority votes -> become leader\n",[45,771,769],{"__ignoreMap":43},[10,773,774],{},"为了减少多个节点同时选举导致冲突，Raft 使用随机 election timeout。这样不同节点超时点不同，更容易选出一个 leader。",[10,776,777],{},"Raft 把共识问题拆得比较容易理解：",[54,779,780,783,786],{},[57,781,782],{},"leader election：谁来当 leader；",[57,784,785],{},"log replication：leader 如何复制日志；",[57,787,788],{},"safety：已经 commit 的日志不能丢。",[29,790,792],{"id":791},"_12-distributed-kv-store分布式系统的最小形态","12. Distributed KV Store：分布式系统的最小形态",[10,794,795],{},"KV store 是理解分布式系统的最好载体。接口很简单：",[37,797,800],{"className":798,"code":799,"language":42,"meta":43},[40],"get(key) -> value\nput(key, value)\ndelete(key)\n",[45,801,799],{"__ignoreMap":43},[10,803,804],{},"但一旦分布式化，问题会立刻变复杂。",[656,806,808],{"id":807},"_121-单副本-kv","12.1 单副本 KV",[10,810,811],{},"最简单：",[37,813,816],{"className":814,"code":815,"language":42,"meta":43},[40],"client -> server -> memory dict\n",[45,817,815],{"__ignoreMap":43},[10,819,820],{},"问题是 server 挂了就不可用，内存丢了数据也没了。",[656,822,824],{"id":823},"_122-主从复制-kv","12.2 主从复制 KV",[10,826,827],{},"一个 leader 接收写请求，复制到 followers：",[37,829,832],{"className":830,"code":831,"language":42,"meta":43},[40],"client -> leader -> followers\n",[45,833,831],{"__ignoreMap":43},[10,835,836],{},"读可以从 leader 读，也可以从 follower 读。读 follower 延迟低，但可能读到旧数据。",[656,838,840],{"id":839},"_123-raft-kv","12.3 Raft KV",[10,842,843],{},"用 Raft 管理复制日志：",[37,845,848],{"className":846,"code":847,"language":42,"meta":43},[40],"put(x=1)\n  -> Raft log\n  -> majority commit\n  -> apply to state machine dict\n",[45,849,847],{"__ignoreMap":43},[10,851,852],{},"这样 KV store 的状态来自 committed log。节点挂了可以通过 log 重放恢复；新节点可以通过 snapshot + log catch up 加入。",[29,854,856],{"id":855},"_13-consistency强一致最终一致与读写取舍","13. Consistency：强一致、最终一致与读写取舍",[10,858,859],{},"分布式 KV store 必须面对一致性取舍。",[656,861,863],{"id":862},"_131-strong-consistency","13.1 Strong Consistency",[10,865,866],{},"写成功后，后续读一定能读到最新值。通常需要读写都经过 leader 或 quorum。",[10,868,869],{},"优点：语义简单。缺点：延迟更高，可用性更受网络影响。",[656,871,873],{"id":872},"_132-eventual-consistency","13.2 Eventual Consistency",[10,875,876],{},"写入后不同副本可能短暂不一致，但如果没有新写入，最终会收敛。",[10,878,879],{},"优点：可用性和性能好。缺点：应用层要能接受读旧值。",[656,881,883],{"id":882},"_133-linearizability","13.3 Linearizability",[10,885,886],{},"Linearizability 是更严格的强一致：每个操作看起来像在某个瞬间原子生效，并且符合真实时间顺序。",[10,888,889],{},"Raft KV 通常追求 linearizable reads\u002Fwrites，因为这让上层应用更容易推理。",[29,891,893],{"id":892},"_14-snapshot日志不能无限长","14. Snapshot：日志不能无限长",[10,895,896],{},"Raft log 如果一直增长，会占用大量磁盘，恢复也很慢。因此系统需要 snapshot。",[37,898,901],{"className":899,"code":900,"language":42,"meta":43},[40],"log 1..100000 已经 apply 到状态机\n  -> 保存 state machine snapshot\n  -> 丢弃旧 log\n  -> 后续只保留 snapshot 之后的 log\n",[45,902,900],{"__ignoreMap":43},[10,904,905],{},"Snapshot 本质是 checkpoint。你会在很多系统里看到类似机制：",[54,907,908,911,914,917,920],{},[57,909,910],{},"数据库 checkpoint；",[57,912,913],{},"workflow 状态快照；",[57,915,916],{},"Ray object spill \u002F checkpoint；",[57,918,919],{},"LLM agent 的 memory snapshot；",[57,921,922],{},"训练中的 model checkpoint。",[10,924,925],{},"思想都是一样的：不要每次从最早事件重放，定期保存一个可恢复状态。",[29,927,929],{"id":928},"_15-workflow-engine分布式状态机","15. Workflow Engine：分布式状态机",[10,931,932],{},"Workflow engine 例如 Temporal、Argo、Airflow，核心是把长流程拆成可恢复 step。",[37,934,937],{"className":935,"code":936,"language":42,"meta":43},[40],"step1: crawl papers\nstep2: parse PDFs\nstep3: run embedding\nstep4: retrieve related work\nstep5: draft report\nstep6: evaluate result\n",[45,938,936],{"__ignoreMap":43},[10,940,941],{},"每个 step 都可能失败、超时、重试。Workflow engine 要保存：",[54,943,944,947,950,953,956,959],{},[57,945,946],{},"当前执行到哪一步；",[57,948,949],{},"每一步输入输出；",[57,951,952],{},"哪些 step 已成功；",[57,954,955],{},"哪些 step 可以重试；",[57,957,958],{},"重试是否幂等；",[57,960,961],{},"worker 挂了后谁接手。",[10,963,964,965,17],{},"这和 MapReduce master、Raft log、KV state 的思想是连在一起的：",[14,966,967],{},"用持久化状态描述系统进度，用可重试任务推进状态变化",[29,969,971],{"id":970},"_16-多-agent-系统为什么是分布式系统","16. 多 Agent 系统为什么是分布式系统",[10,973,974],{},"多 agent 系统表面上是多个 LLM 角色协作：planner、researcher、coder、reviewer、critic。但系统角度看，它就是一组 worker 和消息队列。",[10,976,977],{},"典型结构：",[37,979,982],{"className":980,"code":981,"language":42,"meta":43},[40],"planner agent\n  -> creates tasks\nresearch agents\n  -> fetch evidence\ncoding agent\n  -> edits files\nreview agent\n  -> checks output\ncoordinator\n  -> merges results and decides next step\n",[45,983,981],{"__ignoreMap":43},[10,985,986],{},"它会遇到分布式系统经典问题：",[54,988,989,992,995,998,1001,1004,1007],{},[57,990,991],{},"agent 输出不稳定，相当于 worker nondeterministic；",[57,993,994],{},"tool call 失败，需要 retry；",[57,996,997],{},"多个 agent 可能写同一份状态，需要冲突解决；",[57,999,1000],{},"任务执行时间不一致，需要调度和超时；",[57,1002,1003],{},"中间结果要存入 memory store；",[57,1005,1006],{},"最终结果要聚合，避免重复和矛盾；",[57,1008,1009],{},"如果 coordinator 挂了，要能恢复执行进度。",[10,1011,1012],{},"所以设计 multi-agent 不只是 prompt engineering，还需要 distributed systems thinking。",[29,1014,1016],{"id":1015},"_17-分布式推理模型服务里的分布式问题","17. 分布式推理：模型服务里的分布式问题",[10,1018,1019],{},"分布式推理也不是简单“多放几张 GPU”。它同时涉及模型切分、请求调度和状态管理。",[10,1021,1022],{},"常见问题：",[54,1024,1025,1028,1031,1034,1037,1040,1043],{},[57,1026,1027],{},"一个模型单卡放不下，需要 tensor parallel；",[57,1029,1030],{},"不同层放在不同 GPU，需要 pipeline parallel；",[57,1032,1033],{},"请求动态到达，需要 continuous batching；",[57,1035,1036],{},"每个请求有 KV cache，需要分配、迁移、释放；",[57,1038,1039],{},"GPU worker 可能 OOM 或 crash，需要摘除和重试；",[57,1041,1042],{},"speculative decoding 里 draft model 和 target model 要协作；",[57,1044,1045],{},"多副本服务要做负载均衡和健康检查。",[10,1047,1048],{},"这和分布式 KV store 的相似点是：系统里都有“状态”。KV store 的状态是 key-value；LLM serving 的状态是 request queue、KV cache、batch 状态、生成进度。",[29,1050,1052],{"id":1051},"_18-deepscientist-为什么是-mini-distributed-system","18. DeepScientist 为什么是 mini distributed system",[10,1054,1055],{},"DeepScientist 如果拆成系统组件，大概是：",[37,1057,1060],{"className":1058,"code":1059,"language":42,"meta":43},[40],"User Query\n  -> Planner\n  -> Search \u002F Retrieval Workers\n  -> PDF \u002F Web Parser Workers\n  -> Memory \u002F Vector Store\n  -> Draft Writer\n  -> Critic \u002F Evaluator\n  -> Final Aggregator\n",[45,1061,1059],{"__ignoreMap":43},[10,1063,1064],{},"这就是一个 mini distributed system：",[269,1066,1067,1077],{},[272,1068,1069],{},[275,1070,1071,1074],{},[278,1072,1073],{},"分布式概念",[278,1075,1076],{},"DeepScientist 对应物",[285,1078,1079,1087,1095,1103,1111,1119,1127,1135,1143],{},[275,1080,1081,1084],{},[290,1082,1083],{},"Master \u002F coordinator",[290,1085,1086],{},"planner \u002F orchestrator",[275,1088,1089,1092],{},[290,1090,1091],{},"Worker",[290,1093,1094],{},"searcher、parser、writer、critic",[275,1096,1097,1100],{},[290,1098,1099],{},"Task queue",[290,1101,1102],{},"待检索、待阅读、待总结的子任务",[275,1104,1105,1108],{},[290,1106,1107],{},"KV store",[290,1109,1110],{},"memory、cache、metadata store",[275,1112,1113,1116],{},[290,1114,1115],{},"MapReduce",[290,1117,1118],{},"多路检索与证据聚合",[275,1120,1121,1124],{},[290,1122,1123],{},"Fault tolerance",[290,1125,1126],{},"tool retry、fallback search、partial result",[275,1128,1129,1132],{},[290,1130,1131],{},"Checkpoint",[290,1133,1134],{},"中间笔记、引用、草稿版本",[275,1136,1137,1140],{},[290,1138,1139],{},"Consensus-like decision",[290,1141,1142],{},"多 reviewer \u002F evaluator 投票或打分",[275,1144,1145,1148],{},[290,1146,1147],{},"Workflow log",[290,1149,1150],{},"agent trajectory \u002F event history",[10,1152,1153],{},"它不一定需要 Raft 这种强共识协议，但它一定需要分布式系统的思维：任务可拆、状态可恢复、失败可重试、结果可聚合、冲突可处理。",[29,1155,1157],{"id":1156},"_19-读系统论文的心法","19. 读系统论文的心法",[10,1159,1160],{},"遇到 Ray、vLLM、workflow engine、多 agent 框架，可以按这几个问题拆：",[89,1162,1163,1166,1169,1172,1175,1178,1181,1184,1187,1190],{},[57,1164,1165],{},"系统里的 coordinator 是谁？",[57,1167,1168],{},"worker 是无状态还是有状态？",[57,1170,1171],{},"状态存在哪里，内存、磁盘、KV、object store，还是 log？",[57,1173,1174],{},"任务失败后是重试、跳过、补偿，还是回滚？",[57,1176,1177],{},"重试是否幂等？",[57,1179,1180],{},"是否需要强一致，还是最终一致就够？",[57,1182,1183],{},"调度目标是吞吐、延迟、公平性，还是成本？",[57,1185,1186],{},"有没有 checkpoint \u002F snapshot？",[57,1188,1189],{},"有没有 straggler，如何处理慢节点？",[57,1191,1192],{},"系统如何扩容和缩容？",[10,1194,1195],{},"这些问题比记住某个框架 API 更重要。",[29,1197,1199],{"id":1198},"_20-week-3-学完应该掌握什么","20. Week 3 学完应该掌握什么",[10,1201,1202],{},"学完这一周，目标不是能实现完整 Raft，而是能讲清楚：",[54,1204,1205,1208,1211,1214,1217,1220,1223,1226,1229],{},[57,1206,1207],{},"MapReduce 如何把大任务拆成可重试的小任务；",[57,1209,1210],{},"master \u002F worker \u002F task queue \u002F shuffle 各自解决什么问题；",[57,1212,1213],{},"fault tolerance 为什么离不开 timeout、retry、idempotency、checkpoint；",[57,1215,1216],{},"Raft 为什么需要 leader、log、term、majority；",[57,1218,1219],{},"Raft 写入为什么先复制日志再 apply 状态机；",[57,1221,1222],{},"distributed KV store 如何从单副本演化到 Raft 复制；",[57,1224,1225],{},"强一致和最终一致的取舍；",[57,1227,1228],{},"Ray、workflow engine、多 agent、分布式推理分别对应哪些分布式模式；",[57,1230,1231],{},"DeepScientist 为什么本质是 mini distributed system。",[29,1233,1235],{"id":1234},"_21-最后总结","21. 最后总结",[10,1237,1238],{},"分布式系统的核心不是把代码部署到多台机器上，而是用系统化方式处理不可靠性：网络不可靠、节点不可靠、时间不可靠、状态不可靠。MapReduce 教我们如何拆任务和重试，Raft 教我们如何复制状态并达成一致，fault tolerance 教我们如何面对失败，distributed KV store 则把这些思想浓缩成最小可理解系统。",[10,1240,1241],{},"当你把这套心智模型带回 AI 系统，就会发现 Ray、分布式推理、多 agent、workflow engine 都不再神秘。它们都是在不同场景下回答同一组问题：任务怎么调度，状态怎么保存，失败怎么恢复，多个 worker 如何协作产出一个可靠结果。",[1243,1244,1245],"style",{},"html pre.shiki code .s6PUj, html code.shiki .s6PUj{--shiki-default:#F47067;--shiki-light:#D73A49}html pre.shiki code .swcJU, html code.shiki .swcJU{--shiki-default:#6CB6FF;--shiki-light:#005CC5}html pre.shiki code .ssh_m, html code.shiki .ssh_m{--shiki-default:#ADBAC7;--shiki-light:#24292E}html pre.shiki code .sNjOc, html code.shiki .sNjOc{--shiki-default:#F69D50;--shiki-light:#E36209}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html.light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html pre.shiki code .saVmf, html code.shiki .saVmf{--shiki-default:#DCBDFB;--shiki-light:#6F42C1}",{"title":43,"searchDepth":168,"depth":183,"links":1247},[1248,1249,1250,1251,1252,1253,1254,1255,1256,1263,1264,1265,1270,1275,1276,1277,1278,1279,1280,1281,1282],{"id":31,"depth":168,"text":32},{"id":80,"depth":168,"text":81},{"id":117,"depth":168,"text":118},{"id":263,"depth":168,"text":264},{"id":333,"depth":168,"text":334},{"id":455,"depth":168,"text":456},{"id":584,"depth":168,"text":585},{"id":612,"depth":168,"text":613},{"id":644,"depth":168,"text":645,"children":1257},[1258,1259,1260,1261,1262],{"id":658,"depth":183,"text":659},{"id":685,"depth":183,"text":686},{"id":692,"depth":183,"text":693},{"id":699,"depth":183,"text":700},{"id":723,"depth":183,"text":724},{"id":736,"depth":168,"text":737},{"id":761,"depth":168,"text":762},{"id":791,"depth":168,"text":792,"children":1266},[1267,1268,1269],{"id":807,"depth":183,"text":808},{"id":823,"depth":183,"text":824},{"id":839,"depth":183,"text":840},{"id":855,"depth":168,"text":856,"children":1271},[1272,1273,1274],{"id":862,"depth":183,"text":863},{"id":872,"depth":183,"text":873},{"id":882,"depth":183,"text":883},{"id":892,"depth":168,"text":893},{"id":928,"depth":168,"text":929},{"id":970,"depth":168,"text":971},{"id":1015,"depth":168,"text":1016},{"id":1051,"depth":168,"text":1052},{"id":1156,"depth":168,"text":1157},{"id":1198,"depth":168,"text":1199},{"id":1234,"depth":168,"text":1235},[1284],"技术","2026-05-05 11:00:00","Week 1 我们用 mini autograd 理解了深度学习框架的本质；Week 2 我们从 GPU、Kernel、KV cache 和 batching 理解了推理系统的性能瓶颈。Week 3 要补的是另一块底层能力：分布式系统思想。",false,"md",{},"\u002Fposts\u002Fdistributed-systems-week3",{"title":5,"description":1286},"posts\u002Fdistributed-systems-week3",[1294,1295,1115,1296,1297,300,1298],"分布式系统","MIT 6.824","Raft","KV Store","Agent","yLG-QYDGdSKkfGsYo1IsbWuUwcqrtxQzqfmkgcokMW8",[1301,1315,1328,1335,1348,1358,1368,1378,1389,1398,1407,1419,1432,1443,1452,1462,1475,1478,1488,1496,1507,1513,1519,1525,1533,1542,1550,1556,1564,1572],{"slug":1302,"path":1303,"title":1304,"date":1305,"tags":1306,"description":43,"draft":1287,"hidden":1287,"published":198,"readingTime":1314},"multimodal-rag-from-scratch","\u002Fposts\u002Fmultimodal-rag-from-scratch","从零实现多模态 RAG：BM25、Dense 检索、RRF 融合、MMR 重排全部手写","2026-06-30 18:00:00",[1307,1308,1309,1310,1311,1312,1313],"RAG","多模态","AI Infra","BM25","向量检索","混合检索","实习求职",9,{"slug":1316,"path":1317,"title":1318,"date":1319,"tags":1320,"description":43,"draft":1287,"hidden":1287,"published":198,"readingTime":1327},"mini-llm-engine-deep-dive","\u002Fposts\u002Fmini-llm-engine-deep-dive","讲透 mini-llm-engine：从显存碎片到六大推理优化","2026-06-30 14:00:00",[1321,1309,1322,1323,1324,1325,1326,1313],"LLM","vLLM","PagedAttention","KV Cache","推理优化","投机解码",11,{"slug":1329,"path":1330,"title":1331,"date":1332,"tags":1333,"description":43,"draft":1287,"hidden":1287,"published":198,"readingTime":1334},"mini-llm-engine-from-scratch","\u002Fposts\u002Fmini-llm-engine-from-scratch","从零实现 LLM 推理引擎：深挖 vLLM 的六大核心优化","2026-06-30 10:00:00",[1321,1309,1322,1323,1325,1326,1313],10,{"slug":1336,"path":1337,"title":1338,"date":1339,"tags":1340,"description":1346,"draft":1287,"hidden":1287,"published":198,"readingTime":1347},"bytedance-recommendation-architecture-intern-interview","\u002Fposts\u002Fbytedance-recommendation-architecture-intern-interview","字节推荐架构实习生 Data 面试准备：推荐系统、实时特征与高并发八股","2026-06-09",[1341,1342,1343,1344,1345],"面试","推荐系统","后端架构","实时计算","字节跳动","面向字节跳动推荐架构团队实习岗位的八股准备清单，覆盖推荐系统链路、实时数据、特征服务、高并发后端、分布式系统与项目包装。",18,{"slug":1349,"path":1350,"title":1351,"date":1339,"tags":1352,"description":1356,"draft":1287,"hidden":1287,"published":198,"readingTime":1357},"high-concurrency-rate-limiting-algorithms","\u002Fposts\u002Fhigh-concurrency-rate-limiting-algorithms","高并发限流算法：固定窗口、滑动窗口、漏桶与令牌桶",[1353,1354,1343,1355,1341],"高并发","限流","系统设计","面试和工程都常见的限流算法总结，讲清楚固定窗口、滑动窗口、漏桶、令牌桶、并发数限制以及分布式限流如何落地。",12,{"slug":1359,"path":1360,"title":1361,"date":1339,"tags":1362,"description":1366,"draft":1287,"hidden":1287,"published":198,"readingTime":1367},"kafka-producer-broker-consumer","\u002Fposts\u002Fkafka-producer-broker-consumer","Kafka 入门：生产者、Broker、消费者和“消费”到底是什么意思",[1363,1364,1365,1341,1342],"Kafka","消息队列","后端","用推荐系统里的用户行为日志为例，讲清楚 Kafka 的作用、Producer、Broker、Consumer、Topic、Partition、Offset 和消费语义。",8,{"slug":1369,"path":1370,"title":1371,"date":1339,"tags":1372,"description":1377,"draft":1287,"hidden":1287,"published":198,"readingTime":207},"leetcode-lru-merge-k-reverse-list","\u002Fposts\u002Fleetcode-lru-merge-k-reverse-list","链表与缓存高频题：LRU Cache、合并 K 个有序链表、反转链表",[1373,1374,1375,1376,1341],"算法","链表","LRU","LeetCode","面试高频算法题速记，整理 LRU Cache、合并 K 个有序链表、反转链表的核心思路、复杂度和 C++ 代码。",{"slug":1379,"path":1380,"title":1381,"date":1382,"tags":1383,"description":43,"draft":1287,"hidden":1287,"published":198,"readingTime":1388},"meddococr-interpreter-source-analysis","\u002Fposts\u002Fmeddococr-interpreter-source-analysis","MedDocOCR-Interpreter 源码导读：医疗文档 OCR、结构化抽取与报告解读原型","2026-05-22 10:30:00",[1384,1308,1385,1307,1386,1387],"OCR","医疗 AI","Python","源码分析",16,{"slug":1390,"path":1391,"title":1392,"date":1393,"tags":1394,"description":43,"draft":1287,"hidden":1287,"published":198,"readingTime":1334},"cet6-writing-model-essays","\u002Fposts\u002Fcet6-writing-model-essays","六级写作范文背诵包：10 个高频话题","2026-05-20 09:00:00",[1395,1396,1397],"English","CET6","Writing",{"slug":1399,"path":1400,"title":1401,"date":1402,"tags":1403,"description":43,"draft":1287,"hidden":1287,"published":198,"readingTime":1406},"claude-code-context-management","\u002Fposts\u002Fclaude-code-context-management","Claude Code 上下文管理机制：从 Microcompact 到 Auto Compact","2026-05-19 10:00:00",[1404,1298,1321,1309,1405],"Claude Code","上下文工程",27,{"slug":1408,"path":1409,"title":1410,"date":1411,"tags":1412,"description":43,"draft":1287,"hidden":1287,"published":198,"readingTime":1418},"nanotron-llm-pretraining-framework-analysis","\u002Fposts\u002Fnanotron-llm-pretraining-framework-analysis","Nanotron 项目详解：Hugging Face 的大模型预训练框架怎么做分布式训练","2026-05-10 12:10:00",[1321,1413,1414,1415,1416,1309,1417],"大模型训练","分布式训练","Nanotron","Hugging Face","PyTorch",19,{"slug":1420,"path":1421,"title":1422,"date":1423,"tags":1424,"description":43,"draft":1287,"hidden":1287,"published":198,"readingTime":1431},"gofoundry-go-backend-foundation-framework","\u002Fposts\u002Fgofoundry-go-backend-foundation-framework","GoFoundry 项目详解：基于 Go 的后端基础框架套件设计","2026-05-10 11:20:00",[1425,1426,1427,1428,1429,1364,1430],"Go","后端框架","ORM","分布式缓存","分布式锁","项目架构",25,{"slug":1433,"path":1434,"title":1435,"date":1436,"tags":1437,"description":43,"draft":1287,"hidden":1287,"published":198,"readingTime":1406},"cloudvault-go-cloud-storage-system","\u002Fposts\u002Fcloudvault-go-cloud-storage-system","CloudVault 项目详解：基于 Go 的云端存储与网盘系统架构设计","2026-05-10 10:30:00",[1425,1438,1439,1294,1430,1440,1441,1442],"云存储","网盘系统","Redis","RabbitMQ","Elasticsearch",{"slug":1444,"path":1445,"title":1446,"date":1447,"tags":1448,"description":43,"draft":1287,"hidden":1287,"published":198,"readingTime":1451},"openclaw-source-code-analysis","\u002Fposts\u002Fopenclaw-source-code-analysis","OpenClaw 源码导读：个人 AI 助手的网关、通道、插件与运行时架构","2026-05-08 16:30:00",[1449,1298,1309,1450,1387],"OpenClaw","TypeScript",20,{"slug":1453,"path":1454,"title":1455,"date":1456,"tags":1457,"description":43,"draft":1287,"hidden":1287,"published":198,"readingTime":1334},"flow-matching-generative-models","\u002Fposts\u002Fflow-matching-generative-models","Flow Matching：从噪声到数据的连续流生成模型","2026-05-07 00:00:00",[1458,1459,1460,1461],"生成模型","Diffusion","Flow Matching","深度学习",{"slug":1463,"path":1464,"title":1465,"date":1466,"tags":1467,"description":43,"draft":1287,"hidden":1287,"published":198,"readingTime":1474},"database-ai-week4","\u002Fposts\u002Fdatabase-ai-week4","Week 4：数据库速成——从 Storage、Index、Query Optimization 到 Vector DB 与 RAG","2026-05-05 12:00:00",[1468,1469,1470,1307,1471,1472,1473],"数据库","CMU 15-445","Vector DB","LLM Memory","Query Optimization","Caching",15,{"slug":1476,"path":1290,"title":5,"date":1285,"tags":1477,"description":43,"draft":1287,"hidden":1287,"published":198,"readingTime":1388},"distributed-systems-week3",[1294,1295,1115,1296,1297,300,1298],{"slug":1479,"path":1480,"title":1481,"date":1482,"tags":1483,"description":43,"draft":1287,"hidden":1287,"published":198,"readingTime":1474},"gpu-inference-acceleration-week2","\u002Fposts\u002Fgpu-inference-acceleration-week2","Week 2：GPU 与推理加速——从 Kernel、算子融合到 LLM Serving","2026-05-05 10:00:00",[1461,1484,1485,1321,1486,1322,1487],"GPU","推理加速","CMU 10-414","TensorRT",{"slug":1489,"path":1490,"title":1491,"date":1492,"tags":1493,"description":43,"draft":1287,"hidden":1287,"published":198,"readingTime":1347},"dl-framework-autograd-mini","\u002Fposts\u002Fdl-framework-autograd-mini","Week 1：DL 框架与 Autograd——从计算图、反向传播到 Mini Autograd 实现","2026-05-05 09:00:00",[1461,1494,1417,1486,1495],"Autograd","Mini Framework",{"slug":1497,"path":1498,"title":1499,"date":1500,"tags":1501,"description":43,"draft":1287,"hidden":1287,"published":198,"readingTime":1357},"lock-free-concurrency-notes","\u002Fposts\u002Flock-free-concurrency-notes","无锁并发入门：从 CAS 到 Atomic Ring Buffer","2026-04-25",[1502,1503,1504,1505,1506],"C++","并发","无锁编程","性能优化","量化开发",{"slug":1508,"path":1509,"title":1510,"date":1511,"tags":1512,"description":43,"draft":1287,"hidden":1287,"published":198,"readingTime":202},"agent-memory","\u002Fposts\u002Fagent-memory","Agent 对话记忆化：从原理到实现","2026-04-24",[1321,1298,1307,1341],{"slug":1514,"path":1515,"title":1516,"date":1511,"tags":1517,"description":43,"draft":1287,"hidden":1287,"published":198,"readingTime":207},"llm-context-compression","\u002Fposts\u002Fllm-context-compression","LLM 上下文五层压缩机制详解",[1321,1298,1518,1341],"上下文压缩",{"slug":1520,"path":1521,"title":1522,"date":1523,"tags":1524,"description":43,"draft":1287,"hidden":1287,"published":198,"readingTime":1314},"cpp-concurrency-basics","\u002Fposts\u002Fcpp-concurrency-basics","C++ 并发编程入门：从数据竞争到线程池","2026-04-15",[1502,1503,1341,1506],{"slug":1526,"path":1527,"title":1528,"date":1529,"tags":1530,"description":43,"draft":1287,"hidden":1287,"published":198,"readingTime":153},"travel-in-shenzhen","\u002Fposts\u002Ftravel-in-shenzhen","XCPC 深圳游记","2026-04-13",[1531,1502,1532],"XCPC","比赛",{"slug":1534,"path":1535,"title":1536,"date":1537,"tags":1538,"description":43,"draft":1287,"hidden":1287,"published":198,"readingTime":219},"backend-stack-deep-dive","\u002Fposts\u002Fbackend-stack-deep-dive","后端五件套：FastAPI \u002F Node.js \u002F SQLAlchemy async \u002F PostgreSQL \u002F Docker 面试速通","2026-04-07",[1365,1539,1540,1541,1341],"FastAPI","PostgreSQL","Docker",{"slug":1543,"path":1544,"title":1545,"date":1546,"tags":1547,"description":43,"draft":1287,"hidden":1287,"published":198,"readingTime":207},"deepscientist-tech-stack","\u002Fposts\u002Fdeepscientist-tech-stack","DeepScientist 技术栈全解析：一个 AI 科研平台的架构设计","2026-04-06",[1548,1539,1549,1540,1341],"全栈","Next.js",{"slug":1551,"path":1552,"title":1553,"date":1546,"tags":1554,"description":43,"draft":1287,"hidden":1287,"published":198,"readingTime":1367},"minicode-source-analysis","\u002Fposts\u002Fminicode-source-analysis","MiniCode 源码解析：用 5000 行 TypeScript 实现一个 AI 编程助手",[1450,1555,1321,1387,1341],"CLI",{"slug":1557,"path":1558,"title":1559,"date":1546,"tags":1560,"description":43,"draft":1287,"hidden":1287,"published":198,"readingTime":202},"nova-theme-implementation","\u002Fposts\u002Fnova-theme-implementation","我是怎么从零实现 Nova 主题的",[1561,1562,1563],"Hexo","前端","开源",{"slug":1565,"path":1566,"title":1567,"date":1568,"tags":1569,"description":43,"draft":1287,"hidden":1287,"published":198,"readingTime":168},"git-cheatsheet","\u002Fposts\u002Fgit-cheatsheet","Git 常用操作备忘","2026-04-05 14:00:00",[1570,1571],"Git","工具",{"slug":1573,"path":1574,"title":1575,"date":1576,"tags":1577,"description":43,"draft":1287,"hidden":1287,"published":198,"readingTime":183},"github-actions-intro","\u002Fposts\u002Fgithub-actions-intro","GitHub Actions 入门：自动化你的工作流","2026-04-04 09:00:00",[1578,1579,1580],"GitHub Actions","CI\u002FCD","自动化",1782796012096]