WHAT CHANGES?
每一步,为什么而改变?
并排比较章节的职责与状态,再观察框架伪代码如何变化。
关键代码对比
移除增加s01_request_flow.pys03_kv_cache.py−# Framework pseudocode, not an executable implementation. −for request in inputs: − llm_engine.add_request(request) +# Framework pseudocode for request-owned cache state. +state = sequence_table[request_id] +blocks = block_manager.resolve(state.block_table) +metadata = build_input_metadata( + query_tokens=planned_tokens, + cached_positions=state.computed_positions, + cache_blocks=blocks, +) +worker.execute_model_step(metadata, cache_engine.cache) −while llm_engine.has_unfinished_requests(): − ready_outputs = llm_engine.step() − collect(ready_outputs) − −collect(llm_engine.flush_inflight_reqs()) − −# Inside a scheduled round: −# Scheduler -> Worker -> ModelRunner -> sampling −# -> ready state updates and request outputs −# A round may cover several requests and token positions. +# Execution consumes input positions and updates cache state. +# A selected token becomes input only if the request continues. +# Reclaim blocks only when their references are no longer needed.
对比对象为框架阅读伪代码,用来理解职责与状态依赖。