原帖内容

Sharing two long-horizon sessions on pi + self-hosted models. first: 16 hours long, zero steering, 99 laps. I used pi + local models to port our upcoming Jina model from bf16 PyTorch into MLX+MTP quants (model J in the sequel). The local model is qwen3.8-flash-next-omlx-q4 from @jundotkim, running at the full 262K context window. I have only 2 plugins on pi: - no /goal command or plugin installed. no need. the first user msg is the "goal". - pi-vcc for context compaction. This is a must for long-horizon tasks with local models. Default pi compaction is opinionated conservative, which is fine as it won't easily forget progress, but it gets slower and slower over time. Pi-vcc is a drop-in replacement of the built-in one, and vcc_search/vcc_recall is basically the memory recall tool. - web read and search tools exposed via jina-mcp, because in my first prompt and in the nudge prompt, I told the agent to search the web for inspiration and solutions when it gets stuck. - search_corpus, a simple vector search tool grounded on local disk. I have a data room locally that can help with this task, so I wanted the agent to have access to it. - The swimlanes chart you see is my vibed visualization to help me understand tool-calling preferences. Prefill/reasoning/generation/bash seem to take up the whole timeline, but that's just because they're so frequent that at a 15h scale the visualization gets too dense and looks like that. The whole setup runs on an M3 Ultra 512G: so yes, pi + qwen3.8-flash-next q4 is running there and model J is also running there. But it's fine, peak memory usage is only around 150GB, plenty left. Interestingly, I originally thought these two would compete the hell out of my Mac, but it turns out they rarely compete at the flops level. When model J is being evaluated or debugged, pi is doing tool calling and waiting, so qwen3.8 just hangs there waiting for the tool callback. The two don't really block each other. Now some takes: - First & most important: it's done correctly, and the final ported Q4 MLX-MTP is usable with properly hill-climbed throughput. - The model spent 3.6h on the first lap, fully autonomous, no nudge no steering, which says something about qwen3.8-flas-next capability on long-horizon tasks. I've seen a 9h+ lap on another task. Since I don't have any /goal plugin installed and my pi setup is really minimal, I give all the credit for this self-driving to qwen. So 2026 open-weight models are seriously trained for long-horizon tasks, not just a marketing term. - The remaining laps are uneven, from 5min to 35min, btw they exist because the same "do better" nudge prompt was sent after each previous lap to force the agent to find better quants or a faster MTP implementation for model J. - I looked at some laps and it's really fixing things. Given the surface of this task is pretty big, with so many knobs and levers to tune, I'm not surprised the model can always find things to do even after 90 laps. What confuses me is why it doesn't group them and do them in one lap. Practically the difference is small, whether it's one long lap or many small ones, but the former is probably easier on the eyes and to maintain. - We know pi has a very limited set of built-in tools by design philosophy: bash/edit/read/write. But in practice, bash dominated every other tool. This is probably qwen3.8-flash-next's own preference, doing everything via bash including edit, search, read, write. So the swimlane you see probably understates the other tools' frequencies, unless you actually parse the inline bash commands and look at them. I wonder if this bash-is-all-u-need will change pi's future design philosophy. - The read tool calls are mostly about invoking qwen's vision capability to read images; normal file reads are usually wrapped in bash. - Surprisingly, web search/read was almost never called, despite every nudge prompt encouraging it to search the web when stuck and get inspired.