LLM-generated code that seemingly went without much review or design is always such an interesting dive into just how bloated you can make code. Multiple files are close to 10K LOC, one file contains a switch statement that has so many case statements it spans more than 1000 lines.
I guess it depends on the model you're trying to use, but seems most of them prefer smaller codebases, they work a lot better with less code, which kind of makes sense. With that in mind, I'd probably aim for something way smaller to bootstrap a self-improving agent. Then I'd use this "Prime Agent" as an example to my self-improving agent for what it should not evolve to.
I built one of these RLM harnesses and a local MCP server along with logging, memories, and project rules based on directories. It worked great for a while but the foundational models have largely caught up to the point where they don't need this harness anymore. At least for my use cases. I can basically just store context in .md in the directories we work out of together and accomplish what I need.
The core idea of the RLM paper is to make a regular LLM act more like a coding agent - offload context to something external that needs to be explicitly queried instead of filling up valuable context. The "recursion" part of the paper really only wins because they use a top-tier model for the root agent, and cheaper models for the sub-agents.
Prime Agent just took the RLM idea (which is really just an academic view on how coding agents have always worked) and then added this "continual harness" idea. This part isn't super well described in the blog post, but includes some message passing between the agents, and the ability to share code.
Overall I chalk it up as neat, but not revolutionary. Another version of what most of these systems are already doing.
It'll be really interesting when they run RL training on the harness self-improvement loop. I've tried using LLMs for harness engineering, but it often creates too much bloat that weighs things down in the end. Guessing it's just not something the models are tuned to do by default.
Curious if anyone's tried using RL for harness engineering? I think we're still pretty far away from the optimal harness, especially when it comes to long-context memory management.
For everyone downvoting: It's literally a story about the creation of mankind's first artificial general intelligence, Prime Intellect, and the consequences of that discovery.
It’s a self-improving harness, and ARC-AGI-3 is explicitly a few-shot benchmark. It’s likely that it gave itself more than the maximum number of tries to learn the games, or even hardcoded the answers.
There’s a lot of improvement to be had from the benchmark harnesses, but sometimes, like with ARC-AGI-3, the limitations are intentional.
I guess it depends on the model you're trying to use, but seems most of them prefer smaller codebases, they work a lot better with less code, which kind of makes sense. With that in mind, I'd probably aim for something way smaller to bootstrap a self-improving agent. Then I'd use this "Prime Agent" as an example to my self-improving agent for what it should not evolve to.
https://minimal-agent.com/
https://alexzhang13.github.io/blog/2025/rlm/
Prime Agent just took the RLM idea (which is really just an academic view on how coding agents have always worked) and then added this "continual harness" idea. This part isn't super well described in the blog post, but includes some message passing between the agents, and the ability to share code.
Overall I chalk it up as neat, but not revolutionary. Another version of what most of these systems are already doing.
Curious if anyone's tried using RL for harness engineering? I think we're still pretty far away from the optimal harness, especially when it comes to long-context memory management.
I am curious - how does it fare for other benchmarks, or everyday programming?
Is it that it wasn't accepted yet, or are there issues with how it was run?
There’s a lot of improvement to be had from the benchmark harnesses, but sometimes, like with ARC-AGI-3, the limitations are intentional.