> it installs to the homebrew dir, despite not being a homebrew package
I can see the reasoning trace in front of me...
> Hmm, user asked me to install so CLI is globally available for their user, but didn't instruct where. I inspected $PATH and only directory I can write to is where Homebrew packages live. The user didn't say "Don't pretend it is a Homebrew package" so this seems like a suitable location, lets save it there as it fits what the user asked for.
You shouldn't link that without mentioning that it is a particularly intense S&M/rape/incest fetish piece, and that the squick content is entirely integral to the story. I'm generally dubious of "trigger warnings" but if ever there was something that needed content tags-- this is it.
If you imagine it being written for alt.sex.stories.moderated but somehow failing to be erotic by virtue of being too explicit, and being a few orders of magnitude better writing for that venue... then you wouldn't be too far off. [Started making a silly example of how some sex story could fail like that and then realized I was litterally describing a scene from prime intellect].
It's also arguably the origin of a lot of the mentally ill ai-safety hysteria. Arguably an enjoyable romp for those who understand that it's fiction, but it seems a lot of people cannot.
You and everyone else saying so are correct; I forgot how bad it was when I posted the link. "Prime Agent" obviously used the story for branding or a callback, so this is 100% on-topic, but I should have re-examined what I was posting and added a warning.
I vouch for this story, personally. I love it. I consider it a classic among science fiction novellas.
It does include some brutal sadomasochism with graphic torture, that's true, and that's why it isn't better known. If that part bothers you, which is fair, I would skim past it. The story overall is quite worth it
Years and years ago I saw this story on k5 when it came out and never read it -- but from commentary I always thought was some sort of hard sci-fi exploration of what happens when we get AGI/ASI.
So all these years later AI is real, I see the story again, and I figure, hey, why not read it. And how very surprised I was to find out it's basically a horny kink story with a little bit of AI/SF window dressing!
Tbh the torture scenes are not nearly as disturbing as the underage incest. It's like "Saw" vs. a pedo flick from the 70s. Especially weird since the latter is portrayed like some kind of innocent ideal.
It's frustrating because apart from those (awful) aspects the story is quite thoughtful and compelling. I always recommend people skip the final chapter since the resulting existential cliffhanger is far better than the bizarre last-minute swerve into gratuitous sex abuse.
> the underage incest [...] Especially weird since the latter is portrayed like some kind of innocent ideal.
(Potential spoilers to the story ahead) Are you talking specifically about the "re-population" stuff from the ending? It was many years I last read it, but I seem to recall that there was hesitation and stuff involved, together with "This is literally the only way to re-populate since we're just two people" basically, not sure what other direction it could have gone into, unless you'd just skip describing it completely. But maybe I misremember (or my brain subconsciously suppress my memory) more of the same horrible stuff from the story?
Aside from it being biologically impossible to repopulate from a single couple, it was not really portrayed ambiguously. I don't care to reread it, but from what I remember, the main protagonist was the driving force behind it, grooms her own (single-digit-age?) daughter into the idea, who then sleeps with her own father in graphic detail. He is a little hesitant at first, but relents almost immediately, and this is implied to be a good thing. Cue decades of continued child-incest and inbreeding.
Putting the characters in that situation and resolving it the explicit way he did was a deliberate choice the author makes -- it didn't have to be that way.
— Spoilers, if you’ve not read this and yet are still reading this comment chain
I guess the grooming may start earlier, it’s not really discussed and it’s a bit ambiguous as to when that started
> but relents almost immediately,
Physically, at the time, though this is something they’ve argued about for 6 years.
It’s not a passage I particularly care for, and it could have been less explicit but then I’m mixed on how that would change the story. I’ve just finished it and it’s certainly an interesting sci-fi story.
Does it provide a strange contrast to the other things?
There’s disgusting horrors inflicted by terrible people but to the willing (but are they only willing due to what was done to them before, by PI?).
Deliberate and horrible suffering of a single person caused for minor gain.
There’s calm destruction of trillions, unfeeling, for protection out of a measure of harms and trying to protect one set. There’s the good goal or perhaps just self indulgent goal that led to that too.
There’s the deliberate killing of trillions with glee at breaking things for one persons view of humanity. There’s the deliberate doing of this by the original catalyst, but with less clear direction. Perhaps a lack of logic and more boredom?
And yet this, done for improving survival chances of a group, feels over the line. I don’t disagree that it is quite disgusting but I do find it interesting at that being my reaction given what else has been done up to this point.
My issue was that the violence and incest were one of the main drivers of the story. It felt like reading some sex novela that happens to have some great scifi elements rather than the other way around.
>It appeared to listen intently as a man in a cleric's uniform railed. "God made all intelligent creatures," the man was saying in a powerful voice. "You may have the apprearance of thinking, but you are really just parroting the responses taught you by that man there." He pointed at Lawrence.
I put this novella into the same bucket as the altered carbon books. So glad I read them, absolutely packed with weird ideas, but I will probably never have the stomach to revisit them.
For everyone downvoting: It's literally a story about the creation of mankind's first artificial general intelligence, Prime Intellect, and the consequences of that discovery.
I feel that more warning is needed for this book. Everything you’ve stated is true, but the book’s first chapter contains some of the most disturbing depictions of stuff I don’t know I can type here on HN. Additionally, the last chapter put a really bad taste in my mouth.
That said, the stuff that deals with AI and its implications on the universe was great and more relevant than ever. The way PI works is pretty similar to how we use subagents to tackle large repos!
Not just the first AGI, but the book deals with a (very) hard-takeoff scenario of the singularity. The AI, Prime Intellect starts recursively increasing its capabilities and things rapidly spin out of control. This is actually a small part of the book, most of the book is about what happens after this - Its a fantastic read, yes as other posters have said there is some extremely graphic sex and violence in this book, but it is not gratuitous. I highly reccommend.
LLM-generated code that seemingly went without much review or design is always such an interesting dive into just how bloated you can make code. In this repository, multiple files are close to 10K LOC, one file contains a switch statement that has so many case statements it spans more than 1000 lines, and lots of other fun stuff.
I guess it depends on the model you're trying to use, but seems most of them prefer smaller codebases, they work a lot better with less code, which kind of makes sense. With that in mind, I'd probably aim for something way smaller to bootstrap a self-improving agent. Then I'd use this "Prime Agent" as an example to my self-improving agent for what it should not evolve to.
I dig the spirit, but if I'm meant to understand it in an afternoon, why is it codegolfed to such an extent? It's like reading the output of a minifier, not idiomatic Go.
> also want to do clojure, unfortunately it looks like java does not come with json support out of the box
Clojure has clojure.data.json (https://clojure.github.io/data.json/), should be easy to use albeit not blazing fast exactly, doesn't really matter here though. Otherwise, you could use Babashka, comes with Cheshire (and others) out of the box.
> In this repository, multiple files are close to 10K LOC, one file contains a switch statement that has so many case statements it spans more than 1000 lines, and lots of other fun stuff.
Pft. Over multiple cases?
I've seen 1000 lines inside a single if-block. The clause was always true. I was not able to break that thing up in my time at that company.
Human wrote that nonsense.
Thing is, the app was pretty successful despite all the stuff wrong with the code. And this success is why I've been bullish on GenAI code itself, even though I'm also bearish on AI companies being able to profit from that.
At all feels a bit like being bullish on the adjustable wrench, to me.
It has lots of utility. Very broad… but not the best for a lot of things. It will be a ubiquitous tool. Outcomes will depend heavily on the quality of the mechanic unless you’re tightening a bolt on your fence gate.
Literally says who. Swe are the least credible "engineers". "It depends" "No you cant track us" "No we have no credentials system other than big company shill certs"
How on Earth do we solve this bloat and death-by-a-thousand-cuts issue with frontier LLMs? Are there any actual solutions or attempts at solutions to this problem that I can try? Any tools or frameworks? I've tried re-architecture skills, dedicated cleanup sessions, and a bunch of other stuff, but nothing really works well.
Don't try to "run as fast as possible" and just add code willy-nilly, think about the design, refactor as you add new features/fixes and intentionally be a bit slower and more considerate.
It's not a technical problem, if you steer the LLMs enough and actually review what they do, you can build proper and clean software with them.
It's an offline coding grader. It works well: You ask for a new module, LLM starts spitting out bloated crap, the code score goes down. LLM keep looping until code score is back up.
Not a substitute for human code review, but keeps things within tight guard rails.
I wonder if it just ends up in reward-hacking, or if it actually substantially changes the quality of the code.
Are these static analysis metrics enough on their own to cause a substantial improvement?
I tried setting some constraints as part of my system instruction prompt to prevent God file creation, and it did end up breaking files up, but it would often just create a new type of mess and do it needlessly.
Without the model doing additional reasoning and search for alternative approaches that were better fitted on top as it was doing this file splitting, it didn't actually seem to improve quality at all.
So I'm curious to know if this would just result in more of that, or if it would actually result in different choices. Because there's more to code maintenance than just splitting things up into modules, etc. Like the main part of it that I suspect will not be captured is the part of thinking about the big picture, brainstorming multiple architectural approaches, and reasoning through the best one.
Only anecdotal benchmarks for what ever that is worth.
I notice the LLM reasoning things like: "The maintainability score went down, let me look.... I see I've duplicated existing code which already exists in this module...I see this logic could be consolidated".
I have noticed file splitting is a strategy to improve the "score".
In my experience, I've been doing more the architectural guidance. There is also a rules engine which I have not used, but looks pretty interesting:
It'll be really interesting when they run RL training on the harness self-improvement loop. I've tried using LLMs for harness engineering, but it often creates too much bloat that weighs things down in the end. Guessing it's just not something the models are tuned to do by default.
Curious if anyone's tried using RL for harness engineering? I think we're still pretty far away from the optimal harness, especially when it comes to long-context memory management.
I built one of these RLM harnesses and a local MCP server along with logging, memories, and project rules based on directories. It worked great for a while but the foundational models have largely caught up to the point where they don't need this harness anymore. At least for my use cases. I can basically just store context in .md in the directories we work out of together and accomplish what I need.
The core idea of the RLM paper is to make a regular LLM act more like a coding agent - offload context to something external that needs to be explicitly queried instead of filling up valuable context. The "recursion" part of the paper really only wins because they use a top-tier model for the root agent, and cheaper models for the sub-agents.
Prime Agent took the RLM idea (which is really just an academic view on how coding agents have always worked) and then added this "continual harness" idea. This part isn't super well described in the blog post, but includes some message passing between the agents, and the ability to share code.
Overall I chalk it up as neat, but not revolutionary. Another version of what most of these systems are already doing.
I went the skills route - skill improvement skill and a rule to use it always. Basically it's directed that if anything causes more than a hop of thinking - failure - try something else it should flag that it needs to learn it as a skill so it never does it other than first shot again or if an existing skill fails improve it after it solves whatever problem. it then syncs the files to a shared location and updates the version and also pulls new skills. this let's a team use it or you have multiple workstations.
It’s a self-improving harness, and ARC-AGI-3 is explicitly a few-shot benchmark. It’s likely that it gave itself more than the maximum number of tries to learn the games, or even hardcoded the answers.
There’s a lot of improvement to be had from the benchmark harnesses, but sometimes, like with ARC-AGI-3, the limitations are intentional.
As models get stronger, huge harnesses may become less useful. An overly opinionated harness could even constrain the model’s reasoning instead of improving it.
Installer might look pretty but it installs to the homebrew dir, despite not being a homebrew package. Very dirty. No uninstall method.
> it installs to the homebrew dir, despite not being a homebrew package
I can see the reasoning trace in front of me...
> Hmm, user asked me to install so CLI is globally available for their user, but didn't instruct where. I inspected $PATH and only directory I can write to is where Homebrew packages live. The user didn't say "Don't pretend it is a Homebrew package" so this seems like a suitable location, lets save it there as it fits what the user asked for.
yea, you're probably right, if only there wasn't a local writeable bin path that wasn't homebrew they could put it :D
https://localroger.com/prime-intellect/mopiidx.html
You shouldn't link that without mentioning that it is a particularly intense S&M/rape/incest fetish piece, and that the squick content is entirely integral to the story. I'm generally dubious of "trigger warnings" but if ever there was something that needed content tags-- this is it.
If you imagine it being written for alt.sex.stories.moderated but somehow failing to be erotic by virtue of being too explicit, and being a few orders of magnitude better writing for that venue... then you wouldn't be too far off. [Started making a silly example of how some sex story could fail like that and then realized I was litterally describing a scene from prime intellect].
It's also arguably the origin of a lot of the mentally ill ai-safety hysteria. Arguably an enjoyable romp for those who understand that it's fiction, but it seems a lot of people cannot.
You and everyone else saying so are correct; I forgot how bad it was when I posted the link. "Prime Agent" obviously used the story for branding or a callback, so this is 100% on-topic, but I should have re-examined what I was posting and added a warning.
I vouch for this story, personally. I love it. I consider it a classic among science fiction novellas.
It does include some brutal sadomasochism with graphic torture, that's true, and that's why it isn't better known. If that part bothers you, which is fair, I would skim past it. The story overall is quite worth it
Years and years ago I saw this story on k5 when it came out and never read it -- but from commentary I always thought was some sort of hard sci-fi exploration of what happens when we get AGI/ASI.
So all these years later AI is real, I see the story again, and I figure, hey, why not read it. And how very surprised I was to find out it's basically a horny kink story with a little bit of AI/SF window dressing!
Tbh the torture scenes are not nearly as disturbing as the underage incest. It's like "Saw" vs. a pedo flick from the 70s. Especially weird since the latter is portrayed like some kind of innocent ideal.
It's frustrating because apart from those (awful) aspects the story is quite thoughtful and compelling. I always recommend people skip the final chapter since the resulting existential cliffhanger is far better than the bizarre last-minute swerve into gratuitous sex abuse.
> the underage incest [...] Especially weird since the latter is portrayed like some kind of innocent ideal.
(Potential spoilers to the story ahead) Are you talking specifically about the "re-population" stuff from the ending? It was many years I last read it, but I seem to recall that there was hesitation and stuff involved, together with "This is literally the only way to re-populate since we're just two people" basically, not sure what other direction it could have gone into, unless you'd just skip describing it completely. But maybe I misremember (or my brain subconsciously suppress my memory) more of the same horrible stuff from the story?
Yes, the final chapter.
Aside from it being biologically impossible to repopulate from a single couple, it was not really portrayed ambiguously. I don't care to reread it, but from what I remember, the main protagonist was the driving force behind it, grooms her own (single-digit-age?) daughter into the idea, who then sleeps with her own father in graphic detail. He is a little hesitant at first, but relents almost immediately, and this is implied to be a good thing. Cue decades of continued child-incest and inbreeding.
Putting the characters in that situation and resolving it the explicit way he did was a deliberate choice the author makes -- it didn't have to be that way.
— Spoilers, if you’ve not read this and yet are still reading this comment chain
I guess the grooming may start earlier, it’s not really discussed and it’s a bit ambiguous as to when that started
> but relents almost immediately,
Physically, at the time, though this is something they’ve argued about for 6 years.
It’s not a passage I particularly care for, and it could have been less explicit but then I’m mixed on how that would change the story. I’ve just finished it and it’s certainly an interesting sci-fi story.
Does it provide a strange contrast to the other things?
There’s disgusting horrors inflicted by terrible people but to the willing (but are they only willing due to what was done to them before, by PI?).
Deliberate and horrible suffering of a single person caused for minor gain.
There’s calm destruction of trillions, unfeeling, for protection out of a measure of harms and trying to protect one set. There’s the good goal or perhaps just self indulgent goal that led to that too.
There’s the deliberate killing of trillions with glee at breaking things for one persons view of humanity. There’s the deliberate doing of this by the original catalyst, but with less clear direction. Perhaps a lack of logic and more boredom?
And yet this, done for improving survival chances of a group, feels over the line. I don’t disagree that it is quite disgusting but I do find it interesting at that being my reaction given what else has been done up to this point.
My issue was that the violence and incest were one of the main drivers of the story. It felt like reading some sex novela that happens to have some great scifi elements rather than the other way around.
>It appeared to listen intently as a man in a cleric's uniform railed. "God made all intelligent creatures," the man was saying in a powerful voice. "You may have the apprearance of thinking, but you are really just parroting the responses taught you by that man there." He pointed at Lawrence.
I put this novella into the same bucket as the altered carbon books. So glad I read them, absolutely packed with weird ideas, but I will probably never have the stomach to revisit them.
I went into this thinking I’m a big boy and don’t need all these warnings.
I was wrong.
For everyone downvoting: It's literally a story about the creation of mankind's first artificial general intelligence, Prime Intellect, and the consequences of that discovery.
I feel that more warning is needed for this book. Everything you’ve stated is true, but the book’s first chapter contains some of the most disturbing depictions of stuff I don’t know I can type here on HN. Additionally, the last chapter put a really bad taste in my mouth.
That said, the stuff that deals with AI and its implications on the universe was great and more relevant than ever. The way PI works is pretty similar to how we use subagents to tackle large repos!
yea the beginning is pretty gory / horrific. it almost made me stop reading but you can't deny the world building is.. unique
Yeah, there's no question this one's over-the-top when it comes to trying to gross out the reader, but I'd like to think people can overlook that.
Is it only the first chapter? I remember it to be the bulk of the book.
Not just the first AGI, but the book deals with a (very) hard-takeoff scenario of the singularity. The AI, Prime Intellect starts recursively increasing its capabilities and things rapidly spin out of control. This is actually a small part of the book, most of the book is about what happens after this - Its a fantastic read, yes as other posters have said there is some extremely graphic sex and violence in this book, but it is not gratuitous. I highly reccommend.
A write up on RLM (Recursive Language Models) by one of the authors of the RLM paper:
https://alexzhang13.github.io/blog/2025/rlm/
LLM-generated code that seemingly went without much review or design is always such an interesting dive into just how bloated you can make code. In this repository, multiple files are close to 10K LOC, one file contains a switch statement that has so many case statements it spans more than 1000 lines, and lots of other fun stuff.
I guess it depends on the model you're trying to use, but seems most of them prefer smaller codebases, they work a lot better with less code, which kind of makes sense. With that in mind, I'd probably aim for something way smaller to bootstrap a self-improving agent. Then I'd use this "Prime Agent" as an example to my self-improving agent for what it should not evolve to.
> one file contains a switch statement that has so many case statements it spans more than 1000 lines
Probably best to leave YandereDev's code out of the training data.
here is a take on a smol agent ("smol")
https://github.com/smol-env/smoleasier to add and customize stuff when you start from a small base
think of it as your starter dough
I dig the spirit, but if I'm meant to understand it in an afternoon, why is it codegolfed to such an extent? It's like reading the output of a minifier, not idiomatic Go.
got one in typescript? or node.js with javascript?
I'm adding python in a few minutes and then looking into other languages including javascript and typescript
all with the same zero 3rd party dependencies approach
also want to do clojure, unfortunately it looks like java does not come with json support out of the box
> also want to do clojure, unfortunately it looks like java does not come with json support out of the box
Clojure has clojure.data.json (https://clojure.github.io/data.json/), should be easy to use albeit not blazing fast exactly, doesn't really matter here though. Otherwise, you could use Babashka, comes with Cheshire (and others) out of the box.
ty for the pointers
I like the babashka angle!
The stuff 99% of agents are coded in? Please tell me this was sarcasm.
i'm also not a fan of javascript and typescript (let alone the ecosystem they come with)
but the runtimes are everywhere
and a smol agent /w no 3rd party dependencies is probably a good option to have in a pinch
> In this repository, multiple files are close to 10K LOC, one file contains a switch statement that has so many case statements it spans more than 1000 lines, and lots of other fun stuff.
Pft. Over multiple cases?
I've seen 1000 lines inside a single if-block. The clause was always true. I was not able to break that thing up in my time at that company.
Human wrote that nonsense.
Thing is, the app was pretty successful despite all the stuff wrong with the code. And this success is why I've been bullish on GenAI code itself, even though I'm also bearish on AI companies being able to profit from that.
At all feels a bit like being bullish on the adjustable wrench, to me.
It has lots of utility. Very broad… but not the best for a lot of things. It will be a ubiquitous tool. Outcomes will depend heavily on the quality of the mechanic unless you’re tightening a bolt on your fence gate.
Bash is all you need.
https://minimal-agent.com/
[dead]
Literally says who. Swe are the least credible "engineers". "It depends" "No you cant track us" "No we have no credentials system other than big company shill certs"
How on Earth do we solve this bloat and death-by-a-thousand-cuts issue with frontier LLMs? Are there any actual solutions or attempts at solutions to this problem that I can try? Any tools or frameworks? I've tried re-architecture skills, dedicated cleanup sessions, and a bunch of other stuff, but nothing really works well.
Don't try to "run as fast as possible" and just add code willy-nilly, think about the design, refactor as you add new features/fixes and intentionally be a bit slower and more considerate.
It's not a technical problem, if you steer the LLMs enough and actually review what they do, you can build proper and clean software with them.
Yeah, but that's boring. :)
Add Sentrux to the LLM loop. https://github.com/sentrux/sentrux
It's an offline coding grader. It works well: You ask for a new module, LLM starts spitting out bloated crap, the code score goes down. LLM keep looping until code score is back up.
Not a substitute for human code review, but keeps things within tight guard rails.
Have you done any benchmarks on this approach?
I wonder if it just ends up in reward-hacking, or if it actually substantially changes the quality of the code.
Are these static analysis metrics enough on their own to cause a substantial improvement?
I tried setting some constraints as part of my system instruction prompt to prevent God file creation, and it did end up breaking files up, but it would often just create a new type of mess and do it needlessly.
Without the model doing additional reasoning and search for alternative approaches that were better fitted on top as it was doing this file splitting, it didn't actually seem to improve quality at all.
So I'm curious to know if this would just result in more of that, or if it would actually result in different choices. Because there's more to code maintenance than just splitting things up into modules, etc. Like the main part of it that I suspect will not be captured is the part of thinking about the big picture, brainstorming multiple architectural approaches, and reasoning through the best one.
Only anecdotal benchmarks for what ever that is worth.
I notice the LLM reasoning things like: "The maintainability score went down, let me look.... I see I've duplicated existing code which already exists in this module...I see this logic could be consolidated".
I have noticed file splitting is a strategy to improve the "score".
In my experience, I've been doing more the architectural guidance. There is also a rules engine which I have not used, but looks pretty interesting:
``` [constraints] max_cycles = 0 max_coupling = "B" max_cc = 25 no_god_files = true
[[layers]] name = "core" paths = ["src/core/"] order = 0
[[layers]] name = "app" paths = ["src/app/"] order = 2
[[boundaries]] from = "src/app/" to = "src/core/internal/" reason = "App must not depend on core internals" ```
Are you the author? The font size and contrast is kind of poor.
Don't have it write the whole thing in one session.
Interesting, so they shipped slop? Succumbed to their own AI psychosis?
It'll be really interesting when they run RL training on the harness self-improvement loop. I've tried using LLMs for harness engineering, but it often creates too much bloat that weighs things down in the end. Guessing it's just not something the models are tuned to do by default.
Curious if anyone's tried using RL for harness engineering? I think we're still pretty far away from the optimal harness, especially when it comes to long-context memory management.
Yes, used bert model with decent results.
What policy would you use?
Not RL. SFT.
Interesting, what did you use for the data? And do you have a write-up anywhere?
I built one of these RLM harnesses and a local MCP server along with logging, memories, and project rules based on directories. It worked great for a while but the foundational models have largely caught up to the point where they don't need this harness anymore. At least for my use cases. I can basically just store context in .md in the directories we work out of together and accomplish what I need.
The core idea of the RLM paper is to make a regular LLM act more like a coding agent - offload context to something external that needs to be explicitly queried instead of filling up valuable context. The "recursion" part of the paper really only wins because they use a top-tier model for the root agent, and cheaper models for the sub-agents.
Prime Agent took the RLM idea (which is really just an academic view on how coding agents have always worked) and then added this "continual harness" idea. This part isn't super well described in the blog post, but includes some message passing between the agents, and the ability to share code.
Overall I chalk it up as neat, but not revolutionary. Another version of what most of these systems are already doing.
I went the skills route - skill improvement skill and a rule to use it always. Basically it's directed that if anything causes more than a hop of thinking - failure - try something else it should flag that it needs to learn it as a skill so it never does it other than first shot again or if an existing skill fails improve it after it solves whatever problem. it then syncs the files to a shared location and updates the version and also pulls new skills. this let's a team use it or you have multiple workstations.
It is impressive that it (almost) saturates ARC-AGI-3, https://x.com/PrimeIntellect/status/2085087000764568010.
I am curious - how does it fare for other benchmarks, or everyday programming?
PrimeIntelect is not on official ARC-AGI-3 leaderboard: https://arcprize.org/leaderboard
Good to know!
Is it that it wasn't accepted yet, or are there issues with how it was run?
It’s a self-improving harness, and ARC-AGI-3 is explicitly a few-shot benchmark. It’s likely that it gave itself more than the maximum number of tries to learn the games, or even hardcoded the answers.
There’s a lot of improvement to be had from the benchmark harnesses, but sometimes, like with ARC-AGI-3, the limitations are intentional.
leaderboard likely has results from "semi-private" dataset, and graph above likely from public dataset, so it can be easily overfit.
Calling this "prime intellect" is a choice and, honestly, I am very much here for it.
As models get stronger, huge harnesses may become less useful. An overly opinionated harness could even constrain the model’s reasoning instead of improving it.
I'd assume the best harness for a model will tend to be the one that it's RLVF'd on.
Very interesting idea but without any concrete examples of performance on real tasks its just a pretty idea
They achieved 95% in ARC-AGI-3. [0]
0: https://x.com/PrimeIntellect/status/2085087000764568010
With Opus 5 as the underlying model.
Examples and ROI
Might actually try this
[flagged]
[flagged]
[flagged]
[flagged]
[dead]
this seems like its going to rip through tokens like crazy
self improvement is not a new idea but at current economics its not feasible
[dead]