> The phrase "frontier model" is starting to mean two things. One is a checkpoint. The other is a system boundary.
LLM-isms aside, I don't think we want this to be the case? An LLM, for all its complexity, is something that can be reasoned about. It's picking the next token, until it hits an EOS. The semantics imposed on those tokens (reasoning ,tool call, etc.) are up to the user('s harness) to decide and act on. The more that's pushed behind the facade, the harder it is achieve sufficient understanding of the model's behavior s.t. one can compose it into larger abstractions. Perhaps the performance (and the adherence to an interface/contract) compensate? But swapping from Opus or 5.5 to this or Fugu seems like a much bigger change than swapping between different 'base' models.
I might be wrong, but strongly suspect that Fable 5 is already something in this shape, considering long time to first token while having normal troughput.
They're applying misdirection so that we use their secret-sauce agentic framework, but like a black box and without seeing any of the internal reasoning patterns, cause that would give it away.
That's a deal-breaker for me. I need as much observability and control over my development workflow as possible; that's part of my secret sauce.
This seems to be a new trend. Noticed it with GPT "ultra" in their announcement[1]. I'm with you, a large language model and a system of many language models working together are not the same thing
A sign of system-level optimization starting to overshadow raw/brute-force scaling of foundational models. My view is that foundational models are indeed statistic parrots, just like humans (humans are worse parrots, but human brain's context window is so small that they often do not recognize how broken was human-intermediated intelligence swarm, but such small context window might be a fundamental feature of so-called intelligence).
LLMs to me are better intelligence than humans in 3 aspects:
1. LLMs can somehow entirely do perspective taking, humans cannot even think self in next 10 minutes after making a decision
2. LLMs can somehow be asked to arbitrarily elevate and lower abstraction level (can be seen as a special form of perspective taking)
3. LLMs "think" instantly
All these innate capabilities should be combined with system level optimization to achieve the last 10% to be beyond human intelligence.
This sounds like adding way too much complexity for something that will likely be covered fully by the next gen of frontier models within a single prompt. It also makes it all opaque and difficult to trace.
The next generation of models are currently being withheld from general release. Beyond that, there's still a lot of room to compete on price and also independence from the US labs.
the rename from checkpoint to system boundary is how you lose traceability. a multi-model black box that benchmarks well is great until it fails on your workload and theres no trace to debug why.
Looks nice (slop article aside), but why is VSR Hybrid only benchmarked on Humanity’s Last Exam and not the other two benchmarks (LiveCodeBench and GPQA-Diamond)? Is this an oversight or are the results too terrible to show?
> The phrase "frontier model" is starting to mean two things. One is a checkpoint. The other is a system boundary.
LLM-isms aside, I don't think we want this to be the case? An LLM, for all its complexity, is something that can be reasoned about. It's picking the next token, until it hits an EOS. The semantics imposed on those tokens (reasoning ,tool call, etc.) are up to the user('s harness) to decide and act on. The more that's pushed behind the facade, the harder it is achieve sufficient understanding of the model's behavior s.t. one can compose it into larger abstractions. Perhaps the performance (and the adherence to an interface/contract) compensate? But swapping from Opus or 5.5 to this or Fugu seems like a much bigger change than swapping between different 'base' models.
I might be wrong, but strongly suspect that Fable 5 is already something in this shape, considering long time to first token while having normal troughput.
No, that was because another Mythos 5 instance had to ACK the response before it was sent to the user.
They're applying misdirection so that we use their secret-sauce agentic framework, but like a black box and without seeing any of the internal reasoning patterns, cause that would give it away.
That's a deal-breaker for me. I need as much observability and control over my development workflow as possible; that's part of my secret sauce.
This seems to be a new trend. Noticed it with GPT "ultra" in their announcement[1]. I'm with you, a large language model and a system of many language models working together are not the same thing
[1] https://news.ycombinator.com/item?id=48689338
I thought all model providers are doing this under the hood anyway in their UI?
They certainly seem to when A/B testing different models, and Fable routes to Opus 4.8 when guardrails fail.
Also, openrouter recently released a fusion router - https://openrouter.ai/blog/announcements/fusion-beats-fronti...
A sign of system-level optimization starting to overshadow raw/brute-force scaling of foundational models. My view is that foundational models are indeed statistic parrots, just like humans (humans are worse parrots, but human brain's context window is so small that they often do not recognize how broken was human-intermediated intelligence swarm, but such small context window might be a fundamental feature of so-called intelligence).
LLMs to me are better intelligence than humans in 3 aspects: 1. LLMs can somehow entirely do perspective taking, humans cannot even think self in next 10 minutes after making a decision 2. LLMs can somehow be asked to arbitrarily elevate and lower abstraction level (can be seen as a special form of perspective taking) 3. LLMs "think" instantly
All these innate capabilities should be combined with system level optimization to achieve the last 10% to be beyond human intelligence.
> 2. LLMs can somehow be asked to arbitrarily elevate and lower abstraction level (can be seen as a special form of perspective taking)
yes but from my experience abstracting (at least upward) is something all models really struggle with.
I would argue that the best models are quite away from human intelligence, let alone 10%.
I guess it's a band of written abstract knowledge embedded in LLMs. Beyond that LLMs certainly falls hard than humans.
But in the band of LLMs, human cannot match
[dead]
Solutions like these are really cementing the view that LLMs are becoming a commodity
This sounds like adding way too much complexity for something that will likely be covered fully by the next gen of frontier models within a single prompt. It also makes it all opaque and difficult to trace.
The next generation of models are currently being withheld from general release. Beyond that, there's still a lot of room to compete on price and also independence from the US labs.
Every one has been saying it’s all about the harness. This is an obvious result of that.
I think an optimal solution would be to have more seamless integration between harness and router roles. As each are only half the picture
the rename from checkpoint to system boundary is how you lose traceability. a multi-model black box that benchmarks well is great until it fails on your workload and theres no trace to debug why.
This should help with better utilizing a heterogenous collection of inference hardware.
sakana fugu landed sooo loudly ... I canceled my test subscription in two days.
Can we please stop submitting fully AI-generated text to HN?
at least 50% of the front page would disappear if this were enforced
Don’t threaten me with a good time
I'd be perfectly okay with that.
So be it.
Looks nice (slop article aside), but why is VSR Hybrid only benchmarked on Humanity’s Last Exam and not the other two benchmarks (LiveCodeBench and GPQA-Diamond)? Is this an oversight or are the results too terrible to show?
[flagged]