As old a rust dev. Any interest in the client side of things? I’ve been experimenting with a few ideas in this regard. Especially, true native markdown rendering and a branching conversational data model. (This complicated the UI and required better layout handling.)
Yes — definitely interested in the client side, especially where it intersects with core system design rather than just presentation.
Both native markdown rendering and non-linear / branching conversation models are areas I’ve been thinking about as well, particularly as systems move beyond linear chat and into more exploratory or agent-driven workflows.
Would be curious to hear more about what you experimented with and what tradeoffs you ran into, especially on the data model and UI side.
Cool I sent you an email and included a screen recorded video demo of that project.
Although FYI: the UI in that demo needs some work since updating iOS but it's an interesting proof of concept that took me some time to figure out (in the future I think I'll do full page swipes of the entire downstream conversation tree and not just at the sibling level although this is less power efficient). Also that demo doesn't include my work in markdown rendering since I have yet to integrate the two. I've been working on both of those projects on and off for the past couple years plus some.
> I’m interested in connecting with senior engineers who enjoy building AI systems in Rust, think in terms of orchestration and state, and care deeply about correctness, performance, and privacy.
So something like triton server backends, but using rust rather that python or c++?
Thanks. That’s a fair comparison, but I’m thinking a bit broader than model serving alone.
I’m less interested in building a generic inference backend and more focused on how the system behaves over time — things like agent lifecycles, state, memory, and coordination. Inference is part of it, but not the whole thing.
Correctness, performance, and privacy matter because the system is long-lived and stateful, not just because it’s pushing tokens around.
Sounds ambitious and exciting. How do you intend to provide support to users given that the system will likely evolve and diverge beyond what you can reasonably test for during development? Will you be able to ship software updates without introducing regressions?
I don’t think you can support something like this if it’s a total black box. If behavior drifts, users need some way to see what state the system thinks it’s in and why it did what it did, otherwise support becomes guesswork.
On updates, I’m pretty cautious about changing behavior silently. I’d expect versioned behavior, explicit migrations, and being careful about what auto-updates versus what users opt into.
And if people are paying over time, updates need to be real improvements they can understand, not just maintenance or regressions.
This feels more like a design problem than a testing problem.
I don't see how design plays a role. Imo first and foremost predictability/compatibility/stability isa QA problem. And I would not know where to start even if testing (ie avoiding regressions during development) were properly in place.
For plain old software development, supporting installed base was always far more difficult than designing and building new features. Doable, but hard.
With LLMs it's much harder. For instance, we built a transcript summarization service that customers were happy with. Then we improved the LLM, but we could not just yank the old LLM and replace it with the new version. Some customers would see the improvements, but others would complain that the new tone was ... just different.
I can't imagine how much more complicated it would be if the LLM was not under our control and had evolved to better help the customer.
But maybe I'm just misunderstanding what you're intending to build?
As old a rust dev. Any interest in the client side of things? I’ve been experimenting with a few ideas in this regard. Especially, true native markdown rendering and a branching conversational data model. (This complicated the UI and required better layout handling.)
Yes — definitely interested in the client side, especially where it intersects with core system design rather than just presentation.
Both native markdown rendering and non-linear / branching conversation models are areas I’ve been thinking about as well, particularly as systems move beyond linear chat and into more exploratory or agent-driven workflows.
Would be curious to hear more about what you experimented with and what tradeoffs you ran into, especially on the data model and UI side.
Cool I sent you an email and included a screen recorded video demo of that project.
Although FYI: the UI in that demo needs some work since updating iOS but it's an interesting proof of concept that took me some time to figure out (in the future I think I'll do full page swipes of the entire downstream conversation tree and not just at the sibling level although this is less power efficient). Also that demo doesn't include my work in markdown rendering since I have yet to integrate the two. I've been working on both of those projects on and off for the past couple years plus some.
> I’m interested in connecting with senior engineers who enjoy building AI systems in Rust, think in terms of orchestration and state, and care deeply about correctness, performance, and privacy.
So something like triton server backends, but using rust rather that python or c++?
Thanks. That’s a fair comparison, but I’m thinking a bit broader than model serving alone.
I’m less interested in building a generic inference backend and more focused on how the system behaves over time — things like agent lifecycles, state, memory, and coordination. Inference is part of it, but not the whole thing.
Correctness, performance, and privacy matter because the system is long-lived and stateful, not just because it’s pushing tokens around.
Sounds ambitious and exciting. How do you intend to provide support to users given that the system will likely evolve and diverge beyond what you can reasonably test for during development? Will you be able to ship software updates without introducing regressions?
Good question — this is one of the tricky parts.
I don’t think you can support something like this if it’s a total black box. If behavior drifts, users need some way to see what state the system thinks it’s in and why it did what it did, otherwise support becomes guesswork.
On updates, I’m pretty cautious about changing behavior silently. I’d expect versioned behavior, explicit migrations, and being careful about what auto-updates versus what users opt into.
And if people are paying over time, updates need to be real improvements they can understand, not just maintenance or regressions.
This feels more like a design problem than a testing problem.
I don't see how design plays a role. Imo first and foremost predictability/compatibility/stability isa QA problem. And I would not know where to start even if testing (ie avoiding regressions during development) were properly in place.
For plain old software development, supporting installed base was always far more difficult than designing and building new features. Doable, but hard.
With LLMs it's much harder. For instance, we built a transcript summarization service that customers were happy with. Then we improved the LLM, but we could not just yank the old LLM and replace it with the new version. Some customers would see the improvements, but others would complain that the new tone was ... just different.
I can't imagine how much more complicated it would be if the LLM was not under our control and had evolved to better help the customer.
But maybe I'm just misunderstanding what you're intending to build?