I don't think I'm ready to trust very security sensitive functions to pure vibe-coded software, and that's what this seems to be? Certainly the README is authored by an LLM, and there's a gazillion empty commits and other weirdness that indicates no human is in the loop. It looks like a loop engineered this software.
Models have gotten good, but c'mon. Good idea, maybe even a good implementation, but I don't have confidence in it, and you've got to have confidence in a project that claims to provide security.
Also, even the best models still regularly write C security bugs. It doesn't make sense to have a model write C code when having it write in a memory safe language is only slightly more effort/cost.
How you type is a poor proxy for code quality. Code quality is a good proxy for code quality. Inspect the code, build a verification pipeline for it, use agents to explore the code and the architecture, see if you can unearth anything fowl.
I'm not judging based on how they type. I can't see how they type, they vibed the README.
And, it's not my monkey. You can inspect the code, build a verification pipeline for it, use agents to explore the architecture and see if you can unearth anything fowl.
My heuristic is to dismiss purely vibe-coded apps from people I don't know, particularly for security sensitive stuff. If the README is written by a human and is coherent and exhibits some kind of desire and competence to make good software on the part of the author, I'm more likely to trust they drove their agents with care.
Here's the thing: you can make good software with agents, if you exhibit good judgement and put yourself in the path as a gate on quality. Too many clues point at this being loop engineering. And, C for this task, given 100% agent authorship, gives me the ick. Seems like bad judgement or opting out of making judgement calls.
Took a look at the readme and seems coherent enough. A readme is also a technical entrypoint, no problem in parts of it being generated, specially if you have quick start, tables and loose documentation there you need updating.
How big a video file is a poor proxy for the encode, quality is a good proxy. The problem is finding the actual quality of a video file is a hell of a lot more work and resources than using a proxy to see if doing so is a good use of our resources. See if you can go the extra mile you described for a few hours/dollars tonight and let us know what you find, it would be appreciated!
Oh, I'll just have to stick with what info SwellJoe provided using the proxy on this project for now then as it seems none of us have the time or resources to go much further :/. Thanks!
You are welcome! My point stands though. Evidence of agentic loop is a bad proxy for code quality. If you are interested in a project you need to do the legwork, vibecoded or not.
Sure - that's always valid - but now I know I'm no longer interested before having put the legwork in! A cool security project that has been reviewed multiple ways already is one thing, a C project nobody else (even the author) was very involved with is another. I don't need to put the legwork in on this codebase to know it's not worth putting the manual legwork in for because I can already get such unverified things out of AI the same as the author, so i'd just review my projects the same as you'd review yours. That's what the proxy provided, not an override of an actual review.
That's the spirit. I was thinking about this that you are saying in general, and agents now make for a case that it's often better for you to rollout your own specialized solution than to adopt a more generic project, so it's really important to know when not to waste time with a project like this.
I am currently doing DevOps work describing a legacy infrastructure as code and instead of buying into a paid or opensource tool, it's nicer for me to roll out my own because even though the task isn't trivial, I can custom tailor the solution exactly to how I'm going to use it to conduct the project.
I can't wait until the really good auditing tools are so cheap we can just run them on everything we see! It's justifiable for work projects but not "fun" yet for me :/
I got lucky I think, in which the workflow of using agents is quite fun and addictive for me. But to audit anything you need a basic level of understanding of the underlying architecture, because we aren't there yet to just leave the agent working unattended and have plausible results, so any audit requires careful human involvement.
It's just that... Take this tiny project of mine I did for the fun of it https://gitlab.com/gabriel.chamon/thoracic-atlas-viewer. Basically went to an online interactive thoracic atlas and saw that the files were 404ing for download, but the visualization worked, so I gathered evidence and asked the agent to rebuild the archive for me. I still need to hack together an offline viewer, but it's amazing how much you can do just by having a general idea of what you need to do.
It's not "how you type", it's "whether any human so much as laid eyes on the code". I wouldn't automatically discard code from an LLM, but let's put the goalposts where they actually are.
Have you look at the commits? A bunch of empty ones (seemingly all empty ones, though there must be some real ones in there somewhere). It's a bizarre looking repo. I don't even know how you make a repo look like that, but it certainly doesn't allow tracing of what code was added when and why.
It looks suspicious, even if it isn't. All those empty commits look like an intentional obfuscation of something, though I have no idea what.
It's indeed strange, it's likely some agent running amok. Doesn't look good indeed, but if I was interested in the project I'd still investigate the code itself
It says on their Github profile that they are building some kind of nowhere detection product. Maybe in that context, a very strict syscall allowlist is useful or good?
> It is designed for CI pipelines, CTF jail challenges, and lightweight code evaluation
Looking at the list, it seems pretty good for that. What does a CI runner that just needs to run GCC or whatever really need?
Edit: no open does seem restrictive. Not that it's bad security (not my area of expertise), but how many useful programs use open that are just off limits here?
Who the F* runs a minimizer on friggin C sources? And it's inconsistent too.
Security-related code should be readable and auditable.
> inconsistent
I have a strong feeling that this person just likes this as a style and enforces it by hand…
I don't think I'm ready to trust very security sensitive functions to pure vibe-coded software, and that's what this seems to be? Certainly the README is authored by an LLM, and there's a gazillion empty commits and other weirdness that indicates no human is in the loop. It looks like a loop engineered this software.
Models have gotten good, but c'mon. Good idea, maybe even a good implementation, but I don't have confidence in it, and you've got to have confidence in a project that claims to provide security.
Also, even the best models still regularly write C security bugs. It doesn't make sense to have a model write C code when having it write in a memory safe language is only slightly more effort/cost.
How you type is a poor proxy for code quality. Code quality is a good proxy for code quality. Inspect the code, build a verification pipeline for it, use agents to explore the code and the architecture, see if you can unearth anything fowl.
I'm not judging based on how they type. I can't see how they type, they vibed the README.
And, it's not my monkey. You can inspect the code, build a verification pipeline for it, use agents to explore the architecture and see if you can unearth anything fowl.
My heuristic is to dismiss purely vibe-coded apps from people I don't know, particularly for security sensitive stuff. If the README is written by a human and is coherent and exhibits some kind of desire and competence to make good software on the part of the author, I'm more likely to trust they drove their agents with care.
Here's the thing: you can make good software with agents, if you exhibit good judgement and put yourself in the path as a gate on quality. Too many clues point at this being loop engineering. And, C for this task, given 100% agent authorship, gives me the ick. Seems like bad judgement or opting out of making judgement calls.
Took a look at the readme and seems coherent enough. A readme is also a technical entrypoint, no problem in parts of it being generated, specially if you have quick start, tables and loose documentation there you need updating.
How big a video file is a poor proxy for the encode, quality is a good proxy. The problem is finding the actual quality of a video file is a hell of a lot more work and resources than using a proxy to see if doing so is a good use of our resources. See if you can go the extra mile you described for a few hours/dollars tonight and let us know what you find, it would be appreciated!
I do for projects I have interest in running and for my own projects. For instance I had agro with https://github.com/Mord3rca/gamma-launcher so I pulled my trusted agent, hammered the code with architectural and static analysis and made it my own here https://gitlab.com/gabriel.chamon/yagi
Oh, I'll just have to stick with what info SwellJoe provided using the proxy on this project for now then as it seems none of us have the time or resources to go much further :/. Thanks!
You are welcome! My point stands though. Evidence of agentic loop is a bad proxy for code quality. If you are interested in a project you need to do the legwork, vibecoded or not.
Sure - that's always valid - but now I know I'm no longer interested before having put the legwork in! A cool security project that has been reviewed multiple ways already is one thing, a C project nobody else (even the author) was very involved with is another. I don't need to put the legwork in on this codebase to know it's not worth putting the manual legwork in for because I can already get such unverified things out of AI the same as the author, so i'd just review my projects the same as you'd review yours. That's what the proxy provided, not an override of an actual review.
That's the spirit. I was thinking about this that you are saying in general, and agents now make for a case that it's often better for you to rollout your own specialized solution than to adopt a more generic project, so it's really important to know when not to waste time with a project like this.
I am currently doing DevOps work describing a legacy infrastructure as code and instead of buying into a paid or opensource tool, it's nicer for me to roll out my own because even though the task isn't trivial, I can custom tailor the solution exactly to how I'm going to use it to conduct the project.
I can't wait until the really good auditing tools are so cheap we can just run them on everything we see! It's justifiable for work projects but not "fun" yet for me :/
I got lucky I think, in which the workflow of using agents is quite fun and addictive for me. But to audit anything you need a basic level of understanding of the underlying architecture, because we aren't there yet to just leave the agent working unattended and have plausible results, so any audit requires careful human involvement.
It's just that... Take this tiny project of mine I did for the fun of it https://gitlab.com/gabriel.chamon/thoracic-atlas-viewer. Basically went to an online interactive thoracic atlas and saw that the files were 404ing for download, but the visualization worked, so I gathered evidence and asked the agent to rebuild the archive for me. I still need to hack together an offline viewer, but it's amazing how much you can do just by having a general idea of what you need to do.
Yeah, it's more about having a better 0 effort proxy than replacing the need for other audits after (from the ones I've messed with at least).
It's not "how you type", it's "whether any human so much as laid eyes on the code". I wouldn't automatically discard code from an LLM, but let's put the goalposts where they actually are.
How do you know nobody laid eyes on the code in the project?
Have you look at the commits? A bunch of empty ones (seemingly all empty ones, though there must be some real ones in there somewhere). It's a bizarre looking repo. I don't even know how you make a repo look like that, but it certainly doesn't allow tracing of what code was added when and why.
It looks suspicious, even if it isn't. All those empty commits look like an intentional obfuscation of something, though I have no idea what.
It's indeed strange, it's likely some agent running amok. Doesn't look good indeed, but if I was interested in the project I'd still investigate the code itself
Setting aside that this seems to be pure slop, what’s with all the empty commits?
The seccomp-BPF rules seem almost unusably strict. What is this even designed to be used to run?
It says on their Github profile that they are building some kind of nowhere detection product. Maybe in that context, a very strict syscall allowlist is useful or good?
> It is designed for CI pipelines, CTF jail challenges, and lightweight code evaluation
Looking at the list, it seems pretty good for that. What does a CI runner that just needs to run GCC or whatever really need?
Edit: no open does seem restrictive. Not that it's bad security (not my area of expertise), but how many useful programs use open that are just off limits here?
allowing individual syscall is the sandbox standard today on BSDs and optin on linux. project have some issues but being too restrictive is not one