best answer is probably to have a layered approach - use this to limit what the generated code can do, wrap it in a secure VM to prevent leaking out to other tenants.
there’s no way around VMs for secure, untrusted workloads. everything else, like Monty has too many tradeoffs that makes it non-viable for any real workloads
As discussed on twitter, v8 shows that's not true.
But to be clear, we're not even targeting the same "computer use" use case I think e2b, daytona, cloudflare, modal, fly.io, deno, google, aws are going after - we're aiming to support programmatic tool calling with minimal latency and complexity - it's a fundamentally different offering.
There's been a constant stream of v8 VM sandbox escape discoveries since its dawn of course. Considering those have mostly existed for a long time before publication it's very porous most of the time.
Then there's of course hypervisor based virtualization and the vulnerabilities and VM escapes there.
Browsers use belt-and-suspenders approaches of employing both language runtime VMs and hardware memory protection as layers to some effect, but still are the star act at pwn2own etc.
It's all layers of porous defenses. There'd definitely be room in the world for performant dynamic language implementations with provably secure foundations.
part of why rexec is "historical" is that Guido was looking at some lockdown work and asked (twitter, probably?) the community to come up with attack ideas (on a specific more-locked-down-than-default proposed version.) After a couple of hours, it was clear that "patching the problems" was entirely doomed given how flexible python is and it was better to do something else entirely and stop pretending...
V8 itself is intended to be heavily sandboxed. Not through a microvm, but otherwise it's probably the most heavily sandboxed piece of code ever ie: in Chrome it can make virtually no system calls and runs with every restriction an OS can possibly provide and moreand seccomp-bpf was basically invented for it.
Perhaps you're using v8 isolates, which then you're back into the "heavily restricted environment within the process" and you lose the things you'd want your AI to be able to do, and even then you still have to sandbox the hell out of it to be safe and you have to seriously consider side channel leaks.
And even after all of that you'd better hope you're staying up to date with patches.
MicroVMs are going to just be way simpler IMO. I don't really get the appeal of using V8 for this unless you have platform/ deployment limitations. Talking over Firecracker's vsock is extremely fast. Firecracker is also insanely safe - 3 CVEs ever, and IMO none are exploitable.
> Bubblewrap and Docker are not hardened security isolation mechanisms, but that's okay with me.
Edit to add: my understanding is the major flaw in this approach is potential bugs in Linux kernel that would allow sandbox escape. Would appreciate your insight if there are some easier/more probable attack vectors.
I'm not saying sandboxes are not needed, I'm saying VMs/containers already provide the core tech and it's easy to DIY a sandbox. Would love to understand what value E2B offers over VMs?
That's right. But they (E2B) rely on the underneath Cloud infra to achieve high scalability. Personally, I'm still not sure about the value they add on top of Cloud hosted VMs. GCP/AWS already offer huge discounts to startups, which should be enough for VM-based sandboxing of agents in the MVP phase.
This is exactly what i am building for a friend in a semi amateur fashion with LLMs. Looking at your codebase I would probably end up with something very similar in 6 months. You even have an Air toml and use firecracker, not to mention using go. Great minds think alike I suppose :D. Mine is not for AI but for running unvetted data science scripts. Simple stuff mostly. I am using rootless podman (I think you are using docker? or perhaps packer which is a tool i didn't know about until now.) to create the microvm images and the images have no network access. We're creating a .ext4 disk image to bring in the data/script.
I think I might just "take" this if the resource requirements are not too demanding. Thanks for sharing. Do you have docs for deploying on bare metal?
Not sure what your customers look like, but I'd for one also be fine with "fair source" licenses (there are several - fair source, fair code, Defold license, etc.)
These give customers 100% control but keep Amazon, Google, and other cling-on folks like WP Engine from reselling your work. It avoids the Docker, Elasticsearch, Redis fate.
"OSI" is a submarine from big tech hyperscalers that mostly take. We should have gone full Stallman, but fair source is a push back against big tech.
when we were starting out we figured there was no solution that would satisfy our requirements for running untrusted code. so we had to build our own.
the reason we open-sourced this is because we want everyone to be able to run our Sandboxes - in contrast to the majority of our competitors who’s goal is to lock you in to their offering.
with open-source you have the choice, and luckily Manus, Perplexity, Nvidia choose us for their workloads.
everything that you don’t want your agent to access should live outside of the sandbox.