Hello, I'm building a replacement for docker containers with a virtual machine with the ergonomics of containers + subsecond start times.
I worked in AWS previously in the container space + with firecracker. I realized the container is an unnecessary layer that slowed things down + firecracker was a technology designed for AWS org structure + usecase.
So I ended up building a hybrid taking the best of containers with the best of firecracker.
Let me know your thoughts, thanks!
harshdoesdev 2 hours ago [-]
+1. i built something similar called shuru.run because i wanted an easy way to set up microVM sandboxes to run some of my AI apps, and firecracker wasn't available for macOS (and, as you said, it is just too heavy for normal user-level workloads).
sahil-shubham 46 minutes ago [-]
Nice work on Shuru — I remember looking at it when I was researching this space. You went with a Rust wrapper on Apple’s Virtualization framework right?
I believe anyone with a spare linux box should be able to carve it into isolated programmable machines, without having to worry about provisioning them or their lifecycle.
The documentation’s still early but I have been using it for orchestrating parallel work (with deploy previews), offloading browser automation for my agents etc. An auction bought heztner server is serving me quite well :)
harshdoesdev 7 minutes ago [-]
bhatti's cli looks very ergonomic! great job!
also, yes, shuru was (still) a wrapper over the Virtualization.framework, but it now supports Linux too (wrapper over KVM lol)
fqiao 1 hours ago [-]
Yes, having a light-weight solution for local devices as well is one primary goal of the design. Another one is to make it easy for hosting, self or managed
sdrinf 56 minutes ago [-]
hi, great project! Windows support is sorely lacking, though. As someone working a lot with sandboxed LLMs right now, the options-space on windows for sandboxing is _extremely lacking_. Any plans to support it?
fqiao 47 minutes ago [-]
Hey, thanks so much! yah we will definitely add windows support later. We are exploring how to get this work with WSL and will release it asap.
Stay tuned and thanks!
binsquare 49 minutes ago [-]
Yeah, it's in my mind.
WSL2 runs a linux virtual machine. Need to take some time and care to wire that up, but definitely feasible.
thm 1 hours ago [-]
You could add OrbStack to the comp. table
fqiao 1 hours ago [-]
Will do. Thanks for the suggestion!
lambdanodecore 11 minutes ago [-]
Basically any open source project nowadays run their software stack in containers often requiring docker compose. Unfortunatley Smol machines do not support Docker inside the microvms and they also do not support nested VMs for things that use Vagrant. I think this is a big drawback.
binsquare 9 minutes ago [-]
I can support docker - will ship a compatible kernel with the necessary flags in the next release.
lambdanodecore 3 minutes ago [-]
I tried something like this already, also including nested kvm. I think this will increase the boot time quiet a bit.
Also libkrun is not secure by default. From their README.md:
> The libkrun security model is primarily defined by the consideration that both the guest and the VMM pertain to the same security context. For many operations, the VMM acts as a proxy for the guest within the host. Host resources that are accessible to the VMM can potentially be accessed by the guest through it.
> While defining the security implementation of your environment, you should think about the guest and the VMM as a single entity. To prevent the guest from accessing host's resources, you need to use the host's OS security features to run the VMM inside an isolated context. On Linux, the primary mechanism to be used for this purpose is namespaces. Single-user systems may have a more relaxed security policy and just ensure the VMM runs with a particular UID/GID.
> While most virtio devices allow the guest to access resources from the host, two of them require special consideration when used: virtio-fs and virtio-vsock+TSI.
> When exposing a directory in a filesystem from the host to the guest through virtio-fs devices configured with krun_set_root and/or krun_add_virtiofs, libkrun does not provide any protection against the guest attempting to access other directories in the same filesystem, or even other filesystems in the host.
bch 46 minutes ago [-]
see too[0][1] for projects of a similar* vein, incl historical account.
*yes, FreeBSD is specifically developed against Firecracker which is specifically avoided w "Smol machines", but interesting nonetheless
Great job with the comparison table. Immediately I was like “neat sounds like firecracker” then saw your table to see where it was similar and different. Easy!
Nice job! This looks really cool
fqiao 1 hours ago [-]
Thanks so much
fqiao 2 hours ago [-]
Give it a try folks. Would really love to hear all the feedbacks!
Cheers!
leetrout 1 hours ago [-]
why did you seemingly create two HN accounts?
Edit: I see this appears to be a contributor to the project as well. It was not obvious to me.
its a really innovative idea! very interested in the subsecond coldstart claim, how does it achieve that?
fqiao 2 hours ago [-]
@binsquare basically brute-force trimmed down unnecessary linux kernel modules, tried to get the vm started with just bare minimum. There are more rooms for improvement for sure. We will keep trying!
deivid 47 minutes ago [-]
With this approach I managed to get to sub-10ms start (to pid1), if you can accept a few constraints there's plenty of room!
Though my version was only tested on Linux hosts
binsquare 18 minutes ago [-]
would be interested to see how you do it, how can I connect with you - emotionally?
harshdoesdev 1 hours ago [-]
nice! for most local workloads, it is actually sufficient. so, do you ship a complete disk snapshot of the machines?
fqiao 1 hours ago [-]
Yes. files on the disks are kept across stop and restart. We also have a pack command to compress the machine as a single file so that it can shipped and rehydrated elsewhere
volume_tech 1 hours ago [-]
[dead]
Rendered at 19:31:25 GMT+0000 (Coordinated Universal Time) with Vercel.
Probably a lot of other neat usecases for this, too
https://docs.docker.com/reference/cli/sbx/
I worked in AWS previously in the container space + with firecracker. I realized the container is an unnecessary layer that slowed things down + firecracker was a technology designed for AWS org structure + usecase.
So I ended up building a hybrid taking the best of containers with the best of firecracker.
Let me know your thoughts, thanks!
I have been working on something similar but on top of firecracker, called it bhatti (https://github.com/sahil-shubham/bhatti).
I believe anyone with a spare linux box should be able to carve it into isolated programmable machines, without having to worry about provisioning them or their lifecycle.
The documentation’s still early but I have been using it for orchestrating parallel work (with deploy previews), offloading browser automation for my agents etc. An auction bought heztner server is serving me quite well :)
also, yes, shuru was (still) a wrapper over the Virtualization.framework, but it now supports Linux too (wrapper over KVM lol)
WSL2 runs a linux virtual machine. Need to take some time and care to wire that up, but definitely feasible.
Also libkrun is not secure by default. From their README.md:
> The libkrun security model is primarily defined by the consideration that both the guest and the VMM pertain to the same security context. For many operations, the VMM acts as a proxy for the guest within the host. Host resources that are accessible to the VMM can potentially be accessed by the guest through it.
> While defining the security implementation of your environment, you should think about the guest and the VMM as a single entity. To prevent the guest from accessing host's resources, you need to use the host's OS security features to run the VMM inside an isolated context. On Linux, the primary mechanism to be used for this purpose is namespaces. Single-user systems may have a more relaxed security policy and just ensure the VMM runs with a particular UID/GID.
> While most virtio devices allow the guest to access resources from the host, two of them require special consideration when used: virtio-fs and virtio-vsock+TSI.
> When exposing a directory in a filesystem from the host to the guest through virtio-fs devices configured with krun_set_root and/or krun_add_virtiofs, libkrun does not provide any protection against the guest attempting to access other directories in the same filesystem, or even other filesystems in the host.
*yes, FreeBSD is specifically developed against Firecracker which is specifically avoided w "Smol machines", but interesting nonetheless
[0] https://github.com/NetBSDfr/smolBSD
[1] https://www.usenix.org/publications/loginonline/freebsd-fire...
microvm space is still underserved.
Can you pipe into one? It would be cute if I could wget in machine 1 and send that result to offline machine 2 for processing.
Yes! GPU passthrough is being actively worked on and will land in next major release: https://github.com/smol-machines/smolvm/pull/96
Yea just tried piping, it works:
``` smolvm machine exec --name m1 -- wget -qO- https://example.com/data.csv \ | smolvm machine exec --name m2 -i -- python3 process.py ```
Nice job! This looks really cool
Cheers!
Edit: I see this appears to be a contributor to the project as well. It was not obvious to me.
@binsquare is this one: https://github.com/BinSquare
Though my version was only tested on Linux hosts