This is a big deal because you can now run Firecracker/other microVMs in an AWS VM instead of expensive AWS bare-metal instances.
GCP has had nested virtualization for a while.
iJohnDoe 15 minutes ago [-]
Was hoping this comment would be here. Firecracker and microVMs is a good use-case. Also, being able to simply test and develop is a nice to have.
Nested virtualization can mean a lot of things. Not just full VMs.
parhamn 27 minutes ago [-]
whats the ~ perf hit of something like this?
otterley 3 minutes ago [-]
As a practical matter, anywhere from 5-15%.
largbae 18 minutes ago [-]
Nowadays nested just wastes the extra operating system overhead and I/O performance if your VM doesn't have paravirtualization drivers installed. CPUs all have hardware support.
sitole 2 hours ago [-]
Support for nested virtualization has been added to the main SDKs. In the us-west-2 region, you can already see the "Nested Virtualization" option and use it with the new M8id, C8id, and R8id instance types.
This is really big news for micro-VM sandbox solutions like E2B, which I work on.
gerdesj 42 minutes ago [-]
Could someone explain why this is might be a big deal?
I remember playing with nested virty some years ago and deciding it is a backwards step except for PoC and the like. Given I haven't personally run out of virty gear, I never needed to do a PoC.
paulfurtado 24 minutes ago [-]
It is great for isolation. There are so many VM based containerization solutions at this point, like Kata Containers, gvisor, and Firecracker. With kata, your kubernetes pods run in isolated VMs. It also opens the door for live migration of apps between ec2 instances, making some kinds of maintenance easier when you have persistent workloads. Even if not for security, there are so many ways a workload can break a machine such that you need to reboot or replace (like detaching an ebs volume with a mounted xfs filesystem at the wrong moment).
The place I've probably wanted it the most though is in CI/CD systems: it's always been annoying to build and test system images in EC2 in a generic way.
It also allows for running other third party appliances unmodified in EC2.
But also, almost every other execution environment offers this: GCP, VMWare, KVM, etc, so it's frustrating that EC2 has only offered it on their bare metal instance types. When ec2 was using xen 10+ years ago, it made sense, but they've been on kvm since the inception of nitro.
UltraSane 22 minutes ago [-]
You can now run VMs inside a cheaper AWS instance instead of having to pay for an entire bare-metal instance. This is useful for things like network simulation where you use QEMU to emulate network hardware.
blibble 1 hours ago [-]
welcome AWS to 2018!
ssl-3 34 minutes ago [-]
Yep. It's pretty boring. I've been using it at home for years and years with libvirt on very not-special consumer hardware. I guess the AWS clown is finally catching up on this one little not-new-at-all thing.
otterley 4 minutes ago [-]
I was an Amazon EC2 Specialist SA in a prior role, so I know a little about this.
If EC2 were like your home server, you might be right. And an EC2 bare metal instance is the closest approximation to that. That option was never disabled and we had some customers who rolled their own nested VM implementations on it.
But EC2 is not like your home server. There are some nontrivial considerations and requirements to offer nested virtualization at cloud scale:
1. Ensuring virtualized networking (VPC) works with nested VMs as well as with the primary VM
2. Making sure the environment (VMM etc) is sufficiently hardened to meet AWS's security standards so that nesting doesn't pose unintended threats or weaken EC2's isolation properties
3. Ensuring performance meets customer standards
4. Building a rock-solid control plane around it all
It's not a trivial matter of flipping a bit.
ATechGuy 45 minutes ago [-]
Would love to see performance numbers with nested virtualization, particularly that of IO-bound workloads.
api 43 minutes ago [-]
What's the performance impact for nested virtualization in general? I'd think this would be adding multiple layers of MMU overhead.
dwattttt 27 minutes ago [-]
From memory, the virtualisation operations themselves aren't nested. The VM instructions interact with the external virtualisation hardware, so it's more of a cooperative situation, e.g. a guest can create & manage virtualisation structures that are run alongside it.
I don't know if this applies to the specific nested virtualisation AWS are providing though.
dangoodmanUT 26 minutes ago [-]
hell yes, finally
57 minutes ago [-]
gchamonlive 50 minutes ago [-]
Highly doubt that
farklenotabot 47 minutes ago [-]
Sounds expensive for legacy apps
bagels 46 minutes ago [-]
"* *Feature*: Launching nested virtualization. This feature allows you to run nested VMs inside virtual (non-bare metal) EC2 instances."
Rendered at 01:43:10 GMT+0000 (Coordinated Universal Time) with Vercel.
GCP has had nested virtualization for a while.
Nested virtualization can mean a lot of things. Not just full VMs.
This is really big news for micro-VM sandbox solutions like E2B, which I work on.
I remember playing with nested virty some years ago and deciding it is a backwards step except for PoC and the like. Given I haven't personally run out of virty gear, I never needed to do a PoC.
The place I've probably wanted it the most though is in CI/CD systems: it's always been annoying to build and test system images in EC2 in a generic way.
It also allows for running other third party appliances unmodified in EC2.
But also, almost every other execution environment offers this: GCP, VMWare, KVM, etc, so it's frustrating that EC2 has only offered it on their bare metal instance types. When ec2 was using xen 10+ years ago, it made sense, but they've been on kvm since the inception of nitro.
If EC2 were like your home server, you might be right. And an EC2 bare metal instance is the closest approximation to that. That option was never disabled and we had some customers who rolled their own nested VM implementations on it.
But EC2 is not like your home server. There are some nontrivial considerations and requirements to offer nested virtualization at cloud scale:
1. Ensuring virtualized networking (VPC) works with nested VMs as well as with the primary VM
2. Making sure the environment (VMM etc) is sufficiently hardened to meet AWS's security standards so that nesting doesn't pose unintended threats or weaken EC2's isolation properties
3. Ensuring performance meets customer standards
4. Building a rock-solid control plane around it all
It's not a trivial matter of flipping a bit.
I don't know if this applies to the specific nested virtualisation AWS are providing though.