That's what the license terms say for normal retail licenses of the consumer versions, yes. This is pretty understandable for machines run concurrently. If you have two VMs running 24/7 it makes sense to just consider them "machines" exactly as if they were physical.
The question is what happens if you have two VMs with identical hardware where you only ever have one active? Microsofts licensing probably doesn't consider this scenario, so I think it would be a matter of interpretation. In theory you could transfer the license back and forth between the VMs (assuming that stopping and unmounting the inactive VM disk counts as "removing the software" from that machine).
That's technically correct, but they could still shout at you when carrying out a review, because you cannot technically guarantee you're running only one at the time.
Is it really the customer's responsibility to technically guarantee that the licensing cannot be breached?
I can't technically guarantee that my system hasn't been duplicated (say, by the NSA), but that isn't my responsibility. I'm just not supposed to run two copies simultaneously.
The question is what happens if you have two VMs with identical hardware where you only ever have one active? Microsofts licensing probably doesn't consider this scenario, so I think it would be a matter of interpretation. In theory you could transfer the license back and forth between the VMs (assuming that stopping and unmounting the inactive VM disk counts as "removing the software" from that machine).