Java, not the jvm may be the issue here. Look at the X10 language for example, which focuses more on array, parallel and numerical performance, but is on the jvm. The issue with Java for voxels is that Java does not have fortran arrays. It because nested objects, so that will give a lot of performance hits for a game built entirely on ndimensional arrays.
That's pretty true for stuff like that you want to use sideways arrays typically, where you decompose the fields in your object to each be in their own array (and sometimes recursively if the fields are themselves objects). With that and avoiding allocation (aka cache invalidation / garbage generation) at all costs you can get things fast.
The java ecosystem offers annotation driven bytecode mangling/generation tools for all kinds of stuff, is there something for "sideways arrays"? The compiler plugin model used by immutables.org (generate a bunch of implementations for the fields implied by an interface) could hopefully make that relatively painless to work with.
