Sure this is true, but this also one of things Georgism seeks fix (land speculation.) A georgist LVT appraises land based on the intrinsic value of the resources it contains and not what is build on it. In fact it seeks to fix “land hoarding” to incentivize productive land. Under a proper LVT it would be really expensive for e.g. one person to own 100 acres of land they aren’t utilizing to it’s true economic potential.