Depends on what abstraction level we are discussing.
Existing applications without code changes in mainstream languages with parallelism-restrictive semantics, I agree we're probably close to the limits.
Beyond that we know that most apps are amenable to human perf work and reformulation to get very big speedups.
Besides human reformulation expressed with only low level parallelism primitives like SIMD instrinsics, the field has been using parallelism geared languages like Cuda, Futhark, ISPC etc. And there's a lot of untapped potential in data representation flexibility etc that even those languages aren't tapping, like for example Halide can do.
Human perf work also involves a lot of trial and error, it's a search type process, automation of which hasn't been explored that much. Some work to automating this are approaches like Atlas BLAS.
Existing applications without code changes in mainstream languages with parallelism-restrictive semantics, I agree we're probably close to the limits.
Beyond that we know that most apps are amenable to human perf work and reformulation to get very big speedups.
Besides human reformulation expressed with only low level parallelism primitives like SIMD instrinsics, the field has been using parallelism geared languages like Cuda, Futhark, ISPC etc. And there's a lot of untapped potential in data representation flexibility etc that even those languages aren't tapping, like for example Halide can do.
Human perf work also involves a lot of trial and error, it's a search type process, automation of which hasn't been explored that much. Some work to automating this are approaches like Atlas BLAS.