And it's only using AVX-2 and not AVX-512, AMX or ACE. Or built-in GPUs and NPUs (the M series doesn't emphasize matrix multiplication on the CPU side because it already has matrix multiplication units on the GPU, which is always attached).
No, but if you are guaranteed to have a GPU or NPU packaged together it becomes like the SPUs in the Cell processor - you can’t offload instructions (unless you use a trap mechanism to a subroutine) but you can have code that hides the setup, the different ISA, and the result retrieval, behind an API call.
Before the M5, there was no dedicated matrix multiplication hardware on the Apple Silicon GPU. Their solution was generally using the NPU and AMX coprocessors for tensor and matrix workloads.
