Peak performance is the speed at which a CPU can execute a sequence of instructions. The problem is that on the CPU architectures ubiquitous since the 90's, characterised by a hierarchy of memory with different speeds, transfer from memory to the processor registers is the key limiting factor: most algorithms test the speed of data transfer from memory to the CPU as much as the CPU speed itself. Some orders of magnitudes: 1 cycle to add or multiply two floats stored in register; about 4 cycles to load from L1 cache to register; about 10 cycles from L2 cache; from 40 to a few hundred cycles from L3 cache. Thus the most efficient operations are those such that once a chunk of data has been moved upward through this memory hierarchy, as many floating point operations are performed on it that do not require further data motion.
It turns out that matrix multiplication is well suited here. Consider the multiplication of two square matrices:
$$ C_{ij} = \sum_{p=1}^n A_{ip} B_{pj}. $$
This requires to move $O(n^2)$ matrix elements from memory to the CPU and it will then take $O(n^3)$ flops. Thus the cost of moving each matrix element is amortised over $O(n)$ flops. This means that the speed at which the processor can execute operations is really tested here. Now compare with a matrix-vector product:
$$ y_i = \sum_{p=1}^n A_{ij} x_j. $$
This requires to move $O(n^2)$ elements to the registers, as for matrix multiplication, but now it will take only $O(n^2)$ flops. That means that for each flop there is a memory move. Thus this hardly exercises the speed at which the CPU can operate as it tests the speed of the memory transfers on an equal footing.
GEMM exercises, one helluva lot, several essential features of any computer system: (i.) memory access, since so much fetching of data has to occur for large matrices; (ii.) ALU performance, since there is so much arithmetic to be done, viz. $\sum a[i] * b[i]$ for arrays $a[i], b[i]$; (iii.) control flow , since so much looping and/or (possibly recursive) subroutine calling must generally be done. So GEMM provides a nice mix of at least these three areas of functionality. And since the tests make the most sense for large matrices, it is relatively easy to generate good statistics from running GEMM-like codes.
Edit: Material added in response to oasisweng's comment: The following is too long for even two comments, so I've added it here: it's not just that a large number of operations are required, it is also very important that the codes exercise different functional units such as memory, ALUs, controllers, preferably in a manner which forces them to interact. Only then can we find out if they do so efficiently, don't stall one another, etc. If the matrices are small, then you will exercise things differently than if you have large matrices, so the tests have to be run with a variety of input data. Obviously, if all you have is a small number of small matrices, the test(s) won't tell you very much. But a large number of smallish matrices or a smallish number of large matrices will exercise the machine in different ways, each of which will be telling. We are implicitly assuming that the mix of initial data is both large and heterogeneous enough to significantly exercise the machine. There's no fixed lower bound for "big enough", but as machines get more powerful the size of the data sets tend to grow. In conclusion, there's very little which is absolutely fixed, so you have to design test suites which are sufficiently varied. Finally, there are a couple of books on computer architecture by Patterson and Hennessy which address these kinds of questions very well; google around and you'll find them. End of Edit.
My take on it anyway.
Hope this helps. Cheerio,
and as always,
Fiat Lux!
