Think of two 2x2 matricies. There's no 'intrinsic' orientation either. Depending on your SPECIFIC matrix definitions you INTERPRET a secondary concept of orientation based on you application, but structurally, all that matters is 'associative order' (which matrix is on the left vs right).
Beyond that question, this can be seen as a basic abstraction where all operational structures increase by 1 dimension. So, in 2d, have 1d horizontal slice (row) x 1d verical slice (column) = 0d scalar (element in resulting 2d matrix). Here its 2d horizontal slice (row sheet/plane) x 2d vertical slice (column sheet) = 1d element in resulting 3d matrix).
Similarly, you'd expect absraction to nd via n-1 horizontal slice x n-1 verical slice = n-2 element in resulting nd matrix. Tensorially it'd be like a tensor product followed by a contraction (for tensors, contraction reduces dimension (rank) by 2)
You need to specify what you want the result to be or do; i.e how it is to behave, and/or at least mention how much of the final result should vary when varying each of the points in the multiplicands.
– Mark Hurd Oct 13 '15 at 12:21