Let \(\boldsymbol A\) and \(\boldsymbol B\) both be matrices of order \(k \times m\). Their sum is defined componentwise: \[\boldsymbol{A} + \boldsymbol{B} =\begin{pmatrix} a_{11}+ b_{11} & a_{12}+ b_{12} & \cdots & a_{1m}+ b_{1m} \\ a_{21}+ b_{21} & a_{22}+ b_{22} & \cdots & a_{2m}+ b_{2m} \\ \vdots & \vdots & & \vdots \\ a_{k1}+ b_{k1} & a_{k2}+ b_{k2} & \cdots & a_{km}+ b_{km} \end{pmatrix}.\] Only two matrices of the same order can be added. Example: \[\boldsymbol{A}=\begin{pmatrix}2&0\\1&5\\3&2\end{pmatrix}\,,\quad \boldsymbol{B}=\begin{pmatrix}-1&1\\7&1\\-5&2\end{pmatrix}\,,\quad \boldsymbol{A}+\boldsymbol{B}=\begin{pmatrix}1&1\\8&6\\-2&4\end{pmatrix}\,.\]
The matrix summation satisfies the following rules: \[\begin{array}{@{}rr@{\ }c@{\ }l@{}r@{}} \text{(i)} & \boldsymbol{A}+\boldsymbol{B} &=& \boldsymbol{B}+\boldsymbol{A}\, & \text{(commutativity)} \\ \text{(ii)} & (\boldsymbol{A}+\boldsymbol{B})+\boldsymbol{C} &=& \boldsymbol{A}+(\boldsymbol{B}+\boldsymbol{C})\, & \text{(associativity)} \\ \text{(iii)} & \boldsymbol A + \boldsymbol 0 &=& \boldsymbol A & {\text{(identity element)}} \\ \text{(iv)} & (\boldsymbol A + \boldsymbol B)' &=& \boldsymbol A' + \boldsymbol B' & {\text{(transposition)}} \end{array}\]
The product of a \(k \times m\) matrix \(\boldsymbol{A}\) with a scalar \(\lambda\in\mathbb{R}\) is defined componentwise: \[\lambda \boldsymbol{A} = \begin{pmatrix} \lambda a_{11} & \lambda a_{12} & \cdots & \lambda a_{1n} \\ \lambda a_{21} & \lambda a_{22} & \cdots & \lambda a_{2n} \\ \vdots & \vdots & & \vdots \\ \lambda a_{m1} & \lambda a_{m2} & \cdots & \lambda a_{mn} \end{pmatrix}.\] Example: \[\lambda=2, \quad \boldsymbol{A}=\begin{pmatrix}2&0\\1&5\\3&2\end{pmatrix},\quad \lambda\boldsymbol{A}=\begin{pmatrix}4&0\\2&10\\6&4\end{pmatrix}.\] Scalar-matrix multiplication satisfies the distributivity law: \[\begin{array}{@{}rr@{\ }c@{\ }l@{}r@{}} \text{(i)} & \lambda(\boldsymbol{A}+\boldsymbol{B})&=& \lambda\boldsymbol{A}+\lambda\boldsymbol{B}\, & \\ \text{(ii)} & (\lambda+\mu)\boldsymbol{A} &=& \lambda\boldsymbol{A}+\mu\boldsymbol{A}\, & \end{array}\]
Basic arithmetic operations work on an element-by-element basis in R:
[,1] [,2]
[1,] 1 1
[2,] 8 6
[3,] -2 4A-B #matrix subtraction [,1] [,2]
[1,] 3 -1
[2,] -6 4
[3,] 8 02*A #scalar-matrix product [,1] [,2]
[1,] 4 0
[2,] 2 10
[3,] 6 4A/2 #division of entries by 2 [,1] [,2]
[1,] 1.0 0.0
[2,] 0.5 2.5
[3,] 1.5 1.0A*B #element-wise multiplication [,1] [,2]
[1,] -2 0
[2,] 7 5
[3,] -15 4The inner product (also known as dot product) of two vectors \(\boldsymbol{a},\boldsymbol{b}\in\mathbb{R}^k\) is \[\boldsymbol{a}'\boldsymbol{b} = a_1 b_1+a_2b_2+\ldots+a_kb_k=\sum_{i=1}^k a_ib_i\in\mathbb{R}.\] Example: \[\boldsymbol{a}=\begin{pmatrix}1\\2\\3\end{pmatrix},\quad \boldsymbol{b}=\begin{pmatrix}-2\\0\\2\end{pmatrix},\quad \boldsymbol{a}'\boldsymbol{b}=1\cdot(-2)+2\cdot0+3\cdot2=4.\]
The inner product is commutative: \[\begin{align*} \boldsymbol a' \boldsymbol b = \boldsymbol b' \boldsymbol a. \end{align*}\] Two vectors \(\boldsymbol a\) and \(\boldsymbol b\) are called orthogonal if \(\boldsymbol a' \boldsymbol b = 0\). The vectors \(\boldsymbol a\) and \(\boldsymbol b\) are called orthonormal if, in addition to \(\boldsymbol a'\boldsymbol b\), we have \(\boldsymbol a' \boldsymbol a = 1\) and \(\boldsymbol b' \boldsymbol b=1\).
The outer product (also known as dyadic product) of two vectors \(\boldsymbol{x} \in \mathbb R^k\) and \(\boldsymbol{y}\in\mathbb{R}^m\) is \[\boldsymbol{x}\boldsymbol{y}' = \left(\begin{matrix} x_1 y_1 & x_1 y_2 &\ldots & x_1 y_m \\ x_2 y_1 & x_2 y_2 & \ldots & x_2 y_m \\ \vdots & \vdots & & \vdots \\ x_k y_1 & x_k y_2 & \ldots & x_k y_m \end{matrix}\right)\in \mathbb{R}^{k \times m}.\] Example: \[\boldsymbol{x}=\begin{pmatrix}1\\2\end{pmatrix}\,,\quad \boldsymbol{y}=\begin{pmatrix}-2\\0\\2\end{pmatrix}\,,\quad \boldsymbol{x}\boldsymbol{y}'=\left(\begin{matrix} -2 & 0 & 2 \\ -4 & 0 & 4 \end{matrix}\right).\]
R
For vector multiplication in R, we use the operator %*% (recall that * is already reserved for element-wise multiplication). Let’s implement some multiplications.
[,1]
[1,] 70 [,1] [,2] [,3] [,4]
[1,] 4 14 8 2
[2,] 14 49 28 7
[3,] 8 28 16 4
[4,] 2 7 4 1 [,1] [,2] [,3]
[1,] -2 0 2
[2,] -4 0 4The matrix product of a \(k \times m\) matrix \(\boldsymbol{A}\) and a \(m \times n\) matrix \(\boldsymbol{B}\) is the \(k\times n\) matrix \(\boldsymbol C = \boldsymbol{A}\boldsymbol{B}\) with the components \[c_{ij} = a_{i1}b_{1j}+a_{i2}b_{2j}+\ldots+a_{im}b_{mj}=\sum_{l=1}^m a_{il}b_{lj} = \boldsymbol a_i' \boldsymbol b_j,\] where \(\boldsymbol a_i = (a_{i1}, \ldots, a_{im})'\) is the \(i\)-th row of \(\boldsymbol A\) written as a column vector, and \(\boldsymbol b_j = (b_{1j}, \ldots, b_{mj})'\) is the \(j\)-th column of \(\boldsymbol B\). The full matrix product can be written as \[ \boldsymbol A \boldsymbol B = \begin{pmatrix} \boldsymbol a_1' \\ \vdots \\ \boldsymbol a_k' \end{pmatrix} \begin{pmatrix} \boldsymbol b_1 & \ldots & \boldsymbol b_n \end{pmatrix} = \begin{pmatrix} \boldsymbol a_1' \boldsymbol b_1 & \ldots & \boldsymbol a_1' \boldsymbol b_n \\ \vdots & & \vdots \\ \boldsymbol a_k' \boldsymbol b_1 & \ldots & \boldsymbol a_k' \boldsymbol b_n \end{pmatrix}. \] The matrix product is only defined if the number of columns of the first matrix equals the number of rows of the second matrix. Therefore, we say that the \(k \times m\) matrix \(\boldsymbol A\) and the \(m \times n\) matrix \(\boldsymbol B\) are conformable for matrix multiplication.
Example: Let \[\begin{aligned} \boldsymbol{A}=\begin{pmatrix} 1 & 0\\ 0 & 1\\ 2 & 1 \end{pmatrix}, \quad \boldsymbol{B}=\begin{pmatrix} -1 & 2\\ -3 & 0 \end{pmatrix}.\end{aligned}\] Their matrix product is \[\begin{aligned} \boldsymbol{A} \boldsymbol{B} &= \begin{pmatrix} 1 & 0\\ 0 & 1\\ 2 & 1 \end{pmatrix} \begin{pmatrix} -1 & 2\\ -3 & 0 \end{pmatrix} \\ &= \left(\begin{matrix}1 \cdot (-1) + 0 \cdot (-3) & 1 \cdot 2 + 0 \cdot 0 \\ 0 \cdot (-1) + 1 \cdot (-3) & 0 \cdot 2 + 1 \cdot 0 \\ 2 \cdot (-1) + 1 \cdot (-3) & 2 \cdot 2 + 1 \cdot 0 \end{matrix}\right) = \left(\begin{matrix}-1 & 2 \\ -3 & 0 \\ -5 & 4 \end{matrix}\right).\end{aligned}\]
The %*% operator is used in R for matrix-matrix multiplications:
[,1] [,2]
[1,] -1 2
[2,] -3 0
[3,] -5 4Matrix multiplication is not commutative. In general, we have \(\boldsymbol A \boldsymbol B \neq \boldsymbol B \boldsymbol A\). Example: \[\begin{aligned} \boldsymbol{A}\boldsymbol{B} = \begin{pmatrix} 1 & 2\\ 3 & 4\end{pmatrix} \begin{pmatrix} 1 & 1\\ 1 & 2\end{pmatrix} &= \begin{pmatrix} 3 & 5\\ 7 & 11\end{pmatrix}\,,\\ \boldsymbol{B}\boldsymbol{A} = \begin{pmatrix} 1 & 1\\ 1 & 2\end{pmatrix} \begin{pmatrix} 1 & 2\\ 3 & 4\end{pmatrix} &= \begin{pmatrix} 4 & 6\\ 7 & 10\end{pmatrix}\,.\end{aligned}\] Even if neither of the two matrices contains zeros, the matrix product can give the zero matrix: \[\boldsymbol{A}\boldsymbol{B} = \begin{pmatrix} 1 & 2\\ 2 & 4\end{pmatrix} \begin{pmatrix} 2 & -4\\ -1 & 2\end{pmatrix} = \begin{pmatrix} 0 & 0\\ 0 & 0\end{pmatrix}=\boldsymbol{0}.\]
The following rules of calculation apply (provided the matrices are conformable): \[\begin{array}{rrcl@{}r@{}} \text{(i)} & \boldsymbol{A}(\boldsymbol{B}\boldsymbol{C}) & = & (\boldsymbol{A}\boldsymbol{B})\boldsymbol{C}\, &\text{(associativity)} \\ \text{(ii)} & \boldsymbol{A}(\boldsymbol{B}+\boldsymbol{D}) & = & \boldsymbol{A}\boldsymbol{B}+\boldsymbol{A}\boldsymbol{D}\, & \text{(distributivity)} \\ \text{(iii)} & (\boldsymbol{B}+\boldsymbol{D})\boldsymbol{C} & = & \boldsymbol{B}\boldsymbol{C}+\boldsymbol{D}\boldsymbol{C}\, & \text{(distributivity)} \\ \text{(iv)} & \boldsymbol{A}(\lambda \boldsymbol{B}) & = & \lambda(\boldsymbol{A}\boldsymbol{B})\, & \text{(scalar commutativity)}\\ \text{(v)} & \boldsymbol{A}\boldsymbol{I}_{n} & = & \boldsymbol{A}\,, & \text{(identity element)}\\ \text{(vi)} & \boldsymbol{I}_{m}\boldsymbol{A} & = & \boldsymbol{A}\, & \text{(identity element)} \\ \text{(vii)} & (\boldsymbol{A}\boldsymbol{B})' & = & \boldsymbol{B}'\boldsymbol{A}'\, & \text{(product transposition)} \\ \text{(viii)} & (\boldsymbol{A}\boldsymbol{B} \boldsymbol C)' & = & \boldsymbol C' \boldsymbol{B}'\boldsymbol{A}'\, & \text{(product transposition)} \end{array}\]