Let $X$ be an $m \times n$ matrix, $v$ be $n \times 1$, and $y$ be $m \times 1$. - $(X^T)^T = X$ - $X^T X$ is symmetric, i.e., $(X^TX)^T = X^T X$ - Let $y = X v$. The derivative of $y$ with respect to $v$ is $X$ $\displaystyle \frac{\partial y}{\partial v} = X$ and the derivative of $y$ transposed with respect to $v$ is $X$ transposed $\displaystyle \frac{\partial y^T}{\partial v} = X^T$. - Let $c = v^T(X^TX)v$. Then $\displaystyle \frac{\partial c}{\partial v} = 2X^TXv$.