Maximum Likelihood and Gradients

Partial Derivatives

Let \(f:\mathbb{R}^p \rightarrow \mathbb{R}\) be a function that accepts \(p\) inputs. For example, if \(f:\mathbb{R}^3\rightarrow \mathbb{R}\), then \(f\) accepts three numerical inputs and produces one numerical output. We can write a function call for \(f\) as \(f(x, y, z)\), where \(x, y, z \in \mathbb{R}\) are variable names for the inputs.

The partial derivative of \(f\) with respect to one of its inputs (say, \(x\)) is the derivative of \(f\) when we treat all other inputs as constants. We write the partial derivative of \(f\) with respect to \(x\) as \(\frac{\partial f}{\partial x}\). For example, if \(f(x, y, z) = x^4y + yz\), then the partial derivative of \(f\) with respect to \(x\) is obtained by treating \(y\) and \(z\) as constants and then applying standard differentiation rules. In this case, the result is

\[ \frac{\partial f}{\partial x} = 4x^3y. \]

The gradient of a multivariate function is a vector that contains all of its partial derivatives. If \(f:\mathbb{R}^p \rightarrow \mathbb{R}\), then the gradient of \(f\) is a \(p\)-dimensional vector defined as

\[ \nabla f(x_1, x_2, \ldots, x_p) = \begin{pmatrix} \frac{\partial f}{\partial x_1} \\ \frac{\partial f}{\partial x_2} \\ \vdots \\ \frac{\partial f}{\partial x_p} \end{pmatrix}. \]

In the case of the \(f\) above, for example, the gradient is

\[ \nabla f(x, y, z) = \begin{pmatrix} \frac{\partial f}{\partial x} \\ \frac{\partial f}{\partial y} \\ \frac{\partial f}{\partial z} \end{pmatrix} = \begin{pmatrix} 4x^3y \\ x^4 + z \\ y \end{pmatrix}. \]

Please practice computing partial derivatives and gradients in the warmup problem below.

Warmup Problem

Let \(f:\mathbb{R}^3 \rightarrow \mathbb{R}\) be defined by \(f(x, y, z) = x^2 e^{2y} + y\log (xz)\). You can assume that the the logarithm is the natural logarithm (also sometimes written \(\ln\)) and that \(x, y, z > 0\).

Part A

Compute the partial derivatives \(\frac{\partial f}{\partial x}\), \(\frac{\partial f}{\partial y}\), and \(\frac{\partial f}{\partial z}\).

Part B

Write down the gradient \(\nabla f(x, y, z)\).