Gradients, chain rule, matrix calculus, probability distributions, expectation, Bayes theorem, entropy, KL divergence and maximum likelihood.
41 cards · basic cards · AI-written, checked twice. Edit anything.
- What is the gradient of a function f at a point x?
Show answer
A vector of partial derivatives: grad f = [∂f/∂x1, ∂f/∂x2, ..., ∂f/∂xn]- State the chain rule for composite functions.
Show answer
If y = f(g(x)), then dy/dx = (df/dg) * (dg/dx)- How does the gradient relate to the directional derivative?
Show answer
The directional derivative in direction u is the dot product of the gradient with the unit vector u- What is the Jacobian matrix?
Show answer
A matrix of all first-order partial derivatives of a vector-valued function- Describe the gradient descent algorithm.
Show answer
Iteratively update parameters by subtracting a learning rate times the gradient: x_new = x_old - learning_rate * grad f(x)- Why is the chain rule essential in backpropagation?
Show answer
It computes gradients through multiple layers by chaining partial derivatives from output layer back to input layer- If f(x) = g(h(x)), express ∂f/∂x using the chain rule.
Show answer
∂f/∂x = (∂g/∂h) * (∂h/∂x)- What is a critical point of a function?
Show answer
A point where the gradient is zero, potentially a local minimum, maximum, or saddle point- How do you compute the derivative of a matrix with respect to a scalar?
Show answer
Element-wise: dM/dt is a matrix where each element is the derivative of the corresponding element of M- Define the trace of a square matrix.
Show answer
The sum of the diagonal elements: tr(A) = sum of A_ii- State the cyclic property of the matrix trace.
Show answer
tr(ABC) = tr(CAB) = tr(BCA), cyclic permutations do not change the trace- What is the derivative of x^T A x with respect to x?
Show answer
(A + A^T) x, or 2Ax if A is symmetric- What is the Hessian matrix?
Show answer
The matrix of all second-order partial derivatives of a scalar function: H_ij = ∂²f/∂x_i∂x_j- Define the determinant of a square matrix.
Show answer
A scalar encoding geometric properties of a matrix, including invertibility (det ≠ 0 means invertible)- What is a probability distribution?
Show answer
A function describing the likelihood of different outcomes of a random variable, where all probabilities sum to 1
FlashKeepers