Linear and logistic regression, loss functions, kNN, naive Bayes, SVMs and decision boundaries.
39 cards · basic cards · AI-written, checked twice. Edit anything.
- What loss function does linear regression minimize?
Show answer
Mean squared error (MSE), the average of squared differences between predicted and actual values.- Linear regression output: y = ?
Show answer
w0 + w1*x1 + w2*x2 + ... + wn*xn (or w_T * x, a linear combination of features)- Gradient descent does what in each iteration?
Show answer
Updates parameters in the direction that reduces the loss.- What is cross-entropy loss used for?
Show answer
Measuring error in classification problems, especially with probabilities.- Hinge loss is used by which classifier?
Show answer
Support Vector Machines (SVMs).- Logistic regression uses which function to output probabilities?
Show answer
The sigmoid function: 1 / (1 + e^(-z))- What is the output range of sigmoid function?
Show answer
(0, 1), representing probability of the positive class.- kNN predicts by doing what?
Show answer
Taking the majority vote (or mean for regression) of its k nearest neighbors.- What happens to the decision boundary when k increases in kNN?
Show answer
It becomes smoother and less sensitive to individual noisy points.- The core assumption in Naive Bayes is that features are what, given the class?
Show answer
Conditionally independent: each feature contributes independently to the class probability.- Naive Bayes applies which theorem to compute P(class | features)?
Show answer
Bayes theorem: P(C|X) = P(X|C) * P(C) / P(X)- SVMs maximize what between the classes?
Show answer
The margin, the distance between the decision boundary and the nearest training points.- Support vectors in SVM are which training points?
Show answer
The points closest to the decision boundary, which define the margin.- What does the kernel trick do in SVMs?
Show answer
Maps data to a higher-dimensional space for non-linear separation without explicitly computing the coordinates.- RBF (Radial Basis Function) kernel is good for what type of data?
Show answer
Data that requires non-linear decision boundaries.
FlashKeepers