Perceptrons, activation functions, forward and backpropagation, weight initialization, loss functions and batch normalization.
40 cards · basic cards · AI-written, checked twice. Edit anything.
- What is a perceptron?
Show answer
A linear classifier that outputs 1 or 0 based on a weighted sum of inputs plus a bias, passed through a step activation function.- What does the perceptron update rule do?
Show answer
It adjusts weights by adding the input vector scaled by the learning rate and prediction error.- What is the perceptron's main limitation?
Show answer
It can only solve linearly separable problems and cannot learn the XOR function.- What does the bias term do in a perceptron?
Show answer
It shifts the decision boundary, allowing predictions even when all inputs are zero.- How is a perceptron related to an artificial neuron?
Show answer
A perceptron is the simplest artificial neuron, using a step activation function instead of a smooth one.- What is a layer in a neural network?
Show answer
A collection of neurons that process output from the previous layer and pass results to the next layer.- What are the three main types of layers?
Show answer
Input layer receives raw data, hidden layers process it, and output layer produces the final prediction.- What is a feedforward neural network?
Show answer
A network where data flows strictly from input toward output with no cycles or backward connections.- What is network depth?
Show answer
The number of layers, typically counting hidden layers plus the output layer, excluding the input layer.- What is the purpose of an activation function?
Show answer
It introduces non-linearity into a neuron's output, enabling the network to learn complex patterns.- What is the sigmoid activation function?
Show answer
The function 1 / (1 + exp(-x)), outputting values between 0 and 1.- What is the main weakness of sigmoid activation?
Show answer
It suffers from vanishing gradients because its derivative is at most 0.25, shrinking gradients during backpropagation.- What is the ReLU activation function?
Show answer
The function max(0, x), outputting x if positive and 0 otherwise.- Why is ReLU widely used?
Show answer
It is computationally efficient, avoids vanishing gradients, and often produces faster convergence than sigmoid or tanh.- What is the tanh activation function?
Show answer
The function (exp(x) - exp(-x)) / (exp(x) + exp(-x)), outputting values between -1 and 1.
FlashKeepers