Foundational computational linguistics concepts, such as parsing, tokenization, and language models, from an intro linguistics course.
30 cards · basic cards · AI-written, checked twice. Edit anything.
- What is tokenization?
Show answer
Breaking text into individual tokens (words, punctuation).- Define parsing.
Show answer
Analyzing the grammatical structure of text.- What is a parse tree?
Show answer
A tree diagram showing the hierarchical syntactic structure of a sentence.- What does POS tagging stand for?
Show answer
Part-of-speech tagging; assigning grammatical categories to words.- What is lemmatization?
Show answer
Reducing words to their dictionary form (e.g., 'running' becomes 'run').- Explain stemming.
Show answer
Removing affixes heuristically to reduce words to root form.- What is an n-gram?
Show answer
A sequence of n contiguous tokens from text.- Distinguish lexical ambiguity from syntactic ambiguity.
Show answer
Lexical: a word has multiple meanings. Syntactic: a sentence has multiple grammatical parses.- What is a language model?
Show answer
A probabilistic model that assigns probabilities to sequences of words.- Define corpus in linguistics.
Show answer
A large, structured collection of text used for analysis.- What does NLP stand for?
Show answer
Natural Language Processing.- What is named entity recognition?
Show answer
Identifying and classifying entities like persons, organizations, and locations in text.- What are word embeddings?
Show answer
Vector representations of words that capture semantic relationships.- What is sentiment analysis?
Show answer
Automatically determining the emotional tone or opinion expressed in text.- What is a dependency parse?
Show answer
Syntactic structure where words are linked by labeled directional arcs showing grammatical relations.
FlashKeepers