Foundational computational linguistics concepts, such as parsing, tokenization, and language models, from an intro linguistics course.
30 cards · basic cards · AI-written, checked twice. Edit anything.
- What is tokenization?
- Breaking text into individual tokens (words, punctuation).
- Define parsing.
- Analyzing the grammatical structure of text.
- What is a parse tree?
- A tree diagram showing the hierarchical syntactic structure of a sentence.
- What does POS tagging stand for?
- Part-of-speech tagging; assigning grammatical categories to words.
- What is lemmatization?
- Reducing words to their dictionary form (e.g., 'running' becomes 'run').
- Explain stemming.
- Removing affixes heuristically to reduce words to root form.
- What is an n-gram?
- A sequence of n contiguous tokens from text.
- Distinguish lexical ambiguity from syntactic ambiguity.
- Lexical: a word has multiple meanings. Syntactic: a sentence has multiple grammatical parses.
- What is a language model?
- A probabilistic model that assigns probabilities to sequences of words.
- Define corpus in linguistics.
- A large, structured collection of text used for analysis.
- What does NLP stand for?
- Natural Language Processing.
- What is named entity recognition?
- Identifying and classifying entities like persons, organizations, and locations in text.
- What are word embeddings?
- Vector representations of words that capture semantic relationships.
- What is sentiment analysis?
- Automatically determining the emotional tone or opinion expressed in text.
- What is a dependency parse?
- Syntactic structure where words are linked by labeled directional arcs showing grammatical relations.
FlashKeepers