Structured Prediction
In many supervised learning problems, the output isn’t a single label like “spam” or a single number like a house price. You need a whole set of interlocking decisions—where each choice affects the others—like assigning a label to every word in a sentence or finding the best path through a grid.
What “structured prediction” means
Structured prediction is supervised learning where the target y has internal structure: a sequence, tree, graph, or other composite object. The key idea is that the model must predict a globally consistent output, not just independent pieces. Instead of learning “one input → one label,” you learn “one input → a structured object,” and you score whole candidate outputs using a function like score(x, y). Prediction becomes an inference problem: find the best structure (often written as argmax over possible y), which can require dynamic programming or specialized search.
Concrete examples
- Named Entity Recognition: label each token as PERSON/ORG/LOC/O, where neighboring labels are correlated (e.g., “New” and “York” should agree).
- Part-of-speech tagging: predict a tag sequence where grammar makes some transitions more likely than others.
- Information extraction: fill multiple fields in a form where constraints link fields (dates, totals, IDs).
Why it matters in practice
If you ignore structure and predict each part independently, you get outputs that contradict themselves (illegal tag transitions, broken spans, inconsistent fields). Structured prediction builds dependencies into training and decoding, improving coherence and accuracy. Classic tools include Conditional Random Fields (CRFs) for sequences (e.g., CRFsuite) and structured SVMs; in deep learning, models like BiLSTM/Transformer encoders are often paired with a CRF decoding layer to enforce sequence-level consistency.
Structured prediction is a supervised learning setting where the target output is a structured object (e.g., a sequence, tree, or graph) with interdependent components, so the model must predict the whole structure jointly rather than independent labels. It matters because many real tasks—such as sequence labeling in NLP (tagging every word in a sentence) or image segmentation—require enforcing consistency across outputs to achieve accurate, valid predictions.
Imagine filling out a crossword puzzle: you don’t choose each letter in isolation. Every letter has to fit with the others, so the whole grid makes sense. Structured prediction is like that in AI. Instead of predicting one simple answer (like “spam” or “not spam”), the model predicts a connected set of answers that must work together.
For example, when an app turns speech into text, it’s not just guessing each word separately—it’s choosing a whole sentence that sounds natural. Or in photo apps, it might label every pixel to outline a person. The “structure” is the relationships between the pieces of the output.