What is overfitting in machine learning?
Overfitting in machine learning is when a model fits its training data so closely that it stops working well on any data it has not seen before.
Every model learns from examples called training data. The goal is not to remember those examples. The goal is to find a general pattern that also works on tomorrow's data. An overfit model mixes up the real pattern with noise: random errors, rare outliers and coincidences that only exist in the training set.
A model with a lot of parameters has more room to memorise, which is why overfitting is a common problem in deep learning and large neural networks. The same problem appears in simple models too, such as a decision tree allowed to grow without limits.
The opposite problem, underfitting, is just as real, so the two are almost always taught together.
Overfitting vs underfitting: what is the difference?
Overfitting means the model is too complex and memorises the training data, while underfitting means the model is too simple to capture even the basic pattern.
| Point | Underfitting | Good fit | Overfitting |
|---|---|---|---|
| What happens | Misses the real pattern | Learns the real pattern | Learns the pattern plus noise |
| Training accuracy | Low | High | Very high, often near perfect |
| Validation or test accuracy | Low | High, close to training | Much lower than training |
| Usual cause | Model too simple, too little training | Balanced model and data | Model too complex, too little data, too long training |
| Technical name | High bias | Balanced | High variance |
Think of a graph of points that roughly follow a curve. A straight line through them underfits, because it misses the bend. A smooth curve fits well. A wiggly line that passes exactly through every single point overfits, because those wiggles are chasing noise. That picture is the classic "overfitting and underfitting diagram" you will see in textbooks.
Finding the middle is called the bias and variance trade-off, a favourite question in machine learning interviews.
What is an example of overfitting in real life?
The easiest real life example of overfitting is a student who memorises past exam papers word for word instead of understanding the chapter.
Imagine Riya preparing for a board exam. She mugs up the answers to the last five years of papers. When the same questions appear, she scores full marks. When the examiner twists a question even slightly, she is lost, because she learned the answers, not the concept. Her "training accuracy" is perfect and her "test accuracy" is poor. That is overfitting.
Her friend Arjun only reads the chapter summary and skips practice. He cannot answer old questions or new ones well. That is underfitting.
A student who understands the concept and practises a sensible number of papers does well on both. That student is the well fitted model.
A machine learning version of the same story: a model trained to spot spam emails might learn that every spam email in its training set happened to contain a certain word. It then flags every email with that word, even genuine ones, and misses new spam that does not use it.
How do you know if a model is overfitting?
You know a model is overfitting when its accuracy on training data keeps rising while its accuracy on validation data stalls or starts falling.
To check this, you hold back part of your data that the model never trains on, called the validation set. Then you watch both numbers during training:
- Healthy training: training and validation accuracy rise together and stay close.
- Overfitting: training accuracy reaches 98 or 99 percent while validation accuracy sits much lower, or the validation loss starts climbing after an early dip.
- Underfitting: both numbers stay low, even after many epochs.
Plotting training loss and validation loss on one chart gives a learning curve. The point where validation loss turns upward is usually where overfitting begins. No exact gap counts as "overfit" for every project, so judge the trend and compare with a simple baseline model.
What causes overfitting?
Overfitting is mainly caused by a model that is too complex for the amount and quality of data it is trained on.
The usual causes are:
- Too little training data: with few examples, it is easy to memorise them all.
- A model that is too complex: too many layers, features or parameters for a simple problem.
- Noisy or wrongly labelled data: the model learns the mistakes as if they were rules.
- Training for too long: after a point, extra epochs only help the model memorise.
- Data leakage: test examples sneak into training, so results look better than they really are.
- Unrepresentative data: the training set does not look like the real world the model will face.
How do you prevent overfitting?
You prevent overfitting by giving the model more and better data, limiting how complex it can get, and stopping training at the right time.
- More data or data augmentation: more varied examples make memorising harder. For images, flipping, cropping and rotating photos creates useful new samples.
- Regularisation: techniques like L1 and L2 add a penalty for overly large weights, which pushes the model towards simpler patterns.
- Early stopping: watch the validation loss and stop training when it stops improving.
- Dropout: in neural networks, randomly switching off some neurons during training stops the network from depending too heavily on any single path.
- A simpler model: fewer layers, fewer features or a shallower decision tree.
- Cross-validation: train and test on several different splits of the data so one lucky split cannot fool you.
I always tell students to train a small, boring baseline model first. If your fancy model beats it on training data but not on validation data, you have found overfitting before it reaches users.
The same idea matters when you fine-tune an LLM on a small dataset, where too many epochs can make it repeat your examples instead of generalising.