Day 57 – Underfitting, Overfitting, and Regularization

Underfitting

We have created a model, and you tested in training data, and the model provides the worst result.  So the model is too simple and unable to predict the outputs based on the inputs.

How to overcome underfitting?

• Fine-tune the model which suits the dataset as mentioned in “No Free Lunch Theorem.”
• Use feature extraction to feed only relevant features
• Fine-tune regularization hyperparameters based on the model.”

Overfitting

The model shows excellent results on training data; however, it shows poor results when you check in test/validation data.

For example, you went to an online e-commerce portal, checked out a product, and delivered it to your home.   The product damaged when it arrived at your home, and then you say the online e-commerce portal only sells damaged products.  We overgeneralize with one incident and claim for the whole company.  This is called overfitting.

How to overcome overfitting?

• Take only required features or reduce the number of features that are required.
• Have enough training data, and sometimes we consider very little training data.
• Check errors in data or remove any attributes that have no data.
• Fine-tune regularization hyperparameters based on the model.

Regularization

Restrict the model to make it simple and decrease overfitting on the model.

The regularization helps the model not to overgeneralize.

Posts on SAP: