What is the Bias-Variance Tradeoff?

High-level understanding of finding the sweet spot.

Visual showing targets depicting model accuracy: top-left (ideal, low bias and variance), top-right (high variance), bottom-left (high bias), bottom-right (high bias and variance).

Introduction

Machine learning models are developed to infer predictions, aiding decisions based on data. Every data scientist will at some point face the following question from stakeholders:

How do we create models that are both accurate and reliable?

The answer to this question lies in understanding the bias-variance tradeoff, a concept that sits at the heart of machine learning success — and failure.

What is Bias?

Bias refers to errors introduced in the model due to overly simplistic assumptions (e.g. stating all birds can fly, not factoring in penguins). Should your model suffer from high bias, you’re model is underfitting.

Underfitting insinuates that your model is too simple and struggles to capture the underlying pattern in the data. Models that underfit to he training data leading to poor performance on both training and unseen data.

Note: If your model performs poorly, even on training data, you are likely suffering from a bias problem.

What is Variance?