Skip to main content
Colin Jaffe/3 min read

Understanding Random Forest Classifiers: How They Work

Random Forest Fundamentals

Many Decision Trees

Trains many trees on bootstrapped samples and random feature subsets.

Voting

Each tree votes; majority class wins for classification.

Built-in Feature Importance

Tells you which features mattered most.

Robust to Overfitting

Averaging across trees reduces variance vs single decision tree.

Master Machine Learning at Noble Desktop

Noble Desktop's Python Machine Learning Bootcamp covers scikit-learn, Keras, neural networks, and applied ML.

This lesson is a preview from our Data Science & AI Certificate Online (includes software) and Python Certification Online (includes software & exam). Enroll in a course for detailed lessons, live instructor support, and project-based training.

Explain how random forest classifiers average multiple diverse decision trees for robust prediction. Watch this tutorial to learn the key concepts and techniques.

Let's talk about random forest classifiers. Let's load this image first. These are decision trees.

This is tree 1, tree 2, many other trees, trees 600. Each of these trees takes data and splits it up bit by bit. It says for this piece of data, was it split this way, was it male or female, was it first class or second class, and then it makes a prediction.

For each of these. And it has a different method each time of doing so. A random forest classifier takes this decision tree idea, and as the name would imply, a forest is a collection of trees, and it has many, many trees.

A random forest classifier classifies something such as survival (e.g., survived or not). And it does it by looking at lots of different possibilities, lots of different methods, and averaging them all together. So how is this helpful? How is this a helpful method? Well, it's looking at random subsets of the data.

This means that each tree is diverse. They're looking at lots and lots of different pieces of the data. So there's a lot of diversity of ideas here.

If you can consider this computer model as an idea. And it also has random features, random inputs. For example, one tree might consider age and fare.

Another might consider class and port of embarkation. And so, ultimately, this prevents any single dominant feature, like class (which is probably the most important feature), from being the only factor in the data. Random forest classifiers examine a diverse group of features and combine all of them, rather than relying solely on one feature like class.

So you get a high amount of accuracy because of this robust randomness. And it works with large datasets, small datasets. It also handles outliers really well.

There are certainly some strange outliers in this data. So a random forest classifier is ideal for Titanic data. Now, here are the hyperparameters we will use.

Hyperparameters are not the parameters within the data. They're similar to metadata. They're the parameters of training the model.

Criterion, number of estimators, and random state. We'll use 10 decision trees. There are a couple of different criteria for splitting our data.

Entropy is a good one. And it's the most common one these days, more than the other one, gini impurity. Setting a random state ensures the randomness is reproducible.

These hyperparameters can be tuned later if we decide the model needs improvement. What if we increase the number of trees or change the criteria? Changing the random state should not affect the outcome.

However, these two definitely could. We'll stick with these hyperparameters for now, but tuning them is an important part of working with random forest classifiers and other models.

Alright, we'll start creating a random forest classifier, which of course is not going to be that much code.

And we're going to see how it does.