DEV Community

Cover image for Ensemble Techniques
Ruthvik Raja M.V
Ruthvik Raja M.V

Posted on

Ensemble Techniques

There are two types of Ensemble Techniques namely:
BAGGING & BOOSTING

Ensemble methods are used to obtain better predictive performance (or) accuracy than could be obtained from any of the constituent learning algorithms alone.

BAGGING:-
Bagging gets its name because it combines bootstrapping and aggregation to form one ensemble model.
Given a sample of data, multiple bootstrapped subsamples with same size are pulled.
Note : Repetition of samples is also allowed.
A Decision Tree is formed on each of the bootstrapped subsamples.

An algorithm is used to aggregate over the Decision Trees to form the most efficient predictor. Ex: BaggingClassifier Algorithm.
In Bagging we use all the predictor variables(P) for creating different trees so, always the strong independent variable is used for the first split for most of the trees.
when we average them the variance wont be reduced much.
so, in Random forest we use only a subset of predictor variables and each time different trees are formed with different subset of predictor variables.

Generally for Regression: P/3 and for Classification: sqrt(P) predictor variables are chosen.

BOOSTING:-
It is the process of turning a weak learner into a strong learner.
There are three types of Boosting: Ada Boost, Gradient Boost and XG Boost.
Initially using the whole dataset the 1st Base learner is created and trained.
The data for which it is incorrectly classified will be taken and sent to base learner 2 and it is trained again.
Again for the data to which it classified wrongly will be sent to the next base learner.
This process continues...
Until a stopping criteria is met.

Sentry image

Hands-on debugging session: instrument, monitor, and fix

Join Lazar for a hands-on session where you’ll build it, break it, debug it, and fix it. You’ll set up Sentry, track errors, use Session Replay and Tracing, and leverage some good ol’ AI to find and fix issues fast.

RSVP here →

Top comments (0)

Billboard image

The Next Generation Developer Platform

Coherence is the first Platform-as-a-Service you can control. Unlike "black-box" platforms that are opinionated about the infra you can deploy, Coherence is powered by CNC, the open-source IaC framework, which offers limitless customization.

Learn more

👋 Kindness is contagious

Please leave a ❤️ or a friendly comment on this post if you found it helpful!

Okay