Working on ML-Zoomcamp Week 06 - Decision Trees
- Decision Trees (DTs) are used as a supervised method to do Regression and Classification.
- DTs are based on the decisions taken based on features splits. For example: Number of Rooms in the house ( 0, 1, 2, >3) and based on this the tree will have decision nodes.
- Each leaf will represent a decision. For example: House is Expensive vs. House is not expensive.
- You can tune the parameters of the tree such as: the depth of the tree, ideally, you don't want to do a full depth, as you will end up overfitting.
- Last but not least, you can you ensemble methods, where basically you can have a number of estimators and the decision will be based on averaging the decision of all estimators.
- Yet another interesting technique is the use of XGBoost as an ensemble algorithm, which is recognized by its performance. It works using bootstrap technique for sampling from the original dataset.