Linear regression assumes that the target variable can be expressed as a linear combination of the independent variables (plus error). If the data does not follow this assumption, the model may not capture the true relationship and produce inaccurate or unstable estimates
XGBoost is a type of gradient boosting algorithm that uses decision trees as base learners. It iteratively adds trees to the ensemble and fits them to correct the prediction errors made by prior models. The values of the model depend on the number of trees, the learning rate, the depth of the trees, the regularization parameters, and the loss function used2. These parameters can be tuned to improve the model performance and reduce overfitting.
SVM regression tries to find a hyperplane that separates the data points with a maximum margin. It uses a kernel function to map the data to a higher-dimensional space where the separation is possible. The values of the model depend on the choice of the kernel function, the penalty parameter, and the epsilon parameter that controls the width of the margin3. These parameters can affect the complexity and generalization ability of the model.
Random forest is another type of ensemble method that uses decision trees as base learners. It creates many trees from random subsets of the data and features, and averages their predictions. The values of the model depend on the number of trees, the maximum depth of the trees, the minimum number of samples required to split a node, and the criterion used to measure the quality of a split4. These parameters can influence the diversity and accuracy of the trees.