Which machine learning algorithm should I start with?
Start with a simple, interpretable baseline, such as logistic regression or a decision tree, then try stronger models such as random forests and gradient boosting. On tabular data, tree-based ensembles are hard to beat: Grinsztajn, Oyallon and Varoquaux (2022) found they remained state-of-the-art on medium-sized tabular datasets of around 10,000 samples, outperforming deep learning methods.
How should I split my data?
Hold out a test set that you use only once, at the end. Within the training data, use k-fold cross-validation to compare models and tune settings. Use stratified folds when classes are imbalanced, and time-ordered splits for time-series data, so the model never learns from the future. To tune and evaluate on the same data honestly, use nested cross-validation.
What is data leakage, and how do I avoid it?
Data leakage happens when information that wouldn't be available at prediction time is used to build the model, which inflates your results. The scikit-learn documentation's rule is simple: never fit anything on the test data. Learn scaling, imputation and feature selection from the training data only, and use a scikit-learn Pipeline so this happens automatically inside cross-validation.
How do I handle imbalanced classes?
First, choose the right metrics: with imbalanced data, accuracy misleads, so report precision, recall, F1 and the precision–recall curve. Then try class weights, or resampling methods such as SMOTE (Chawla et al., 2002). Apply any resampling only to the training folds, inside your pipeline; resampling before splitting leaks information into the test set.
How do I tune hyperparameters?
Search over a defined set of values using cross-validation on the training data only. Random search is often more efficient than an exhaustive grid search, because only a few hyperparameters usually matter (Bergstra and Bengio, 2012). Record the search space and the final settings, so your results can be reproduced.
How do I explain what my model has learned?
Use permutation importance to see which features matter most for performance, and SHAP values (Lundberg and Lee, 2017) to explain individual predictions and overall patterns. Treat explanations as describing the model, not proving cause and effect in the real world, and check that they make sense in your domain.