Being successful on Kaggle using mlr

R r-bloggers

Tutorial on how to be successful on Kaggle using mlr

Achieving a good score on a Kaggle competition is typically quite difficult. This blog post outlines 7 tips for beginners to improve their ranking on the Kaggle leaderboards. For this purpose, I also created a Kernel for the Kaggle bike sharing competition that shows how the R package, mlr, can be used to tune a xgboost model with random search in parallel (using 16 cores). The R script scores rank 90 (of 3251) on the Kaggle leaderboard.

7 Rules

  1. Use good software
  2. Understand the objective
  3. Create and select features
  4. Tune your model
  5. Validate your model
  6. Ensemble different models
  7. Track your progress

1. Use good software

Whether you choose R, Python or another language to work on Kaggle, you will most likely need to leverage quite a few packages to follow best practices in machine learning. To save time, you should use ‘software’ that offers a standardized and well-tested interface for the important steps in your workflow:

Examples of ‘software’ that implement the steps above and more:

2. Understand the objective

To develop a good understanding of the Kaggle challenge, you should:

Make sure you choose an approach that directly optimizes the measure of interest! Example:

3. Create and select features:

In many kaggle competitions, finding a “magic feature” can dramatically increase your ranking. Sometimes, better data beats better algorithms! You should therefore try to introduce new features containing valuable information (which can’t be found by the model) or remove noisy features (which can decrease model performance):

4. Tune your model

Typically you can focus on a single model (e.g. xgboost) and tune its hyperparameters for optimal performance.

5. Validate your model

Good machine learning models not only work on the data they were trained on, but also on unseen (test) data that was not used for training the model. When you use training data to make any kind of decision (like feature or model selection, hyperparameter tuning, …), the data becomes less valuable for generalization to unseen data. So if you just use the public leaderboard for testing, you might overfit to the public leaderboard and lose many ranks once the private leaderboard is revealed. A better approach is to use validation to get an estimate of performane on unseen data:

6. Ensemble different models (see, e.g. this guide):

After training many different models, you might want to ensemble them into one strong model using one of these methods:

7. Track your progress

A kaggle project might get quite messy very quickly, because you might try and prototype many different ideas. To avoid getting lost, make sure to keep track of:

If you do not want to use a tool like git, at least make sure you create subfolders for each prototype. This way you can later analyse which models you might want to ensemble or use for your final commits for the competition.

Citation

For attribution, please cite this work as

Casalicchio (2017, March 9). mlr-org: Being successful on Kaggle using mlr. Retrieved from https://mlr-org.github.io/mlr-org-website/posts/2017-03-09-being-successful-on-kaggle-using-mlr/

BibTeX citation

@misc{casalicchio2017being,
  author = {Casalicchio, Giuseppe},
  title = {mlr-org: Being successful on Kaggle using mlr},
  url = {https://mlr-org.github.io/mlr-org-website/posts/2017-03-09-being-successful-on-kaggle-using-mlr/},
  year = {2017}
}