library(mlr3verse)Loading required package: mlr3
The website features runtime benchmarks of the paradox package now.
Augment a Random Forest with automatic imputation.
Florian Pfisterer
January 31, 2020
This tutorial assumes familiarity with the basics of mlr3pipelines. Consult the mlr3book if some aspects are not fully understandable. It deals with the problem of missing data.
The random forest implementation in the package ranger unfortunately does not support missing values. Therefore, it is required to impute missing features before passing the data to the learner.
We show how to use mlr3pipelines to augment the ranger learner with automatic imputation.
We load the mlr3verse package which pulls in the most important packages for this example.
We initialize the random number generator with a fixed seed for reproducibility, and decrease the verbosity of the logger to keep the output clearly represented.
First, we take an example task with missing values (diabetes) and create the ranger learner:
── <TaskClassif> (768x9): Synthetic Diabetes ───────────────────────────────────────────────────────────────────────────
• Target: diabetes
• Properties: twoclass
• Features (8):
• dbl (8): age, glucose, insulin, mass, pedigree, pregnant, pressure, triceps
• Target classes: pos (positive class, 36%), neg (64%)
── <LearnerClassifRanger> (classif.ranger): Random Forest ──────────────────────────────────────────────────────────────
• Model: -
• Parameters: num.threads=1
• Packages: mlr3, mlr3learners, and ranger
• Predict Types: [response] and prob
• Feature Types: logical, integer, numeric, character, factor, and ordered
• Encapsulation: none (fallback: -)
• Properties: hotstart_backward, importance, missings, multiclass, oob_error, selected_features, twoclass, and weights
• Other settings: use_weights = 'use', predict_raw = 'FALSE'
We can now inspect the task for missing values. task$missings() returns the count of missing values for each variable.
diabetes age glucose insulin mass pedigree pregnant pressure triceps
0 0 5 405 13 0 0 35 251
Additionally, we can see that the ranger learner can not handle missing values:
[1] "hotstart_backward" "importance" "missings" "multiclass" "oob_error"
[6] "selected_features" "twoclass" "weights"
For comparison, other learners, e.g. the rpart learner can handle missing values internally.
[1] "importance" "missings" "multiclass" "selected_features" "twoclass"
[6] "weights"
Before we dive deeper, we quickly try to visualize two of the columns with missing values:
An overview over implemented PipeOps for imputation can be obtained like so:
mlr3pipelines contains several imputation methods. We focus on rather simple ones, and show how to impute missing values for factor features and numeric features respectively.
Since our task only has numeric features, we do not need to deal with imputing factor levels, and can instead concentrate on imputing numeric values:
We do this in a two-step process: * We create new indicator columns, that tells us whether the value of a feature is “missing” or “present”. We achieve this using the missind PipeOp.
imputehist PipeOp.We also have to make sure to apply the pipe operators in the correct order!
In order to better understand we can look at the results of every PipeOp separately.
We can manually trigger the PipeOp to test the operator on our task:
diabetes missing_glucose missing_insulin missing_mass missing_pressure missing_triceps
<fctr> <fctr> <fctr> <fctr> <fctr> <fctr>
1: neg present missing present present present
2: pos present missing present present missing
3: neg present missing present present present
4: neg present missing present present missing
5: neg present missing present present present
---
764: pos present missing present present present
765: pos present present present present present
766: neg present present present present present
767: neg present missing present present missing
768: neg present missing present present missing
For imputehist, we can do the same:
diabetes age pedigree pregnant glucose insulin mass pressure triceps
<fctr> <num> <num> <num> <num> <num> <num> <num> <num>
1: neg 28 0.240 0 194 181.02651 28.5 50 45.00000
2: pos 44 0.828 8 129 129.30320 27.6 84 23.56236
3: neg 26 0.696 2 132 253.29451 38.2 72 42.00000
4: neg 39 0.192 6 184 599.13042 24.7 62 40.82062
5: neg 22 0.178 4 108 162.01169 31.6 58 32.00000
---
764: pos 43 0.346 10 92 233.45051 33.1 50 28.00000
765: pos 26 1.162 0 181 474.00000 35.8 58 27.00000
766: neg 60 0.460 6 130 540.00000 34.2 106 27.00000
767: neg 24 0.206 2 146 8.83319 27.7 72 28.42715
768: neg 41 0.187 6 115 138.42919 27.6 72 28.06583
This time we obtain the imputed data set without missing values.
Now we have to put all PipeOps together in order to form a graph that handles imputation automatically.
We do this by creating a Graph that copies the data twice, processes each copy using the respective imputation method and afterwards unions the features. For this we need the following two PipeOps : * copy: Creates copies of the data. * featureunion Merges the two tasks together.
as a last step we append the learner we planned on using:
We can now visualize the resulting graph:
Correct imputation is especially important when applying imputation to held-out data during the predict step. If applied incorrectly, imputation could leak info from the test set, which potentially skews our performance estimates. mlr3pipelines takes this complexity away from the user and handles correct imputation internally.
By wrapping this graph into a GraphLearner, we can now train resample the full graph, here with a 3-fold cross validation:
In some cases, we have missing values only in the data we want to predict on. In order to showcase this, we create a copy of the task with several more missing columns.
And now we learn on task, while trying to predict on task_2.
For factor features, the process works analogously. Instead of using imputehist, we can for example use imputeoor. This will simply replace every NA in each factor variable with a new value missing.
A full graph might the look like this:
Note that we specify the parameter affect_columns = NULL when initializing missind, because we also want indicator columns for our factor features. By default, affect_columns would be set to selector_invert(selector_type(c("factor", "ordered", "character"))). We also set the parameter which to "all" to add indicator columns for all features, regardless whether values were missing during training or not.
In order to test out our new graph, we again create a situation where our task has missing factor levels. As the (diabetes) task does not have any factor levels, we use the famous (boston_housing) task.
Now we train on task_bh_1 and predict on task_bh_2:
── <PredictionRegr> for 506 observations: ──────────────────────────────────────────────────────────────────────────────
row_ids truth response
1 24.0 25.08769
2 21.6 22.31111
3 34.7 32.08418
--- --- ---
504 23.9 24.08323
505 22.0 22.58690
506 19.0 20.43212
Success! We learned how to deal with missing values in less than 10 minutes.
---
title: Impute Missing Variables
categories:
- classification
- imputation
- pipelines
author:
- name: Florian Pfisterer
date: 01-31-2020
description: |
Augment a Random Forest with automatic imputation.
aliases:
- ../../../gallery/2020-01-30-impute-missing-levels/index.html
---
{{< include ../../_setup.qmd >}}
## Prerequisites
This tutorial assumes familiarity with the basics of `r ref_pkg("mlr3pipelines")`.
Consult the [mlr3book](https://mlr3book.mlr-org.com/chapters/chapter7/sequential_pipelines.html) if some aspects are not fully understandable.
It deals with the problem of missing data.
The random forest implementation in the package `r ref_pkg("ranger")` unfortunately does not support missing values.
Therefore, it is required to impute missing features before passing the data to the learner.
We show how to use `r ref_pkg("mlr3pipelines")` to augment the `r ref("mlr_learners_classif.ranger", "ranger learner ")` with automatic imputation.
We load the `r ref_pkg("mlr3verse")` package which pulls in the most important packages for this example.
```{r index-001}
library(mlr3verse)
```
We initialize the random number generator with a fixed seed for reproducibility, and decrease the verbosity of the logger to keep the output clearly represented.
```{r index-002}
set.seed(7832)
lgr::get_logger("mlr3")$set_threshold("warn")
```
## Construct the Base Objects
First, we take an example task with missing values (`r ref("mlr_tasks_diabetes", "diabetes")`) and create the `r ref("mlr_learners_classif.ranger", "ranger learner")`:
```{r index-003}
library(mlr3learners)
task = tsk("diabetes")
print(task)
learner = lrn("classif.ranger")
print(learner)
```
We can now inspect the task for missing values.
`task$missings()` returns the count of missing values for each variable.
```{r index-004}
task$missings()
```
Additionally, we can see that the `r ref("mlr_learners_classif.ranger", "ranger learner")` can not handle missing values:
```{r index-005}
learner$properties
```
For comparison, other learners, e.g. the `r ref("mlr_learners_classif.rpart", "rpart learner")` can handle missing values internally.
```{r index-006}
lrn("classif.rpart")$properties
```
Before we dive deeper, we quickly try to visualize two of the columns with missing values:
```{r index-007, message = FALSE, warning = FALSE, fig.width=10, fig.height=8}
autoplot(task$clone()$select(c("insulin", "triceps")), type = "pairs")
```
## Operators overview
An overview over implemented `r ref("PipeOp")`s for imputation can be obtained like so:
```{r index-008}
mlr_pipeops$keys("^impute")
```
## Construct Operators
`r ref_pkg("mlr3pipelines")` contains several imputation methods.
We focus on rather simple ones, and show how to impute missing values for `factor` features and
`numeric` features respectively.
Since our task only has numeric features, we do not need to deal with imputing factor levels,
and can instead concentrate on imputing numeric values:
We do this in a two-step process:
* We create new indicator columns, that tells us whether the value of a feature is "missing" or "present".
We achieve this using the `r ref("mlr_pipeops_missind", "missind")` `r ref("PipeOp")`.
* Afterwards, we impute every missing value by sampling from the histogram of the respective column.
We achieve this using the `r ref("mlr_pipeops_imputehist", "imputehist")` `r ref("PipeOp")`.
We also have to make sure to apply the pipe operators in the correct order!
```{r index-009}
imp_missind = po("missind")
imp_num = po("imputehist", affect_columns = selector_type("numeric"))
```
In order to better understand we can look at the results of every `r ref("PipeOp")` separately.
We can manually trigger the `r ref("PipeOp")` to test the operator on our task:
```{r index-010}
task_ext = imp_missind$train(list(task))[[1]]
task_ext$data()
```
For `r ref("mlr_pipeops_imputehist", "imputehist")`, we can do the same:
```{r index-011}
task_ext = imp_num$train(list(task))[[1]]
task_ext$data()
```
This time we obtain the imputed data set without `missing` values.
```{r index-012}
task_ext$missings()
```
## Putting everything together
Now we have to put all `r ref("PipeOp")`s together in order to form a graph that handles imputation automatically.
We do this by creating a `r ref("Graph")` that copies the data twice, processes each copy using the respective imputation method and afterwards unions the features.
For this we need the following two `r ref("PipeOp")`s :
* `r ref("mlr_pipeops_copy", "copy")`: Creates copies of the data.
* `r ref("mlr_pipeops_featureunion", "featureunion")` Merges the two tasks together.
```{r index-013}
graph = po("copy", 2) %>>%
gunion(list(imp_missind, imp_num)) %>>%
po("featureunion")
```
as a last step we append the learner we planned on using:
```{r index-014}
graph = graph %>>% po(learner)
```
We can now visualize the resulting graph:
```{r index-015, fig.width=6, fig.height=8}
graph$plot()
```
## Resampling
Correct imputation is especially important when applying imputation to held-out data during the `predict` step.
If applied incorrectly, imputation could leak info from the test set, which potentially skews our performance estimates.
`r ref_pkg("mlr3pipelines")` takes this complexity away from the user and handles correct imputation internally.
By wrapping this graph into a `r ref("GraphLearner")`, we can now train resample the full graph, here with a 3-fold cross validation:
```{r index-016}
graph_learner = as_learner(graph)
rr = resample(task, graph_learner, rsmp("cv", folds = 3))
rr$aggregate()
```
## Missing values during prediction
In some cases, we have missing values only in the data we want to predict on.
In order to showcase this, we create a copy of the task with several more missing columns.
```{r index-017}
dt = task$data()
dt[1:10, "age"] = NA
dt[30:70, "pedigree"] = NA
task_2 = as_task_classif(dt, id = "diabetes2", target = "diabetes")
```
And now we learn on `task`, while trying to predict on `task_2`.
```{r index-018}
graph_learner$train(task)
graph_learner$predict(task_2)
```
## Missing factor features
For `factor` features, the process works analogously.
Instead of using `r ref("mlr_pipeops_imputehist", "imputehist")`, we can for example use `r ref("mlr_pipeops_imputeoor", "imputeoor")`.
This will simply replace every `NA` in each factor variable with a new value `missing`.
A full graph might the look like this:
```{r index-019}
imp_missind = po("missind", affect_columns = NULL, which = "all")
imp_fct = po("imputeoor", affect_columns = selector_type("factor"))
graph = po("copy", 2) %>>%
gunion(list(imp_missind, imp_num %>>% imp_fct)) %>>%
po("featureunion")
```
Note that we specify the parameter `affect_columns = NULL` when initializing `r ref("mlr_pipeops_missind", "missind")`, because we also want indicator columns for our `factor` features.
By default, `affect_columns` would be set to `selector_invert(selector_type(c("factor", "ordered", "character")))`.
We also set the parameter `which` to `"all"` to add indicator columns for all features, regardless whether values were missing during training or not.
In order to test out our new graph, we again create a situation where our task has missing factor levels.
As the (`r ref("mlr_tasks_diabetes", "diabetes")`) task does not have any factor levels, we use the
famous (`r ref("mlr_tasks_boston_housing", "boston_housing")`) task.
```{r index-020}
# task_bh_1 is the training data without missings
task_bh_1 = tsk("boston_housing")
# task_bh_2 is the prediction data with missings
dt = task_bh_1$data()
dt[1:10, chas := NA][20:30, rm := NA]
task_bh_2 = as_task_regr(dt, id = "bh", target = "cmedv")
```
Now we train on `task_bh_1` and predict on `task_bh_2`:
```{r index-021}
graph_learner = as_learner(graph %>>% po(lrn("regr.ranger")))
graph_learner$train(task_bh_1)
graph_learner$predict(task_bh_2)
```
Success! We learned how to deal with missing values in less than 10 minutes.