library(mlr3verse)Intro
mlr3pipelines offers a very flexible way to create data preprocessing steps. This is achieved by a modular approach using PipeOps. For detailed overview check the mlr3book.
Recommended prior readings:
This post covers:
- How to apply different preprocessing steps on different features
- How to branch different preprocessing steps, which allows to select the best performing path
- How to tune the whole pipeline
Prerequisites
We load the mlr3verse package which pulls in the most important packages for this example.
We initialize the random number generator with a fixed seed for reproducibility, and decrease the verbosity of the logger to keep the output clearly represented.
set.seed(7832)
lgr::get_logger("mlr3")$set_threshold("warn")
lgr::get_logger("bbotk")$set_threshold("warn")The diabetes classification task will be used.
task_diabetes = tsk("diabetes")
skimr::skim(task_diabetes$data())| Name | task_diabetes$data() |
| Number of rows | 128 |
| Number of columns | 9 |
| Key | NULL |
| _______________________ | |
| Column type frequency: | |
| factor | 1 |
| numeric | 8 |
| ________________________ | |
| Group variables | None |
Variable type: factor
| skim_variable | n_missing | complete_rate | ordered | n_unique | top_counts |
|---|---|---|---|---|---|
| diabetes | 0 | 1 | FALSE | 2 | neg: 83, pos: 45 |
Variable type: numeric
| skim_variable | n_missing | complete_rate | mean | sd | p0 | p25 | p50 | p75 | p100 | hist |
|---|---|---|---|---|---|---|---|---|---|---|
| age | 0 | 1.00 | 34.48 | 3.29 | 24.00 | 32.00 | 34.00 | 37.00 | 44.00 | ▁▅▇▅▁ |
| glucose | 6 | 0.95 | 121.61 | 24.45 | 64.00 | 104.50 | 121.00 | 137.00 | 176.00 | ▁▅▇▆▂ |
| insulin | 5 | 0.96 | 101.79 | 77.29 | 14.00 | 57.00 | 82.00 | 125.50 | 598.00 | ▇▂▁▁▁ |
| mass | 5 | 0.96 | 32.00 | 5.52 | 19.80 | 28.45 | 32.10 | 35.40 | 45.70 | ▂▆▇▃▂ |
| pedigree | 0 | 1.00 | 0.48 | 0.40 | 0.04 | 0.22 | 0.34 | 0.65 | 2.18 | ▇▃▁▁▁ |
| pregnant | 0 | 1.00 | 3.83 | 1.85 | 0.00 | 2.00 | 4.00 | 5.00 | 8.00 | ▂▇▅▅▂ |
| pressure | 6 | 0.95 | 71.83 | 10.52 | 46.00 | 64.25 | 73.00 | 79.00 | 102.00 | ▂▆▇▅▁ |
| triceps | 5 | 0.96 | 25.17 | 8.68 | 5.00 | 19.00 | 26.00 | 30.00 | 49.00 | ▂▆▇▃▁ |
Selection of features for preprocessing steps
Several features of the diabetes task have missing values:
task_diabetes$missings()diabetes age glucose insulin mass pedigree pregnant pressure triceps
0 0 6 5 5 0 0 6 5 A common approach in such situations is to impute the missing values and to add a missing indicator column as explained in the Impute missing variables post. Suppose we want to use
PipeOpImputeHiston the features “glucose”, “mass” and “pressure” andPipeOpImputeMedianon the features “insulin” and “triceps”.
In the following subsections, we show two approaches to implement this.
1. Consider all features and apply the preprocessing step only to certain features
Using the affect_columns argument of a PipeOp to define the variables on which a PipeOp will operate with an appropriate Selector function:
# imputes values based on histogram
imputer_hist = po("imputehist",
affect_columns = selector_name(c("glucose", "mass", "pressure")))
# imputes values using the median
imputer_median = po("imputemedian",
affect_columns = selector_name(c("insulin", "triceps")))
# adds an indicator column for each feature with missing values
miss_ind = po("missind")When PipeOps are constructed this way, they will perform the specified preprocessing step on the appropriate features and pass all the input features to the subsequent steps:
# no missings in "glucose", "mass" and "pressure"
imputer_hist$train(list(task_diabetes))[[1]]$missings()diabetes age insulin pedigree pregnant triceps glucose mass pressure
0 0 5 0 0 5 0 0 0 # no missings in "insulin" and "triceps"
imputer_median$train(list(task_diabetes))[[1]]$missings()diabetes age glucose mass pedigree pregnant pressure insulin triceps
0 0 6 5 0 0 6 0 0 We construct a pipeline that combines imputer_hist and imputer_median. Here, imputer_hist will impute the features “glucose”, “mass” and “pressure”, and imputer_median will impute “insulin” and “triceps”. In each preprocessing step, all the input features are passed to the next step. In the end, we obtain a data set without missing values:
# combine the two impuation methods
impute_graph = imputer_hist %>>% imputer_median
impute_graph$plot(html = FALSE)
impute_graph$train(task_diabetes)[[1]]$missings()diabetes age pedigree pregnant glucose mass pressure insulin triceps
0 0 0 0 0 0 0 0 0 The PipeOpMissInd operator replaces features with missing values with a missing value indicator:
miss_ind$train(list(task_diabetes))[[1]]$data() diabetes missing_glucose missing_insulin missing_mass missing_pressure missing_triceps
<fctr> <fctr> <fctr> <fctr> <fctr> <fctr>
1: pos missing present present present present
2: pos present present present missing present
3: neg present present present present missing
4: pos present missing present present present
5: neg present present missing present present
---
124: neg present present present present present
125: neg present present present present present
126: neg present present present present present
127: neg present present present missing present
128: neg present present present present presentObviously, this step can not be applied to the already imputed data as there are no missing values. If we want to combine the previous two imputation steps with a third step that adds missing value indicators, we would need to PipeOpCopy the data two times and supply the first copy to impute_graph and the second copy to miss_ind using gunion(). Finally, the two outputs can be combined with PipeOpFeatureUnion:
impute_missind = po("copy", 2) %>>%
gunion(list(impute_graph, miss_ind)) %>>%
po("featureunion")
impute_missind$plot(html = FALSE)
impute_missind$train(task_diabetes)[[1]]$data() diabetes age pedigree pregnant glucose mass pressure insulin triceps missing_glucose missing_insulin
<fctr> <int> <num> <int> <num> <num> <num> <num> <num> <fctr> <fctr>
1: pos 38 0.106 4 111.0298 21.90000 84.00000 15 30 missing present
2: pos 32 0.646 4 159.0000 32.50000 63.05159 84 15 present present
3: neg 34 0.067 7 66.0000 32.50000 89.00000 62 26 present present
4: pos 37 1.778 4 128.0000 35.90000 58.00000 82 18 present missing
5: neg 33 0.218 3 111.0000 29.67166 79.00000 206 37 present present
---
124: neg 36 0.299 2 91.0000 36.40000 86.00000 82 19 present present
125: neg 35 0.245 4 95.0000 39.90000 102.00000 69 44 present present
126: neg 37 1.754 1 100.0000 21.30000 74.00000 51 26 present present
127: neg 36 0.584 2 111.0000 37.60000 74.53819 210 24 present present
128: neg 29 0.302 6 116.0000 28.80000 63.00000 66 6 present present
missing_mass missing_pressure missing_triceps
<fctr> <fctr> <fctr>
1: present present present
2: present missing present
3: present present missing
4: present present present
5: missing present present
---
124: present present present
125: present present present
126: present present present
127: present missing present
128: present present present2. Select the features for each preprocessing step and apply the preprocessing steps to this subset
We can use the PipeOpSelect to select the appropriate features and then apply the desired impute PipeOp on them:
imputer_hist_2 = po("select",
selector = selector_name(c("glucose", "mass", "pressure")),
id = "slct1") %>>% # unique id so we can combine it in a pipeline with other select PipeOps
po("imputehist")
imputer_hist_2$plot(html = FALSE)
imputer_hist_2$train(task_diabetes)[[1]]$data() diabetes glucose mass pressure
<fctr> <num> <num> <num>
1: pos 110.1189 21.90000 84.00000
2: pos 159.0000 32.50000 68.87389
3: neg 66.0000 32.50000 89.00000
4: pos 128.0000 35.90000 58.00000
5: neg 111.0000 33.52207 79.00000
---
124: neg 91.0000 36.40000 86.00000
125: neg 95.0000 39.90000 102.00000
126: neg 100.0000 21.30000 74.00000
127: neg 111.0000 37.60000 79.58754
128: neg 116.0000 28.80000 63.00000imputer_median_2 =
po("select", selector = selector_name(c("insulin", "triceps")), id = "slct2") %>>%
po("imputemedian")
imputer_median_2$train(task_diabetes)[[1]]$data() diabetes insulin triceps
<fctr> <num> <num>
1: pos 15 30
2: pos 84 15
3: neg 62 26
4: pos 82 18
5: neg 206 37
---
124: neg 82 19
125: neg 69 44
126: neg 51 26
127: neg 210 24
128: neg 66 6To reproduce the result of the fist example (1.), we need to copy the data four times and apply imputer_hist_2, imputer_median_2 and miss_ind on each of the three copies. The fourth copy is required to select the features without missing values and to append it to the final result. We can do this as follows:
other_features = task_diabetes$feature_names[task_diabetes$missings()[-1] == 0]
imputer_missind_2 = po("copy", 4) %>>%
gunion(list(imputer_hist_2,
imputer_median_2,
miss_ind,
po("select", selector = selector_name(other_features), id = "slct3"))) %>>%
po("featureunion")
imputer_missind_2$plot(html = FALSE)
imputer_missind_2$train(task_diabetes)[[1]]$data() diabetes glucose mass pressure insulin triceps missing_glucose missing_insulin missing_mass missing_pressure
<fctr> <num> <num> <num> <num> <num> <fctr> <fctr> <fctr> <fctr>
1: pos 119.039 21.90000 84.00000 15 30 missing present present present
2: pos 159.000 32.50000 76.47987 84 15 present present present missing
3: neg 66.000 32.50000 89.00000 62 26 present present present present
4: pos 128.000 35.90000 58.00000 82 18 present missing present present
5: neg 111.000 22.62252 79.00000 206 37 present present missing present
---
124: neg 91.000 36.40000 86.00000 82 19 present present present present
125: neg 95.000 39.90000 102.00000 69 44 present present present present
126: neg 100.000 21.30000 74.00000 51 26 present present present present
127: neg 111.000 37.60000 61.69390 210 24 present present present missing
128: neg 116.000 28.80000 63.00000 66 6 present present present present
missing_triceps age pedigree pregnant
<fctr> <int> <num> <int>
1: present 38 0.106 4
2: present 32 0.646 4
3: missing 34 0.067 7
4: present 37 1.778 4
5: present 33 0.218 3
---
124: present 36 0.299 2
125: present 35 0.245 4
126: present 37 1.754 1
127: present 36 0.584 2
128: present 29 0.302 6Note that when there is one input channel, it is automatically copied as many times as needed for the downstream PipeOps. In other words, the code above works also without po("copy", 4):
imputer_missind_3 = gunion(list(imputer_hist_2,
imputer_median_2,
miss_ind,
po("select", selector = selector_name(other_features), id = "slct3"))) %>>%
po("featureunion")
imputer_missind_3$train(task_diabetes)[[1]]$data() diabetes glucose mass pressure insulin triceps missing_glucose missing_insulin missing_mass missing_pressure
<fctr> <num> <num> <num> <num> <num> <fctr> <fctr> <fctr> <fctr>
1: pos 91.9305 21.90000 84.00000 15 30 missing present present present
2: pos 159.0000 32.50000 62.98685 84 15 present present present missing
3: neg 66.0000 32.50000 89.00000 62 26 present present present present
4: pos 128.0000 35.90000 58.00000 82 18 present missing present present
5: neg 111.0000 36.91103 79.00000 206 37 present present missing present
---
124: neg 91.0000 36.40000 86.00000 82 19 present present present present
125: neg 95.0000 39.90000 102.00000 69 44 present present present present
126: neg 100.0000 21.30000 74.00000 51 26 present present present present
127: neg 111.0000 37.60000 87.60931 210 24 present present present missing
128: neg 116.0000 28.80000 63.00000 66 6 present present present present
missing_triceps age pedigree pregnant
<fctr> <int> <num> <int>
1: present 38 0.106 4
2: present 32 0.646 4
3: missing 34 0.067 7
4: present 37 1.778 4
5: present 33 0.218 3
---
124: present 36 0.299 2
125: present 35 0.245 4
126: present 37 1.754 1
127: present 36 0.584 2
128: present 29 0.302 6Usually, po("copy") is required when there are more than one input channels and multiple output channels, and their numbers do not match.
Branching
We can not know if the combination of a learner with this preprocessing graph will benefit from the imputation steps and the added missing value indicators. Maybe it would have been better to just use imputemedian on all the variables. We could investigate this assumption by adding an alternative path to the graph with the mentioned imputemedian. This is possible using the “branch” PipeOp:
imputer_median_3 = po("imputemedian", id = "simple_median") # add the id so it does not clash with `imputer_median`
branches = c("impute_missind", "simple_median") # names of the branches
graph_branch = po("branch", branches) %>>%
gunion(list(impute_missind, imputer_median_3)) %>>%
po("unbranch")
graph_branch$plot(html = FALSE)
Tuning the pipeline
To finalize the graph, we combine it with a rpart learner:
graph = graph_branch %>>%
lrn("classif.rpart")
graph$plot(html = FALSE)
To define the parameters to be tuned, we first check the available ones in the graph:
as.data.table(graph$param_set)[, .(id, class, lower, upper, nlevels)] id class lower upper nlevels
<char> <char> <num> <num> <num>
1: branch.selection ParamFct NA NA 2
2: imputehist.affect_columns ParamUty NA NA Inf
3: imputemedian.affect_columns ParamUty NA NA Inf
4: missind.which ParamFct NA NA 2
5: missind.type ParamFct NA NA 4
6: missind.affect_columns ParamUty NA NA Inf
7: simple_median.affect_columns ParamUty NA NA Inf
8: classif.rpart.cp ParamDbl 0 1 Inf
9: classif.rpart.keep_model ParamLgl NA NA 2
10: classif.rpart.maxcompete ParamInt 0 Inf Inf
11: classif.rpart.maxdepth ParamInt 1 30 30
12: classif.rpart.maxsurrogate ParamInt 0 Inf Inf
13: classif.rpart.minbucket ParamInt 1 Inf Inf
14: classif.rpart.minsplit ParamInt 1 Inf Inf
15: classif.rpart.surrogatestyle ParamInt 0 1 2
16: classif.rpart.usesurrogate ParamInt 0 2 3
17: classif.rpart.xval ParamInt 0 Inf InfWe decide to jointly tune the "branch.selection", "classif.rpart.cp" and "classif.rpart.minbucket" hyperparameters:
search_space = ps(
branch.selection = p_fct(c("impute_missind", "simple_median")),
classif.rpart.cp = p_dbl(0.001, 0.1),
classif.rpart.minbucket = p_int(1, 10))In order to tune the graph, it needs to be converted to a learner:
graph_learner = as_learner(graph)
cv3 = rsmp("cv", folds = 3)
cv3$instantiate(task_diabetes) # to generate folds for cross validation
instance = tune(
tuner = tnr("random_search"),
task = task_diabetes,
learner = graph_learner,
resampling = rsmp("cv", folds = 3),
measure = msr("classif.ce"),
search_space = search_space,
term_evals = 5)
as.data.table(instance$archive, unnest = NULL, exclude_columns = c("x_domain", "uhash", "resample_result")) branch.selection classif.rpart.cp classif.rpart.minbucket classif.ce runtime_learners timestamp warnings
<char> <num> <int> <num> <num> <POSc> <int>
1: simple_median 0.052247533 9 0.2729790 2.361 2026-08-10 15:18:32 0
2: simple_median 0.009352281 4 0.3671096 2.346 2026-08-10 15:18:33 0
3: impute_missind 0.042075931 5 0.3514212 2.708 2026-08-10 15:18:35 0
4: simple_median 0.053105368 7 0.3117386 1.406 2026-08-10 15:18:35 0
5: simple_median 0.046968016 3 0.3672942 1.573 2026-08-10 15:18:36 0
errors batch_nr
<int> <int>
1: 0 1
2: 0 2
3: 0 3
4: 0 4
5: 0 5The best performance in this short tuned experiment was achieved with:
instance$result branch.selection classif.rpart.cp classif.rpart.minbucket learner_param_vals x_domain classif.ce
<char> <num> <int> <list> <list> <num>
1: simple_median 0.05224753 9 <list[9]> <list[3]> 0.272979Conclusion
This post shows ways on how to specify features on which preprocessing steps are to be performed. In addition it shows how to create alternative paths in the learner graph. The preprocessing steps that can be used are not limited to imputation. Check the list of available PipeOp.