Skip to content

Variable importance fails in superlearner containing interactions or feature screener #435

@acousticus

Description

@acousticus

Description:

sl3::importance() works with a simple SL, but fails when using interactions and/or feature screeners. Using sl3_1.4.4.

Steps to reproduce:

library(sl3)
data(cpp_imputed)

covars <- c("apgar1", "apgar5", "parity", "gagebrth", "mage", "meducyrs", "sexn")
outcome <- "haz"
task <- sl3_Task$new(cpp_imputed, covariates = covars, outcome = outcome)

interactions <- list(c("apgar1", "parity"), c("apgar5", "parity"))
keepme <- c("mage", "meducyrs")

# BASE LEARNERS
lrnr_glm <- make_learner(Lrnr_glm)
lrnr_mean <- make_learner(Lrnr_mean)
learners <- c(lrnr_glm, lrnr_mean)
stack <- make_learner(Stack, learners)

# RF SCREENER
miniforest <- Lrnr_ranger$new(importance = "impurity")
screen_rf <- Lrnr_screener_importance$new(learner = miniforest)

# INTERACTIONS
lrn_interaction <- make_learner(Lrnr_define_interactions, interactions)


### 1. SL WITHOUT SCREEN OR INTERACTIONS
sl <- make_learner(Lrnr_sl, learners)
sl_fit <- sl$train(task)
importance(sl_fit)

## Example output:
#   covariate   MSE_difference
# 1:  gagebrth    3.173805e-02
# 2:    parity    1.323431e-02
# 3:      mage    5.495150e-03
# 4:      sexn   -2.788994e-05
# 5:    apgar1   -2.859870e-04
# 6:  meducyrs   -1.242081e-03
# 7:    apgar5   -3.459809e-03


### 2. SL + INTERACTIONS + SCREENER
ints_screen_rf <- make_learner(Pipeline, lrn_interaction, screen_rf, stack)
sl <- make_learner(Lrnr_sl, learners = ints_screen_rf)
sl_fit <- sl$train(task)
importance(sl_fit)
# Error : Passed a vector of type 'list'. Needs to be type 'character'.
# Failed on chain
# Error in self$compute_step() :
#   Error : Passed a vector of type 'list'. Needs to be type 'character'.


### 3. SL + INTERACTIONS
ints_only <- make_learner(Pipeline, lrn_interaction, stack)
sl <- make_learner(Lrnr_sl, learners = ints_only)
sl_fit <- sl$train(task)
importance(sl_fit)
# Error : Passed a vector of type 'list'. Needs to be type 'character'.
# Failed on chain
# Error in self$compute_step() :
#   Error : Passed a vector of type 'list'. Needs to be type 'character'.


### 4. SL + SCREENER
ints_only <- make_learner(Pipeline, lrn_interaction, stack)
sl <- make_learner(Lrnr_sl, learners = ints_only)
sl_fit <- sl$train(task)
importance(sl_fit)
# Error : Passed a vector of type 'list'. Needs to be type 'character'.
# Failed on chain
# Error in self$compute_step() :
#   Error : Passed a vector of type 'list'. Needs to be type 'character'.

Question

Is this an issue with the way I am running importance() on the fits, with how I code the different superlearner variants, or with sl3 itself? Many thanks

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions