New iterative method uses predictor correlations to improve variable selection when predictors far outnumber samples
This paper introduces Structured Screen-and-Select Variable Selection (S3VS), a flexible recipe for finding important predictors when the number of candidate variables is much larger than the number of samples. In such “ultra-high-dimensional” settings, common screening methods that look only at each predictor’s individual link to the outcome can miss relevant variables. That problem is especially bad when predictors are strongly correlated and an important variable has a weak individual signal.
S3VS works in short cycles. At each step it picks a set of leading variables based on their association with the current outcome quantity. Around each leader it builds a small local set of other predictors that are strongly associated with that leader. The method then applies a model-specific selector inside each local set (for example, LASSO, a penalized regression method) and combines the selected and excluded variables to update the pool of candidates and, when appropriate, the outcome representation. The framework is modular: users can choose how to pick leaders, how to form local sets, how to aggregate results, and which selection method to use. The authors give implementations for linear models, generalized linear models, and two common survival models (accelerated failure time and Cox proportional hazards).
The paper gives a theoretical guarantee for one particular one-step linear configuration of S3VS. That result says S3VS can have a “sure screening” property under several conditions. One key condition is the presence of useful proxies. A proxy is a predictor that is correlated with an active (truly relevant) predictor and has a stronger marginal association with the outcome. If an active predictor appears in the local set of a stronger proxy and the within-set selector keeps it, S3VS can recover the active predictor even when its own marginal signal is weak. The authors also identify conditions that must hold for this advantage and show a counterexample where no proxy advantage exists. The paper stresses that “proxy” here is a screening concept only, not a statement about causation or biological substitution.