Feature importance measures how much a model's predictions change when an input is perturbed or removed — a property of the fitted model, not of the world — and because correlated inputs substitute for one another, an important feature may be a proxy for an unmeasured driver while the causal variable scores low: the documented pattern in correlated data is that importance splits, swaps or concentrates arbitrarily among stand-ins depending on the fitting algorithm. UZU NEWS publishes information, not investment advice, and keeps the causal language out of correlational tools.
"The model found X most important" is a sentence that sounds like discovery and is, so far, a sentence about the model. Importance rankings are among the most requested outputs of applied machine learning and among the most routinely over-interpreted — in finance especially, where features are mutually correlated by construction and the temptation to read rankings as market mechanisms is strongest.
What do importance scores measure?
Different algorithms measure different things, which is the first problem. Split-gain importance in tree ensembles measures the total improvement in loss from splits on a feature — biased toward high-cardinality and continuous features. Permutation importance measures the degradation in performance when a feature's values are randomly shuffled — sensitive to correlation among features, because shuffling one of two redundant columns changes less than either alone would seem to deserve. SHAP values decompose individual predictions into additive feature contributions with game-theoretic guarantees among the features present — attributing among the inputs the model saw, not among causes in the world. Every method answers a question about the model's function; none answers a question about the data-generating process, which is what causal claims are about.
Why do correlations break the reading?
Two mechanisms. Substitution: when features correlate, the model can use either one, and which one it uses depends on fitting noise, regularization, ordering — the ranking can flip between runs of the same algorithm on resampled data, a diagnostic worth running. Splitting: importance divides among correlated stand-ins, so a truly causal driver can appear modest because its proxies absorbed shares of the credit. In financial data — where valuation ratios, momentum at several horizons, sector memberships and macro aggregates all intercorrelate — both mechanisms operate on almost every input set. The empirical tell: recompute importance across bootstrap resamples and across algorithms; unstable rankings indicate correlation-driven attribution, and stability across resamples is a minimum condition before any stronger reading.
What would establish causation instead?
Different instruments entirely, none of them importance scores. Randomized intervention — changing the input and observing outcomes — is the standard; in markets, that mostly does not exist for features, which is the structural difficulty of the domain. Natural experiments and quasi-experimental designs exploit events that moved one variable and not others. Causal discovery algorithms test conditional-independence structure, with assumptions that financial data routinely violate. Formal sensitivity analysis asks how strong an unmeasured confounder would need to be to overturn a conclusion — publishing the answer alongside the estimate. None of these is a button in a machine-learning library; all of them are research designs, and the honest workflow uses importance for what it is — model diagnostics and hypothesis generation — and reaches for designs when the question becomes causal.
Where does the error show up in practice?
Three documented patterns in applied finance. Narrative laundering: a model ranks a macro feature first, and the write-up explains the economic mechanism — the mechanism was never tested, only decorated. Action reversal: because importance identifies reliance, not effect, interventions guided by importance can move nothing — the model was using the feature as a proxy, and the causal variable remained untouched. Regime fragility: importance is measured on the fitted distribution; when correlations shift — as they do across market regimes — yesterday's ranking rearranges, and models live or die by proxies that stopped substituting. Each pattern is a case of asking a correlational instrument a causal question and reading the answer as delivered.
| Method | Actually measures | Known bias | Valid use |
|---|---|---|---|
| Split-gain (trees) | Loss improvement from splits | Favors continuous, high-cardinality features | Model diagnostics |
| Permutation | Performance drop when shuffled | Distorted by correlated features | Diagnostics with caution |
| SHAP | Additive contribution per prediction | Attribution among inputs, not causes | Explaining predictions |
| Intervention / experiment | Causal effect | Cost, feasibility | Answering causal questions |
How should importance be reported?
With its method named, its stability assessed across resamples, and its claims bounded to the model: "the fitted ensemble relied most on X" — a statement about reliance, checkable by recomputation — rather than "X drives outcomes," a statement about the world that importance cannot support. This boundary is the same one this site draws around every correlational instrument, from factor loadings to GEX aggregates: report what was measured, and let the causal vocabulary wait for causal evidence. Model-risk guidance for exactly this class of interpretation questions in production settings is documented in supervisory material such as the Federal Reserve's SR 11-7 letter at federalreserve.gov.
For more context, read What does out-of-sample mean in a forecasting paper?.
For more context, read regime detection models.
For more context, read Why random cross-validation breaks on financial time series.
