Skip to content

Random Forests Variable Importance Question

2 messages · Paul Fisch, Liaw, Andy

Paul Fisch
# 
I am trying to use the random forests package for classification in R.

The Variable Importance Measures listed are:

-mean raw importance score of variable x for class 0

-mean raw importance score of variable x for class 1

-MeanDecreaseAccuracy

-MeanDecreaseGini

Now I know what these "mean" as in I know their definitions. What I
want to know is how to use them.

What I am trying to figure out is what these values mean in only the
context of how accurate they are, what is a good value, what is a bad
value, what are the maximums and minimums, etc.

If a variable has a high MeanDecreaseAccuracy or MeanDecreaseGini does
that mean it is important or unimportant? Also any information on the
raw scores would be really helpful too. I want to know everything
there is to know about these numbers that is relevant to the
application of them.

I don't really want a technical explanation that uses words like
'error', 'summation', or 'permutated', but rather a simpler
explanation that didn't involve any discussion of how random forests
works(I have read all about that and didn't find it very helpful.)

Like if I wanted someone to explain to me how to use a radio, I
wouldn't expect the explanation to involve how a radio converts radio
waves into sound.

If anyone can help me out at all it would be really great.? I have
read many many lectures on random forests and other data mining
lectures but I have never found simple answers about how to read the
variable importance measures.

Thanks,
Paul Fisch
Liaw, Andy
# 
I'll take a shot.

Let me try to explain the 3rd measure first.  A RF model tries to predict an outcome variable (the classes) from a group of potential predictor variables (the "x").  If a predictor variable is "important" in making the prediction accurate, then by messing with it (e.g., giving it random values) should have a larger impact on how well the prediction can be made, compared to a variable that contributes little.  The variable importance measure tries to capture this.  (If you throw a wrench into the trunk of a car, it probably doesn't affect how the car drives.  However, if you throw the wrench into the engine compartment, that _may_ be a different story.)

I don't know about others, but I only look at the relative importance of the variables, rather than trying to interpret the numbers (raw or scaled).  Any number below 0 should be treated as the same as 0 (if I recall, Breiman & Cutler's code truncate the values at 0).  Any variable with importance value smaller than the absolute value of the minimum is probably not worth much looking.

The first two measures (you must be predicting an outcome variable with two classes) are the analogous measures that address each of the two classes specifically, rather than over all of the data.

Andy


From: Paul Fisch
Notice:  This e-mail message, together with any attachme...{{dropped:12}}