RuleFit & quantreg: partial dependence plots; showing an effect

Dear Roger,

Is it possible to combine the two ideas that you mentioned: (1)  
algorithmic
approaches of Breiman, Friedman, and others that achieve  
flexibility in the
predictor space, and (2) robust and flexible regression like QR  
that achieve
flexibility in the response space, so as to achieve complete  
flexibility?
If it is possible, are you or anyone else in the R community  
working on
this?

-----Original Message-----
From: r-help-bounces at stat.math.ethz.ch
[mailto:r-help-bounces at stat.math.ethz.ch] On Behalf Of roger koenker
Sent: Wednesday, December 20, 2006 8:57 AM
To: Mark Difford
Cc: R-help list
Subject: Re: [R] RuleFit & quantreg: partial dependence plots;  
showing an
effect

They are entirely different:  Rulefit is a fiendishly clever
combination of decision tree  formulation
of models and L1-regularization intended to select parsimonious fits
to very complicated
responses yielding e.g. piecewise constant functions.  Rulefit
estimates the  conditional
mean of the response over the covariate space, but permits a very
flexible, but linear in
parameters specifications of the covariate effects on the conditional
mean.  The quantile
regression plotting you refer to adopts a fixed, linear specification
for conditional quantile
functions and given that specification depicts how the covariates
influence the various
conditional quantiles of the response.   Thus, roughly speaking,
Rulefit is focused on
flexibility in the x-space, maintaining the classical conditional
mean objective; while
QR is trying to be more flexible in the y-direction, and maintaining
a fixed, linear
in parameters specification for the covariate effects at each  
quantile.


url:    www.econ.uiuc.edu/~roger            Roger Koenker
email    rkoenker at uiuc.edu            Department of Economics
vox:     217-333-4558                University of Illinois
fax:       217-244-6678                Champaign, IL 61820


On Dec 20, 2006, at 4:17 AM, Mark Difford wrote:

Dear List,

I would greatly appreciate help on the following matter:

The RuleFit program of Professor Friedman uses partial dependence
plots
to explore the effect of an explanatory variable on the response
variable, after accounting for the average effects of the other
variables.  The plot method [plot(summary(rq(y ~ x1 + x2,
t=seq(.1,.9,.05))))] of Professor Koenker's quantreg program
appears to
do the same thing.


Question:
Is there a difference between these two types of plot in the manner
in which they depict the relationship between explanatory variables
and the response variable ?

Thank you inav for your help.

Regards,
Mark Difford.

-------------------------------------------------------------
Mark Difford
Ph.D. candidate, Botany Department,
Nelson Mandela Metropolitan University,
Port Elizabeth, SA.