Skip to content

regression constraints?

7 messages · Strumila, John, Adam Gehr, Brian Ripley +2 more

Strumila, John
# 
gday R gurus,

I have a multivariate regression for which I want to constrain the
coefficients to be > 0.  Is this possible?

I've check the doco and searched CRAN but can't find anything. 

thanks,
John Strumila
-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To: r-help-request at stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._
Adam Gehr
#
"Strumila, John" wrote:
I've been doing something like that (regression coefficients
constrained 
to be > 0 and also forced to sum to 1) using the quadratic
programming program solve.QP in package quadprog. 

    Adam Gehr
-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To: r-help-request at stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._
Brian Ripley
#
On Fri, 5 Jan 2001, Strumila, John wrote:

            

      
      
      
    

        
Is that multiple linear regression?  (one y, lots of x's?)  Multivariate
regression might be thought to have many y's and only model part of the
relationship amongst them by many x's.

Fitting by least-squares, it is a quadratic programming problem.
So either use the quadprog package on CRAN, or use a general
minimizer with constraints: set forward optim().

Here's an example from the VR script ch08.R:

library(MASS)
data(whiteside)
attach(whiteside)
Gas <- Gas[Insul=="Before"]
Temp <- -Temp[Insul=="Before"]
#nnls.fit(cbind(1, -1, Temp), Gas) : this is an S-PLUS function
# can use box-constrained optimizer
fn <- function(par) sum((Gas - par[1] - par[2]*Temp)^2)
optim(rep(0,2), fn, lower=0, method="L-BFGS-B")$par
rm(Gas, Temp)
detach()

You can do non-linear regression and non-LS fitting the same way.

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      John P. Burkett
    

    

      
      #
    

  


  

      
A Bayesian approach to regression with inequality constraints on the
coefficients is to estimate the regression without constraints, sample from
the distribution of the coefficients, discard draws that violate the
inequality constraints, and finally compute summary statistics from the
remaining subsample.  Andrew Gelman et al. explain how to do that in
Bayesian Data Analysis.  I believe they implemented their procedures in
S-Plus.  If any one has written similar programs in R, please let us know.
----- Original Message -----
From: Strumila, John <John.Strumila at team.telstra.com>
To: <R-help at stat.math.ethz.ch>
Sent: Thursday, January 04, 2001 4:56 PM
Subject: [R] regression constraints?

            

      
      
      
    

        
-.-.-

            

      
      
      
    

        
http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html

            

      
      
      
    

        
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._.
_._

-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To: r-help-request at stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._

  
  


  


      

    

    

      
      

        


  

        

      John P. Burkett
    

    

      
      #
    

  


  

      
A Bayesian approach to regression with inequality constraints on the
coefficients is to estimate the regression without constraints, sample from
the distribution of the coefficients, discard draws that violate the
inequality constraints, and finally compute summary statistics from the
remaining subsample.  Andrew Gelman et al. explain how to do that in
Bayesian Data Analysis.  I believe they implemented their procedures in
S-Plus.  If any one has written similar programs in R, please let us know.

----- Original Message -----
From: Strumila, John <John.Strumila at team.telstra.com>
To: <R-help at stat.math.ethz.ch>
Sent: Thursday, January 04, 2001 4:56 PM
Subject: [R] regression constraints?

            

      
      
      
    

        
-.-.-

            

      
      
      
    

        
http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html

            

      
      
      
    

        
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._.
_._

-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To: r-help-request at stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._

  
  


  


      

    

    

      
      

        


  

        

      John P. Burkett
    

    

      
      #
    

  


  

      
A Bayesian approach to regression with inequality constraints on 
coefficients is to first estimate the regression without constraints, 
then sample from the distribution of the coefficients, discarding all 
draws that violate the constraints, and finally calculate summary 
statistics from the subsample that is consistent with the constraints.  
Andrew Gelman et al. explain how to do that in their Bayesian Data 
Analysis.  I believe they implemented their procedures in S-Plus.  If 
any one has written similar programs in R, I would like very much to 
hear them.

        
Adam Gehr wrote:

        

            

      
      
      
    

        

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Thomas Stockton
    

    

      
      #
    

  


  

      
Neptune and Company is currently looking for statisticians who are interested in
working with a dynamic group of statisticians and other scientists to perform
multi-disciplinary work in the environmental arena.  We are looking for
expertise in the following areas:  statistical planning and design, data
analysis, data mining, spatial statistics, decision analysis, Bayesian methods,
and probabilistic modeling of environmental systems.  Expertise in S-Plus/R is
strongly preferred, and familiarity with other statistical software packages
such as Arcview, spreadsheet and data base software, modeling software,
web-based tools and computer graphics is desired.  Knowledge of environmental
regulations is a plus.  Excellent written and interpersonal communication skills
are necessary to excel in this position. The successful candidate will be able
to convey statistical concepts to technical and non-technical audiences, will be
adept at abstract problem solving, and will have a strong desire to work with
others to solve environmental problems.  The statisticians we are seeking will
be self-starters who are detail minded and broad thinking.  MS/Ph.D or
equivalent experience is required.  Potential job locations are Denver,
Washington DC, Albuquerque, or Los Alamos.

For further information on Neptune and Company check our web site:
www.neptuneandco.com

Send resume, cover letter and salary requirements to
Paul Black
2031 Kerr Gulch Road
Evergreen, CO  80439
or send email to pblack at neptuneandco.com.

-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To: r-help-request at stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._