Skip to content

Linear Regressions with constraint coefficients

11 messages · Aleksandrovic, Aljosa (Pfaeffikon), Bert Gunter, Gabor Grothendieck +1 more

#
Hi all,

I hope you are doing well?

I?m currently using the lm() function from the package stats to fit linear multifactor regressions.

Unfortunately, I didn?t yet find a way to fit linear multifactor regressions with constraint coefficients? I would like the slope coefficients to be all inside an interval, let?s say, between 0 and 1. Further, if possible, the slope coefficients should add up to 1.

Is there an elegant and not too complicated way to do such a constraint regression estimation in R?

I would very much appreciate if you could help me with my issue?

Thanks a lot in advance and kind regards, 
Aljosa Aleksandrovic



Aljosa Aleksandrovic, FRM, CAIA
Quantitative Analyst - Convertibles
aljosa.aleksandrovic at man.com
Tel +41 55 417 7603

Man Investments (CH) AG
Huobstrasse 3 | 8808 Pf?ffikon SZ | Switzerland


-----Original Message-----
From: Kevin E. Thorpe [mailto:kevin.thorpe at utoronto.ca] 
Sent: Dienstag, 26. April 2016 14:35
To: Aleksandrovic, Aljosa (Pfaeffikon)
Subject: Re: Linear Regressions with constraint coefficients

You need to send it to r-help at r-project.org however.

Kevin
On 04/26/2016 08:32 AM, Aleksandrovic, Aljosa (Pfaeffikon) wrote:

  
    
#
If the slope coefficients sum to a constant, the regressors are
dependent and so a unique solution is impossible (an infinity of
solutions would result). So I think you have something going on that
you don't understand and should consult a local statistician to help
you formulate your problem appropriately.

Cheers,
Bert


Bert Gunter

"The trouble with having an open mind is that people keep coming along
and sticking things into it."
-- Opus (aka Berkeley Breathed in his "Bloom County" comic strip )


On Tue, Apr 26, 2016 at 5:39 AM, Aleksandrovic, Aljosa (Pfaeffikon)
<Aljosa.Aleksandrovic at man.com> wrote:
#
Ok, and if I would just like to force my slope coefficients to be inside an interval, let's say, between 0 and 1? Is there a way in R to formulate such a constraint regression?

Thanks in advance and kind regards,
Aljosa



Aljosa Aleksandrovic, FRM, CAIA
Quantitative Analyst - Convertibles
aljosa.aleksandrovic at man.com
Tel +41 55 417 7603

Man Investments (CH) AG
Huobstrasse 3 | 8808 Pf?ffikon SZ | Switzerland


-----Original Message-----
From: Bert Gunter [mailto:bgunter.4567 at gmail.com] 
Sent: Dienstag, 26. April 2016 16:51
To: Aleksandrovic, Aljosa (Pfaeffikon)
Cc: r-help at r-project.org
Subject: Re: [R] Linear Regressions with constraint coefficients

If the slope coefficients sum to a constant, the regressors are dependent and so a unique solution is impossible (an infinity of solutions would result). So I think you have something going on that you don't understand and should consult a local statistician to help you formulate your problem appropriately.

Cheers,
Bert


Bert Gunter

"The trouble with having an open mind is that people keep coming along and sticking things into it."
-- Opus (aka Berkeley Breathed in his "Bloom County" comic strip )
On Tue, Apr 26, 2016 at 5:39 AM, Aleksandrovic, Aljosa (Pfaeffikon) <Aljosa.Aleksandrovic at man.com> wrote:
#
Have you tried web searching on " R constrained linear regression" or
similar. There seemed to be resources related to your issues when I
looked. You might also search on rseek.org  . There are apparently
several packages that do regression with constraints, but I don't know
if they fit your situation.

Cheers,
Bert


Bert Gunter

"The trouble with having an open mind is that people keep coming along
and sticking things into it."
-- Opus (aka Berkeley Breathed in his "Bloom County" comic strip )


On Tue, Apr 26, 2016 at 8:29 AM, Aleksandrovic, Aljosa (Pfaeffikon)
<Aljosa.Aleksandrovic at man.com> wrote:
#
Ok, will try!

Thanks a kind regards,
Aljosa


Aljosa Aleksandrovic, FRM, CAIA
Quantitative Analyst - Convertibles
aljosa.aleksandrovic at man.com
Tel +41 55 417 76 03

Man Investments (CH) AG
Huobstrasse 3 | 8808 Pf?ffikon SZ | Switzerland


-----Original Message-----
From: Bert Gunter [mailto:bgunter.4567 at gmail.com] 
Sent: Dienstag, 26. April 2016 17:49
To: Aleksandrovic, Aljosa (Pfaeffikon)
Cc: r-help at r-project.org
Subject: Re: [R] Linear Regressions with constraint coefficients

Have you tried web searching on " R constrained linear regression" or similar. There seemed to be resources related to your issues when I looked. You might also search on rseek.org  . There are apparently several packages that do regression with constraints, but I don't know if they fit your situation.

Cheers,
Bert


Bert Gunter

"The trouble with having an open mind is that people keep coming along and sticking things into it."
-- Opus (aka Berkeley Breathed in his "Bloom County" comic strip )
On Tue, Apr 26, 2016 at 8:29 AM, Aleksandrovic, Aljosa (Pfaeffikon) <Aljosa.Aleksandrovic at man.com> wrote:
#
This is a quadratic programming problem that you can solve using
either a quadratic programming solver with constraints or a general
nonlinear solver with constraints.  See
https://cran.r-project.org/web/views/Optimization.html
for more info on what is available.

Here is an example using a nonlinear least squares solver and
non-negative bound constraints. The constraint that the coefficients
sum to 1 is implied by dividing them by their sum and then dividing
the coefficients found by their sum at the end:

# test data
set.seed(123)
n <- 1000
X1 <- rnorm(n)
X2 <- rnorm(n)
X3 <- rnorm(n)
Y <- .2 * X1 + .3 * X2 + .5 * X3 + rnorm(n)

# fit
library(nlmrt)
fm <- nlxb(Y ~ (b1 * X1 + b2 * X2 + b3 * X3)/(b1 + b2 + b3),
     data = list(Y = Y, X1 = X1, X2 = X2, X3 = X3),
     lower = numeric(3),
     start = list(b1 = 1, b2 = 2, b3 = 3))

giving the following non-negative coefficients which sum to 1 that are
reasonably close to the true values of 0.2, 0.3 and 0.5:
b1      b2      b3
0.18463 0.27887 0.53650


On Tue, Apr 26, 2016 at 8:39 AM, Aleksandrovic, Aljosa (Pfaeffikon)
<Aljosa.Aleksandrovic at man.com> wrote:

  
    
#
Hi Gabor,

Thanks a lot for your input!

Kind regards,
Aljosa



Aljosa Aleksandrovic, FRM, CAIA
Quantitative Analyst - Convertibles
aljosa.aleksandrovic at man.com
Tel +41 55 417 76 03

Man Investments (CH) AG
Huobstrasse 3 | 8808 Pf?ffikon SZ | Switzerland


-----Original Message-----
From: Gabor Grothendieck [mailto:ggrothendieck at gmail.com] 
Sent: Dienstag, 26. April 2016 17:59
To: Aleksandrovic, Aljosa (Pfaeffikon)
Cc: r-help at r-project.org
Subject: Re: [R] Linear Regressions with constraint coefficients

This is a quadratic programming problem that you can solve using either a quadratic programming solver with constraints or a general nonlinear solver with constraints.  See https://cran.r-project.org/web/views/Optimization.html
for more info on what is available.

Here is an example using a nonlinear least squares solver and non-negative bound constraints. The constraint that the coefficients sum to 1 is implied by dividing them by their sum and then dividing the coefficients found by their sum at the end:

# test data
set.seed(123)
n <- 1000
X1 <- rnorm(n)
X2 <- rnorm(n)
X3 <- rnorm(n)
Y <- .2 * X1 + .3 * X2 + .5 * X3 + rnorm(n)

# fit
library(nlmrt)
fm <- nlxb(Y ~ (b1 * X1 + b2 * X2 + b3 * X3)/(b1 + b2 + b3),
     data = list(Y = Y, X1 = X1, X2 = X2, X3 = X3),
     lower = numeric(3),
     start = list(b1 = 1, b2 = 2, b3 = 3))

giving the following non-negative coefficients which sum to 1 that are reasonably close to the true values of 0.2, 0.3 and 0.5:
b1      b2      b3
0.18463 0.27887 0.53650
On Tue, Apr 26, 2016 at 8:39 AM, Aleksandrovic, Aljosa (Pfaeffikon) <Aljosa.Aleksandrovic at man.com> wrote:

  
    
#
Any help with exporting anova output in R to csv or xlsx?

      From: "Aleksandrovic, Aljosa (Pfaeffikon)" <Aljosa.Aleksandrovic at man.com>
 To: Bert Gunter <bgunter.4567 at gmail.com> 
Cc: "r-help at r-project.org" <r-help at r-project.org>
 Sent: Tuesday, April 26, 2016 8:29 AM
 Subject: Re: [R] Linear Regressions with constraint coefficients
   
Ok, and if I would just like to force my slope coefficients to be inside an interval, let's say, between 0 and 1? Is there a way in R to formulate such a constraint regression?

Thanks in advance and kind regards,
Aljosa



Aljosa Aleksandrovic, FRM, CAIA
Quantitative Analyst - Convertibles
aljosa.aleksandrovic at man.com
Tel +41 55 417 7603

Man Investments (CH) AG
Huobstrasse 3 | 8808 Pf?ffikon SZ | Switzerland


-----Original Message-----
From: Bert Gunter [mailto:bgunter.4567 at gmail.com] 
Sent: Dienstag, 26. April 2016 16:51
To: Aleksandrovic, Aljosa (Pfaeffikon)
Cc: r-help at r-project.org
Subject: Re: [R] Linear Regressions with constraint coefficients

If the slope coefficients sum to a constant, the regressors are dependent and so a unique solution is impossible (an infinity of solutions would result). So I think you have something going on that you don't understand and should consult a local statistician to help you formulate your problem appropriately.

Cheers,
Bert


Bert Gunter

"The trouble with having an open mind is that people keep coming along and sticking things into it."
-- Opus (aka Berkeley Breathed in his "Bloom County" comic strip )
On Tue, Apr 26, 2016 at 5:39 AM, Aleksandrovic, Aljosa (Pfaeffikon) <Aljosa.Aleksandrovic at man.com> wrote:
______________________________________________
R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
1 day later
#
Hi Gabor,

Thanks a lot for your help!

I tried to implement your nonlinear least squares solver on my data set. I was just wondering about the argument start. If I would like to force all my coefficients to be inside an interval, let?s say, between 0 and 1, what kind of starting values are normally recommended for the start argument (e.g. Using a 4 factor model with b1, b2, b3 and b4, I tried start = list(b1 = 0.5, b2 = 0.5, b3 = 0.5, b4 = 0.5))? I also tried other starting values ... Hence, the outputs are very sensitive to that start argument?     

Thanks a lot for your answer in advance!

Kind regards,
Aljosa



Aljosa Aleksandrovic, FRM, CAIA
Quantitative Analyst - Convertibles
aljosa.aleksandrovic at man.com
Tel +41 55 417 76 03

Man Investments (CH) AG
Huobstrasse 3 | 8808 Pf?ffikon SZ | Switzerland

-----Original Message-----
From: Gabor Grothendieck [mailto:ggrothendieck at gmail.com] 
Sent: Dienstag, 26. April 2016 17:59
To: Aleksandrovic, Aljosa (Pfaeffikon)
Cc: r-help at r-project.org
Subject: Re: [R] Linear Regressions with constraint coefficients

This is a quadratic programming problem that you can solve using either a quadratic programming solver with constraints or a general nonlinear solver with constraints.  See https://cran.r-project.org/web/views/Optimization.html
for more info on what is available.

Here is an example using a nonlinear least squares solver and non-negative bound constraints. The constraint that the coefficients sum to 1 is implied by dividing them by their sum and then dividing the coefficients found by their sum at the end:

# test data
set.seed(123)
n <- 1000
X1 <- rnorm(n)
X2 <- rnorm(n)
X3 <- rnorm(n)
Y <- .2 * X1 + .3 * X2 + .5 * X3 + rnorm(n)

# fit
library(nlmrt)
fm <- nlxb(Y ~ (b1 * X1 + b2 * X2 + b3 * X3)/(b1 + b2 + b3),
     data = list(Y = Y, X1 = X1, X2 = X2, X3 = X3),
     lower = numeric(3),
     start = list(b1 = 1, b2 = 2, b3 = 3))

giving the following non-negative coefficients which sum to 1 that are reasonably close to the true values of 0.2, 0.3 and 0.5:
b1      b2      b3
0.18463 0.27887 0.53650
On Tue, Apr 26, 2016 at 8:39 AM, Aleksandrovic, Aljosa (Pfaeffikon) <Aljosa.Aleksandrovic at man.com> wrote:

  
    
#
The nls2 package can be used to get starting values.

On Thu, Apr 28, 2016 at 8:42 AM, Aleksandrovic, Aljosa (Pfaeffikon)
<Aljosa.Aleksandrovic at man.com> wrote:

  
    
#
Thx a lot Gabor!

Aljosa Aleksandrovic, FRM, CAIA
Quantitative Analyst - Convertibles
aljosa.aleksandrovic at man.com
Tel +41 55 417 76 03

Man Investments (CH) AG
Huobstrasse 3 | 8808 Pf?ffikon SZ | Switzerland


-----Original Message-----
From: Gabor Grothendieck [mailto:ggrothendieck at gmail.com] 
Sent: Donnerstag, 28. April 2016 14:48
To: Aleksandrovic, Aljosa (Pfaeffikon)
Cc: r-help at r-project.org
Subject: Re: [R] Linear Regressions with constraint coefficients

The nls2 package can be used to get starting values.
On Thu, Apr 28, 2016 at 8:42 AM, Aleksandrovic, Aljosa (Pfaeffikon) <Aljosa.Aleksandrovic at man.com> wrote: