Hi, I just started exploring PortfolioAnalytics package. Similar to setting up custom objective functions, is there a way to set up custom constraints too? I would like to know how to set up cardinality constraint (i.e., limiting number of assets in a portfolio). Thanks!
PortfolioAnalytics: Custom Constraint
6 messages · Abhay Bhadani, Ross Bennett, Brian G. Peterson
On Mon, 2016-09-19 at 20:22 +0530, Abhay Bhadani wrote:
I just started exploring PortfolioAnalytics package. Similar to setting up custom objective functions, is there a way to set up custom constraints too? I would like to know how to set up cardinality constraint (i.e., limiting number of assets in a portfolio).
cardinality constraints are already supported via the 'position_limit'
constraint which is an integer constraint limiting the maximum number of
non-zero weight positions in the portfolio. It may be added like this:
pspec <- add.constraint(portfolio=pspec,
type="position_limit",
max_pos=3,
enabled=TRUE)
assuming that your portfolio specification object is 'pspec'.
As with other constraint types, this may not be efficiently supported by
all optimization engines. (This is a limitation of the underlying
optimizers/solvers, not of PortfolioAnalytics).
On a more general note, any constraint may be expressed as an objective
by creating a penalty for violating the constraint. As noted above,
this may lead to very inefficient or non-converging optimization.
Regards,
Brian
Brian G. Peterson http://braverock.com/brian/ Ph: 773-459-4973 IM: bgpbraverock
Thanks, Brian!
I implemented the following:
----------------------------------------------------------------------------------------------
data("edhec")
returns <- edhec[,1:12]
colnames(returns) <-
c("CA","CTAG","DS","EM","EN","ED","FIA","GMLS","MA","RV","SS","FF")
print(head(returns,5))
fund.names <- colnames(returns)
#Giving Portfolio Specifications
pspec <- portfolio.spec(assets=fund.names)
print.default(pspec)
#Adding Constraints
#Full investment constraint: sum of all x_i is 1
pspec <- add.constraint(portfolio= pspec, type =
"weight_sum",min=0.99,max=1.01)
#Box constraint: value of x_i varies between 0.2 to 0.8
pspec <- add.constraint(portfolio= pspec, type="box", min = 0.01, max =
0.25)
#Cardinality constraint
pspec <- add.constraint(portfolio= pspec, type="position_limit",max_pos =
6,enabled=TRUE)
#Adding Objective
pspec <- add.objective(portfolio=pspec, type="risk", name="var")
pspec <- add.objective(portfolio=pspec, type="return", name="mean")
opt_meanvar <- optimize.portfolio(R=returns,portfolio = pspec,
optimize_method="pso", trace=TRUE)
------------------------------------------------------------------------------------------
Results:
Optimal Weights:
CA CTAG DS EM EN ED FIA GMLS MA RV
SS FF
0.1319 0.0551 0.0312 0.1312 0.1127 0.1467 0.0641 0.0218 0.0805 0.1101
0.0296 0.0952
_____________________________________________________
I obtained weights for all 12 assets. That's why I was not sure whether
position_limit constraint is same as cardinality constraint.
On Mon, Sep 19, 2016 at 8:32 PM, Brian G. Peterson <brian at braverock.com>
wrote:
On Mon, 2016-09-19 at 20:22 +0530, Abhay Bhadani wrote:
I just started exploring PortfolioAnalytics package. Similar to setting up custom objective functions, is there a way to set
up
custom constraints too? I would like to know how to set up cardinality constraint (i.e., limiting number of assets in a portfolio).
cardinality constraints are already supported via the 'position_limit'
constraint which is an integer constraint limiting the maximum number of
non-zero weight positions in the portfolio. It may be added like this:
pspec <- add.constraint(portfolio=pspec,
type="position_limit",
max_pos=3,
enabled=TRUE)
assuming that your portfolio specification object is 'pspec'.
As with other constraint types, this may not be efficiently supported by
all optimization engines. (This is a limitation of the underlying
optimizers/solvers, not of PortfolioAnalytics).
On a more general note, any constraint may be expressed as an objective
by creating a penalty for violating the constraint. As noted above,
this may lead to very inefficient or non-converging optimization.
Regards,
Brian
--
Brian G. Peterson
http://braverock.com/brian/
Ph: 773-459-4973
IM: bgpbraverock
Hi Abhay, The cardinality constraint is not directly support by the pso solver so we have to implement that constraint as a penalized objective. Also note that your box constraint is greater than 0, so the box constraint and position limit constraint are fighting each other (i.e your problem is overconstrained). I recommend using DEoptim or random portfolios as a solver when dealing with position limit constraints. Position limit constraints are supported directly by the algorithm in the random portfolios 'sample' method. We use the mapping function supported by DEoptim to handle more complex constraints such as position limit when using DEoptim as the solver. Hope that helps. Ross
On Mon, Sep 19, 2016 at 10:25 AM, Abhay Bhadani <abhadani at gmail.com> wrote:
Thanks, Brian!
I implemented the following:
------------------------------------------------------------
----------------------------------
data("edhec")
returns <- edhec[,1:12]
colnames(returns) <-
c("CA","CTAG","DS","EM","EN","ED","FIA","GMLS","MA","RV","SS","FF")
print(head(returns,5))
fund.names <- colnames(returns)
#Giving Portfolio Specifications
pspec <- portfolio.spec(assets=fund.names)
print.default(pspec)
#Adding Constraints
#Full investment constraint: sum of all x_i is 1
pspec <- add.constraint(portfolio= pspec, type =
"weight_sum",min=0.99,max=1.01)
#Box constraint: value of x_i varies between 0.2 to 0.8
pspec <- add.constraint(portfolio= pspec, type="box", min = 0.01, max =
0.25)
#Cardinality constraint
pspec <- add.constraint(portfolio= pspec, type="position_limit",max_pos =
6,enabled=TRUE)
#Adding Objective
pspec <- add.objective(portfolio=pspec, type="risk", name="var")
pspec <- add.objective(portfolio=pspec, type="return", name="mean")
opt_meanvar <- optimize.portfolio(R=returns,portfolio = pspec,
optimize_method="pso", trace=TRUE)
------------------------------------------------------------
------------------------------
Results:
Optimal Weights:
CA CTAG DS EM EN ED FIA GMLS MA RV
SS FF
0.1319 0.0551 0.0312 0.1312 0.1127 0.1467 0.0641 0.0218 0.0805 0.1101
0.0296 0.0952
_____________________________________________________
I obtained weights for all 12 assets. That's why I was not sure whether
position_limit constraint is same as cardinality constraint.
On Mon, Sep 19, 2016 at 8:32 PM, Brian G. Peterson <brian at braverock.com>
wrote:
On Mon, 2016-09-19 at 20:22 +0530, Abhay Bhadani wrote:
I just started exploring PortfolioAnalytics package.
Similar to setting up custom objective functions, is there a way to set
up
custom constraints too?
I would like to know how to set up cardinality constraint (i.e.,
limiting
number of assets in a portfolio).
cardinality constraints are already supported via the 'position_limit'
constraint which is an integer constraint limiting the maximum number of
non-zero weight positions in the portfolio. It may be added like this:
pspec <- add.constraint(portfolio=pspec,
type="position_limit",
max_pos=3,
enabled=TRUE)
assuming that your portfolio specification object is 'pspec'.
As with other constraint types, this may not be efficiently supported by
all optimization engines. (This is a limitation of the underlying
optimizers/solvers, not of PortfolioAnalytics).
On a more general note, any constraint may be expressed as an objective
by creating a penalty for violating the constraint. As noted above,
this may lead to very inefficient or non-converging optimization.
Regards,
Brian
--
Brian G. Peterson
http://braverock.com/brian/
Ph: 773-459-4973
IM: bgpbraverock
[[alternative HTML version deleted]]
_______________________________________________
R-SIG-Finance at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions
should go.
The attached .R file should demonstrate what Ross was talking about. The cardinality constraint is directly supported by the stochastic global solvers 'DEoptim' and 'random', as described in the documentation. It can also be formulated as a mixed integer linear problem for certain objective functions, but because you're looknig for the Markowitz mean/variance portfolio, you have a quadratic problem, and can't formulate it using an MILP. The attached file fixes the minimum position box constraint to 0, which aalows all solvers to converge. pso still doesn't get a good solution, but this solution may be improved by increasing the maximum number of iterations via the parameter 'maxit'. After increasing maxit, it gets closer to the solutions returned by 'random' and 'DEoptim'. Both 'random' and 'DEoptim' produce similar solutions which meet the constraints and objectives, but they are not identical, and change over multiple runs. (This is stochastic optimization, after all, so the only way to get the same answer would be to artificially, and incorrectly, set the random seed.). This suggests two things: 1> that there are not enough random portfolios (for 'random') or that the population size and number of generations are insufficient to find the global solution (for DEoptim). 2> that the problem may still be somewhat over constrained, most likely by the max weight box constraint. You might want to use random portfolios and portfolio sets to compare and plot the difference between the unconstrained search space and the constrained search space to sort out whether this is really what you want to do. Regards, Brian
Brian G. Peterson http://braverock.com/brian/ Ph: 773-459-4973 IM: bgpbraverock On Tue, 2016-09-20 at 07:18 -0500, Ross Bennett wrote: > Hi Abhay, > > The cardinality constraint is not directly support by the pso solver so we > have to implement that constraint as a penalized objective. Also note that > your box constraint is greater than 0, so the box constraint and position > limit constraint are fighting each other (i.e your problem is > overconstrained). > > I recommend using DEoptim or random portfolios as a solver when dealing > with position limit constraints. Position limit constraints are supported > directly by the algorithm in the random portfolios 'sample' method. We use > the mapping function supported by DEoptim to handle more complex > constraints such as position limit when using DEoptim as the solver. > > Hope that helps. > > Ross > > > On Mon, Sep 19, 2016 at 10:25 AM, Abhay Bhadani <abhadani at gmail.com> wrote: > > > Thanks, Brian! > > > > I implemented the following: > > ------------------------------------------------------------ > > ---------------------------------- > > data("edhec") > > returns <- edhec[,1:12] > > colnames(returns) <- > > c("CA","CTAG","DS","EM","EN","ED","FIA","GMLS","MA","RV","SS","FF") > > print(head(returns,5)) > > fund.names <- colnames(returns) > > > > > > #Giving Portfolio Specifications > > pspec <- portfolio.spec(assets=fund.names) > > print.default(pspec) > > > > #Adding Constraints > > #Full investment constraint: sum of all x_i is 1 > > pspec <- add.constraint(portfolio= pspec, type = > > "weight_sum",min=0.99,max=1.01) > > > > #Box constraint: value of x_i varies between 0.2 to 0.8 > > pspec <- add.constraint(portfolio= pspec, type="box", min = 0.01, max = > > 0.25) > > > > #Cardinality constraint > > pspec <- add.constraint(portfolio= pspec, type="position_limit",max_pos = > > 6,enabled=TRUE) > > > > #Adding Objective > > pspec <- add.objective(portfolio=pspec, type="risk", name="var") > > pspec <- add.objective(portfolio=pspec, type="return", name="mean") > > > > opt_meanvar <- optimize.portfolio(R=returns,portfolio = pspec, > > optimize_method="pso", trace=TRUE) > > > > ------------------------------------------------------------ > > ------------------------------ > > > > Results: > > > > Optimal Weights: > > CA CTAG DS EM EN ED FIA GMLS MA RV > > SS FF > > 0.1319 0.0551 0.0312 0.1312 0.1127 0.1467 0.0641 0.0218 0.0805 0.1101 > > 0.0296 0.0952 > > > > _____________________________________________________ > > > > I obtained weights for all 12 assets. That's why I was not sure whether > > position_limit constraint is same as cardinality constraint. > > > > > > On Mon, Sep 19, 2016 at 8:32 PM, Brian G. Peterson <brian at braverock.com> > > wrote: > > > > > On Mon, 2016-09-19 at 20:22 +0530, Abhay Bhadani wrote: > > > > I just started exploring PortfolioAnalytics package. > > > > > > > > Similar to setting up custom objective functions, is there a way to set > > > up > > > > custom constraints too? > > > > > > > > I would like to know how to set up cardinality constraint (i.e., > > limiting > > > > number of assets in a portfolio). > > > > > > cardinality constraints are already supported via the 'position_limit' > > > constraint which is an integer constraint limiting the maximum number of > > > non-zero weight positions in the portfolio. It may be added like this: > > > > > > > > > pspec <- add.constraint(portfolio=pspec, > > > type="position_limit", > > > max_pos=3, > > > enabled=TRUE) > > > > > > assuming that your portfolio specification object is 'pspec'. > > > > > > As with other constraint types, this may not be efficiently supported by > > > all optimization engines. (This is a limitation of the underlying > > > optimizers/solvers, not of PortfolioAnalytics). > > > > > > On a more general note, any constraint may be expressed as an objective > > > by creating a penalty for violating the constraint. As noted above, > > > this may lead to very inefficient or non-converging optimization. > > > > > > Regards, > > > > > > Brian > > > > > > -- > > > Brian G. Peterson > > > http://braverock.com/brian/ > > > Ph: 773-459-4973 > > > IM: bgpbraverock > > > > > > > > > > > > > > > > > > > [[alternative HTML version deleted]] > > > > _______________________________________________ > > R-SIG-Finance at r-project.org mailing list > > https://stat.ethz.ch/mailman/listinfo/r-sig-finance > > -- Subscriber-posting only. If you want to post, subscribe first. > > -- Also note that this is not the r-help list where general R questions > > should go. > > > > [[alternative HTML version deleted]] > > _______________________________________________ > R-SIG-Finance at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-sig-finance > -- Subscriber-posting only. If you want to post, subscribe first. > -- Also note that this is not the r-help list where general R questions should go. -------------- next part -------------- A non-text attachment was scrubbed... Name: cardinality_test.R Type: text/x-r-source Size: 1590 bytes Desc: not available URL: <https://stat.ethz.ch/pipermail/r-sig-finance/attachments/20160920/04b22810/attachment.bin>
1 day later
Thank you, Brian and Ross. That was very helpful. DEoptim and random solvers gave good results. I tested the same using GenSA, which performed worse than pso. Thank you again! :) On Tue, Sep 20, 2016 at 8:53 PM, Brian G. Peterson <brian at braverock.com> wrote:
The attached .R file should demonstrate what Ross was talking about. The cardinality constraint is directly supported by the stochastic global solvers 'DEoptim' and 'random', as described in the documentation. It can also be formulated as a mixed integer linear problem for certain objective functions, but because you're looknig for the Markowitz mean/variance portfolio, you have a quadratic problem, and can't formulate it using an MILP. The attached file fixes the minimum position box constraint to 0, which aalows all solvers to converge. pso still doesn't get a good solution, but this solution may be improved by increasing the maximum number of iterations via the parameter 'maxit'. After increasing maxit, it gets closer to the solutions returned by 'random' and 'DEoptim'. Both 'random' and 'DEoptim' produce similar solutions which meet the constraints and objectives, but they are not identical, and change over multiple runs. (This is stochastic optimization, after all, so the only way to get the same answer would be to artificially, and incorrectly, set the random seed.). This suggests two things: 1> that there are not enough random portfolios (for 'random') or that the population size and number of generations are insufficient to find the global solution (for DEoptim). 2> that the problem may still be somewhat over constrained, most likely by the max weight box constraint. You might want to use random portfolios and portfolio sets to compare and plot the difference between the unconstrained search space and the constrained search space to sort out whether this is really what you want to do. Regards, Brian -- Brian G. Peterson http://braverock.com/brian/ Ph: 773-459-4973 IM: bgpbraverock On Tue, 2016-09-20 at 07:18 -0500, Ross Bennett wrote:
Hi Abhay, The cardinality constraint is not directly support by the pso solver so
we
have to implement that constraint as a penalized objective. Also note
that
your box constraint is greater than 0, so the box constraint and position limit constraint are fighting each other (i.e your problem is overconstrained). I recommend using DEoptim or random portfolios as a solver when dealing with position limit constraints. Position limit constraints are supported directly by the algorithm in the random portfolios 'sample' method. We
use
the mapping function supported by DEoptim to handle more complex constraints such as position limit when using DEoptim as the solver. Hope that helps. Ross On Mon, Sep 19, 2016 at 10:25 AM, Abhay Bhadani <abhadani at gmail.com>
wrote:
Thanks, Brian!
I implemented the following:
------------------------------------------------------------
----------------------------------
data("edhec")
returns <- edhec[,1:12]
colnames(returns) <-
c("CA","CTAG","DS","EM","EN","ED","FIA","GMLS","MA","RV","SS","FF")
print(head(returns,5))
fund.names <- colnames(returns)
#Giving Portfolio Specifications
pspec <- portfolio.spec(assets=fund.names)
print.default(pspec)
#Adding Constraints
#Full investment constraint: sum of all x_i is 1
pspec <- add.constraint(portfolio= pspec, type =
"weight_sum",min=0.99,max=1.01)
#Box constraint: value of x_i varies between 0.2 to 0.8
pspec <- add.constraint(portfolio= pspec, type="box", min = 0.01, max =
0.25)
#Cardinality constraint
pspec <- add.constraint(portfolio= pspec,
type="position_limit",max_pos =
6,enabled=TRUE)
#Adding Objective
pspec <- add.objective(portfolio=pspec, type="risk", name="var")
pspec <- add.objective(portfolio=pspec, type="return", name="mean")
opt_meanvar <- optimize.portfolio(R=returns,portfolio = pspec,
optimize_method="pso", trace=TRUE)
------------------------------------------------------------
------------------------------
Results:
Optimal Weights:
CA CTAG DS EM EN ED FIA GMLS MA RV
SS FF
0.1319 0.0551 0.0312 0.1312 0.1127 0.1467 0.0641 0.0218 0.0805 0.1101
0.0296 0.0952
_____________________________________________________ I obtained weights for all 12 assets. That's why I was not sure whether position_limit constraint is same as cardinality constraint. On Mon, Sep 19, 2016 at 8:32 PM, Brian G. Peterson <
brian at braverock.com>
wrote:
On Mon, 2016-09-19 at 20:22 +0530, Abhay Bhadani wrote:
I just started exploring PortfolioAnalytics package. Similar to setting up custom objective functions, is there a way
to set
up
custom constraints too? I would like to know how to set up cardinality constraint (i.e.,
limiting
number of assets in a portfolio).
cardinality constraints are already supported via the
'position_limit'
constraint which is an integer constraint limiting the maximum
number of
non-zero weight positions in the portfolio. It may be added like
this:
pspec <- add.constraint(portfolio=pspec,
type="position_limit",
max_pos=3,
enabled=TRUE)
assuming that your portfolio specification object is 'pspec'.
As with other constraint types, this may not be efficiently
supported by
all optimization engines. (This is a limitation of the underlying optimizers/solvers, not of PortfolioAnalytics). On a more general note, any constraint may be expressed as an
objective
by creating a penalty for violating the constraint. As noted above, this may lead to very inefficient or non-converging optimization. Regards, Brian -- Brian G. Peterson http://braverock.com/brian/ Ph: 773-459-4973 IM: bgpbraverock
[[alternative HTML version deleted]]
_______________________________________________ R-SIG-Finance at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-finance -- Subscriber-posting only. If you want to post, subscribe first. -- Also note that this is not the r-help list where general R questions should go.
[[alternative HTML version deleted]]
_______________________________________________ R-SIG-Finance at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-finance -- Subscriber-posting only. If you want to post, subscribe first. -- Also note that this is not the r-help list where general R questions
should go.