random portfolios
The process you describe is pretty standard for an investment-committee driven process. I'm going to suggest that you don't really want to change the constraints that often. For example, box constraints should be as large as your overall investment mandate allows to give you the greatest possible room for allocations. Sector or Factor constraints likewise should be as minimal as possible just to guarantee the degree of diversification described in your investment mandate. The reason I'm suggesting this minimal constraint set is one of the reasons we wrote the random portfolio code in the first place. To see what I mean, generate a set of unconstrained random portfolios (or e.g. only with a full-investment constraint). Then generate sets of constrained random portfolios, adding your various constraint sets. Plot the different sets on the same risk/return scatter plot, using different colors for each set. Note how small the feasible space becomes, very quickly. This shrinkage of the feasible space has some good shrinkage properties... moderate shrinkage actually decreases the possible impact of estimation error in the various inputs, a little. Large amounts of shrinkage (overly restrictive constraints) will do the opposite, and magnify the negative out of sample impact of estimation error. The academic literature mostly focuses on analytical solvers (e.g. quadratic, linear, etc) and simple constraint sets. We've cited papers by Patrick Burns as well as papers on the simplex models in PortfolioAnalytics, but the literature is not vast. Numerical solvers become important as the feasible space becomes non-smooth. One of the things that can create a non-smooth feasible space is a complex, overlapping constraint set. The rportfolios package proposed by Frederick Novomestky also seems to be an R-only implementation, at a glance relying on truncated random binomial vectors rather than truncated random uniform vectors. I believe it will have similar performance characteristics to the Burns-style random sample portfolios, and it seems to support fewer constraint sets (no overlapping sector, group, or factor constraints that I see). In any case, it generates matrices of weights that are likely compatible with the PortfolioAnalytics random or seed portfolio inputs. So if it works for you, that's great. You also discuss using rejection after generating the portfolios. This is the method used internally by random.portfolios to reject individual weights if a constraint is violated. I'll have to evaluate whether the truncdist package used by rportfolios could be more efficient than the runif that is used by the current code. PortfolioAnalytics also allows portfolios to be penalized in the solver, so that more complex cases can be considered, or interactions between constraints and objectives. To answer the question of whether Rcpp will help is somewhat complex. I'm confident that some of the nested loops in the generation code will be sped up by Rcpp. It is possible that more efficient algorithms are available for constructing the weight vectors. A reason that this hasn't been a huge priority though is that construction of the random portfolio matrix is usually not the time limiter in a large optimization: your objective function is. I think it will be possible to improve the efficiency of this step, though it is unclear how much of an impact this should have in practice to a large and complicated numerically solved portfolio optimization problem. Regards, Brian
On 03/20/2017 07:06 PM, Kevin Dhingra wrote:
Brian, Yes I think that will be a good starting point. My universe would not change a lot (I will be working with 10-15 benchmarks at a time and I guess I can generate a reusable set for each independently before running it through my main algorithm). Having said that, I envision the investment mandates/constraints changing quite a lot (both in the cross section and also over time for the same manager). I am hoping there must be a way around it using rejection sampling but have not done enough research to comment on how that solution works for such big dimensions. It will be really helpful if you could point me to any specific resources from academia for the same (Haven't been able to find much about random portfolios myself except Portfolio Analytics and Patrick Burns work on Portfolio Probe). As a side note - Do you think translating it using Rcpp would be time well spent or you think there must be a smarter way to get around it still using R? I really appreciate your help on this thread. Regards, Kshitij Dhingra On Mon, Mar 20, 2017 at 7:21 PM, Brian G. Peterson <brian at braverock.com> wrote:
For this type of problem, I would probably generate one set of random portfolios and just reuse that set of feasible portfolios... My usual rule is n-assets + 1-2k feasible portfolios. You can get a better number e.g. from sampling theory, but this should be enough. Once you have this weights matrix rp, you only need to regenerate rp if your universe changes. Still interested in a more efficient implementation, of course, or we can work with you to see if we can find resources to work on it, e.g. from academia. Regards, Brian On 03/20/2017 05:28 PM, Kevin Dhingra wrote:
Hi Ross, Sure. Even though I have not profiled the bottlenecks quite in detail as of yet, i will give you a decent idea of the problem I am working with. I can have multiple indices with as much as 2000 assets with group, position and turnover limits (Not sure if i can increase the speed by removing constraints and doing rejection sampling later). In order to generate a daily possible set for the market in this case, I was playing around with ~4-5 thousand permutations. Also I think I will end up using the "sample" method because of the type of constraints we have and as you already have mentioned that method is the slowest (takes about 30 times the time using "simplex" for the same constraints). Adding box and position limit constraints are causing it to run a bit slower (but its not a big difference). I can always provide a more thorough analysis of the potential bottlenecks with a lot more detail when I have a chance to start working on translating it to cpp Thank you, On Mon, Mar 20, 2017 at 4:04 PM, Ross Bennett <rossbennett34 at gmail.com> wrote: Kevin,
Can you give us a sense of the number of assets in the portfolio and the constraints? That will help us understand where the potential bottlenecks are in the random portfolio generation. For example, generating a set of random portfolios for box and weight constraints if relatively fast, but adding group or position limit constraints makes the algorithm more complicated and slower. Thanks, Ross On Mon, Mar 20, 2017 at 2:35 PM, Kevin Dhingra <kevin.dhingra at appliedacademics.com> wrote:
Brian, Thank you for a quick reply. I will soon be working on that problem and from what I have played with so far, it is unlikely that for our example ~2k portfolios will be enough (really hoping it would) to get a good
sense
of the feasible space and seems like I need to implement an Rcpp version
of
the random portfolios function. I will be happy to collaborate and share
my
code once i get a decent handle on it locally for the purposes of our current project. Regards, Kshitij Dhingra On Mon, Mar 20, 2017 at 3:17 PM, Brian G. Peterson <brian at braverock.com
wrote: On Mon, 2017-03-20 at 15:09 -0400, Kevin Dhingra wrote:
I have been using the random_portfolios function from the `PortfolioAnalytics` package to simulate the range of possibilities for return paths at each step under various portfolio constraints / mandates for evaluating mutual fund managers. As more managers are added to the universe, however, and more simulations are needed, the pure R implementations get pretty heavy and hard to scale. I was wondering if there has been any work out there thus far on implementing any of the three random portfolio generation methods (sample, simplex, and grid search) at a lower level, using something like `Rcpp` to enhance the efficiency of these algorithms?
We've discussed it, but I can't say that it is terribly high on our list of priorities. In most cases, no more than 1-2k portfolios should be required to get a fair view of the feasible space given your constraints and objectives. We'd be happy to work with you if you want to craft a patch to use C or Rcpp for this. Regards, Brian
-- Kshitij Dhingra Applied Academics LLC Office: +1.917.262.0516 Mobile: +1.206.696.5945 Email: kshitij.dhingra at appliedacademics.com Website: http://www.AppliedAcademics.com [[alternative HTML version deleted]]
_______________________________________________ R-SIG-Finance at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-finance -- Subscriber-posting only. If you want to post, subscribe first. -- Also note that this is not the r-help list where general R questions
_______________________________________________ R-SIG-Finance at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-finance -- Subscriber-posting only. If you want to post, subscribe first. -- Also note that this is not the r-help list where general R questions should go.
Brian G. Peterson http://braverock.com/brian/ Ph: 773-459-4973 IM: bgpbraverock