portfolio.optim and error in solve.QP: matrix D not positive definite

The problem could be due to few reasons the papers by Higham & Rebonato
are a good read as to what the recipe does. The simplest recipe is to
set the negative eigenvalues to a small positive number and rescale the
matrix.

There are a few causes for this if you create corr matrices from
bivariate estimation and slap them together that may not be PD. In some
cases in derivative pricing the matrix is hand glued together from
implied correlations between two assets. Now a matrix of such implied
corrs does not have to be PD and fails.

Sample corr matrices are by definition PD however often times if you
have a lot of missing data(illiquid names in your universe?) and if you
do a na.locf (creating a const asset) this could do it as well.

It might be useful to do multivariate imputation with some o the R
packages rather than deletion or carrying fwd on your data.

There are other reasons besides all of the above for your corr matrix
being non PD. I think if you are working with such a large universe it
might be easier to have factor corr matrices using PCA or ICA. This way
if you manage to label the factors then you can see what bets your
optimizer is taking.


HTH

Best
Krishna



On Jan 29, 2011, at 3:52 PM, "Lui ##" <lui.r.project at googlemail.com> wrote:

Hello everybody,

sorry for my delayed "thanks" note - I was travelling.

@Arun: Debugging the underlying code is a little bit difficult since
the optimizer was written in FORTRAN. I think going for the nearest PD
(as Krishna also suggested) might be the best way. However, I honestly
don't understand why it is not PD... Does anybody have an explanation
for that?
@Guy: The weird thing is that I got the error code without the t(x) in
the first place. t(x) solved the problem (for some assets) and the
result indicated that it took a look at the assets and not the
observations... I am going to give it a try with the covariance matrix
again and let you know if it worked out... strange though.
@Krishna: Thanks for your link! I think that really helps... I am
going to try it out! Do you have an explanation why this is a common
problem with a large number of assets?

Thank you! Have a nice weekend!

Lui

On Fri, Jan 28, 2011 at 3:51 PM, krishna <kriskumar at earthlink.net> wrote:

Also the link below might help with a large number of assets this is a
common problem.

https://stat.ethz.ch/pipermail/r-sig-finance/2008q3/002854.html


Cheers
Krishna


On Jan 27, 2011, at 12:03 PM, Guy Yollin <gyollin at r-programming.org>
wrote:

Hi Lui,

Without seeing the data this is just speculation but...

Are you sure you want t(x)? If you're mixing up your observations
versus
your assets this may explain the error.

The first parameter of portfolio.optim (in the tseries package) is a
returns matrix, one column for each asset and one row for each day
(assuming
daily returns). If you have this wrong then for your small datasets
you'd
have more columns than rows and this could produce that error.

Also, you don't have to pass the entire returns matrix to
portfolio.optim,
you could pass just the covariance matrix you calculate yourself and a
vector (1-row matrix) of mean returns as follows:


library(tseries)
set.seed(2)
R <- matrix(rnorm(100*10),nrow=100,ncol=10) # 10 assets, 100
observations
averet <- matrix(apply(R,2,mean),nrow=1)
rcov <- cov(R)
current_er <- 0.05
(op <- portfolio.optim(x=averet,pm=current_er,covmat=rcov,riskless =
FALSE,shorts = FALSE, rf = 0.0))

Hope this helps.

Best,

Guy


On 1/26/2011 7:51 PM, Lui ## wrote:

Dear Group,

I have a large set of stocks and want to determine the efficient
frontier. The data set covers approx. 1.5 years and S&P 500 companies
(nothing weird). portfolio.optim from the PerformanceAnalytics package
works very well and fast. However, whenever I decrease the number of
stocks in the portfolio (to 10 or 400), I receive an error message:

"solve.QP(Dmat, dvec, Amat, bvec = b0, meq = 2) :
matrix D in quadratic function is not positive definite!"

My command settings for portfolio.optim were:

seed<- portfolio.optim(t(x), pm = current_er, riskless = FALSE,
shorts = FALSE, rf = 0.0)

Even when I tried it with shorts = TRUE the error would still remain.
x is the set of stocks (stocks in columns, time in rows), current_er
is the target return (lies between the minimal mean and the maximum
mean of a long only portfolio).
I can not post the stock data here - so maybe you have some general
suggestions for me of what could have gone wrong... The covariance
matrix is positive definite. What could cause the problem? It works
fine with the large data set but does not work at all with the small
one...
Thanks a lot for your suggestions!

Lui