Filling a covariance matrix

Hi Eric,
useRs ?
I?m working with the attached data that contains one year?s worth of
sub-daily observations of flow (?Q?) and specific conductance (?SC?, a
surrogate for concentration) at a point in a stream.  The R code posted
below shows the extent of data processing thus far.  My goal is to create a
covariance matrix that takes on the following form:
      Q1 Q2 ? Q365  SC1 SC2 ? SC365
Q1
Q2
?
Q365
SC1
SC2
...
SC365

Where the covariance between Q1 (flow on day 1) & Q2 (flow on day 2) is
determined using the sub-daily data contained in the variable ?x? of the R
code below.  Similarly, the covariance between Q1 & SC1 (specific
conductance on day 1) would be made using the sub-daily observations of flow
and specific conductance.  Covariance between observations that are more
than 5 days distant from one another are likely meaningless.  Thus, the
covariance matrix should reflect this limitation with zeros.  For example,
the covariance between Q1 & Q6, or between Q1 & SC6, or between SC359
(specific conductance on day 359) & SC365 (specific conductance on day 365)
would be zero as these observations are more than 5 days apart.  Here is the
R code that reads the attached files containing Q and SC and puts the
processed data into ?x?:

07130500_BelowJM_q_2004.txt
<http://r.789695.n4.nabble.com/file/n4647170/07130500_BelowJM_q_2004.txt>
07130500_BelowJM_2004.txt
<http://r.789695.n4.nabble.com/file/n4647170/07130500_BelowJM_2004.txt>

library(xts)
Q_subDaily<-read.table("C:/temp/07130500_BelowJM_q_2004.rdb",col.names=c('date','time','tz','Q','rating','unknown'),colClasses=c("character","character","character","numeric","character","character"))
SC_subDaily<-read.table("C:/temp/07130500_BelowJM_2004.rdb",col.names=c('date','time','tz','SC','rating','unknown'),colClasses=c("character","character","character","numeric","character","character"))

Q_subDaily$datetime.str <- paste(Q_subDaily$date, Q_subDaily$time)
SC_subDaily$datetime.str <- paste(SC_subDaily$date, SC_subDaily$time)

fmt <- "%Y%m%d %H%M%S"
xQ <- xts(as.matrix(Q_subDaily["Q"]), as.POSIXct(Q_subDaily$datetime.str,
format=fmt))
xSC <- xts(as.matrix(SC_subDaily["SC"]),
as.POSIXct(SC_subDaily$datetime.str, format=fmt))

x <- merge(xQ,xSC)

And here?s where I?m stuck, I?m not sure how to create the covariance matrix
I?ve described above?  I would appreciate and greatly benefit from the sort
of help often found in the useR community.

Thanks for the reproducible example.  I don't have time to provide a
complete solution, but this should get you started and hopefully give
you some ideas:

# create a list of xts objects that only contain data for one day
xs <- split(x, "days")
# initialize the covariance matrix
xcov <- matrix(0, length(xs)*2, length(xs)*2)
rownames(xcov) <- colnames(xcov) <-
  c(sprintf("Q%d",seq_along(xs)), sprintf("SC%d",seq_along(xs)))
# use a double-for loop to fill it in
for(i in seq_along(xs)) {
  for(j in seq_along(xs)) {
    xcov[paste0("Q",i),paste0("Q",j)] <- # same as below
    xcov[paste0("Q",j),paste0("Q",i)] <- cov(xs[[i]]$Q, xs[[j]]$Q, use="pair")
    xcov[paste0("Q",i),paste0("SC",j)] <- # same as below
    xcov[paste0("Q",j),paste0("SC",i)] <- cov(xs[[i]]$Q, xs[[j]]$SC, use="pair")
    xcov[paste0("SC",i),paste0("SC",j)] <- # same as below
    xcov[paste0("SC",j),paste0("SC",i)] <- cov(xs[[i]]$SC, xs[[j]]$SC,
use="pair")
  }
}

--
View this message in context: http://r.789695.n4.nabble.com/Filling-a-covariance-matrix-tp4647170.html
Sent from the R help mailing list archive at Nabble.com.

Please include some context if you reply via nabble.  This is a
mailing list, not a forum, and nabble doesn't quote the previous post
by default, which can be frustrating for mailing list subscribers.
______________________________________________
R-help at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Best,
--
Joshua Ulrich  |  about.me/joshuaulrich
FOSS Trading  |  www.fosstrading.com

Filling a covariance matrix

Thread (3 messages)