Message-ID: <CAPPM_gQCBXYBjqdTiGtWU8r7brY9UPpUte+bd=wro1rVEB0hPQ@mail.gmail.com>
Date: 2012-10-27T17:15:00Z
From: Joshua Ulrich
Subject: Filling a covariance matrix
In-Reply-To: <1351007949106-4647170.post@n4.nabble.com>
Hi Eric,
On Tue, Oct 23, 2012 at 10:59 AM, emorway <emorway at usgs.gov> wrote:
> useRs ?
> I?m working with the attached data that contains one year?s worth of
> sub-daily observations of flow (?Q?) and specific conductance (?SC?, a
> surrogate for concentration) at a point in a stream. The R code posted
> below shows the extent of data processing thus far. My goal is to create a
> covariance matrix that takes on the following form:
> Q1 Q2 ? Q365 SC1 SC2 ? SC365
> Q1
> Q2
> ?
> Q365
> SC1
> SC2
> ...
> SC365
>
> Where the covariance between Q1 (flow on day 1) & Q2 (flow on day 2) is
> determined using the sub-daily data contained in the variable ?x? of the R
> code below. Similarly, the covariance between Q1 & SC1 (specific
> conductance on day 1) would be made using the sub-daily observations of flow
> and specific conductance. Covariance between observations that are more
> than 5 days distant from one another are likely meaningless. Thus, the
> covariance matrix should reflect this limitation with zeros. For example,
> the covariance between Q1 & Q6, or between Q1 & SC6, or between SC359
> (specific conductance on day 359) & SC365 (specific conductance on day 365)
> would be zero as these observations are more than 5 days apart. Here is the
> R code that reads the attached files containing Q and SC and puts the
> processed data into ?x?:
>
> 07130500_BelowJM_q_2004.txt
> <http://r.789695.n4.nabble.com/file/n4647170/07130500_BelowJM_q_2004.txt>
> 07130500_BelowJM_2004.txt
> <http://r.789695.n4.nabble.com/file/n4647170/07130500_BelowJM_2004.txt>
>
> library(xts)
> Q_subDaily<-read.table("C:/temp/07130500_BelowJM_q_2004.rdb",col.names=c('date','time','tz','Q','rating','unknown'),colClasses=c("character","character","character","numeric","character","character"))
> SC_subDaily<-read.table("C:/temp/07130500_BelowJM_2004.rdb",col.names=c('date','time','tz','SC','rating','unknown'),colClasses=c("character","character","character","numeric","character","character"))
>
> Q_subDaily$datetime.str <- paste(Q_subDaily$date, Q_subDaily$time)
> SC_subDaily$datetime.str <- paste(SC_subDaily$date, SC_subDaily$time)
>
> fmt <- "%Y%m%d %H%M%S"
> xQ <- xts(as.matrix(Q_subDaily["Q"]), as.POSIXct(Q_subDaily$datetime.str,
> format=fmt))
> xSC <- xts(as.matrix(SC_subDaily["SC"]),
> as.POSIXct(SC_subDaily$datetime.str, format=fmt))
>
> x <- merge(xQ,xSC)
>
> And here?s where I?m stuck, I?m not sure how to create the covariance matrix
> I?ve described above? I would appreciate and greatly benefit from the sort
> of help often found in the useR community.
>
Thanks for the reproducible example. I don't have time to provide a
complete solution, but this should get you started and hopefully give
you some ideas:
# create a list of xts objects that only contain data for one day
xs <- split(x, "days")
# initialize the covariance matrix
xcov <- matrix(0, length(xs)*2, length(xs)*2)
rownames(xcov) <- colnames(xcov) <-
c(sprintf("Q%d",seq_along(xs)), sprintf("SC%d",seq_along(xs)))
# use a double-for loop to fill it in
for(i in seq_along(xs)) {
for(j in seq_along(xs)) {
xcov[paste0("Q",i),paste0("Q",j)] <- # same as below
xcov[paste0("Q",j),paste0("Q",i)] <- cov(xs[[i]]$Q, xs[[j]]$Q, use="pair")
xcov[paste0("Q",i),paste0("SC",j)] <- # same as below
xcov[paste0("Q",j),paste0("SC",i)] <- cov(xs[[i]]$Q, xs[[j]]$SC, use="pair")
xcov[paste0("SC",i),paste0("SC",j)] <- # same as below
xcov[paste0("SC",j),paste0("SC",i)] <- cov(xs[[i]]$SC, xs[[j]]$SC,
use="pair")
}
}
>
>
>
> --
> View this message in context: http://r.789695.n4.nabble.com/Filling-a-covariance-matrix-tp4647170.html
> Sent from the R help mailing list archive at Nabble.com.
>
Please include some context if you reply via nabble. This is a
mailing list, not a forum, and nabble doesn't quote the previous post
by default, which can be frustrating for mailing list subscribers.
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
Best,
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com