An embedded and charset-unspecified text was scrubbed... Name: not available URL: <https://stat.ethz.ch/pipermail/r-help/attachments/20110508/d5a1087f/attachment.pl>
Another quantmod question
10 messages · Russ Abbott, David Winsemius, Joshua Wiley +2 more
On May 8, 2011, at 3:07 PM, Russ Abbott wrote:
I'm having troubles with the names of columns. quantmod deal with stock quotes. I've created an array of the first 5 closing prices from Jan 2007. (Is there a problem that the name is the same as the variable name? There shouldn't be.)
close
close
2007-01-03 1416.60
2007-01-04 1418.34
2007-01-05 1409.71
2007-01-08 1412.84
2007-01-09 1412.11
When I try to create a more complex array by adding columns, the
names get
fouled up. Here's a simple example.
cbind(changed.close = close+1, zero = 0, close)
I suspect that you are actually using xts objects that you are
incorrectly calling 'array's. If something is puzzling about the
behavior of an R object the first thing to do is see what you are
really dealing with so ... str(object)
If you load the xts package and type ?cbind.xts , you get a help page
for merge.xts.
(In base R I do not know of a way to assign columns the way you
propose within a `merge` call.)
Here is the code for cbind.xts:
> cbind.xts
function (..., all = TRUE, fill = NA, suffixes = NULL)
{
merge.xts(..., all = all, fill = fill, suffixes = suffixes)
}
<environment: namespace:xts>
close zero close.1
2007-01-03 1417.60 0 1416.60
2007-01-04 1419.34 0 1418.34
2007-01-05 1410.71 0 1409.71
2007-01-08 1413.84 0 1412.84
2007-01-09 1413.11 0 1412.11
The first column should be called "changed.close", but it's called
"close".
The second column has the right name. The third column should be
called
"close" but it's called "close.1". Why is that? Am I missing
something?
If I change the order of the columns and let close have its original
name,
there is still a problem.
cbind(close, zero = 0, changed.close = close+1)
close zero close.1
2007-01-03 1416.60 0 1417.60
2007-01-04 1418.34 0 1419.34
2007-01-05 1409.71 0 1410.71
2007-01-08 1412.84 0 1413.84
2007-01-09 1412.11 0 1413.11
Now the names on the first two columns are ok, but the third column
is still
wrong. Again, why is that? Apparently it's not letting me assign a
name to
a column that comes from something that already has a name. Is that
the way
it should be?
I don't get that same problem on a simpler example.
IX <- cbind(I=0, X=(1:3))
IX
I X
[1,] 0 1
[2,] 0 2
[3,] 0 3
cbind(Y = 1, Z = IX[, "I"], W = IX[, "X"])
Y Z W
[1,] 1 0 1
[2,] 1 0 2
[3,] 1 0 3
Is this a peculiarity to xts objects?
Thanks.
*-- Russ *
*
*
P.S. Once again I feel frustrated because it's taken me far more
time than
it deserves to track down and characterize this problem. I can fix
it by
using the names function. But I shouldn't have to do that.
[[alternative HTML version deleted]]
______________________________________________ R-help at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
David Winsemius, MD West Hartford, CT
Hi Russ,
On Sun, May 8, 2011 at 12:07 PM, Russ Abbott <russ.abbott at gmail.com> wrote:
I'm having troubles with the names of columns. quantmod deal with stock quotes. ?I've created an array of the first 5 closing prices from Jan 2007. (Is there a problem that the name is the same as the variable name? There shouldn't be.)
close
? ? ? ? ? ? close 2007-01-03 1416.60 2007-01-04 1418.34 2007-01-05 1409.71 2007-01-08 1412.84 2007-01-09 1412.11
It would be appreciated in the future if you provided the object via dput() or some such that is easy to paste in.
When I try to create a more complex array by adding columns, the names get fouled up. ?Here's a simple example.
cbind(changed.close = close+1, zero = 0, close)
? ? ? ? ? ? close zero close.1 2007-01-03 1417.60 ? ?0 1416.60 2007-01-04 1419.34 ? ?0 1418.34 2007-01-05 1410.71 ? ?0 1409.71 2007-01-08 1413.84 ? ?0 1412.84 2007-01-09 1413.11 ? ?0 1412.11 The first column should be called "changed.close", but it's called "close". The second column has the right name. The third column should be called "close" but it's called "close.1". Why is that? Am I missing something?
Yes. mat <- matrix(1:10, dimnames = list(NULL, "A")) cbind(X = 11:20, Y = mat + 1) cbind(X = 11:20, Y = mat[, "A"] + 1) In particular note that: class(mat) class(mat[, "A"]) It is silly to expect to be able to pass a single name for an entire matrix, while it makes complete sense that that would work for a vector. The problem with naming the column the same thing as the object containing it, is our limited human minds get ramfeezled (R does just fine, as you can see).
If I change the order of the columns and let close have its original name, there is still a problem.
cbind(close, zero = 0, changed.close = close+1)
? ? ? ? ? ? close zero close.1 2007-01-03 1416.60 ? ?0 1417.60 2007-01-04 1418.34 ? ?0 1419.34 2007-01-05 1409.71 ? ?0 1410.71 2007-01-08 1412.84 ? ?0 1413.84 2007-01-09 1412.11 ? ?0 1413.11 Now the names on the first two columns are ok, but the third column is still wrong. Again, why is that? ?Apparently it's not letting me assign a name to a column that comes from something that already has a name. ?Is that the way it should be? I don't get that same problem on a simpler example.
IX <- cbind(I=0, X=(1:3))
?> IX ? ? I X [1,] 0 1 [2,] 0 2 [3,] 0 3
cbind(Y = 1, Z = IX[, "I"], W = IX[, "X"])
? ? Y Z W [1,] 1 0 1 [2,] 1 0 2 [3,] 1 0 3 Is this a peculiarity to xts objects?
Nope, but do check the type of object you are working with---there are often different methods for different objects so I would not assumet that all things would work the same. Cheers, Josh
Thanks. *-- Russ * * * P.S. Once again I feel frustrated because it's taken me far more time than it deserves to track down and characterize this problem. I can fix it by using the names function. But I shouldn't have to do that. ? ? ? ?[[alternative HTML version deleted]]
Please post using plain text.
______________________________________________ R-help at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Joshua Wiley Ph.D. Student, Health Psychology University of California, Los Angeles http://www.joshuawiley.com/
An embedded and charset-unspecified text was scrubbed... Name: not available URL: <https://stat.ethz.ch/pipermail/r-help/attachments/20110508/c2afefe6/attachment.pl>
An embedded and charset-unspecified text was scrubbed... Name: not available URL: <https://stat.ethz.ch/pipermail/r-help/attachments/20110508/f520c93f/attachment.pl>
An embedded and charset-unspecified text was scrubbed... Name: not available URL: <https://stat.ethz.ch/pipermail/r-help/attachments/20110508/4677b3ae/attachment.pl>
Russ,
On May 8, 2011 6:29 PM, "Russ Abbott" <russ.abbott at gmail.com> wrote:
Hi Jeff, The xts class has some very nice features, and you have done a valuable service in developing it. My primary frustration is how difficult it seems to be to find out what went wrong when my code doesn't work. ?I've been writing quite sophisticated code for a fairly long time. It's not that I'm new to software development. The column name rule is a good example. ?I'm willing to live with the rule that column names are not changed for efficiency sake. ?What's difficult for me is that I never saw that rule anywhere before. ?Of course, I'm not an R expect. I've been using it for only a couple of months. But still, I would have expected to run into a rule like that. Worse, since the rule is in conflict with the explicit intent of cbind--one can name columns when using cbind; in fact the examples illustrate how to do it--it would really be nice of cbind would issue a warning when one attempts to rename a column in violation of that rule. ?Instead, cbind is silent, giving no hint about what went wrong.
Naming columns is not the explicit intent of cbind. The explicit intent is to combine objects by columns. Please don't overstate the case. While the examples for the generic show naming columns, neither ?cbind.zoo or ?cbind.xts have such examples. That's a hint.
It's those sorts of things that have caused me much frustration. And it's these sorts of things that seem pervasive in R. ?One never knows what one is dealing with. Did something not work because there is a special case rule that I haven't heard of? Did it not work because a special convenience was programmed into a function in a way that conflicted with normal use? ?Since these sorts of things seem to come up so often, I find myself feeling that there is no good way to track down problems, which leads to a sense of helplessness and confusion. That's not what one wants in a programming language.
If that's not what one wants, one can always write their own programming language. Seriously, it seems like you want to rant more than understand what's going on. You have the R and xts help pages and the source code. The "Note" section of help(cbind) tells you that the method dispatch is different. It even tells you what R source file to look at to see how dispatching is done. Compare the relevant source files from base::cbind and xts::cbind.xts, look at the "R Language Definition" manual to see how method dispatch is normally done. But you've been writing quite sophisticated code for a fairly long time, so I'm not telling you anything you don't know... you just don't think you should have to do the legwork.
-- Russ
-- Joshua Ulrich | FOSS Trading: www.fosstrading.com
On Sun, May 8, 2011 at 2:42 PM, Jeff Ryan <jeff.a.ryan at gmail.com> wrote:
Hi Russ, Colnames don't get rewritten if they already exist. The reason is due to performance and how cbind is written at the R level. It isn't perfect per se, but the complexity and variety of dispatch that can take place for cbind in R, as it isn't a generic, is quite challenging to get to behave as one may hope. ?After years of trying I'd say it is nearly impossible to do what you want without causing horrible memory issues on non trivial objects they are use in production systems **using** xts on objects with billions of rows. ?Your simple case that has a simple workaround would cost everyone using in the other 99.999% of cases to pay a recurring cost that isn't tolerable. If this is frustrating to you you should stop using the class. Jeff Jeffrey Ryan ? ?| ? ?Founder ? ?| ? ? <jeffrey.ryan at lemnica.com> jeffrey.ryan at lemnica.com www.lemnica.com On May 8, 2011, at 2:07 PM, Russ Abbott <russ.abbott at gmail.com> wrote: I'm having troubles with the names of columns. quantmod deal with stock quotes. ?I've created an array of the first 5 closing prices from Jan 2007. (Is there a problem that the name is the same as the variable name? There shouldn't be.)
close
? ? ? ? ? ? ?close 2007-01-03 1416.60 2007-01-04 1418.34 2007-01-05 1409.71 2007-01-08 1412.84 2007-01-09 1412.11 When I try to create a more complex array by adding columns, the names get fouled up. ?Here's a simple example.
cbind(changed.close = close+1, zero = 0, close)
? ? ? ? ? ? ?close zero close.1 2007-01-03 1417.60 ? ?0 1416.60 2007-01-04 1419.34 ? ?0 1418.34 2007-01-05 1410.71 ? ?0 1409.71 2007-01-08 1413.84 ? ?0 1412.84 2007-01-09 1413.11 ? ?0 1412.11 The first column should be called "changed.close", but it's called "close". The second column has the right name. The third column should be called "close" but it's called "close.1". Why is that? Am I missing something? If I change the order of the columns and let close have its original name, there is still a problem.
cbind(close, zero = 0, changed.close = close+1)
? ? ? ? ? ? ?close zero close.1 2007-01-03 1416.60 ? ?0 1417.60 2007-01-04 1418.34 ? ?0 1419.34 2007-01-05 1409.71 ? ?0 1410.71 2007-01-08 1412.84 ? ?0 1413.84 2007-01-09 1412.11 ? ?0 1413.11 Now the names on the first two columns are ok, but the third column is still wrong. Again, why is that? ?Apparently it's not letting me assign a name to a column that comes from something that already has a name. ?Is that the way it should be? I don't get that same problem on a simpler example.
IX <- cbind(I=0, X=(1:3))
?> IX ? ? ?I X [1,] 0 1 [2,] 0 2 [3,] 0 3
cbind(Y = 1, Z = IX[, "I"], W = IX[, "X"])
? ? ?Y Z W [1,] 1 0 1 [2,] 1 0 2 [3,] 1 0 3 Is this a peculiarity to xts objects? Thanks. *-- Russ * * * P.S. Once again I feel frustrated because it's taken me far more time than it deserves to track down and characterize this problem. I can fix it by using the names function. But I shouldn't have to do that.
? ? ? ?[[alternative HTML version deleted]]
______________________________________________ R-help at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
On Sun, May 8, 2011 at 1:17 PM, Russ Abbott <russ.abbott at gmail.com> wrote:
I understand Josh's example: mat <- matrix(1:10, dimnames = list(NULL, "A")) cbind(X = 11:20, Y = mat + 1) cbind(X = 11:20, Y = mat[, "A"] + 1) In the line,?cbind(X = 11:20, Y = mat + 1), it would be nice if an error or warning message were issued to the effect that the "Y = " part is ignored or not applicable or something to that effect. I still don't understand why cbind.xts doesn't do what I expect. ?If I ?try what Josh suggests on my xts example, the result is still confusing.
cbind.xts(close, A = close[,"close"])
close close.1
2007-01-03 1416.60 1416.60
2007-01-04 1418.34 1418.34
2007-01-05 1409.71 1409.71
2007-01-08 1412.84 1412.84
2007-01-09 1412.11 1412.11
If this helps:
dput(close)
structure(c(1416.6, 1418.34, 1409.71, 1412.84, 1412.11), .indexCLASS =
"Date", .indexTZ = structure("", .Names = "TZ"), src = "yahoo", updated =
structure(1304875186.625, class = c("POSIXct",
"POSIXt")), class = c("xts", "zoo"), index = structure(c(1167811200,
1167897600, 1167984000, 1168243200, 1168329600), tzone = structure("",
.Names = "TZ"), tclass = "Date"), .Dim = c(5L,
1L), .Dimnames = list(NULL, "close"))
It does. Thank you. It sounds like you want to keep the xts class, so this may not be useful to you, but if you are willing to drop it at the point you are cbind()ing things together: cbind(changed.close = as.matrix(close + 1)[, "close"], zero = 0, close) should do the trick. Cheers, Josh
I was also unable to convert Josh's matrix example to xts.
as.xts(mat, order.by = rownames())
Error in dimnames(x) : 'x' is missing The order.by parameter is required, but I don't understand the error message and haven't been able to figure out how to fix it. If this helps:
dput(mat)
structure(1:10, .Dim = c(10L, 1L), .Dimnames = list(NULL, "A")) This suggests that mat already has defined dimnames. So why does as.xts give an error message about dimnames? ?I was unable to figure out the problem by looking at any of the help files. In particular the help file for?as.xts.methods didn't seem to offer any clarifications--at least none that I could understand. -- Russ On Sun, May 8, 2011 at 12:26 PM, Joshua Wiley <jwiley.psych at gmail.com> wrote:
class(mat) class(mat[, "A"])
Joshua Wiley Ph.D. Student, Health Psychology University of California, Los Angeles http://www.joshuawiley.com/
An embedded and charset-unspecified text was scrubbed... Name: not available URL: <https://stat.ethz.ch/pipermail/r-help/attachments/20110508/5936bcfe/attachment.pl>
An embedded and charset-unspecified text was scrubbed... Name: not available URL: <https://stat.ethz.ch/pipermail/r-help/attachments/20110508/5a848e39/attachment.pl>