Skip to content

R 2.8.0 qqnorm produces error with object of class zoo?

14 messages · Achim Zeileis, Peter Dalgaard, Pfaff, Bernhard Dr. +1 more

Original
Pfaff, Bernhard Dr.
# 
Dear list-reader,

by running the following script:

library(zoo)
sessionInfo()
search()
packageDescription("zoo")
data(EuStockMarkets)
dax <- as.zoo(EuStockMarkets[1:10, "DAX"])
daxr <- diff(log(dax))
identical(as.vector(qnorm(daxr)), qnorm(coredata(daxr)))
qqnorm(coredata(daxr))
qqnorm(daxr)


qqnorm() produces an error:
Fehler in if (xi == xj) 0L else if (xi > xj) 1L else -1L : 
  Argument hat L?nge 0
Calls: qqnorm ... xtfrm.default -> as.vector -> rank -> switch -> .gt
Ausf?hrung angehalten


However, this is not witnessed by running the script through R 2.7.2 (attached are the two output files). Incidentally, the QQ-plot is produced if one uses "qqnorm(coredata(daxr))" under R version 2.8.0. After having read both NEWS on CRAN (General and Windows specific) I still could not figure out what causes this result. The version of package zoo is the same and the results do not change if I use the zoo package built under R version 2.7.2 with R version 2.7.2 or the one that has been built under R version 2.8.0.

Any hints and enlightenment are most welcome.

Best,
Bernhard


*****************************************************************
Confidentiality Note: The information contained in this message,
and any attachments, may contain confidential and/or privileged
material. It is intended solely for the person(s) or entity to
which it is addressed. Any review, retransmission, dissemination,
or taking of any action in reliance upon this information by
persons or entities other than the intended recipient(s) is
prohibited. If you received this in error, please contact the
sender and delete the material from any computer.
*****************************************************************
Pfaff, Bernhard Dr.
# 
It seems, that in my previous emails the attached output files got deleted, hence these are now copied below:

For R version 2.7.2:
====================
R version 2.7.2 (2008-08-25)
Copyright (C) 2008 The R Foundation for Statistical Computing
ISBN 3-900051-07-0

R ist freie Software und kommt OHNE JEGLICHE GARANTIE.
Sie sind eingeladen, es unter bestimmten Bedingungen weiter zu verbreiten.
Tippen Sie 'license()' or 'licence()' f?r Details dazu.

R ist ein Gemeinschaftsprojekt mit vielen Beitragenden.
Tippen Sie 'contributors()' f?r mehr Information und 'citation()',
um zu erfahren, wie R oder R packages in Publikationen zitiert werden k?nnen.

Tippen Sie 'demo()' f?r einige Demos, 'help()' f?r on-line Hilfe, oder
'help.start()' f?r eine HTML Browserschnittstelle zur Hilfe.
Tippen Sie 'q()', um R zu verlassen.


Invesco colors have been created!
Usage: "col = invcolor[1]", or "col = invcolor['blue']" etc.
Type: 'invcolor' to see its definition.

[Vorher gesicherter Workspace wiederhergestellt]
Attache Paket: 'zoo'


	The following object(s) are masked from package:base :

	 as.Date.numeric 

Warning message:
Paket 'zoo' wurde unter R Version 2.8.0 gebaut
R version 2.7.2 (2008-08-25) 
i386-pc-mingw32 

locale:
LC_COLLATE=German_Germany.1252;LC_CTYPE=German_Germany.1252;LC_MONETARY=German_Germany.1252;LC_NUMERIC=C;LC_TIME=German_Germany.1252

attached base packages:
[1] stats     graphics  utils     datasets  grDevices methods   base     

other attached packages:
[1] zoo_1.5-4

loaded via a namespace (and not attached):
[1] grid_2.7.2      lattice_0.17-13
[1] ".GlobalEnv"        "package:zoo"       "package:stats"    
 [4] "package:graphics"  "package:utils"     "package:datasets" 
 [7] "package:grDevices" "package:methods"   "Autoloads"        
[10] "package:base"
Package: zoo
Version: 1.5-4
Date: 2008-07-09
Title: Z's ordered observations
Author: Achim Zeileis, Gabor Grothendieck
Maintainer: Achim Zeileis <Achim.Zeileis at R-project.org>
Description: An S3 class with methods for totally ordered indexed
        observations. It is particularly aimed at irregular time series
        of numeric vectors/matrices and factors. zoo's key design goals
        are independence of a particular index/date/time class and
        consistency with with ts and base R by providing methods to
        extend standard generics.
Depends: R (>= 2.4.1), stats
Suggests: chron, fCalendar, fSeries, its, tseries, lattice,
        strucchange, DAAG, xts
Imports: stats, utils, graphics, grDevices, lattice
LazyLoad: yes
License: GPL-2
URL: http://R-Forge.R-project.org/projects/zoo/
Packaged: Wed Jul 9 18:28:07 2008; zeileis
Built: R 2.8.0; ; 2008-10-16 00:01:11; windows

-- File: C:/R/package/zoo/Meta/package.rds
[1] TRUE
Warning messages:
1: In qnorm(p, mean, sd, lower.tail, log.p) : NaNs wurden erzeugt
2: In qnorm(p, mean, sd, lower.tail, log.p) : NaNs wurden erzeugt
User      System verstrichen 
       1.65        0.13        2.97 

For R version 2.8.0:
====================
 version 2.8.0 (2008-10-20)
Copyright (C) 2008 The R Foundation for Statistical Computing
ISBN 3-900051-07-0

R ist freie Software und kommt OHNE JEGLICHE GARANTIE.
Sie sind eingeladen, es unter bestimmten Bedingungen weiter zu verbreiten.
Tippen Sie 'license()' or 'licence()' f?r Details dazu.

R ist ein Gemeinschaftsprojekt mit vielen Beitragenden.
Tippen Sie 'contributors()' f?r mehr Information und 'citation()',
um zu erfahren, wie R oder R packages in Publikationen zitiert werden k?nnen.

Tippen Sie 'demo()' f?r einige Demos, 'help()' f?r on-line Hilfe, oder
'help.start()' f?r eine HTML Browserschnittstelle zur Hilfe.
Tippen Sie 'q()', um R zu verlassen.


Invesco colors have been created!
Usage: "col = invcolor[1]", or "col = invcolor['blue']" etc.
Type: 'invcolor' to see its definition.

[Vorher gesicherter Workspace wiederhergestellt]
Attache Paket: 'zoo'


	The following object(s) are masked from package:base :

	 as.Date.numeric
R version 2.8.0 (2008-10-20) 
i386-pc-mingw32 

locale:
LC_COLLATE=German_Germany.1252;LC_CTYPE=German_Germany.1252;LC_MONETARY=German_Germany.1252;LC_NUMERIC=C;LC_TIME=German_Germany.1252

attached base packages:
[1] stats     graphics  utils     datasets  grDevices methods   base     

other attached packages:
[1] zoo_1.5-4

loaded via a namespace (and not attached):
[1] grid_2.8.0      lattice_0.17-15
[1] ".GlobalEnv"        "package:zoo"       "package:stats"    
 [4] "package:graphics"  "package:utils"     "package:datasets" 
 [7] "package:grDevices" "package:methods"   "Autoloads"        
[10] "package:base"
Package: zoo
Version: 1.5-4
Date: 2008-07-09
Title: Z's ordered observations
Author: Achim Zeileis, Gabor Grothendieck
Maintainer: Achim Zeileis <Achim.Zeileis at R-project.org>
Description: An S3 class with methods for totally ordered indexed
        observations. It is particularly aimed at irregular time series
        of numeric vectors/matrices and factors. zoo's key design goals
        are independence of a particular index/date/time class and
        consistency with with ts and base R by providing methods to
        extend standard generics.
Depends: R (>= 2.4.1), stats
Suggests: chron, fCalendar, fSeries, its, tseries, lattice,
        strucchange, DAAG, xts
Imports: stats, utils, graphics, grDevices, lattice
LazyLoad: yes
License: GPL-2
URL: http://R-Forge.R-project.org/projects/zoo/
Packaged: Wed Jul 9 18:28:07 2008; zeileis
Built: R 2.8.0; ; 2008-10-16 00:01:11; windows

-- File: C:/R/package/zoo/Meta/package.rds
[1] TRUE
Warning messages:
1: In qnorm(p, mean, sd, lower.tail, log.p) : NaNs wurden erzeugt
2: In qnorm(p, mean, sd, lower.tail, log.p) : NaNs wurden erzeugt
Fehler in if (xi == xj) 0L else if (xi > xj) 1L else -1L : 
  Argument hat L?nge 0
Calls: qqnorm ... xtfrm.default -> as.vector -> rank -> switch -> .gt
Ausf?hrung angehalten
Peter Dalgaard
#
Pfaff, Bernhard Dr. wrote:
(If the MIME type is wrong, then that will happen.)

Anyways, the root cause seems to be the new function .gt() which is
related to

    o   New generic function xtfrm() as an auxiliary helper for
        sort(), order() and rank().  This should return a numeric
        vector that sorts in the same way as its input.  The default
        method supports any class with ==, > and is.na() methods but
        specific methods can be much faster.

        As a side-effect, rank() will now work better on classed
        objects, although possibly rather slowly.

Here, "better" may be in the eyes of the beholder, for
Data:
logical(0)

Index:
integer(0)

and accordingly
Error in if (xi == xj) 0L else if (xi > xj) 1L else -1L :
  argument is of length zero

which is the error that you are seeing.

What to do about it is a bit dubious. Obviously, we don't want to "fix"
.gt() so that it automatically unclasses objects, and I assume that zoo
has its reasons for not wanting to compare series with different
indices. So I suppose that either the user must unclass, or zoo define
rank.zoo.

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Pfaff, Bernhard Dr.
    

    

      
      #
    

  


  

      
Dear Peter,

many thanks for your swift reply and the enlightenment; much appreciated.  
I have cc'ed Achim as the package maintainer of zoo, in case this thread should have slipt his notice. Just curious, Achim, will there be a rank.zoo() function in a future release of your package?

Best,
Bernhard

            

      
      
      
    

        

      
      
    

            

      
      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Gabor Grothendieck
    

    

      
      #
    

  


  

      
xtfrm.zoo has been added to the development version of zoo and should
address this.  It is available on R-Forge or in the meantime you can just
add this line to your code:

xtfrm.zoo <- coredata


On Wed, Oct 22, 2008 at 8:49 AM, Peter Dalgaard

        
<P.Dalgaard at biostat.ku.dk> wrote:

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Achim Zeileis
    

    

      
      #
    

  


  

      
On Wed, 22 Oct 2008, Pfaff, Bernhard Dr. wrote:

        

            

      
      
      
    

        
...and I have added Gabor, just in case.

            

      
      
      
    

        
I'm not sure - my first reaction would be "no". With time series, there 
might be confusion between sorting/ordering/ranking with respect to the 
observations vs. with respect to the time index. To clarify that you want 
some operation with respect to the data, you can easily say
   sort(coredata(z))
etc. This is rather evocative and not that much longer to type than 
sort(z). And it would be unclear what the latter should mean exactly.

In the case of rank(z), one could argue that it is meaningful...but 
providing rank() without sort() or order() methods might again be 
confusing. My first impression would be that the cons outweigh the pros.
Z

  
  


  


      

    

    

      
      

        


  

        

      Gabor Grothendieck
    

    

      
      #
    

  


  

      
On Wed, Oct 22, 2008 at 8:49 AM, Peter Dalgaard

        
<P.Dalgaard at biostat.ku.dk> wrote:

            

      
      
      
    

        
Actually qqnorm does not use rank but it does use order and with the
xtfrm.zoo method I mentioned qqnorm works with zoo; however, I think
rank needs to be fixed in R to make use of xtfrm as well since I would have
expected that supplying an xtfrm method for zoo would be sufficient to
get both order and rank to work without giving errors.  Also note that rank
is not generic.

  
  


  


      

    

    

      
      

        


  

        

      Peter Dalgaard
    

    

      
      #
    

  


  

      
Gabor Grothendieck wrote:

            

      
      
      
    

        
Notice that xtfrm.default() uses rank()....

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Gabor Grothendieck
    

    

      
      #
    

  


  

      
Yes, I noticed that but rank is not generic.  An xtfrm.zoo
method has been added to zoo on R-Forge but rank still
fails:

            

      
      
      
    

        
[1] "R version 2.8.0 Patched (2008-10-21 r46766)"

            

      
      
      
    

        
[1] "1.5-3"

            

      
      
      
    

        
[1] 1 2 3 4

            

      
      
      
    

        
Error in if (xi == xj) 0L else if (xi > xj) 1L else -1L :
  argument is of length zero

            

      
      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Gabor Grothendieck
    

    

      
      #
    

  


  

      
And one other point.

z <- zoo(1:4)
.gt(z, 1, 2)

fails because z[1] and z[2] are at different time points so

   z[1] == z[2]

is logical(0) because when zoo compares objects it aligns them
first.

On Wed, Oct 22, 2008 at 2:19 PM, Gabor Grothendieck

        
<ggrothendieck at gmail.com> wrote:

            

      
      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Peter Dalgaard
    

    

      
      #
    

  


  

      
Gabor Grothendieck wrote:

            

      
      
      
    

        
Yes, that was the point that I was trying to make. Well, arguably it 
doesn't "fail", it just does what it is supposed to do. Things would 
"work" with [[ or a preceding unclass(z), but that would break 
comparisons involving, say, POSIXlt objects. So you're sort of stuck 
between a rock and a hard place.

            

      
      
      
    

        

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Gabor Grothendieck
    

    

      
      #
    

  


  

      
I don't think its hopeless.  order works ok provided the underlying class
defines an xtfrm method.  I think rank should follow that route too.  Its
arguably the inconsistency between rank and order (order but not the
rank uses xtfrm) that causes the inconsistent behavior between the two.
If rank were also built on top of xtfrm then it would work as desired as
well.

On Wed, Oct 22, 2008 at 5:03 PM, Peter Dalgaard

        
<p.dalgaard at biostat.ku.dk> wrote:

            

      
      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Pfaff, Bernhard Dr.
    

    

      
      #
    

  


  

      
Dear Achim, Gabor, Jeff, Peter and remaining list-subscribers,

first, please accept my apologies for having started this thread. Given the avalanche of responses (some off-list) and the associated time stamps, some of you have almost pulled an all-nighter about this topic. I might have not have started this thread in the first place, if I would have known this.

I agree with Achim's view that a "not working for zoo objects" strategy is preferable compared to a "half-working for zoo objects" strategy.  I do not have either a problem by employing coredata(z) when necessary. Now, Gabor, you pointed out nicely that the culprit, namely that order() resides on top of xtfrm whereas rank() does not (this was voiced by you in an email to Peter and R-Devel, hence I inlucde these recipients again in this thread and the reason for doing this is also motivated by the following proposition):

You further pointed out that the problems wrt to rank and zoo-objects could be solved if rank() would, like order() does, reside on top of xtfrm. My question/proposal would then be to follow this approach, i.e. use xtfrm in rank. Now, I am not that deep into R nor an expert to judge whether this would cause problems/breaking existing R code in other instances; hence I appreciate feedback if this would be a feasible/desirable change in R-Devel.


Best,
Bernhard

            

      
      
      
    

        
*****************************************************************
Confidentiality Note: The information contained in this ...{{dropped:10}}

  
  


  


      

    

    

      
      

        


  

        

      Gabor Grothendieck
    

    

      
      #
    

  


  

      
Based on the quote that Peter gave:

   o   New generic function xtfrm() as an auxiliary helper for
       sort(), order() and rank().  This should return a numeric
       vector that sorts in the same way as its input.  The default
       method supports any class with ==, > and is.na() methods but
       specific methods can be much faster.

       As a side-effect, rank() will now work better on classed
       objects, although possibly rather slowly.

it seems that it was the intention to have rank(x) use x only via xtfrm(x)
but that is not, in fact, how it works as defining xtfrm.zoo still
results in the
discussed error.

On Thu, Oct 23, 2008 at 5:13 AM, Pfaff, Bernhard Dr.

        
<Bernhard_Pfaff at fra.invesco.com> wrote: