Skip to content

Possible POSIXlt / wday glitch & bugs.r-project.org status

11 messages · Joshua Ulrich, Scott Kostyshak, Imanuel Costigan +2 more

#
Wanted to raise two questions:

1. Is bugs.r-project.org down? I haven't been able to reach it for two or three days:

```
ping bugs.r-project.org
PING rbugs.research.att.com (207.140.168.137): 56 data bytes
Request timeout for icmp_seq 0
Request timeout for icmp_seq 1
Request timeout for icmp_seq 2
Request timeout for icmp_seq 3
Request timeout for icmp_seq 4
Request timeout for icmp_seq 5
Request timeout for icmp_seq 6
```

2. Is wday element of POSIXlt meant to be timezone invariant? You would expect the wday element to be invariant to the timezone of a date. That is, the same date/time instant of 5th October 2013 in both Australia/Sydney and UTC should be a Saturday (i.e. wday = 6). And indeed that is the case with 1 min past midnight on 5 October 2013:

```
library(lubridate)
d_utc <- ymd_hms(20131005000001, tz='UTC')
d_local <- ymd_hms(20131005000001, tz='Australia/Sydney')
as.POSIXlt(x=d_utc, tz=tz(d_utc))$wday # 6
as.POSIXlt(x=d_local, tz=tz(d_local))$wday # 6
```

But this isn't always the case. For example,

```
d_utc <- ymd_hms(20381002000001, tz='UTC')
d_local <- ymd_hms(20381002000001, tz='Australia/Sydney')
as.POSIXlt(x=d_utc, tz=tz(d_utc))$wday # 6
as.POSIXlt(x=d_local, tz=tz(d_local))$wday # 5
```

Is this expected behaviour? I would have expected a properly encoded date/time of 2 Oct 2038 to be a Saturday irrespective of its time zone.

Obligatory system dump:

```
R version 3.0.1 (2013-05-16)
Platform: x86_64-apple-darwin12.4.0 (64-bit)

locale:
[1] en_AU.UTF-8/en_AU.UTF-8/en_AU.UTF-8/C/en_AU.UTF-8/en_AU.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] lubridate_1.3.0 testthat_0.7.1  devtools_1.3   

loaded via a namespace (and not attached):
 [1] colorspace_1.2-4   dichromat_2.0-0    digest_0.6.3       evaluate_0.5.1    
 [5] ggplot2_0.9.3.1    grid_3.0.1         gtable_0.1.2       httr_0.2          
 [9] labeling_0.2       MASS_7.3-29        memoise_0.1        munsell_0.4.2     
[13] parallel_3.0.1     plyr_1.8           proto_0.3-10       RColorBrewer_1.0-5
[17] RCurl_1.95-4.1     reshape2_1.2.2     scales_0.2.3       stringr_0.6.2     
[21] tools_3.0.1        whisker_0.3-2     

```

Using R compiled by homebrew [1]. But also experiencing the same bug using R installed on Windows 7 from the CRAN binaries.

For those interested, I've also noted this on the `lubridate` Github issues page [2], even though this doesn't appear to be a lubridate issue.

Thanks for any help.

[1] http://brew.sh
[2] https://github.com/hadley/lubridate/issues/209
#
Quoting from ?timezone:

     Note that except on Windows, the operation of time zones is an OS
     service, and even on Windows a third-party database is used and
     can be updated (see the section on ?Time zone names?).  Incorrect
     results will never be an R issue, so please ensure that you have
     the courtesy not to blame R for them.

Best,
--
Joshua Ulrich  |  about.me/joshuaulrich
FOSS Trading  |  www.fosstrading.com
On Fri, Oct 4, 2013 at 5:11 AM, Imanuel Costigan <i.costigan at me.com> wrote:
#
On Fri, Oct 4, 2013 at 6:11 AM, Imanuel Costigan <i.costigan at me.com> wrote:
Yes. Quote from Duncan:

    ... the server is currently down. The volunteer who runs the server is
    currently away from his office, so I expect it won't get fixed until he
    gets back in a few days.

https://stat.ethz.ch/pipermail/r-help/2013-October/360958.html

Scott
--
Scott Kostyshak
Economics PhD Candidate
Princeton University
#
Thanks for the responses and quoting the timezone help file. 

I am assuming that in order to determine the wday element of POSIXlt, R does the necessary calculations in Julian time (via POSIXct). Based on this excerpt from ?DateTimeClasses, it looks like R is responsible for determining time zones post 2037 (the example I gave was in 2038). So it could be an R issue.

        
On 05/10/2013, at 12:59 AM, Scott Kostyshak <skostysh at princeton.edu> wrote:

            
#
On Fri, Oct 4, 2013 at 8:02 PM, Imanuel Costigan <i.costigan at me.com> wrote:
It's an issue with size of the largest number you can store in a
signed integer, which is not specific to R.
[1] "2038-01-19 03:14:07 UTC"

Dates larger than that cannot be represented by a signed integer.  It
could be worked around, but it's not trivial because R would have to
use something other than the tm C struct.  Luckily, there's a decade
or two before it starts to become a pressing issue. :)
#
Ok thanks for that explanation. That's bad news for me. I need to generate dates out past that limit regularly. It means that many operations past that limit are unreliable. Any suggested work arounds / alternatives?
On 05/10/2013, at 11:37 AM, Joshua Ulrich <josh.m.ulrich at gmail.com> wrote:

            
#
On Oct 4, 2013, at 4:59 PM, Scott Kostyshak <skostysh at princeton.edu> wrote:

            
FWIW it's available at

https://rbugs.urbanek.info/bugzilla3/

until the official R-project.org DNS is updated (but I have no idea how long that may take).

Cheers,
Simon
#
Some people (luckily not me anymore!) working with mortgages and
pensions need to calculate up to 40 years into the future for the
payment schedule.
On 5 October 2013 02:37, Joshua Ulrich <josh.m.ulrich at gmail.com> wrote:
#
On Oct 5, 2013, at 4:51 PM, Sean O'Riordain <seanpor at acm.org> wrote:

            
Just to clarify since the Joshua's comment was ambiguous (and in part plain wrong) - R's POSIXct has no such limit since it doesn't use integers, so that is not really the issue here. As the original post suggested there may be a bug in handing some cases where conversions hit the system libraries (that may truncate to integers) and some cases may be worked around - and that remains to be investigated.

Cheers,
Simon
#
Thanks for the clarification Simon. Having a look at the code in datetime.c, I can't say that I envy its maintainer. Looks like there are a lot of "hacks" to get things working. 

For the time being, I am going to override the TZ environment variable to be UTC as at this time I don't need to worry about converting between time zones - although I will in the future.

Also - I can't submit a new bug to your website's hosting of R bugs. The "submit new bug" page doesn't load. I also get a certificate error when visiting your site's mirror.

Thanks!
On 06/10/2013, at 5:18 AM, Simon Urbanek <simon.urbanek at r-project.org> wrote:

            
9 days later
#
In an effort to redeem myself, I have found and submitted a patch for
what seems to be causing this issue.

Best,
--
Joshua Ulrich  |  about.me/joshuaulrich
FOSS Trading  |  www.fosstrading.com


On Sat, Oct 5, 2013 at 1:18 PM, Simon Urbanek
<simon.urbanek at r-project.org> wrote: