Skip to content

Floating Point with POSIXct

5 messages · Martin Maechler, John Muschelli, Duncan Murdoch

Original
John Muschelli
# 
I see in ?POSIXct and I'm trying to understand the note:
Mainly, I'm trying to understand printing of POSIXct with fractional
seconds.  I see print.POSIXct calls format.POSIXct and eventually
calls format.POSIXlt, which then takes into account `digits.secs` for
printing. The format uses %OS3, which strptime indicates (* added):
So I'm seeing it truncates the seconds to 3 digits, so I think that is
why the below is printing 0.024.

I think this is especially relevant even if you set
`options(digits.secs = 6)`, then the code in
format.POSIXlt would still return np=3 as the following condition
would break at i = 3

            for (i in seq_len(np) - 1L) if (all(abs(secs - round(secs,
                i)) < 1e-06)) {
                np <- i
                break
            }

as sub_seconds - round(sub_seconds,3) < 1e-06.   This seems to be
expected behavior given the docs, but would any consider this a bug?


Example:

options(digits.secs = 4)
x = structure(947016000.025, class = c("POSIXct", "POSIXt"), tzone = "UTC")
summary(x, digits = 20)
#>                      Min.                   1st Qu.                    Median
#> "2000-01-04 20:00:00.024" "2000-01-04 20:00:00.024" "2000-01-04 20:00:00.024"
#>                      Mean                   3rd Qu.                      Max.
#> "2000-01-04 20:00:00.024" "2000-01-04 20:00:00.024" "2000-01-04 20:00:00.024"
x
#> [1] "2000-01-04 20:00:00.024 UTC"
format.POSIXct(x, format = "%Y-%m-%d %H:%M:%OS3")
#> [1] "2000-01-04 20:00:00.024"
format.POSIXct(x, format = "%Y-%m-%d %H:%M:%OS4")
#> [1] "2000-01-04 20:00:00.0249"
sub_seconds = as.numeric(x) %% 1
sub_seconds
#> [1] 0.02499998
round(sub_seconds, 3)
#> [1] 0.025

rounded = as.POSIXct(
  floor(as.numeric(x)) +
    round(as.numeric(x) %% 1, 3),
  origin = "1970-01-01")
rounded
#> [1] "2000-01-04 20:00:00.024 UTC"
as.numeric(rounded) %% 1
#> [1] 0.02499998

R.version
               _
platform       x86_64-pc-linux-gnu
arch           x86_64
os             linux-gnu
system         x86_64, linux-gnu
status
major          4
minor          1.2
year           2021
month          11
day            01
svn rev        81115
language       R
version.string R version 4.1.2 (2021-11-01)
nickname       Bird Hippie



Best,
John
Martin Maechler
# 
> I see in ?POSIXct and I'm trying to understand the note:
    >> Classes "POSIXct" and "POSIXlt" are able to express fractions of a second. (Conversion of fractions between the two forms may not be exact, but will have better than microsecond accuracy.)

    > Mainly, I'm trying to understand printing of POSIXct with fractional
    > seconds.  I see print.POSIXct calls format.POSIXct and eventually
    > calls format.POSIXlt, which then takes into account `digits.secs` for
    > printing. The format uses %OS3, which strptime indicates (* added):

    >> Specific to R is %OSn, which for output gives the seconds *truncated* to 0 <= n <= 6 decimal places (and if %OS is not followed by a digit, it uses the setting of getOption("digits.secs"), or if that is unset, n = 0).

    > So I'm seeing it truncates the seconds to 3 digits, so I think that is
    > why the below is printing 0.024.

    > I think this is especially relevant even if you set
    > `options(digits.secs = 6)`, then the code in
    > format.POSIXlt would still return np=3 as the following condition
    > would break at i = 3

    > for (i in seq_len(np) - 1L) 
    >   if (all(abs(secs - round(secs, > i)) < 1e-06)) {
    >     np <- i
    >     break
    > }

    > as sub_seconds - round(sub_seconds,3) < 1e-06.   This seems to be
    > expected behavior given the docs, but would any consider this a bug?


    > Example:

    > options(digits.secs = 4)
    > x = structure(947016000.025, class = c("POSIXct", "POSIXt"), tzone = "UTC")

I think you've fallen into the R FAQ 7.31 trap :
[1] 0.02499998
Of course, the issue may still be somewhat interesting, ...

Yes, POSIXct is of limited precision and I think the help page
you mentioned did document that that's one reason for using
POSIXlt instead, as there, sub second accuracy can be much better.

But FAQ 7.31 and the fact that all numbers are base 2 and in
base 2,  no decimal   <n>.025   can be represented in full accuracy.

Also, as you've noticed the R POSIX[cl]t  code just truncates,
i.e. rounds towards 0 unconditionally, and I tend to agree that it
should rather round than truncate.

But we should carefully separate the issues here, from the
underlying inherent FAQ 7.31 truth that most decimal numbers in
a computer are not quite what they look like ...

Martin Maechler
ETH Zurich and  R Core Team (also author of the CRAN package 'round')


    > summary(x, digits = 20)
    > #>                      Min.                   1st Qu.                    Median
    > #> "2000-01-04 20:00:00.024" "2000-01-04 20:00:00.024" "2000-01-04 20:00:00.024"
    > #>                      Mean                   3rd Qu.                      Max.
    > #> "2000-01-04 20:00:00.024" "2000-01-04 20:00:00.024" "2000-01-04 20:00:00.024"
    > x
    > #> [1] "2000-01-04 20:00:00.024 UTC"
    > format.POSIXct(x, format = "%Y-%m-%d %H:%M:%OS3")
    > #> [1] "2000-01-04 20:00:00.024"
    > format.POSIXct(x, format = "%Y-%m-%d %H:%M:%OS4")
    > #> [1] "2000-01-04 20:00:00.0249"
    > sub_seconds = as.numeric(x) %% 1
    > sub_seconds
    > #> [1] 0.02499998
    > round(sub_seconds, 3)
    > #> [1] 0.025

    > rounded = as.POSIXct(
    > floor(as.numeric(x)) +
    > round(as.numeric(x) %% 1, 3),
    > origin = "1970-01-01")
    > rounded
    > #> [1] "2000-01-04 20:00:00.024 UTC"
    > as.numeric(rounded) %% 1
    > #> [1] 0.02499998

    > R.version
    > _
    > platform       x86_64-pc-linux-gnu
    > arch           x86_64
    > os             linux-gnu
    > system         x86_64, linux-gnu
    > status
    > major          4
    > minor          1.2
    > year           2021
    > month          11
    > day            01
    > svn rev        81115
    > language       R
    > version.string R version 4.1.2 (2021-11-01)
    > nickname       Bird Hippie



    > Best,
    > John

    > ______________________________________________
    > R-devel at r-project.org mailing list
    > https://stat.ethz.ch/mailman/listinfo/r-devel
John Muschelli
# 
Martin - I agree that the floating point representation of 0.025 is a
piece of the puzzle, but yes the heart was whether format/%OS3 should
potentially round vs. truncate.  As it's spelled out in ?strptime that
it will truncate, there is no bug per se, but simply unexpected
behavior and I wanted to know if that was the intent in this case.
Best,
John

On Thu, Mar 3, 2022 at 11:52 AM Martin Maechler
<maechler at stat.math.ethz.ch> wrote:
Duncan Murdoch
#
On 03/03/2022 11:52 a.m., Martin Maechler wrote:
If you print the hour and minute at 01:59:59, you get 1 and 59, not 2 
and 0.  That may be the motivation for doing the same for fractional 
seconds.  Should 1:59:59.9 really print as 2:00:00?

Duncan Murdoch
John Muschelli
# 
That?s a good point. I think that?s fair and why rounding may not be the
appropriate default.  Oddly enough, I think 1:59:60 may be more appropriate
though wrong.  The way the seconds are separated in POSIXlt however, I
don?t think that would ever happen, but the big downside would be if that
would round to 1:59:59.00

On Thu, Mar 3, 2022 at 2:38 PM Duncan Murdoch <murdoch.duncan at gmail.com>
wrote:
Best,
John