Skip to content

cumulative data monthly

8 messages · Jeff Newmiller, Diego Avesani, Rui Barradas

Diego Avesani
# 
Dear all,

I have a set of data with has hourly value:

# ID
# Lo
# L
# Q
Time,    T, RH,PSFC,DIR,VEL10, PREC, RAD, CC,FOG
yyyy-mm-dd hh:mm,   ?C,  %, hPa, ?N,  m/s, mm/h,W/m?,  %,-
2012-01-01 06:00, -0.1,100, 815,313,  2.6,  0.0,   0,  0,0
2012-01-01 07:00, -1.2, 93, 814,314,  4.8,  0.0,   0,  0,0
2012-01-01 08:00,  1.7, 68, 815,308,  7.5,  0.0,  41, 11,0
2012-01-01 09:00,  2.4, 65, 815,308,  7.4,  0.0, 150, 33,0
.....
.....

I was able to read it,  create my-own data frame and to plot the total
cumulative function.
This is basically what I have done:

dati <- read.csv(file="116.txt", header=FALSE, sep="," ,
na.strings="-999",skip = 6)
colnames(dati)=c("DATAORA","T", "RH","PSFC","DIR","VEL10", "PREC", "RAD",
"CC","FOG")

dati$DATAORA<-as.POSIXct(strptime(dati$DATAORA,format="%Y-%m-%d %H:%M"))


P <- cumsum(dati$PREC)
plot(dati$DATAORA, P)

I would like to select the data according to an starting and ending date.
In addition, I would like to plot the monthly and not the total one.
I mean, I would like to have a cumulative plot for each month of the
selected year.

I am struggling with "ddply" but probably it is the wrong way.

Could someone help me?  Really Really thanks,


Diego
Jeff Newmiller
# 
Are you looking for a plot where each point represents a month? Or a plot  where each point represents the accumulated precipitation so far that month? The latter seems closer to your computations so far, but doesn't seem like a typical way to present precipitation data...
On January 27, 2019 7:25:17 AM PST, Diego Avesani <diego.avesani at gmail.com> wrote:

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Rui Barradas
    

    

      
      #
    

  


  

      
Hello,

See if the following can get you started.
It uses package CRAN zoo, function as.yearmon.

dati$MES <- zoo::as.yearmon(dati$DATAORA)
PMES <- ave(dati$PREC, dati$MES, FUN = cumsum)

plot(dati$DATAORA, PMES)


Hope this helps,

Rui Barradas

?s 15:25 de 27/01/2019, Diego Avesani escreveu:

            

      
      
      
    

        

      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Jeff Newmiller
    

    

      
      #
    

  


  

      
Very succinct, Rui!

One warning to Diego.... automatic data recorders tend to use the local standard timezone year-round. R by default assumes that timestamps converted from character to POSIXct using the current timezone on your computer... which may not be in the same zone that the logger was in but even more commonly the computer follows daylight savings time. This leads to NAs showing up in your converted timestamps in spring and duplicated values in autumn as the data are misinterpreted. The easiest solution can be to use

Sys.setenv( TZ="GMT" )

though if you need the actual timezone you can use a zone name of the form "Etc/GMT+5" (5 hrs west of GMT).

Note that Rui's solution will only work correctly near the month transition if you pretend the data timezone is GMT or UTC. (Technically these are different so your mileage may vary but most implementations treat them as identical and I have not encountered any cases where they differ.)

        
On January 27, 2019 10:03:44 AM PST, Rui Barradas <ruipbarradas at sapo.pt> wrote:

            

      
      
      
    

        

      
      
    

        

      
      
    

        

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Diego Avesani
    

    

      
      #
    

  


  

      
Dear  Jeff, Dear Rui, Dear all,

I will try Rui's solution as soon as possible.
If I could ask:
As a first step, I would like to follow Jeff's suggestion. I will represent
the precipitation data with a cumulative distribution, one for each year.
This follow that I would like to select the starting date and the ending
date properly form dati$DATA in order to perform the cumulative function.

Could you help me on that.

Again, really really thanks

Diego



On Sun, 27 Jan 2019 at 21:37, Jeff Newmiller <jdnewmil at dcn.davis.ca.us>
wrote:

            

      
      
      
    

        

      
      
    

        

      
      
    

            

      
      
      
    

        

  
  


  


      

    

    

      
      

        


  

        

      Jeff Newmiller
    

    

      
      #
    

  


  

      
I have no idea what you mean when you say "select starting date and ending 
date properly form [sic] datai$DATA". For one thing there is no column 
called DATA, and for another I don't know what starting dates and ending 
dates you might be interested in. If you need help to subset by time, 
perhaps you should ask a question about that instead.

Here is a reproducible example of making monthly data and manipulating it 
using artificial data:

###############
library(zoo)
Sys.setenv( TZ = "GMT" )
set.seed(42)
dati <- data.frame( DATAORA = as.POSIXct( "2012-01-01" )
                             + as.difftime( seq( 0, 365*3*24
                                          ), units="hours" )
                   )
# terrible simulation of precipitation
dati$PREC <- 0.1 * trunc( 50 * rbeta( nrow( dati ), 1, 80 ) )
dati$ym <- as.yearmon( dati$DATAORA )
# aggregate usually reduces the number of rows given to it
datim <- aggregate( list( PREC = dati$PREC ) # data to summarize
                   , dati[ , "ym", drop=FALSE ] # columns to group on
                   , FUN = sum  # calculation on data
                   )
plot(PREC ~ ym, data=datim) # This is how I would usually look at it
as.year <- function(x) floor( as.numeric( x ) ) # from help file on as.yearmon
datim$y <- as.year( datim$ym )
# ave typically does not change the number of rows given to it
datim$PMES <- ave( datim$PREC, datim$y, FUN = cumsum)
plot(PMES ~ ym, data=datim) # My guess as to what you asked for?
###############

        
On Sun, 27 Jan 2019, Diego Avesani wrote:

        

            

      
      
      
    

        

      
      
    

            
        

      
      
    

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        
---------------------------------------------------------------------------
Jeff Newmiller                        The     .....       .....  Go Live...
DCN:<jdnewmil at dcn.davis.ca.us>        Basics: ##.#.       ##.#.  Live Go...
                                       Live:   OO#.. Dead: OO#..  Playing
Research Engineer (Solar/Batteries            O.O#.       #.O#.  with
/Software/Embedded Controllers)               .OO#.       .OO#.  rocks...1k
---------------------------------------------------------------------------

  
  


  


      

    

    

      
      

        


  

        

      Diego Avesani
    

    

      
      #
    

  


  

      
Dear Jeff, Dear Rui, Dear all,

Forget about the monthly things. I was trying to do two things at the same
time.
I try to explain myself. Thanks for your time and I really appreciate your
help.

I have  a long file with hourly precipitation from 2000 to 2018. I would
like to select only on e year or even half of a year and plot the
cumulative precipitation of it in order to compare it with the simulation
data that I have.

So far I was able only to read all the file:
dati <- read.csv(file="116.txt", header=FALSE, sep="," ,
na.strings="-999",skip = 6)

and to plot the entire cumulative:
P <- cumsum(dati$PREC)
plot(dati$DATAORA, P)

How can I choose only, for example, 2013 in order to have P?
thanks again


Diego



On Mon, 28 Jan 2019 at 02:36, Jeff Newmiller <jdnewmil at dcn.davis.ca.us>
wrote:

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

  
  


  


      

    

    

      
      

        


  

        

      Rui Barradas
    

    

      
      #
    

  


  

      
Hello,

With on?bjects of class "Date" or "POSIXt", POSIXct" you can do

lubridate::year(date_obj)

to extract the year. Then aggregate by it.

Hope this helps,

Rui Barradas

?s 08:25 de 28/01/2019, Diego Avesani escreveu: