Skip to content

Incorrect month order in zApply function

9 messages · Dominik Schneider, Vijay Lulla, Kilpatrick, Katherine A +2 more

Thiago V. dos Santos
# 
Dear all,

I am using the raster package to calculate monthly averages of climatic variables.

Essentially, this is the function I use:

library(raster)

# Create date sequence
idx <- seq(as.Date("1996/1/1"), as.Date("2010/12/31"), by = "day")

# Create raster stack and assign dates
r <- raster(ncol=5, nrow=5)
s <- stack(lapply(1:length(idx), function(x) setValues(r, runif(ncell(r)))))
s <- setZ(s, idx)

# Calculate monthly average
x <- zApply(s, by=months, mean, name=month.abb[])

names(x)
[1] "April" "August" "December" "February" "January" "July" "June" 
[8] "March" "May" "November" "October" "September"
getZ(x)
[1] "April" "August" "December" "February" "January" "July" "June" 
[8] "March" "May" "November" "October" "September"


The problem here is the output of both names(x) and getZ(x). It looks like a random month order is returned (even though I provide the labels), which makes me confused about the results.


By performing the same calculation in a different software and comparing the results, I came to realize that the order of months for the results by raster should, in fact, be Jan-Feb-Mar-Apr-May-Jun-Jul-Aug-Sep-Oct-Nov-Dec
 
How can I control the way raster delivers the object names after using zApply, in order to sort the months in the "natural" order?

Greetings, -- Thiago V. dos Santos

PhD student
Land and Atmospheric Science
University of Minnesota
Dominik Schneider
# 
Sorry I'm not sure how to fix this, but it looks like the months are  in
alphanumeric order, indicating they are being treated as a factor. Might be
worth a bug report?



On Thursday, July 28, 2016, Thiago V. dos Santos via R-sig-Geo <
r-sig-geo at r-project.org> wrote:

            

      
      
      
    

        

      
      
    

        

  
  


  


      

    

    

      
      

        


  

        

      Ben Tupper
    

    

      
      #
    

  


  

      
Hi,

Might you simply reorder the stack after zApply()?

            

      
      
      
    

        
[1] "January"   "February"  "March"     "April"     "May"       "June"     
 [7] "July"      "August"    "September" "October"   "November"  "December" 

Cheers,
Ben

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

        
Ben Tupper
Bigelow Laboratory for Ocean Sciences
60 Bigelow Drive, P.O. Box 380
East Boothbay, Maine 04544
http://www.bigelow.org

Report Gulf of Maine jellyfish sightings to jellyfish at bigelow.org or tweet them to #MaineJellies -- include date, time, and location, as well as any descriptive information such as size or type.  Learn more at https://www.bigelow.org/research/srs/nick-record/nick-record-laboratory/mainejellies/

  
  


  


      

    

    

      
      

        


  

        

      Vijay Lulla
    

    

      
      #
    

  


  

      
Maybe like this?

R> m <- function(x) ordered(months(x),labels=month.name)
R> x1 <- zApply(s, by=m, mean, name=month.abb[])
R> names(x1) # Appears to have natural ordering
R> all(x[] == x1[]) # TRUE

I'm not sure how using factors will impact other aspects of your project.
HTH,
Vijay.

On Thu, Jul 28, 2016 at 10:52 PM, Thiago V. dos Santos via R-sig-Geo <

        
r-sig-geo at r-project.org> wrote:

        

            

      
      
      
    

        

      
      
    

        

  
  


  


      

    

    

      
      

        


  

        

      Kilpatrick, Katherine A
    

    

      
      #
    

  


  

      
Hi

The various Apply functions always return the output in simple 
alphabetical order when the factor label is a character, regardless of 
the order of the factor in the calling argument.

Ben had a correct solution --- simply reorder the output from Zapply.

Alternatively you could use a numeric representation for the month label 
using by = as.numeric(months(x). The output would then be in increasing 
numeric order from 1 -12.

I do not believe Vijay's suggestion is proper. It does not reorder the 
output --- instead it changes the label of a given monthly mean eg. The 
value for Jan. would then be labeled as Apr.

k

        
On 7/28/16 10:52 PM, Thiago V. dos Santos via R-sig-Geo wrote:

            

      
      
      
    

        

      
      
    

        

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Thiago V. dos Santos
    

    

      
      #
    

  


  

      
Thanks everyone for their suggestions.

Reordering the output of zApply would be a WRONG solution, because the layers are already ordered from Jan to Dec.

It is just the NAMES that are misaligned, which can cause a bit of confusion.

I already contacted Robert about this issue and we should see a solution soon.
 Greetings,
 -- Thiago V. dos Santos

PhD student
Land and Atmospheric Science
University of Minnesota

        
On Friday, July 29, 2016 2:09 PM, Kay Kilpatrick <kkilpatrick at rsmas.miami.edu> wrote:

        
Hi

The various Apply functions always return the output in simple 
alphabetical order when the factor label is a character, regardless of 
the order of the factor in the calling argument.

Ben had a correct solution --- simply reorder the output from Zapply.

Alternatively you could use a numeric representation for the month label 
using by = as.numeric(months(x). The output would then be in increasing 
numeric order from 1 -12.

I do not believe Vijay's suggestion is proper. It does not reorder the 
output --- instead it changes the label of a given monthly mean eg. The 
value for Jan. would then be labeled as Apr.

k

        
On 7/28/16 10:52 PM, Thiago V. dos Santos via R-sig-Geo wrote:

            

      
      
      
    

        

      
      
    

        

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Ben Tupper
    

    

      
      #
    

  


  

      
Hi,

I reworked your example a bit so that it encompasses just one year, and the value of each layer is the month number for the Z index.  Sure enough, the mean month number values look like they are mismatched relative to the layer name.  Example below - sorry about line-wrapping.  Uhoh.

Cheers,
Ben

library(raster)

# Create date sequence
idx <- seq(as.Date("1996/1/1"), as.Date("1996/12/31"), by = "day")

mon_num <- as.numeric(format(idx, "%m"))

# Create raster stack and assign dates
r <- raster(ncol=5, nrow=5)
s <- stack(
    lapply(1:length(idx), 
        function(x) setValues(r, rep(mon_num[x],ncell(r)))
        )
    )
s <- setZ(s, idx)

# Calculate monthly average
x <- zApply(s, by=months, mean, name=month.abb[])

x
# class       : RasterBrick 
# dimensions  : 5, 5, 25, 12  (nrow, ncol, ncell, nlayers)
# resolution  : 72, 36  (x, y)
# extent      : -180, 180, -90, 90  (xmin, xmax, ymin, ymax)
# coord. ref. : +proj=longlat +datum=WGS84 +ellps=WGS84 +towgs84=0,0,0 
# data source : in memory
# names       : April, August, December, February, January, July, June, March, May, November, October, September 
# min values  :     1,      2,        3,        4,       5,    6,    7,     8,   9,       10,      11,        12 
# max values  :     1,      2,        3,        4,       5,    6,    7,     8,   9,       10,      11,        12

            

      
      
      
    

        
R version 3.3.1 (2016-06-21)
Platform: x86_64-apple-darwin13.4.0 (64-bit)
Running under: OS X 10.11.6 (El Capitan)

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] raster_2.5-8 sp_1.2-3    

loaded via a namespace (and not attached):
[1] rgdal_1.1-10    Rcpp_0.12.6     grid_3.3.1      lattice_0.20-33

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

        
Ben Tupper
Bigelow Laboratory for Ocean Sciences
60 Bigelow Drive, P.O. Box 380
East Boothbay, Maine 04544
http://www.bigelow.org

Report Gulf of Maine jellyfish sightings to jellyfish at bigelow.org or tweet them to #MaineJellies -- include date, time, and location, as well as any descriptive information such as size or type.  Learn more at https://www.bigelow.org/research/srs/nick-record/nick-record-laboratory/mainejellies/

  
  


  


      

    

    

      
      

        


  

        

      Vijay Lulla
    

    

      
      #
    

  


  

      
Ben,
You're right about a mistake in my code but it is not use of ordered
function.  It is that I shouldn't have used the name argument.  Following
your example this is what I get:

            

      
      
      
    

        
# class       : RasterBrick
# dimensions  : 5, 5, 25, 12  (nrow, ncol, ncell, nlayers)
# resolution  : 72, 36  (x, y)
# extent      : -180, 180, -90, 90  (xmin, xmax, ymin, ymax)
# coord. ref. : +proj=longlat +datum=WGS84 +ellps=WGS84 +towgs84=0,0,0
# data source : in memory
# names       : January, February, March, April, May, June, July, August,
September, October, November, December
# min values  :       1,        2,     3,     4,   5,    6,    7,
8,         9,      10,       11,       12
# max values  :       1,        2,     3,     4,   5,    6,    7,
8,         9,      10,       11,       12
#             : January, February, March, April, May, June, July, August,
September, October, November, December

Thanks for pointing out the mistake,
Vijay.

        
On Fri, Jul 29, 2016 at 7:17 PM, Ben Tupper <btupper at bigelow.org> wrote:

        

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

        

  
  


  


      

    

    

      
      

        


  

        

      Ben Tupper
    

    

      
      #
    

  


  

      
Hi Vijay,

Whoops!  I think you misunderstood my message - I didn't think I was pointing out a mistake on your part.  My 'uhoh' was pointing out surprising behavior on the part of zApply.

If I use my earlier edited example and remove the "name=" argument to zApply, the results still make my forehead get all wrinkled up as the month labels don't align with the results.

library(raster)

# Create date sequence
idx <- seq(as.Date("1996/1/1"), as.Date("1996/12/31"), by = "day")
mon_num <- as.numeric(format(idx, "%m"))

# Create raster stack and assign dates
r <- raster(ncol=5, nrow=5)
s <- stack(
    lapply(1:length(idx), 
        function(x) setValues(r, rep(mon_num[x],ncell(r)))
        )
    )
s <- setZ(s, idx)

# Calculate monthly average
x <- zApply(s, by=months, fun = mean)
x
#class       : RasterBrick 
#dimensions  : 5, 5, 25, 12  (nrow, ncol, ncell, nlayers)
#resolution  : 72, 36  (x, y)
#extent      : -180, 180, -90, 90  (xmin, xmax, ymin, ymax)
#coord. ref. : +proj=longlat +datum=WGS84 +ellps=WGS84 +towgs84=0,0,0 
#data source : in memory
#names       : April, August, December, February, January, July, June, March, May, November, October, September 
#min values  :     1,      2,        3,        4,       5,    6,    7,     8,   9,       10,      11,        12 
#max values  :     1,      2,        3,        4,       5,    6,    7,     8,   9,       10,      11,        12 
#            : April, August, December, February, January, July, June, March, May, November, October, September 


So, what the heck does the "by=" do if not group the layers by month?  I can't see why you need to define your own aggregation m() to get it to work (and I am impressed that you did get it to work.)

Cheers,
Ben

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

        
Ben Tupper
Bigelow Laboratory for Ocean Sciences
60 Bigelow Drive, P.O. Box 380
East Boothbay, Maine 04544
http://www.bigelow.org

Report Gulf of Maine jellyfish sightings to jellyfish at bigelow.org or tweet them to #MaineJellies -- include date, time, and location, as well as any descriptive information such as size or type.  Learn more at https://www.bigelow.org/research/srs/nick-record/nick-record-laboratory/mainejellies/