Skip to content

Extract VIIRS data in R

7 messages · Frederico Faleiro, Washington Correia Filho, Michael Sumner +3 more

Original
Miluji Sb
# 
Dear all,

Hope everyone is keeping safe.

I am trying to extract/read VIIRS nighttime lights data but the output
seems rather strange.

I have uploaded the file here
<https://drive.google.com/open?id=1zpqXJ8AlEcnk6ApRb75tfQc4-Y8AfK5h> so as
not exceed the size limit of the email.

## Code ##
library(ncdf4)
library(rgdal)

netcdf_file <- c("VNP02DNB_NRT.A2020069.1048.001.nc")
nl <- brick(netcdf_file, lvar=0, values=TRUE,
varname="observation_data/DNB_observations")

## Detail ##
class      : RasterLayer
dimensions : 3232, 4064, 13134848  (nrow, ncol, ncell)
resolution : 1, 1  (x, y)
extent     : 0.5, 4064.5, 0.5, 3232.5  (xmin, xmax, ymin, ymax)
crs        : NA
source     : C:/Users/shour/Desktop/VNP02DNB_NRT.A2020069.1048.001.nc
names      : DNB.observations.at.pixel.locations
zvar       : observation_data/DNB_observations

The spatial resolution is supposed to be 750m but this shows 1?. What am I
doing wrong? Any help will be greatly appreciated. Thank you.

Sincerely,

Millu
Frederico Faleiro
# 
Hi Miluji,

the R in general do not handle very well with netCDF. However I
recommend you use the netcdf4 package before try others.
This link is a good start:
http://geog.uoregon.edu/bartlein/courses/geog490/week04-netCDF.html#get-coordinate-including-time-variables
.
The coordinates of this file is not in longitude and latitude, it use
"number_of_lines" for X Axis and "number_of_pixels" for Y Axis. This way
the difference between the cells is always one and raster package
interprete the difference as one degree and return an warning about it.
I open the file in the software Panoply (design to work with netCDF)  and
everything is OK apparently.
Check bellow some exploration of the data in R.

library(raster)
library(ncdf4)
library(rgdal)

# netcdf4 package
netcdf_file <- "VNP02DNB_NRT.A2020069.1048.001.nc"
nc <- nc_open(netcdf_file)
print(nc)
# check your variable is structured as:
observation_data/DNB_observations[number_of_pixels,number_of_lines]
# variables available
names(nc[["var"]])
# properties from your variable
ncatt_get(nc, "observation_data/DNB_observations")
# get the variables values and check the structure
obs_matrix <- ncvar_get(nc, "observation_data/DNB_observations")
str(obs_matrix)

# raster package
nc_r <- brick(netcdf_file, lvar=0, values=TRUE,
varname="observation_data/DNB_observations")
# [1] "vobjtovarid4: **** WARNING **** I was asked to get a varid for
dimension named number_of_pixels BUT this dimension HAS NO DIMVAR! Code
will probably fail at this point"
# [1] "vobjtovarid4: **** WARNING **** I was asked to get a varid for
dimension named number_of_lines BUT this dimension HAS NO DIMVAR! Code will
probably fail at this point"

Best regards,

Frederico Faleiro
Postdoctoral Researcher in the INCT-EECBio (https://www.eecbio.ufg.br/)
Federal University of Goi?s | Brazil
RG: https://www.researchgate.net/profile/Frederico_Faleiro
On Tue, Mar 17, 2020 at 11:39 AM Miluji Sb <milujisb at gmail.com> wrote:

            

      
      
      
    

        

      
      
    

        

  
  


  


      

    

    

      
      

        


  

        

      Washington Correia Filho
    

    

      
      #
    

  


  

      
Dear Millu,

The data they use is not with the CRS set, so the displayed result is showing the grid points (extent: 0.5, 4064.5, 0.5, 3232.5). Then you should consult the VIIRS data guide so that you can organize your data, to which you need to know the start data for both latitude and longitude, as well as the multiplication factor and so you will have your data readjusted. I also have a similar problem to solve, if I can, I can contact you and give you the data and the scheme I used. See you soon.

Sincerely,
Washington L.F. Correia Filho
Postdoctoral Research Fellowship in Meteorology, UFAL, Brazil
Dr. in Climate Sciences, UFRN, Brazil
http://lattes.cnpq.br/7596712599262929
ORCID: https://orcid.org/0000-0002-4029-4491/

        

      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Michael Sumner
    

    

      
      #
    

  


  

      
You won't find much on this kind of file in the R community, this is
relatively new NetCDF-4 with "groups", and contains a satellite "granule" -
a kind of pre-mapping raw data array with only technical information about
where the scanning occurred.

In short, raster and stars package (and RNetCDF and ncdf4) will read the
data from this file, but give very little or no help for how to treat it
like a map. The spatial resolution and extent are purely nominal as shown
by raster, a spacing of 1 in two dimensions and extending from 0 to nrows,
0 to ncols (with a  half cell shift). There aren't any coordinate arrays in
the file, or any simple way to transform from this matrix-space to
geography as far as I know.  (Would love to be corrected on how to do
that).

You'd need to pursue domain-specific expertise to discover the mapping of
this, unless some kind soul turns up to help here. I expect you'll want to
find a product that has been converted into simpler form for these data.

Cheers, Mike.

        
On Wed, Mar 18, 2020 at 1:39 AM Miluji Sb <milujisb at gmail.com> wrote:

        

            

      
      
      
    

        

      
      
    

        

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Zia Ahmed
    

    

      
      #
    

  


  

      
This is might help you:

https://zia207.github.io/geospatial-r-github.io/netCDF-data-processing.html

Zia

        
On Wed, Mar 18, 2020, 9:34 AM Frederico Faleiro <fvfaleiro at gmail.com> wrote:

        

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

        

  
  


  


      

    

    

      
      

        


  

        

      Miluji Sb
    

    

      
      #
    

  


  

      
Dear all,

Thank you everyone. The issue (as pointed out) is the following;

# [1] "vobjtovarid4: **** WARNING **** I was asked to get a varid for
dimension named number_of_pixels BUT this dimension HAS NO DIMVAR! Code
will probably fail at this point"

I have contacted the VIIRS people and will keep you posted here. Thank you
again!

Best,

Milu

        
On Wed, Mar 18, 2020 at 3:34 PM Zia Ahmed <zia207 at gmail.com> wrote:

        

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

  
  


  


      

    

    

      
      

        


  

        

      Edzer Pebesma
    

    

      
      #
    

  


  

      
If the product info is in this document:
https://ladsweb.modaps.eosdis.nasa.gov/missions-and-measurements/viirs/NASAVIIRSL1BUGAug2019.pdf

then it sounds like these data are distributed as curvilinear grids,
meaning that there are two additional raster with for each pixel the
longitude and the latitude. The file you pointed to does not contain
these two grids, but you should be able to get them at the place where
you got the data. With that, you can use package stars to load them, and
e.g. resample to some (more) regular grid.

See e.g.
https://r-spatial.github.io/stars/articles/stars4.html#curvilinear-grids

        
On 3/18/20 3:29 PM, Michael Sumner wrote: