Skip to content

Coordinate/grid for ncdf data

6 messages · Michael Sumner, Thomas Szegvary, David W. Pierce

Original
Thomas Szegvary
# 
Hi!

I have a table with 3 columns, 2 for long/lat coordinates and 1 for values
(radon concentration). I have data for every week of the year
2006,distributed in 52 tables/files. I want to create a NetCDF file, which
is much easier to handle than extracting the values from 52 tables. I tried
the following for the first dataset (i.e first week):

#extracting coordinates and values from my tables (which have also other
information I don't need)
W01<-read.table("RN_weekly/KW01_RN.dat")
long<-W01$V1
lat<-W01$V2
RN01<-W01$V3

#defining dimensions for coordinates
dim1 <- dim.def.ncdf( "EW","degrees", as.double(long))
dim2 <- dim.def.ncdf( "SN","degrees", as.double(lat))

#defining variable for my values I want to have the time series
varz <- var.def.ncdf("Radon","Bq/m2/h1", list(dim1,dim2), -1, 
          longname="Radon flux rate")

#creating the netcdf file and filling the variable varz with the first
data-week from my time series
nc.rn <- create.ncdf("rn_weekly.nc",varz)
put.var.ncdf(nc.rn,varz,RN01)
close.ncdf(nc.rn)


The problem now is that the last step (put.var.ncdf) doesn't work, because
it says I am trying to "error: you asked to write 111788329 values, but the
passed data array only has 10573* entries!". So I think the problem is that
I need an array with two dimensions (coordinates...) for my values. But how
do I get this from my tables??

 *10573 are the pixels for my area

Thanks for any help!
Thomas


__

Thomas Szegvary
Institute of Environmental Geosciences
Department of Geosciences
University of Basel
Bernoullistrasse 30
CH - 4056 Basel

Tel.  41-61-267 04 82
Fax. 41-61-267 04 79
Email: t.szegvary at unibas.ch
www.radon.unibas.ch
www.unibas.ch/environment
Michael Sumner
# 
Hi, can you provide an example file?

I'd recommend avoiding NetCDF for various reasons, why don't you simply add
each value column from the separate files to a new column in a
SpatialGridDataFrame?

That would be the recommend sp way to proceed here. If you really need to
create a NetCDF file, please let us know more about your OS and the
package/s and versions you are using (ncdf I presume?) - however such
questions should include the ncdf(?) package author at least.  

Cheers, Mike. 

-----Original Message-----
From: r-sig-geo-bounces at stat.math.ethz.ch
[mailto:r-sig-geo-bounces at stat.math.ethz.ch] On Behalf Of Thomas Szegvary
Sent: Wednesday, 8 August 2007 11:13 PM
To: r-sig-geo at stat.math.ethz.ch
Subject: [R-sig-Geo] Coordinate/grid for ncdf data

Hi!

I have a table with 3 columns, 2 for long/lat coordinates and 1 for values
(radon concentration). I have data for every week of the year
2006,distributed in 52 tables/files. I want to create a NetCDF file, which
is much easier to handle than extracting the values from 52 tables. I tried
the following for the first dataset (i.e first week):

#extracting coordinates and values from my tables (which have also other
information I don't need)
W01<-read.table("RN_weekly/KW01_RN.dat")
long<-W01$V1
lat<-W01$V2
RN01<-W01$V3

#defining dimensions for coordinates
dim1 <- dim.def.ncdf( "EW","degrees", as.double(long))
dim2 <- dim.def.ncdf( "SN","degrees", as.double(lat))

#defining variable for my values I want to have the time series varz <-
var.def.ncdf("Radon","Bq/m2/h1", list(dim1,dim2), -1, 
          longname="Radon flux rate")

#creating the netcdf file and filling the variable varz with the first
data-week from my time series nc.rn <- create.ncdf("rn_weekly.nc",varz)
put.var.ncdf(nc.rn,varz,RN01)
close.ncdf(nc.rn)


The problem now is that the last step (put.var.ncdf) doesn't work, because
it says I am trying to "error: you asked to write 111788329 values, but the
passed data array only has 10573* entries!". So I think the problem is that
I need an array with two dimensions (coordinates...) for my values. But how
do I get this from my tables??

 *10573 are the pixels for my area

Thanks for any help!
Thomas


__

Thomas Szegvary
Institute of Environmental Geosciences
Department of Geosciences
University of Basel
Bernoullistrasse 30
CH - 4056 Basel

Tel.  41-61-267 04 82
Fax. 41-61-267 04 79
Email: t.szegvary at unibas.ch
www.radon.unibas.ch
www.unibas.ch/environment

_______________________________________________
R-sig-Geo mailing list
R-sig-Geo at stat.math.ethz.ch
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Thomas Szegvary
# 
I use ncdf V1.6 on Windows XP Sp2, I hava attached the sample file (tab
delim) which contains 2 columns for coordinates long/lat and 10 columns for
weekly averages of radon. Sp would be the best solution to produce map I
guess, but for time series analysis I think netcdf would perform better. 

Thanks,thoams


 
-----Urspr?ngliche Nachricht-----
Von: Michael Sumner [mailto:mdsumner at utas.edu.au] 
Gesendet: Donnerstag, 9. August 2007 01:49
An: 'Thomas Szegvary'; r-sig-geo at stat.math.ethz.ch
Betreff: RE: [R-sig-Geo] Coordinate/grid for ncdf data

Hi, can you provide an example file?

I'd recommend avoiding NetCDF for various reasons, why don't you simply add
each value column from the separate files to a new column in a
SpatialGridDataFrame?

That would be the recommend sp way to proceed here. If you really need to
create a NetCDF file, please let us know more about your OS and the
package/s and versions you are using (ncdf I presume?) - however such
questions should include the ncdf(?) package author at least.  

Cheers, Mike. 

-----Original Message-----
From: r-sig-geo-bounces at stat.math.ethz.ch
[mailto:r-sig-geo-bounces at stat.math.ethz.ch] On Behalf Of Thomas Szegvary
Sent: Wednesday, 8 August 2007 11:13 PM
To: r-sig-geo at stat.math.ethz.ch
Subject: [R-sig-Geo] Coordinate/grid for ncdf data

Hi!

I have a table with 3 columns, 2 for long/lat coordinates and 1 for values
(radon concentration). I have data for every week of the year
2006,distributed in 52 tables/files. I want to create a NetCDF file, which
is much easier to handle than extracting the values from 52 tables. I tried
the following for the first dataset (i.e first week):

#extracting coordinates and values from my tables (which have also other
information I don't need)
W01<-read.table("RN_weekly/KW01_RN.dat")
long<-W01$V1
lat<-W01$V2
RN01<-W01$V3

#defining dimensions for coordinates
dim1 <- dim.def.ncdf( "EW","degrees", as.double(long))
dim2 <- dim.def.ncdf( "SN","degrees", as.double(lat))

#defining variable for my values I want to have the time series varz <-
var.def.ncdf("Radon","Bq/m2/h1", list(dim1,dim2), -1, 
          longname="Radon flux rate")

#creating the netcdf file and filling the variable varz with the first
data-week from my time series nc.rn <- create.ncdf("rn_weekly.nc",varz)
put.var.ncdf(nc.rn,varz,RN01)
close.ncdf(nc.rn)


The problem now is that the last step (put.var.ncdf) doesn't work, because
it says I am trying to "error: you asked to write 111788329 values, but the
passed data array only has 10573* entries!". So I think the problem is that
I need an array with two dimensions (coordinates...) for my values. But how
do I get this from my tables??

 *10573 are the pixels for my area

Thanks for any help!
Thomas


__

Thomas Szegvary
Institute of Environmental Geosciences
Department of Geosciences
University of Basel
Bernoullistrasse 30
CH - 4056 Basel

Tel.  41-61-267 04 82
Fax. 41-61-267 04 79
Email: t.szegvary at unibas.ch
www.radon.unibas.ch
www.unibas.ch/environment

_______________________________________________
R-sig-Geo mailing list
R-sig-Geo at stat.math.ethz.ch
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: sample_rn.txt
URL: <https://stat.ethz.ch/pipermail/r-sig-geo/attachments/20070809/97647e70/attachment.txt>
David W. Pierce
# 
Hello,

the reason you are getting that error from the ncdf package is because you
have defined variable "Radon" to be a *two* dimensional array in the
netcdf file.  The number of entries in dimension 1 is length(long) =
10573, and the number of entries in dimension 2 is length(lat) = 10573. 
So, the total number of entries in the array in the netcdf file is
length(long) * length(lat) = 10573 * 10573 = 111788329.

You only want to define a 2-d array when the location of data[i,j] can be
described as (lon[i],lat[j]).  That is not the case for your data, because
it is not on any sort of grid.  Instead, you will want to create a set of
3 1-dimensional arrays along the following lines:

n_stations <- length(RN01)
dim_station <- dim.def.ncdf( 'Radon_station', '-', 1:n_stations )

var_lon <- var.def.ncdf( 'Longitude', 'degreesE', dim_station, 1.e30 )
var_lat <- var.def.ncdf( 'Latitude',  'degreesN', dim_station, 1.e30 )
var_radon <- var.def.ncdf( 'Radon',   'Bq/m2/h1', dim_stattion, 1.e30 )

ncid_out <- create.ncdf( 'rn.weekly.nc', list(var_lon,var_lat,var_radon))

put.var.ncdf( ncid_out, var_lon, long )
put.var.ncdf( ncid_out, var_lat, lat  )
put.var.ncdf( ncid_out, var_radon, RN01 )

close.ncdf( ncid_out )

Now, the location of data point RN01[i] is correctly described as
(lon[i],lat[i]); compare this to what you were trying before, which was
RN01[i,j] having the location (lon[i],lat[j]).

Regards,

--Dave
Thomas Szegvary wrote:
---------------------------------------------------------------
David W. Pierce        / Climate Research Division
Scripps Inst. Oceanog. / (858) 534-8276 (voice)
dpierce at ucsd.edu       / (858) 534-8561 (fax)
Thomas Szegvary
# 
Dear Dave

Great, that worked! Many thanks. Now I want to put more weekly data (as RN01
was data for the first week of the year 2006) into the var_radon, but this
is not possible for dim_station, as there are only 3 dimensions. Do I have
to redefine 51 other dimensions? I tried to use list() to define the other
dimensions, but that didn't work.

Many thanks
Thomas




 
-----Urspr?ngliche Nachricht-----
Von: David Pierce [mailto:dpierce at ucsd.edu] 
Gesendet: Donnerstag, 9. August 2007 18:25
An: Thomas Szegvary
Cc: 'Michael Sumner'; r-sig-geo at stat.math.ethz.ch
Betreff: Re: AW: [R-sig-Geo] Coordinate/grid for ncdf data

Hello,

the reason you are getting that error from the ncdf package is because you
have defined variable "Radon" to be a *two* dimensional array in the netcdf
file.  The number of entries in dimension 1 is length(long) = 10573, and the
number of entries in dimension 2 is length(lat) = 10573. 
So, the total number of entries in the array in the netcdf file is
length(long) * length(lat) = 10573 * 10573 = 111788329.

You only want to define a 2-d array when the location of data[i,j] can be
described as (lon[i],lat[j]).  That is not the case for your data, because
it is not on any sort of grid.  Instead, you will want to create a set of
3 1-dimensional arrays along the following lines:

n_stations <- length(RN01)
dim_station <- dim.def.ncdf( 'Radon_station', '-', 1:n_stations )

var_lon <- var.def.ncdf( 'Longitude', 'degreesE', dim_station, 1.e30 )
var_lat <- var.def.ncdf( 'Latitude',  'degreesN', dim_station, 1.e30 )
var_radon <- var.def.ncdf( 'Radon',   'Bq/m2/h1', dim_stattion, 1.e30 )

ncid_out <- create.ncdf( 'rn.weekly.nc', list(var_lon,var_lat,var_radon))

put.var.ncdf( ncid_out, var_lon, long )
put.var.ncdf( ncid_out, var_lat, lat  )
put.var.ncdf( ncid_out, var_radon, RN01 )

close.ncdf( ncid_out )

Now, the location of data point RN01[i] is correctly described as
(lon[i],lat[i]); compare this to what you were trying before, which was
RN01[i,j] having the location (lon[i],lat[j]).

Regards,

--Dave
Thomas Szegvary wrote:
---------------------------------------------------------------
David W. Pierce        / Climate Research Division
Scripps Inst. Oceanog. / (858) 534-8276 (voice)
dpierce at ucsd.edu       / (858) 534-8561 (fax)
David W. Pierce
# 
If your data is weekly, then you'll want to define an unlimited time
dimension, so that your data structure looks something like this:

Radon[station_number,week_number]

And your dimensions look like something like this:

station_number[1:n_stations]
week_number[1:n_weeks] (make this unlimited with dim.def.ncdf(,unlim=TRUE)
latitude[1:n_stations]
longitude[1:n_stations]

Regards,

--Dave
Thomas Szegvary wrote:
---------------------------------------------------------------
David W. Pierce        / Climate Research Division
Scripps Inst. Oceanog. / (858) 534-8276 (voice)
dpierce at ucsd.edu       / (858) 534-8561 (fax)