An embedded and charset-unspecified text was scrubbed... Name: not available URL: <https://stat.ethz.ch/pipermail/r-sig-geo/attachments/20061205/2d6fd51f/attachment.pl>
Errore in sprintf(fmt, ...) ?
2 messages · Massimo Di Stefano, Roger Bivand
On Tue, 5 Dec 2006, massimodisasha wrote:
Please respect the time of people who reply to your questions. You send messages which indicate that you are just fumbling in ignorance. You are trying to run a very large problem on difficult hardware (yes, recently we even found a case of console R on Intel Mac working when R.App failed), using somebody else's script. 0) Why such large gaps in time between your replies - are you really interested in a resolution? My suggestion of using reasRAST6() wasn't answering your problem anyway, but was sent Sat, 2 Dec 2006 16:03:53, after a previous longish gap. 1) Why such a large area? 2) Why this script and methodology - why clara() and this choice of variables - you are classifying 5M observations on 7 variables? Wouldn't it be possible to set up criteria for classification using a sample or sub-region, and use the same rules in for example r.mapcalc? 3) Why an Intel Mac using R.app(?)? 4) Why not try harder to resolve the problems yourself, since you are trying to do things that probably nobody else on the list can reproduce on a similar platform? You say Dylan's script works for a smaller region, doesn't that tell you something? Please try to do this on a Linux platform, then at least some of us could try to reproduce it. 4a) Or make the difficult object available on a website as: save(x, file="very_large_geomorp.RData", compress=TRUE) if this doesn't cause memory problems too, that is. 5) Instead of copying to morph, use x at data directly - it is already a data frame. str(x at data) summary(x at data) In any case in GRASS doing r.stats or whatever would show that there are NAs in your region, wouldn't it? 6) Instead of R.app(?), use R in a regular console (but maybe you are doing that anyway); recent experience indicates that Intel Mac is not necessarily well supported for larger tasks, especially R.app. Roger
hi, i've tryied to use readRAST6 instead of readFLOAT6sp this is the log:
> library(cluster) > library(cluster) > x <- readRAST6(c
("er","crosc","longc","slope","profc","minic","maxic"))
.....
.......
........
Percent complete: 100%
r.out.bin complete.
Creating BIL support files ...
Header File = /grassdata/remote/onearth/.tmp/powerbook-g4-12-di-
epifanio.local/maxic.hdr
World File = /grassdata/remote/onearth/.tmp/powerbook-g4-12-di-
epifanio.local/maxic.wld
Exporting Raster as double values(bytes=8)
Using the Current Region settings ...
north=4516560.000000
south=4483080.000000
east=527440.000000
west=485740.000000
r=1674
c=2085
Percent complete: 100%
r.out.bin complete.
> gc()
used (Mb) gc trigger (Mb) max used (Mb) Ncells 416932 11.2 741108 19.8 692659 18.5 Vcells 23007923 175.6 194536624 1484.2 242896455 1853.2
> morph <- as(x, "data.frame")
R(22751) malloc: *** vm_allocate(size=8421376) failed (error code=3) R(22751) malloc: *** error: can't allocate region R(22751) malloc: *** set a breakpoint in szone_error to debug Errore: memoria 'vector' esaurita (raggiunto il limite?) R(22751) malloc: *** vm_allocate(size=8421376) failed (error code=3) R(22751) malloc: *** error: can't allocate region R(22751) malloc: *** set a breakpoint in szone_error to debug
>
... R not respond, i force to quit the shell. i cant continue to check morph: str(morph) # look inside morph summary(morph) sapply(morph, function(x) any(!is.finite(x))) # check morph for NAs etc. Massimo Il giorno 03/dic/06, alle ore 12:00, r-sig-geo- request at stat.math.ethz.ch ha scritto:
On Sat, 2 Dec 2006, Massimo Di Stefano wrote:
Roger: This is the log,
library(spgrass6)
Carico il pacchetto richiesto: sp Carico il pacchetto richiesto: maptools Carico il pacchetto richiesto: foreign Carico il pacchetto richiesto: rgdal Geospatial Data Abstraction Library extensions to R successfully loaded
library(cluster) sessionInfo()
R version 2.4.0 (2006-10-03) powerpc-apple-darwin8.7.0 locale: it_IT/it_IT/it_IT/C/it_IT/it_IT attached base packages: [1] "methods" "stats" "graphics" "grDevices" "utils" "datasets" [7] "base" other attached packages: cluster spgrass6 rgdal maptools foreign sp "1.11.2" "0.3-3" "0.5-1" "0.6-3" "0.8-18" "0.9-4"
gc()
used (Mb) gc trigger (Mb) max used (Mb) Ncells 408935 11.0 597831 16 531268 14.2 Vcells 320324 2.5 786432 6 472905 3.7
gmeta6 <- gmeta6() x <-
readFLOAT6sp(c("er","crosc","longc","slope","profc","minic","maxic"))
Warning messages:
1: perl = TRUE ? implementato solo nei locale UTF-8
2: perl = TRUE ? implementato solo nei locale UTF-8
3: perl = TRUE ? implementato solo nei locale UTF-8
4: perl = TRUE ? implementato solo nei locale UTF-8
5: perl = TRUE ? implementato solo nei locale UTF-8
6: perl = TRUE ? implementato solo nei locale UTF-8
7: perl = TRUE ? implementato solo nei locale UTF-8
Nowadays I would suggest readRAST6() instead of readFLOAT6sp(), because it uses r.out.bin rather than r.out.arc from GRASS. The perl=TRUE warnings seems to be coming from string comparison somewhere inside a function called by readFLOAT6sp(), I don't think they are the problem.
gc()
used (Mb) gc trigger (Mb) max used (Mb) Ncells 418204 11.2 741108 19.8 699974 18.7 Vcells 22643234 172.8 191414408 1460.4 238993684 1823.4
morph <- data.frame(cbind(x$er, x$crosc, x$longc,
x$slope, x$profc, x$minic, x$maxic))
morph <- as(x, "data.frame") str(morph) # look inside morph summary(morph) sapply(morph, function(x) any(!is.finite(x))) # check morph for NAs etc.
gc()
used (Mb) gc trigger (Mb) max used (Mb) Ncells 418451 11.2 741108 19.8 699974 18.7 Vcells 46682176 356.2 153131526 1168.4 238993684 1823.4 <er, x$crosc, x$longc, x$slope/max(x$slope), x$profc, x$minic, x$maxic))
gc()
used (Mb) gc trigger (Mb) max used (Mb) Ncells 418453 11.2 741108 19.8 699974 18.7 Vcells 46682176 356.2 137841862 1051.7 238993684 1823.4
names(morph) <-
c("er","crosc","longc","slope_n","profc","minic","maxic")
gc()
used (Mb) gc trigger (Mb) max used (Mb) Ncells 418456 11.2 741108 19.8 699974 18.7 Vcells 46682180 356.2 137841862 1051.7 238993684 1823.4
morph.clara <- clara(morph, k=5, stand=F)
Errore in clara(morph, k = 5, stand = F) : Observations 1929241,1931311,1933381,1935451,1937521,1939591,1941661,1943731,19458 01,1947871,1949941,1952011,1954081,1956151,1958221,1960291,1962361,19 64431,1966501,1968571,1970641,1972711,1974781,1976851,1978921,1980991 , 1983061,1985131,1987201,1989271,1991341,1993411,1995481,1997551,19996 21,2001691,2003761,2005831,2007901,2009971,2012041,2014111,2016181,20 18251,2020321,2022391,2024461,2026531,2028601,2030671,2032741,2034811 , 2036881,2038951,2041021,2043091,2045161,2047231,2049301,2051371,20534 41,2055511,2057581,205965 1,2061721,2063791,2065861,2067931,2070001,2072071,2074141,2076211,207 8281,2080351,2082421,2084491,2086561,2088631,2090701,2092771,2094841, 2096911,2098981,2101051,2103121,2105191,2107261,2109331,2111401,21134 71,2115541,2117611,2119681,2121751,2123821,21258 91,2127961,2130031,2132101,2134171,2136241,2138311,2140381,2142451,21 44521,2146591,2148661,2150731,2152801,2154871,2156941,2159011,2161081 ,2163151,2165221,2167291,2169361,2171431
traceback()
2: stop(ngettext(length(i), sprintf("Observation %d
has", i[1]),
sprintf("Observations %s have", paste(i,
collapse = ","))),
" *only* NAs --> omit for clustering")
1: clara(morph, k = 5, stand = F)
is this log usefool ?
Yes, I think the problem is firstly that there are NAs in morph, and then that the string listing the row numbers of the NAs exceeds 8K characters (your earlier message had: Errore in sprintf(fmt, ...) : La lunghezza della stringa eccede la dimensione del buffer di 8192) So try clara on morph once you have omitted the NAs. If the NAs are for all variables in a row, coercing from SpatialGridDataFrame to SpatialPixelsDataFrame will work, if only some, you need to set the whole row to NA, coerce, cluster, add the clara results to the SpatialPixelsDataFrame, coerce back to SpatialGridDataFrame, then back to GRASS. Once this is resolved, please post back to the list as well as to me, NAs do happen, expecially in geomorphometrics, for example around the edges where the filter is not fully within the window and/or mask. Best wishes, Roger
[[alternative HTML version deleted]]
_______________________________________________ R-sig-Geo mailing list R-sig-Geo at stat.math.ethz.ch https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Roger Bivand Economic Geography Section, Department of Economics, Norwegian School of Economics and Business Administration, Helleveien 30, N-5045 Bergen, Norway. voice: +47 55 95 93 55; fax +47 55 95 95 43 e-mail: Roger.Bivand at nhh.no