Skip to content

zip code map - huge files!

7 messages · Enrico Rossi, Hadley Wickham, Dylan Beaudette +1 more

Original
Enrico Rossi
# 
Hello,

I have some data at the zip code level, and I'm using the shapefiles
downloaded from the Census TigerLine website
(http://www2.census.gov/geo/tiger/TIGER2008/tl_2008_us_zcta5.zip) to
plot a shaded map of the US. However, the files generated in this way
are enormous, and take a long time to process, even on a fast machine
with lots of memory. I'm wondering if there's a more efficient way to
do this. Maybe rasterize before plotting somehow?

If anyone on this list has experience working with zip-level data, I'd
appreciate any advice.

Here's some example code like what I'm doing:

# This works, and produces a 1.2GB PDF file! After it's done, I can
rasterize it using gs to reduce file size, but it takes almost an hour
library(maptools)
zip<-readShapePoly("tl_2008_us_zcta5")  # This takes a while!
val<-runif(length(zip[[1]])) # there are about 32000 zip codes
pdf("zipplot.pdf")
plot(zip,xlim=c(-130,-65),ylim=c(20,50),col=grey(val),lty=0)
dev.off()
system("gs -dSAFER -dBATCH -dNOPAUSE -sDEVICE=png16m -r300
-dTextAlphaBits=4 -dGraphicsAlphaBits=4 -dMaxStripSize=8192
-sOutputFile=zipplot.png zipplot.pdf")

# I've tried plotting directly to png, but it just seems to hang, my
patience ran out after two hours
png("zipplot.png")
plot(zip,xlim=c(-130,-65),ylim=c(20,50),col=grey(val),lty=0)
dev.off()

# This also takes too long, I never got any output out of it
library(lattice)
zip$val<-val
pdf("zipplot2.pdf")
spplot(zip,"val",xlim=c(-130,-65),ylim=c(20,50),lty=0)
dev.off()

Many thanks!
Enrico Rossi
Hadley Wickham
# 
Hi Enrico,

I have some code to do map generalisation (reducing map resolution
without visible loss in detail) at
http://github.com/hadley/data-counties/tree/master.  It's applied to
counties data, but would be trivial (if slow) to modify to work with
zip codes instead

Hadley
On Sun, Apr 26, 2009 at 3:06 PM, Enrico Rossi <enrico.a.rossi at gmail.com> wrote:

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Dylan Beaudette
    

    

      
      #
    

  


  

      
Hi,

You can use the v.generalize command in GRASS to reduce the complexity
of vectors like these.

Cheers,
Dylan

        
On Sun, Apr 26, 2009 at 3:53 PM, hadley wickham <h.wickham at gmail.com> wrote:

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Enrico Rossi
    

    

      
      #
    

  


  

      
Hi all,

Thanks for your responses! Dylan, unfortunately I can't install GRASS
(although I'm working on getting it approved).
Hadley, your code sounds very promising. Which bit of code should I
look at? thin? thin-better? Will these methods work with SpatialPoly
objects?

Many thanks,
Enrico

On Sun, Apr 26, 2009 at 7:13 PM, Dylan Beaudette

        
<dylan.beaudette at gmail.com> wrote:

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

            

      
      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Alex Mandel
    

    

      
      #
    

  


  

      
Enrico Rossi wrote:

            

      
      
      
    

        
Enrico,

What size map are you making, and how is it going to be used?
Zip code seems like a fairly small size area to plot over such a large
area, especially when you think about small cities that are only 5-10
miles wide but have 2-4 zip codes.

Have you considered aggregating neighbors based on some geographic or
other parameter.

Alex

  
  


  


      

    

    

      
      

        


  

        

      Hadley Wickham
    

    

      
      #
    

  


  

      
Yes, this is basically the method that my code implements, although I
definitely preserve all junction points, and it's not clear from the
description whether v.generalize does or not.

Hadley

On Sun, Apr 26, 2009 at 6:13 PM, Dylan Beaudette

        
<dylan.beaudette at gmail.com> wrote:

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Hadley Wickham
    

    

      
      #
    

  


  

      
Hi Enrico,

Start with county-to-csv.r - that converts the shape files to normal
csv files.  Then look at thin-all.r, which actually does the thinning
- although I see that I've forgotten to source thin-better.csv.  I
don't have any code to convert it back to into shapefiles because I
don't use them myself.

Hadley

        
On Sun, Apr 26, 2009 at 7:54 PM, Enrico Rossi <enrico.a.rossi at gmail.com> wrote: