Managing R CPU/memory usage on Linux

Control groups is probably the way to go if you have it, as Etienne 
suggested, but also consider increasing swappiness, e.g.

echo 100 >> /proc/sys/vm/swappiness

(or more likely the equivalent in whatever /etc/sysctl.conf is on 
Ubuntu).  You are quite low on RAM.  You will of course want a 
decent-sized swap file.

And without control croups you should set resource limits, cf. man 
limits.conf.

You could also try 'ionice -c 3' but the benefit is likely to be low 
(nice -19 already drops your scheduling class to the equivalent of -c 2 
-n 7).

Allan