Skip to content

Summary of colSums

8 messages · a296180 at mica.fmr.com (David Kane, Douglas Bates, Mark Myatt +4 more

Original
a296180 at mica.fmr.com (David Kane
# 
Thanks to Andy Liaw, Douglas Bates and Marcel Wolters for spending time on my
problem. In summary, there does not seem to be a built in colSums in R as there
is in S+ 6.0. However, Doug Bates was kind enough to send me a simple math
utilities package that has a version of colSums in C. The package installed
without a hitch and produced similar performance to the S+ colSums function (in
my simple tests). Both Andy Liaw and Marcel Wolters pointed out that the built
in matrix operators (like %*%) are likely to be faster than apply, especially
if one has compiled R with the Atlas library (which I have not yet done).

Thanks again to all,

Dave Kane
-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To: r-help-request at stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._
Douglas Bates
# 
"David Kane  <David Kane" <a296180 at mica.fmr.com> writes:
Actually I called the package MatUtils for "Matrix Utilities".

Some on the list may be interested in how the package was created.
Although the information on how to do this in the "Writing R
Extensions" Manual, available as a PDF file in the doc section on
CRAN, it is good to see a short example.

Because the calculation is simple and we want it to run fast, I used C
code and the .Call interface.  

The first step is to write the R function colSums.
In R-1.3.0 you can use the package.skeleton function to create the
skeleton of a package directory.
This creates a directory called MatUtils along with the R, man, data,
and src subdirectories and the DESCRIPTION, README R/colSums.R and
man/colSums.Rd files.

I then added src/colSums.c
--- begin colSums.c ---
#include <R.h>
#include <Rdefines.h>

SEXP colSums(SEXP m)
{
    int i, j, *mdims, n, p;
    double *mm, sum;
    SEXP val;

    if (!isMatrix(m)) error("m must be a matrix");
    mdims = INTEGER(coerceVector(getAttrib(m, R_DimSymbol), INTSXP));
    n = mdims[0]; p = mdims[1];
    PROTECT(val = allocVector(REALSXP, p));
    PROTECT(m = AS_NUMERIC(m));
    mm = REAL(m);
    for (j = 0; j < p; j++) {
        sum = 0.;
        for (i = 0; i < n; i++) sum += mm[i];
        REAL(val)[j] = sum;
        mm += n;
    }
    UNPROTECT(2);
    return val;
}
--- end colSums.c ---
edited the DESCRIPTION file so it reads
--- begin DESCRIPTION ---
Package: MatUtils
Title: Some utility functions for Matrices
Version: 1.0
Author: Douglas Bates <bates at stat.wisc.edu>
Description: Matrix Utility functions such as colSums
Maintainer: Douglas Bates <bates at stat.wisc.edu>
License: GPL version 2 or later
--- end DESCRIPTION ---
edited man/colSums.Rd to read
--- begin man/colSums.Rd ---
\name{colSums}
\alias{colSums}
\title{Compute the column sums of a matrix}
\description{
  \code{colSums} computes the column sums of a matrix
}
\usage{
colSums(m)
}
\arguments{
  \item{m}{a numeric matrix.}
}
\value{
  a numeric vector of the column sums of the matrix \code{m}
}
\author{Douglas Bates \email{bates at stat.wisc.edu}}
\examples{
colSums(matrix(1:20, ncol = 4))
}
\keyword{array}
\keyword{algebra}
\keyword{math}
--- end colSums.Rd ---
and - this is important - added a file R/zzz.R with contents
--- begin R/zzz.R ---
.First.lib <- function(lib, pkg) {
    library.dynam("MatUtils", pkg, lib)
}
--- end R/zzz.R ---

I think I would have forgotten to write the .First.lib function but
Thomas has thoughtfully provided a README file in the src directory
that reminds you to do this.

On Linux/Unix one can then cd to the parent directory of MatUtils and
run

R CMD build --force MatUtils   # this creates the INDEX file
R CMD check MatUtils

If the check fails (and, in fact, I did have mistakes in the C code in
the first version I tried to compile) then go through the usual
edit/debug cycle.

As for timings, you wrote (and I hope it is ok to send this to the list)

 Thanks for the answer and the package. It installed without a hitch on Solaris
 2.6. Here is a summary of the *very* brief comparison that I did.

 > library(MatUtils)
 > m <- matrix(rnorm(10000000), 100000, 100)
 > system.time(a1 <- apply(m, 2, sum))
 [1] 6.77 0.00 6.87 0.00 0.00
 > system.time(a2 <- colSums(m))
 [1] 0.43 0.00 0.43 0.00 0.00
 > all.equal(a1, a2)
 [1] TRUE

 Compare these to the results in S+ 6.0.

 > m <- matrix(rnorm(10000000), 100000, 100)
 > sys.time(a1 <- apply(m, 2, sum))
 [1] 18.74 26.09
 > sys.time(a2 <- colSums(m))
 [1] 0.49 0.55
 > all.equal(a1, a2)
 [1] T
-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To: r-help-request at stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._
Mark Myatt
# 
Dear All,

Just a note to say that I have just posted a new version of my R notes
to my web-site:

        http://www.myatt.demon.co.uk

As a Zip archive including RTF and PDF versions of the document as well
as example datasets.

This is just a minor revision. Changes are:

        1. References to "survival5" now reference "survival" - the
        package name changed with v1.3.0.

        2. Information on install.packages() is supplemented with a note
        of the use of the "Packages" menu on GUI versions.

        3. The document is now released under the GPL/FDL license ...
        share and enjoy!

        4. One reference (Selvin, S, "Modern Applied Biostatistical
        Methods Using S-Plus", Oxford University Press, New York, 1998)
        has been added.

        5. Document naming now follows R versions (i.e. Rex1030.pdf).

I will have some time later in the year to revise this document. I am
thinking of including:

        1. Introduction to writing your own (simple) graphics functions
        using a "Bland & Altman" plot as an example.

        2. Examples of using sample() &c. to do bootstraps and jack-
        knives.

Does anyone think that this is worth it or have any other suggestions as
to what could be usefully added? I would like to hear from anyone who
has used this material in the classroom ... did it work?

Best wishes,

Mark


--
Mark Myatt
-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To: r-help-request at stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._
Tim Hoar
# 
I am building R-1.3.0 on a Sun running Solaris 2.6 and have sorted
out all the "hard" stuff ... save this ...

When trying to make install-pdf, I run into an unknown file:



This is pdfTeX, Version 3.14159-14f-released-20000525 (Web2C 7.3.2x)
(/tmp/tmp.1742.16125/t2d13975/xtr/R-admin.texi{/opt/local/teTeX/share/texmf/pdf
tex/config/pdftex.cfg}
Babel <v3.6x> and hyphenation patterns for american, french, german, ngerman,
n
ohyphenation, loaded.
(texinfo.tex Loading texinfo [version 2001-05-24.08]: Basics, pdf,
! I can't find file `pdfcolor'.
l.926   \input pdfcolor

Please type another input file name:


I searched through the CTAN site to no avail, downloaded and inflated
pdftex ... in short ... made quite an effort to locate pdfcolor to no avail.

Any hints?

This is preventing me from building the .pdf versions of the manuals.
I am still tracking down why I have lots of zero-length files in my
doc/manual directory.

-rw-r--r--   1 thoar    cgdcas         0 Jul 19 10:51 R-FAQ.html
-rw-r--r--   1 thoar    cgdcas         0 Jul 19 10:51 R-lang.html
-rw-r--r--   1 thoar    cgdcas         0 Jul 19 10:51 R-intro.html
-rw-r--r--   1 thoar    cgdcas         0 Jul 19 10:51 R-exts.html
-rw-r--r--   1 thoar    cgdcas         0 Jul 19 10:51 R-data.html
-rw-r--r--   1 thoar    cgdcas         0 Jul 19 10:51 R-admin.html
-rw-r--r--   1 thoar    cgdcas         0 Jul 19 11:03 stamp-refman-dvi
-rw-r--r--   1 thoar    cgdcas         0 Jul 19 11:03 R-admin.tp
-rw-r--r--   1 thoar    cgdcas         0 Jul 19 11:03 R-admin.pg
-rw-r--r--   1 thoar    cgdcas         0 Jul 19 11:03 R-admin.ky
-rw-r--r--   1 thoar    cgdcas         0 Jul 19 11:03 R-admin.fn
-rw-r--r--   1 thoar    cgdcas         0 Jul 19 11:03 R-data.tp
-rw-r--r--   1 thoar    cgdcas         0 Jul 19 11:03 R-data.pg
-rw-r--r--   1 thoar    cgdcas         0 Jul 19 11:03 R-data.ky
-rw-r--r--   1 thoar    cgdcas         0 Jul 19 11:03 R-data.fn
-rw-r--r--   1 thoar    cgdcas         0 Jul 19 11:04 R-exts.tp
-rw-r--r--   1 thoar    cgdcas         0 Jul 19 11:04 R-exts.pg
-rw-r--r--   1 thoar    cgdcas         0 Jul 19 11:04 R-exts.ky
-rw-r--r--   1 thoar    cgdcas         0 Jul 19 11:04 R-exts.fn
-rw-r--r--   1 thoar    cgdcas         0 Jul 19 11:04 stamp-images-eps
-rw-r--r--   1 thoar    cgdcas         0 Jul 19 11:05 R-intro.tp
-rw-r--r--   1 thoar    cgdcas         0 Jul 19 11:05 R-intro.pg
-rw-r--r--   1 thoar    cgdcas         0 Jul 19 11:05 R-intro.ky
-rw-r--r--   1 thoar    cgdcas         0 Jul 19 11:05 R-intro.fn
-rw-r--r--   1 thoar    cgdcas         0 Jul 19 11:06 R-lang.tp
-rw-r--r--   1 thoar    cgdcas         0 Jul 19 11:06 R-lang.pg
-rw-r--r--   1 thoar    cgdcas         0 Jul 19 11:06 R-lang.ky
-rw-r--r--   1 thoar    cgdcas         0 Jul 19 11:06 R-lang.fn
-rw-r--r--   1 thoar    cgdcas         0 Jul 19 11:18 stamp-refman-pdf
-rw-r--r--   1 thoar    cgdcas         0 Jul 19 11:29 R-FAQ.vr
-rw-r--r--   1 thoar    cgdcas         0 Jul 19 11:29 R-FAQ.tp
-rw-r--r--   1 thoar    cgdcas         0 Jul 19 11:29 R-FAQ.pg
-rw-r--r--   1 thoar    cgdcas         0 Jul 19 11:29 R-FAQ.ky
-rw-r--r--   1 thoar    cgdcas         0 Jul 19 11:29 R-FAQ.fn
-rw-r--r--   1 thoar    cgdcas         0 Jul 19 11:29 R-FAQ.cp


## Tim Hoar, Associate Scientist              email: thoar at ucar.edu     ##
## Geophysical Statistics Project             phone: 303-497-1708       ##
## National Center for Atmospheric Research   FAX  : 303-497-1333       ##
## Boulder, CO  80307                    http://www.cgd.ucar.edu/~thoar ##

-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To: r-help-request at stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._
Kurt Hornik
# 
This is part of pdftex.

See e.g.
	http://www.tug.org/applications/pdftex/

-k
-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To: r-help-request at stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._
2 days later
Tim Hoar
# 
Perhaps at one time it was included in pdftex ...

I downloaded     /pub/tex/local/cstug/thanh/pdftex/0.14h/pdftex-20010417.tgz
(The source of the distribution mentioned at the TUG site Kurt references.)

installed it ... and lo ... there is    NO   pdfcolor.tex   file in the
distribution. I downloaded it from the DUKE mirror with essentially the
same result.

As you can see from my original post ... I am using pdfTeX circa May, 2000
and the newest one (without pdfcolor.tex) is circa April 2001.

I also used the CTAN search for pdfcolor.tex to no avial, so I am guessing it
is not currently in the distribution.

What now?

In a message from Han The Thanh
http://www.tug.org/pipermail/pdftex/1999-December/006060.html

   "pdfcolor.tex redefines \makeheadline and \makefootline. Please notice
    that this file is for demonstration purpose only: LaTeX users should
    use graphicx/color package, and plain users are supposed to know what
    they are doing :). The first line of pdfcolor.tex says:

    % Very simple macros to show how to use colors with pdftex

    Perhaps I should remove the re-definition of \makeheadline and
    \makefootline from pdfcolor.tex, as it might be dangerous.

    Thanh"

So I wonder about building in a dependency.

The URL was pretty much "the only" hit for a google on pdfcolor.tex

Tim
## Tim Hoar, Associate Scientist              email: thoar at ucar.edu     ##
## Geophysical Statistics Project             phone: 303-497-1708       ##
## National Center for Atmospheric Research   FAX  : 303-497-1333       ##
## Boulder, CO  80307                    http://www.cgd.ucar.edu/~thoar ##


-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To: r-help-request at stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._
Peter Dalgaard
# 
Tim Hoar <thoar at cgd.ucar.edu> writes:
Hmmm. Sounds like we need to investigate a little. The file *is*
present in tetex 1.0.7:

[pd at butterfly R]$ rpm -qf /usr/share/texmf/pdftex/plain/misc/pdfcolor.tex
tetex-1.0.7-15

The file itself has a bunch of color definition, as in

\def\cmykGreenYellow{0.15 0 0.69 0}
\def\cmykYellow{0 0 1 0}
...
\def\GreenYellow{\pdfsetcolor{\cmykGreenYellow}}
\def\Yellow{\pdfsetcolor{\cmykYellow}}
...

and the following stuff at the end:

\def\pdfsetcolor#1{\pdfliteral{#1 k}}
\def\setcolor#1{\mark{#1}\pdfsetcolor{#1}}

\pdfoutput=1
\def\maincolor{\cmykBlack}
\pdfsetcolor{\maincolor}

\def\makefootline{
    \baselineskip24pt
    \line{\pdfsetcolor{\maincolor}\the\footline}}

\def\makeheadline{%
    \edef\M{\topmark}
    \ifx\M\empty\let\M=\maincolor\fi
    \vbox to 0pt{\vskip-22.5pt
        \line{\vbox to8.5pt{}%
        \pdfsetcolor{\maincolor}\the\headline\pdfsetcolor{\M}}\vss}%
    \nointerlineskip}


You might want to investigate whether that code got migrated to other
parts of pdftex...
Brian Ripley
#
On 23 Jul 2001, Peter Dalgaard BSA wrote:

            

      
      
      
    

        
Install a current teTeX, fptex or MikTeX, all of which AFAIK have the file?

AFAIK pdfcolor.tex is only used by R in texi2dvi, and so one probably needs
to look to the texinfo maintainers for what *they* are expecting.
Given the low level of support for texi2dvi --pdf, perhaps we need to ship
pdfcolor.tex (as we already ship texinfo.tex).