[Bioc-devel] affxparser: Core dump with R 2.14.x on OSX [take #2]

On Fri, Jan 20, 2012 at 3:20 PM, Henrik Bengtsson <hb at biostat.ucsf.edu> wrote:

[bringing back to the list, because we could need some help from other
developers with access to various OSX versions]

Hi Dan,

thanks for looking into this.

On Fri, Jan 20, 2012 at 2:17 PM, Dan Tenenbaum <dtenenba at fhcrc.org> wrote:

On Fri, Jan 20, 2012 at 1:57 PM, Dan Tenenbaum <dtenenba at fhcrc.org> wrote:

On Fri, Jan 20, 2012 at 1:44 PM, Dan Tenenbaum <dtenenba at fhcrc.org> wrote:

Hi Henrik,


On Fri, Jan 20, 2012 at 10:33 AM, Henrik Bengtsson <hb at biostat.ucsf.edu> wrote:

Hi,

this is a kind request for the BioC team to have another look at
fixing the binary affxparser builds. ?Quite a few OSX users on R
v2.14.0 have R crashing because of this problem.

Thanks for the prompt and the detailed problem report.

Since thread 'Re: [Bioc-devel] affxparser: Core dump with R 2.14.0 on
OSX' on Nov 7, 2011
[https://stat.ethz.ch/pipermail/bioc-devel/2011-November/002969.html]
became cluttered with mistakes, I'm starting a new thread on the same
topic.

PROBLEM:
The binary build of affxparser v1.26.2 for OSX provided by
Bioconductor is broken and causes R v2.14.0 to crash ("core dump",
"abort trap", ...) on OSX 10.6 ("Snow Leopard") and (I assume; someone
please confirm) OSX 10.7 ("Lion"),

I can confirm that it happens on Lion too.

but not OSX 10.5 ("Leopard"). ?A
reproducible example is:

library("affxparser");
readCdfHeader("Mapping10K_Xba142.cdf");

which should return a named header. (Download CDF file:
http://www.aroma-project.org/data/annotationData/chipTypes/Mapping10K_Xba142/Mapping10K_Xba142.CDF.gz
; 2.2Mb). ?Another example is
[http://www.aroma-project.org/data/annotationData/chipTypes/Mapping250K_Nsp/Mapping250K_Nsp.cdf.gz]:

library("affxparser");
readCdfHeader("Mapping250K_Nsp.cdf");


CURRENT WORKAROUNDS:
- Install affxparser from source
[http://bioconductor.org/packages/2.9/bioc/src/contrib/affxparser_1.26.2.tar.gz].
- Install Kasper Hansen's binary build (not universal?)
[http://www.braju.com/R/repos/osx_10.6/affxparser_1.26.2.tgz] that
works on (at least) OSX 10.6.8.

See also aroma.affymetrix thread 'OSX 10.6 & 10.7 users: Workaround
for faulty BioC build of affxparser v1.26.2' on Jan 14, 2012
[https://groups.google.com/forum/#!topic/aroma-affymetrix/lEfDanThLEA/discussion]


TROUBLESHOOTING:
I can confirm that installing from source, works on an OSX 10.6.8
machine with R v2.14.1
(http://cran.r-project.org/bin/macosx/R-2.14.1.pkg). ?Installing
Kasper's binary build also works. ?I've a limited understanding on the
different types of OSX package binaries, only access to OSX 10.6.8,
making it hard for me to do any more troubleshooting, but as far as I
understand there is something wrong with the way affxparser is build
on the Bioconductor servers.

An important fact to bear in mind is that the BioC Mac build servers
are running Leopard (OS X 10.5.8).

It's a bit tricky to debug since it works fine on the platform it's
built on...but using primitive means (Rprintf() statements), I was
able to narrow down the problem to the
FileHeaderReader::ReadMagicNumber()
function in
affxparser/src/fusion_sdk/calvin_files/parsers/src/FileHeaderReader.cpp

In that function, the expression
if (fileMagicNumber != DATA_FILE_MAGIC_NUMBER)
evaluates to true, and therefore an
affymetrix_calvin_exceptions::InvalidFileTypeException is thrown.

I don't really know why the magic number is wrong, or would vary
between operating systems, but perhaps this gives you something to go
on?

BTW, the trace is:
R: readCdfHeader()
C++:
R_affx_get_cdf_file_header()
FusionCDFData::ReadHeader()
FusionCDFData::CreateObject()
FusionCDFData::IsCalvinCompatibleFile()
GenericFileReader::ReadFileHeaderNoDataGroupHeader()
FileHeaderReader::Read()
FileHeaderReader::ReadMagicNumber()

Hope this helps. If I can be of assistance in further debugging this,
please let me know.

I should also have mentioned that in the ReadMagicNumber() function,
fileMagicNumber == 67
with the file Mapping10K_Xba142.cdf, and the expected magic number is 59.

Sorry for all the emails, but here's one more piece of info:
If I run the package with the debug statements on pitt, our Leopard
build machine, it works fine, as expected, but it also reports that
fileMagicNumber is 67. So the exception is still thrown, but execution
continues.

So when "execution continues" despite the incorrect magic number, do
still get a valid CDF header readout at the R prompt?

Yes. Or at least, I assume it is valid. But no other errors are displayed.

Here is what it displays:

$ncols
[1] 658

$nrows
[1] 658

$nunits
[1] 10208

$nqcunits
[1] 9

$refseq
[1] ""

$chiptype
[1] "Mapping10K_Xba142"

$filename
[1] "./Mapping10K_Xba142.CDF"

$rows
[1] 658

$cols
[1] 658

$probesets
[1] 10208

$qcprobesets
[1] 9

$reference
[1] ""

Whereas on my Lion machine, execution ends (after a pause) with "Abort trap: 6".
So I am not sure whether this exception is really part of the problem,
or just a red herring.

It could be a red herring; the incorrectly read magic header (first
byte in the file) is just a side effect of something more complicated,
but it is definitely a start. ?It is also a hint that we could/should
update affxparser to at least catch this and give an error instead of
crashing (but I'm sure if we should play with such updates, while
troubleshooting the real cause).

There is one more important clue available. This problem started to
occur with BioC 2.9 and R v2.14.x. ?Previous BioC builds of affxparser
did not cause this, and by even forcing an installation of the old
affxparser v1.24.0 binaries on R v2.14.1 on OSX 10.6.8:

?http://bioconductor.org/packages/2.8/bioc/bin/macosx/leopard/contrib/2.13/affxparser_1.24.0.tgz

it works. ?So, something "happened" between:

affxparser_1.24.0.tgz:
Packaged: 2011-04-15 09:35:06 UTC; biocbuild
Built: R 2.13.0; universal-apple-darwin9.8.0; 2011-04-15 16:46:28 UTC; unix
Archs: i386, ppc, x86_64

and

affxparser_1.26.2.tgz
Packaged: 2011-11-17 06:38:13 UTC; biocbuild
Built: R 2.14.0; universal-apple-darwin9.8.0; 2011-11-17 15:39:53 UTC; unix
Archs: i386, ppc, x86_64

(the first known report on this problem is from Nov 7, 2011
[http://goo.gl/ZqBsW], which is before the date of the latter). ?There
is only one real update in affxparser v1.26.1, but that is in pure R
code and more importantly not in code used in this bug report. ?So,
rebuilding affxparser v1.24.0 on the BioC server will most likely
cause the same crash as affxparser v1.26.2 does.


BTW, are you planning to update to R v2.14.1 on the BioC OSX servers?
With some luck, maybe that will fix it.

pitt is already running R 2.14.1, see:
http://bioconductor.org/checkResults/release/bioc-LATEST/pitt-NodeInfo.html

It would be great if someone else with OSX 10.5.8 ("Leopard") could
build/install affxparser v1.26.2 from source are share it with us for
testing on OSX 10.6 & 10.7; that would help narrow down the source of
the problem. ?If such a build works, then it is much more likely that
there is something with the BioC OSX 10.5.8 server setup, whereas if
it also crashes, then we might have to search for the problem
elsewhere.

Sounds good, although we already tried this test, with another Leopard
machine we had, and the resulting package also crashed on newer OSes.
But it is probably still valuable to have someone else try this, and
if they create a package that works on newer OSes, then we can start
to look at differences in compilers, etc.

BTW, the last known 'good' version was built on pelham, here is the
node info for that machine:
http://bioconductor.org/checkResults/2.8/bioc-20111021/pelham-NodeInfo.html

It looks like the same versions of C and C++ compilers were used on both.
One difference is that R CMD config CXX on pelham is "g++-4.2 -arch
i386" whereas on pitt it is "g++ -arch i386", however, on pitt,
ls -l /usr/bin/g++
lrwxr-xr-x ?1 root ?wheel ?7 Jun 29 ?2011 /usr/bin/g++ -> g++-4.2
so I guess they are really equivalent.
(The same is true of gfortran, if that matters.)

This makes me wonder if the "something" that "happened" could have
happened between R 2.13 and 2.14. Of course, the problem seems to be
deep inside C++ code that is not even using SEXPs, but it's possible
something could have changed in .Call()...

I'm in the process of doing further testing (trying to build my debug
version on Lion and seeing if the magic number mismatch occurs). Will
let you know what happens.

Dan

Thanks,

Henrik

Hope this is helpful....
Dan

Dan

Thanks,
Dan

Thanks

Henrik