[Bioc-devel] vignette problems

I'll try to provide you with a pull request addressing issues. Martin

On 04/09/2018 08:42 AM, campos wrote:

Dear devel team,

I am still puzzled with the problems with mac compiling. I am really 
lost and have no idea how to continue or how to be able to check 
about this problems with my linux machine in order to fix it faster. 
Could you please help me with that??

Best,

Rafael


On 05.04.2018 14:29, Shepherd, Lori wrote:

In order for changes to be propagated a version bump in the 
DESCRIPTION file is needed.? Please bump the version in the 
DESCRIPTION file to 2.7.2.




Lori Shepherd

Bioconductor Core Team

Roswell Park Cancer Institute

Department of Biostatistics & Bioinformatics

Elm & Carlton Streets

Buffalo, New York 14263

------------------------------------------------------------------------ 

*From:* Bioc-devel <bioc-devel-bounces at r-project.org> on behalf of 
campos <campos at mpipz.mpg.de>
*Sent:* Thursday, April 5, 2018 7:45:57 AM
*To:* Morgan, Martin; bioc-devel
*Subject:* Re: [Bioc-devel] vignette problems
Hey Martin,

I pushed new changes since last friday but in
https://bioconductor.org/checkResults/3.7/bioc-LATEST/STAN/ says that
the last change date was friday. Any idea what is the problem?

I have tried to fix the problems with memory and all you told me.

Best,

Rafael


On 03.04.2018 17:06, Martin Morgan wrote:

Please use 'reply all' so that the mailing list remains engaged.

Check out the release schedule

http://bioconductor.org/developers/release-schedule/

in particular

Wednesday April 25

- Deadline for packages passing ??R CMD build?? and ??R CMD check??
without errors or warnings.

so you still have time to get your package in order.

Using the same techniques as before, I still see valgrind problems,
the first being

hmm_fitted_poilog = fitHMM(trainRegions, hmm_poilog,

sizeFactors=sizeFactors, maxIters=10)
==24916== Invalid write of size 4
==24916==??? at 0x4BA93FD7:
TransitionMatrix::updateAuxiliaries(double**, double***, double*,
int*, int, int**, int, double, int) (TransitionMatrix.cpp:292)
==24916==??? by 0x4BA77934: HMM::updateSampleAux(double***, int*, 

int,

double**, double**, double**, double***, double*, int*, int*, int*,
int**, double***, SEXPREC*, SEXPREC*, int, double, int, int, int)
(HMM.cpp:771)
==24916==??? by 0x4BA7896D: HMM::BaumWelch[abi:cxx11](double***, 

int*,

int, int, int**, int*, int*, int*, int, int, int**, double***,
SEXPREC*, SEXPREC*, int, double, double, int, int) (HMM.cpp:1076)
==24916==??? by 0x4BA8FEFD: RHMMFit (RWrapper.cpp:1494)
==24916==??? by 0x4F2992D: R_doDotCall (dotcode.c:692)
==24916==??? by 0x4F339D5: do_dotcall (dotcode.c:1252)
==24916==??? by 0x4F81BA6: bcEval (eval.c:6771)
==24916==??? by 0x4F6E963: Rf_eval (eval.c:624)
==24916==??? by 0x4F71188: R_execClosure (eval.c:1764)
==24916==??? by 0x4F70E7C: Rf_applyClosure (eval.c:1692)
==24916==??? by 0x4F6F18B: Rf_eval (eval.c:747)
==24916==??? by 0x4F74B12: do_set (eval.c:2774)
==24916==? Address 0x2e73a294 is 4 bytes inside a block of size 5 

alloc'd

==24916==??? at 0x4C2DB8F: malloc (in
/usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==24916==??? by 0x4BA93FA6:
TransitionMatrix::updateAuxiliaries(double**, double***, double*,
int*, int, int**, int, double, int) (TransitionMatrix.cpp:289)
==24916==??? by 0x4BA77934: HMM::updateSampleAux(double***, int*, 

int,

double**, double**, double**, double***, double*, int*, int*, int*,
int**, double***, SEXPREC*, SEXPREC*, int, double, int, int, int)
(HMM.cpp:771)
==24916==??? by 0x4BA7896D: HMM::BaumWelch[abi:cxx11](double***, 

int*,

int, int, int**, int*, int*, int*, int, int, int**, double***,
SEXPREC*, SEXPREC*, int, double, double, int, int) (HMM.cpp:1076)
==24916==??? by 0x4BA8FEFD: RHMMFit (RWrapper.cpp:1494)
==24916==??? by 0x4F2992D: R_doDotCall (dotcode.c:692)
==24916==??? by 0x4F339D5: do_dotcall (dotcode.c:1252)
==24916==??? by 0x4F81BA6: bcEval (eval.c:6771)
==24916==??? by 0x4F6E963: Rf_eval (eval.c:624)
==24916==??? by 0x4F71188: R_execClosure (eval.c:1764)
==24916==??? by 0x4F70E7C: Rf_applyClosure (eval.c:1692)
==24916==??? by 0x4F6F18B: Rf_eval (eval.c:747)
==24916==

This seems to be the exact same code as in the problem that you fixed
at another location. Actually, I would guess that all of these

grep --color -nH -e ")\*ncores+1" *
HMM.cpp:784:??? int *myStateBuckets = 

(int*)malloc(sizeof(int)*ncores+1);

MultivariateGaussian.cpp:295:??? int *myDimBuckets =
(int*)malloc(sizeof(int)*ncores+1);
MultivariateGaussian.cpp:475:??? int *myDimBuckets =
(int*)malloc(sizeof(int)*ncores+1);
TransitionMatrix.cpp:132:??????? int *myStateBuckets =
(int*)malloc(sizeof(int)*ncores+1);
TransitionMatrix.cpp:289:??? int *myStateBuckets =
(int*)malloc(sizeof(int)*ncores+1);

are the same problem. Also, usually code that has been copy/pasted
like this can instead be refactored to? a single function call, so a
bug can be fixed in one place.

I still see a number of compiler warnings, the first of which is

STAN master$ R CMD INSTALL .
Bioconductor version 3.7 (BiocInstaller 1.29.5), ?biocLite for help
* installing to library
'/home/mtmorgan/R/x86_64-pc-linux-gnu-library/3.5-Bioc-3.7'
* installing *source* package 'STAN' ...
** libs
g++? -I"/home/mtmorgan/bin/R-3-5-branch/include" -DNDEBUG
-I/usr/local/include? -D_RDLL_ -fopenmp? -fpic? -g -O0 -Wall 

-pedantic

-c HMM.cpp -o HMM.o
HMM.cpp: In member function ?virtual void
HMM::calcEmissionProbs(double***, double**, int*, int, int**, int*,
int*, int**, int, int, int*)?:
HMM.cpp:112:15: warning: unused variable ?j? [-Wunused-variable]
???????? int i,j,t,k;
?????????????? ^
It really pays to clean these up; most are harmless, but they obscure
the more important warnings.

Martin


On 04/03/2018 09:58 AM, campos wrote:

Hi Martin,

when I run now valgrind seems that there are no problems. I pushed
yesterday the changes, was I still on time for the new release? We
want to publish the changes and it would be really helpful if the
package is running on Bioconductor.

Thank you very much,

Rafael


On 02.04.2018 02:51, Martin Morgan wrote:

On 04/01/2018 08:06 PM, Martin Morgan wrote:

On 04/01/2018 03:53 PM, campos wrote:

Dear Martin,

I am trying to fix this problem but I am really lost... Do you
mean C++ code? Becasue there is no C code in the whole package. I
really don't know what the problem might be.

Please keep the conversation on the bioc-devel mailing list, so
that others can learn or help.

I use Linux, not Mac, but C (which I used to mean your C++ code)
errors often occur on all platforms but are only visible as a
segfault on one. I created the vignette R code with

?? cd vignettes
?? R CMD Stangle STAN-knitr.Rmd

This produces a file STAN-knitr.R. I then ran your R code with
valgrind

?? R -d valgrind -f STAN-knitr.R

this runs much slower than without valgrind. The first error
reported by valgrind was


?> ##

----STAN-PoiLog----------------------------------------------------------- 

?> nStates = 10
?> hmm_poilog = initHMM(trainRegions, nStates, "PoissonLogNormal",
sizeFactors)
?> hmm_fitted_poilog = fitHMM(trainRegions, hmm_poilog,
sizeFactors=sizeFactors, maxIters=10)
[1] 6
==22304== Invalid write of size 4
==22304==??? at 0x4B489316: HMM::BaumWelch[abi:cxx11](double***,
int*, int, int, int**, int*, int*, int*, int, int, int**,
double***, SEXPREC*, SEXPREC*, int, double, double, int, int)
(HMM.cpp:998)
==22304==??? by 0x4B4A0EFF: RHMMFit (RWrapper.cpp:1494)
==22304==??? by 0x4F2992D: R_doDotCall (dotcode.c:692)
==22304==??? by 0x4F339D5: do_dotcall (dotcode.c:1252)
==22304==??? by 0x4F81BA6: bcEval (eval.c:6771)
==22304==??? by 0x4F6E963: Rf_eval (eval.c:624)
==22304==??? by 0x4F71188: R_execClosure (eval.c:1764)
==22304==??? by 0x4F70E7C: Rf_applyClosure (eval.c:1692)
==22304==??? by 0x4F6F18B: Rf_eval (eval.c:747)
==22304==??? by 0x4F74B12: do_set (eval.c:2774)
==22304==??? by 0x4F6EDF5: Rf_eval (eval.c:699)
==22304==??? by 0x4FB7BEE: Rf_ReplIteration (main.c:258)
==22304==? Address 0x238b28f4 is 4 bytes inside a block of size 5
alloc'd
==22304==??? at 0x4C2DB8F: malloc (in
/usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==22304==??? by 0x4B4892E5: HMM::BaumWelch[abi:cxx11](double***,
int*, int, int, int**, int*, int*, int*, int, int, int**,
double***, SEXPREC*, SEXPREC*, int, double, double, int, int)
(HMM.cpp:995)
==22304==??? by 0x4B4A0EFF: RHMMFit (RWrapper.cpp:1494)
==22304==??? by 0x4F2992D: R_doDotCall (dotcode.c:692)
==22304==??? by 0x4F339D5: do_dotcall (dotcode.c:1252)
==22304==??? by 0x4F81BA6: bcEval (eval.c:6771)
==22304==??? by 0x4F6E963: Rf_eval (eval.c:624)
==22304==??? by 0x4F71188: R_execClosure (eval.c:1764)
==22304==??? by 0x4F70E7C: Rf_applyClosure (eval.c:1692)
==22304==??? by 0x4F6F18B: Rf_eval (eval.c:747)
==22304==??? by 0x4F74B12: do_set (eval.c:2774)
==22304==??? by 0x4F6EDF5: Rf_eval (eval.c:699)

'Invalid write' suggests that you are writing after the end of
memory that you'd allocated. I looked at the C code at the line
where the error occurs as indicated in the stack trace, 

HMM.cpp:998

which is the assigment myStateBucks[i] = 0 in the loop

???? int *myStateBuckets = (int*)malloc(sizeof(int)*ncores+1);
???? for(i=0; i<=ncores; i++)
???? {
???????? myStateBuckets[i] = 0;
???? }

The argument to malloc (where he memory was allocated, at line 

995)

should be the number of bytes to allocate and it should have been
memory for ncores + 1 'int'

?? malloc(sizeof(int) * (ncores + 1))

rather than what you wrote, which is memory for ncores ints plus 1
byte.

C++ code would avoid the need for such explicit memory management,
e.g., using a vector from the standard template library

?? std::vector<int> myStateBuckets(ncores);

oops, std::vector<int> myStateBuckets(ncores + 1); !

There were may other valgrind errors, but I do not know whether
these are from similar programming errors, or a consequence of 

this

one.

Martin

Thanks,
Rafael

On 03/29/2018 01:07 PM, campos wrote:

Dear bioc-devel team,

I have made some changes in the package STAN and although it
seems to install correctly, I have problems with timeout and
error in windows... Could someone help me to improve the time?

https://bioconductor.org/checkResults/3.7/bioc-LATEST/STAN/

it looks like, with your most recent commit (at the top of the
page, 'Snapshot Date', 'Last Commit', 'Last Changed Date'), the
package built on Windows and Linux.

There is a segfault on Mac, which is likely a programming error
in your C code. It could be debugged perhaps using valgrind or
similar tools, but the first step would be to isolate the 

code to

something more easily reproduced than the full vignette. It 

would

also help to clean up the C code so that it compiles without
warnings with the -Wall -pedantic flags

Martin

Best,

Rafael


On 28.03.2018 01:08, Martin Morgan wrote:

When I try and install the version on the master branch of the
Bioconductor git repository I get

STAN master$ Rdev --vanilla CMD INSTALL .
* installing to library
?/home/mtmorgan/R/x86_64-pc-linux-gnu-library
...
** testing if installed package can be loaded
Error: package or namespace load failed for 'STAN' in
namespaceExport(ns, exports):
?undefined exports: viterbi2Gviz


This comes about in a rather interesting way because the body
of plotViterbi is not defined

plotViterbi <- function(viterbi, regions, gen, chrom, from, 

to,

statecols, col)

#'
...

Can you please commit a version of the package that installs?

Martin

On 03/27/2018 06:42 PM, campos wrote:

Dear bioc-devel team,

I am developing the STAN packages and I am running into
problems when trying to build my package. The problem is the
following:

Error in vignette_type(Outfile) :
?? Vignette product 'STAN.tex' does not have a known filename
extension ('NA')
ERROR: installing vignettes failed
* removing '/tmp/Rtmp925Iru/Rinst63471ff1efdc/STAN'
I tried to build the package in old versions (which they used
to work) and I run in other problems but in this case is:

Error in texi2dvi(file = file, pdf = TRUE, clean = clean,
quiet = quiet,? :
?? Running 'texi2dvi' on 'STAN.tex' failed.
LaTeX errors:
! LaTeX Error: File `beramono.sty' not found.

Type X to quit or <RETURN> to proceed,
or enter new name. (Default extension: sty)

Could you help me with this problem?

Thank you very much,

Rafael

This email message may contain legally privileged and/or
confidential information.? If you are not the intended
recipient(s), or the employee or agent responsible for the
delivery of this message to the intended recipient(s), you are
hereby notified that any disclosure, copying, distribution, or
use of this email message is prohibited.? If you have received
this message in error, please notify the sender immediately by
e-mail and delete this email message from your computer. Thank
you.

This email message may contain legally privileged and/or
confidential information.? If you are not the intended
recipient(s), or the employee or agent responsible for the
delivery of this message to the intended recipient(s), you are
hereby notified that any disclosure, copying, distribution, or
use of this email message is prohibited. If you have received
this message in error, please notify the sender immediately by
e-mail and delete this email message from your computer. 

Thank you.

This email message may contain legally privileged
and/or...{{dropped:2}}

This email message may contain legally privileged and/or
confidential information.? If you are not the intended 

recipient(s),

or the employee or agent responsible for the delivery of this
message to the intended recipient(s), you are hereby notified that
any disclosure, copying, distribution, or use of this email message
is prohibited.? If you have received this message in error, please
notify the sender immediately by e-mail and delete this email
message from your computer. Thank you.

This email message may contain legally privileged and/or confidential
information.? If you are not the intended recipient(s), or the
employee or agent responsible for the delivery of this message to the
intended recipient(s), you are hereby notified that any disclosure,
copying, distribution, or use of this email message is 

prohibited.? If

you have received this message in error, please notify the sender
immediately by e-mail and delete this email message from your
computer. Thank you.

This email message may contain legally privileged and/or confidential 
information.? If you are not the intended recipient(s), or the 
employee or agent responsible for the delivery of this message to the 
intended recipient(s), you are hereby notified that any disclosure, 
copying, distribution, or use of this email message is prohibited.? If 
you have received this message in error, please notify the sender 
immediately by e-mail and delete this email message from your 
computer. Thank you.