Skip to content

Problem with WARNING...headers with CRLF line endings

17 messages · William Dunlap, Simon Urbanek, Duncan Murdoch +3 more

Original
cstrato
# 
Dear all,

When running R CMD check on Windows XP to test my package I get the 
following warning message:

"* checking line endings in C/C++/Fortran sources/headers ... WARNING
Found the following sources/headers with CR or CRLF line endings:
   src/xpsDict.h"

The problem is that this file is created by the compiler AUTOMATICALLY 
during the compilation process, and since the file is created by VC++ on 
WinXP, it will always have CRLF line endings.

Thus my question is:
- Is it really necessary to issues this warning message?
- If yes, could it be suppressed on Windows XP, since there it should 
obviously be no problem.

One more issue:
While I have always received this warning on my WinXP installation, for 
some lucky reason the warning did until now not appear on the 
Bioconductor Windows server, see BioC 2.6 with R-2.11.1:
http://bioconductor.org/checkResults/2.6/bioc-LATEST/xps/liverpool-checksrc.html

However, for some reason on BioC 2.7 running R-2.12.0 this warning does 
appear, see:
http://bioconductor.org/checkResults/2.7/bioc-LATEST/xps/liverpool-checksrc.html
For this reason I would  appreciate if there would be a possibility to 
suppress this warning message.

Thank you in advance.
Best regards
Christian
_._._._._._._._._._._._._._._._._._
C.h.r.i.s.t.i.a.n   S.t.r.a.t.o.w.a
V.i.e.n.n.a           A.u.s.t.r.i.a
e.m.a.i.l:        cstrato at aon.at
_._._._._._._._._._._._._._._._._._
Dirk Eddelbuettel
#
On 13 September 2010 at 23:07, cstrato wrote:
| Dear all,
| 
| When running R CMD check on Windows XP to test my package I get the 
| following warning message:
| 
| "* checking line endings in C/C++/Fortran sources/headers ... WARNING
| Found the following sources/headers with CR or CRLF line endings:
|    src/xpsDict.h"
| 
| The problem is that this file is created by the compiler AUTOMATICALLY 
| during the compilation process, and since the file is created by VC++ on 
| WinXP, it will always have CRLF line endings.
| 
| Thus my question is:
| - Is it really necessary to issues this warning message?
| - If yes, could it be suppressed on Windows XP, since there it should 
| obviously be no problem.
| 
| One more issue:
| While I have always received this warning on my WinXP installation, for 
| some lucky reason the warning did until now not appear on the 
| Bioconductor Windows server, see BioC 2.6 with R-2.11.1:
| http://bioconductor.org/checkResults/2.6/bioc-LATEST/xps/liverpool-checksrc.html
| 
| However, for some reason on BioC 2.7 running R-2.12.0 this warning does 
| appear, see:
| http://bioconductor.org/checkResults/2.7/bioc-LATEST/xps/liverpool-checksrc.html
| For this reason I would  appreciate if there would be a possibility to 
| suppress this warning message.

I once had the warning to in project and just added another filtering step
using this


## simple 0d 0a -> 0a converter to suppress a warning on Windows

filename <- commandArgs(trailingOnly=TRUE)[1]
if (!file.exists(filename)) q()

con <- file(filename, "rb")
bin <- readBin(con, raw(), 100000)
bin <- bin[ which(bin != "0d") ]
close(con)

Sys.sleep(1)

con <- file(filename, "wb")
writeBin(bin, con)
close(con)


Maybe you can use something like this and have the generated file transformed.

Dirk
William Dunlap
# 
Older versions of Sun C compilers would refuse to
compile code with Windows-style line endings.  I don't
know if that is still the case.  One reason to run check
is to see if there are any platform-dependencies in
code on CRAN so the warning should not be suppressed.

(The S+ package system tries to avoid the problem by changing
line endings on text files when it compiles the package.
It is not trivial to reliably figure out which files are
meant to be text files.)

Bill Dunlap
Spotfire, TIBCO Software
wdunlap tibco.com
pool-checksrc.html
pool-checksrc.html
cstrato
# 
Dear Dirk,

Thank you for this suggestion, however I have no idea where this code 
could be used.

As I have said, this file is created automatically during the 
compilation by the compiler, and I have no idea at which point "R CMD 
check" does check for CRLF line endings, and how to interfere with this 
step. Do you have any ideas?

Best regards
Christian
On 9/13/10 11:31 PM, Dirk Eddelbuettel wrote:
cstrato
# 
Dear Bill,

It would be great if this warning message could at least be suppressed 
on Windows.

Best regards
Christian
On 9/14/10 2:06 AM, William Dunlap wrote:
Simon Urbanek
#
On Sep 14, 2010, at 2:27 PM, cstrato wrote:

            

      
      
      
    

        
I think you're missing the point - you should be fixing that file instead - it doesn't matter whether you're on Windows or not. If the file is created automatically then it has no business being in the package. Otherwise you should simply post-process it (e.g., as Dirk suggested) after it has been generated but before you create the package - that is the common practice with generated files that need to be part of the distribution.

Cheers,
Simon

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

  
  


  


      

    

    

      
      

        


  

        

      cstrato
    

    

      
      #
    

  


  

      
Dear Simon,

Thank you for this clarification/suggestion, however I am confused. What 
do you mean with "If the file is created automatically then it has no 
business being in the package."?

If you download the source code of my package from:
http://bioconductor.org/packages/2.7/bioc/html/xps.html
you will see that this file is not in the package.

However, I could indeed add the corrected files xpsDict.h and 
xpsDict.cxx to the source code of my package. There is only one problem. 
Currently the source code "xps_1.9.6.tar.gz" has a total size of 4MB. 
Adding the corrected files will add another 4MB to the source code.

Best regards
Christian

        
On 9/14/10 9:59 PM, Simon Urbanek wrote:

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

            

      
      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Hervé Pagès
    

    

      
      #
    

  


  

      
Hi Christian,

        
On 09/14/2010 02:12 PM, cstrato wrote:

            

      
      
      
    

        
He means it shouldn't be in the source tarball. We run 'R CMD check'
on source trees or source tarballs. Source tarballs are platform
independent. If I understand correctly when you run 'R CMD build'
on your source tree, you use some trick to generate this src/xpsDict.h
file on Windows and this file ends up in the source tarball.
But the source tarball created on Linux won't have this file.
That doesn't sound like a good thing to me to have the content
of the source tarball depending on the machine you've run
'R CMD build'. In other words, you should try to generate this
src/xpsDict.h file at installation time but it shouldn't end up
in the source tarball.
My 2 cents...

Cheers,
H.

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

        

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      cstrato
    

    

      
      #
    

  


  

      
Dear Herve,

Thank you for your reply, however maybe I was not quite clear.

The files xpsDict.h and xpsDict.cxx are automatically created by the 
ROOT framework during compilation on every architecture. This means they 
are created on Linux and Mac with LF line endings, but on Windows with 
CRLF line endings. However, they are created only if they do not already 
exist, and thus are not in the source tarball.

For testing purposes I have just added both files with LF line endings 
to the source tarball and compiled it on Windows w/o problems. 
Furthermore, the  size of "xps_1.9.6.tar.gz" increases only from 4MB to 
4.3MB. Thus in principle I could upload both files to SVN for BioC 2.7, 
and this should eliminate the warning message. What is your opinion?

Best regards
Christian

        
On 9/14/10 11:32 PM, Herv? Pag?s wrote:

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

            

      
      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Hervé Pagès
    

    

      
      #
    

  


  

      
On 09/14/2010 02:58 PM, cstrato wrote:

            

      
      
      
    

        
on every architecture... ok
But if they are created during compilation, why do they need to be
included in the source tarball? They are just temporary files right?
Or I'm missing something...

            

      
      
      
    

        
I guess you mean they are not part of the source *tree*.

            

      
      
      
    

        
I still don't understand why you want to have them in the source
tarball.

H.

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Duncan Murdoch
    

    

      
      #
    

  


  

      
On 14/09/2010 6:08 PM, Herv? Pag?s wrote:

            

      
      
      
    

        
I think he doesn't want to put them in the source tarball, but because 
of the way R CMD check works, he may have to.

It appears that R CMD check builds those files, and then checks for CRLF 
endings on all files.  If it did the CRLF check first, it wouldn't see 
them and complain.  The problem with this change is that some packages 
might create files with CRLF endings on all platforms, and then check 
*should* complain about them.

My advice would be not to put them in the tarball, and ignore the 
warning.  Or write a Makevars.win that fixes the line endings so that 
check is happy.

Duncan Murdoch

  
  


  


      

    

    

      
      

        


  

        

      Hervé Pagès
    

    

      
      #
    

  


  

      
On 09/14/2010 03:30 PM, Duncan Murdoch wrote:

            

      
      
      
    

        
I see your point but, on the other hand, and more generally speaking,
you expect 'R CMD check' to check the source files i.e. the files that
belong to the source tarball, and not temporary compilation/installation
products that 'R CMD INSTALL' didn't remove (for whatever reason).

It's weird to get a message like:

   * checking line endings in C/C++/Fortran sources/headers ... WARNING
   Found the following sources/headers with CR or CRLF line endings:
     src/xpsDict.h

if there is no such file in the source tarball.

            

      
      
      
    

        
Yes and since he already uses a Makefile, it should be easy to remove
those files at the end of installation so they don't end up in the
tarball anymore. Wouldn't that also be enough to silent 'R CMD check'?
Perhaps package authors should really make sure that src/ gets
cleaned after the installation step of 'R CMD build' (this step is
performed only if the package has vignettes). Then it shouldn't matter
whether 'R CMD check' checks for CRLF endings after or before
installing the package.

Cheers,
H.

            

      
      
      
    

        

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      cstrato
    

    

      
      #
    

  


  

      
Dear Duncan, dear Herve,

Thank you both for your help and suggestions. I think that you are both 
right:

In principle I do not want to put these files in the source tarball (and 
I did not in the past), however because of the way R CMD check works 
this seems to be the only possibility to get rid of the warning message.

I agree that it is weird to get this warning message although these 
files are not in the source tarball.

Since I have a Makefile, as Herve mentions, I can try to remove these 
files in the clean step, which currently is:

clean:
	rm -f $(MYOBJ) *.a *.d *.rc

I will try to change this part but I am not sure if this will solve the 
problem.

Best regards
Christian

        
On 9/15/10 1:58 AM, Herv? Pag?s wrote:

            

      
      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Dirk Eddelbuettel
    

    

      
      #
    

  


  

      
On 15 September 2010 at 22:39, cstrato wrote:

        
| Dear Duncan, dear Herve,
| 
| Thank you both for your help and suggestions. I think that you are both 
| right:
| 
| In principle I do not want to put these files in the source tarball (and 
| I did not in the past), however because of the way R CMD check works 
| this seems to be the only possibility to get rid of the warning message.
| 
| I agree that it is weird to get this warning message although these 
| files are not in the source tarball.
| 
| Since I have a Makefile, as Herve mentions, I can try to remove these 

If you have a Makefile, then you have implicit rules as well as explicit
ones.  That is how the file gets created.

Now modify the rules and insert another layer which will do the filtering. As
the saying goes:  "there is no problem that cannot be solved by adding
another layer of indirection".  Instead of requiring the generated file,
require the generated and filtered file.

| files in the clean step, which currently is:
| 
| clean:
| 	rm -f $(MYOBJ) *.a *.d *.rc
| 
| I will try to change this part but I am not sure if this will solve the 
| problem.

I fear that the clean step runs too late.  You want the file filtered at the
compile stage.

Dirk

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      cstrato
    

    

      
      #
    

  


  

      
Dear Dirk,

Thank you for this suggestion, however since I am not very good in 
creating Makefiles I would appreciate if you could explain how to filter 
at the compile stage.

These are the lines which I think are essential:

xps.dll:  $(MYOBJ)
	$(LINK) /dll /def:xps.def /out:xps.dll fp10.obj $(SOFLAGS) $(LDFLAGS) 
$(GLIBS) *.obj
	
xpsDict.cxx: $(MYHDR) xpsLinkDef.h
	@echo "Generating dictionary $@..."
	@rootcint -f $@ -c $(MYHDR) xpsLinkDef.h
	
xpsDict.obj: xpsDict.cxx xpsDict.h
	$(CL) /I"${ROOTSYS}/include" /FIw32pragma.h /MT /EHsc /Ox /D "MSVC" /D 
"WIN32" /c xpsDict.cxx

clean:
	rm -f $(MYOBJ) *.a *.d *.rc

The line @rootcint creates the xpsLinkDef.h file.

Do I have to add something after the compilation step of xpsDict.obj?

Thank you in advance.
Best regards
Christian

        
On 9/15/10 10:51 PM, Dirk Eddelbuettel wrote:

            

      
      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Hervé Pagès
    

    

      
      #
    

  


  

      
On 09/15/2010 01:51 PM, Dirk Eddelbuettel wrote:

            

      
      
      
    

        
But shouldn't 'R CMD check' run the clean step (if it runs it at all)
right after the 'R CMD INSTALL' step?

Cheers,
H.

  
    

      
      
    

  


  


      

    

    
9 days later

    

      
      

        


  

        

      cstrato
    

    

      
      #
    

  


  

      
Dear Herve,

I have just tested the modification of Makefile.win:

clean:
	rm -f $(MYOBJ) *.a *.d *.rc xpsLinkDef.h

and it was sufficient to eliminate the warning message.
Thus R CMD check seems indeed to run the clean step.

I have just uploaded the new version to BioC2.7.

Best regards
Christian

        
On 9/15/10 11:53 PM, Herv? Pag?s wrote: