Greetings, I have a package which provides headers for a C++ library (similar to the BH package). However, the C++ library has some heavily nested components within its' structure so when I run R CMD check I get the following NOTE: Tarballs are only required to store paths of up to 100 bytes and cannot store those of more than 256 bytes, with restrictions including to 100 bytes for the final component. Is this a major problem? As this is a 'NOTE' I am under the impression that the long paths are not critical but I want to make sure. Thank you, Regards, Charles
[R-pkg-devel] R CMD check NOTE - Long paths in package
8 messages · Dirk Eddelbuettel, Barry Rowlingson, Uwe Ligges +3 more
On 12 October 2015 at 13:13, Charles Determan wrote:
| Greetings, | | I have a package which provides headers for a C++ library (similar to the | BH package). However, the C++ library has some heavily nested components | within its' structure so when I run R CMD check I get the following NOTE: | | Tarballs are only required to store paths of up to 100 bytes and cannot | store those of more than 256 bytes, with restrictions including to 100 | bytes for the final component. | | Is this a major problem? Yes. | As this is a 'NOTE' I am under the impression | that the long paths are not critical but I want to make sure. Wrong. 'BH' is called 'BH', believe it or not, to save a few letters, or else I would have a dozen or more files complain for the very same reason (and CRAN reject it for non-suitable path/filenames). Hence not 'BoostHeaders' but just 'BH'. And see the ChangeLog of BH and revel in how _each_ release has to rename one of the Runge-Kutta integration files to make the 'net' path length be suitable for R packing. There is no point in ignoring the output of R CMD check unless you _really_ know better. Dirk
http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org
On Tue, Oct 13, 2015 at 4:16 AM, Dirk Eddelbuettel <edd at debian.org> wrote:
On 12 October 2015 at 13:13, Charles Determan wrote: | Greetings, | | I have a package which provides headers for a C++ library (similar to the | BH package). However, the C++ library has some heavily nested components | within its' structure so when I run R CMD check I get the following NOTE: | | Tarballs are only required to store paths of up to 100 bytes and cannot | store those of more than 256 bytes, with restrictions including to 100 | bytes for the final component. | | Is this a major problem? Yes. | As this is a 'NOTE' I am under the impression | that the long paths are not critical but I want to make sure. Wrong. 'BH' is called 'BH', believe it or not, to save a few letters, or else I would have a dozen or more files complain for the very same reason (and CRAN reject it for non-suitable path/filenames). Hence not 'BoostHeaders' but just 'BH'. And see the ChangeLog of BH and revel in how _each_ release has to rename one of the Runge-Kutta integration files to make the 'net' path length be suitable for R packing. There is no point in ignoring the output of R CMD check unless you _really_ know better.
But is it worth asking if the NOTE's restriction could be lifted, given that from what I read on wikipedia that the "tar" file format has been happy with long path names since 2001? Given that R-Core/CRAN sometimes refer to R versions from a year ago as "obsolete", how can they complain about sticking with a file format restriction from the last century? I've just happily made a tar file with a path that is 1300 chars long containing a file with a name of 160 chars . Or are there still old versions of tar around, or other file systems with restrictive naming practices (in which case the error should be about the names/paths of the files themselves, not that old versions of tar can't cope with them)? It now seems to have bothered at least two people and Dirk sounds like he's had to implement a little hack to keep the path names down. Barry
On 13.10.2015 09:01, Barry Rowlingson wrote:
On Tue, Oct 13, 2015 at 4:16 AM, Dirk Eddelbuettel <edd at debian.org> wrote:
On 12 October 2015 at 13:13, Charles Determan wrote: | Greetings, | | I have a package which provides headers for a C++ library (similar to the | BH package). However, the C++ library has some heavily nested components | within its' structure so when I run R CMD check I get the following NOTE: | | Tarballs are only required to store paths of up to 100 bytes and cannot | store those of more than 256 bytes, with restrictions including to 100 | bytes for the final component. | | Is this a major problem? Yes. | As this is a 'NOTE' I am under the impression | that the long paths are not critical but I want to make sure. Wrong. 'BH' is called 'BH', believe it or not, to save a few letters, or else I would have a dozen or more files complain for the very same reason (and CRAN reject it for non-suitable path/filenames). Hence not 'BoostHeaders' but just 'BH'. And see the ChangeLog of BH and revel in how _each_ release has to rename one of the Runge-Kutta integration files to make the 'net' path length be suitable for R packing. There is no point in ignoring the output of R CMD check unless you _really_ know better.
But is it worth asking if the NOTE's restriction could be lifted, given that from what I read on wikipedia that the "tar" file format has been happy with long path names since 2001? Given that R-Core/CRAN sometimes refer to R versions from a year ago as "obsolete", how can they complain about sticking with a file format restriction from the last century? I've just happily made a tar file with a path that is 1300 chars long containing a file with a name of 160 chars . Or are there still old versions of tar around, or other file systems with restrictive naming practices (in which case the error should be about the names/paths of the files themselves, not that old versions of tar can't cope with them)?
Yes, some Unixes seem to have old tar versions installed. Best, Uwe Ligges
It now seems to have bothered at least two people and Dirk sounds like he's had to implement a little hack to keep the path names down. Barry
______________________________________________ R-package-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
On 13 Oct 2015, at 09:42 , Uwe Ligges <ligges at statistik.tu-dortmund.de> wrote:
On 13.10.2015 09:01, Barry Rowlingson wrote:
On Tue, Oct 13, 2015 at 4:16 AM, Dirk Eddelbuettel <edd at debian.org> wrote:
On 12 October 2015 at 13:13, Charles Determan wrote: | Greetings, | | I have a package which provides headers for a C++ library (similar to the | BH package). However, the C++ library has some heavily nested components | within its' structure so when I run R CMD check I get the following NOTE: | | Tarballs are only required to store paths of up to 100 bytes and cannot | store those of more than 256 bytes, with restrictions including to 100 | bytes for the final component. | | Is this a major problem? Yes. | As this is a 'NOTE' I am under the impression | that the long paths are not critical but I want to make sure. Wrong. 'BH' is called 'BH', believe it or not, to save a few letters, or else I would have a dozen or more files complain for the very same reason (and CRAN reject it for non-suitable path/filenames). Hence not 'BoostHeaders' but just 'BH'. And see the ChangeLog of BH and revel in how _each_ release has to rename one of the Runge-Kutta integration files to make the 'net' path length be suitable for R packing. There is no point in ignoring the output of R CMD check unless you _really_ know better.
But is it worth asking if the NOTE's restriction could be lifted, given that from what I read on wikipedia that the "tar" file format has been happy with long path names since 2001? Given that R-Core/CRAN sometimes refer to R versions from a year ago as "obsolete", how can they complain about sticking with a file format restriction from the last century? I've just happily made a tar file with a path that is 1300 chars long containing a file with a name of 160 chars . Or are there still old versions of tar around, or other file systems with restrictive naming practices (in which case the error should be about the names/paths of the files themselves, not that old versions of tar can't cope with them)?
Yes, some Unixes seem to have old tar versions installed.
Yes, it is somewhat harder to deem entire OSs obsolete... But: How old versions and how common is the issue? Mainstream Unix is essentially Linux and OS X these days. OS X seems to be stuck in 2010 (BSD tar 2.8.3), at least up until Yosemite. Linux distros usually keep themselves up to date, except for the enterprise/long upgrade cycle ones that can be like 5 years outdated. Less common, but still in use, are FreeBSD, Solaris and AIX. It _is_ a valid question for how long we want out-of-the-box support for OSs that use obsolete tools. At some point we could decide that the more obscure Unices would have to install a recent version (with the R configuration arranging to set TAR=/usr/local/bin/gtar or somesuch). -pd
Best, Uwe Ligges
It now seems to have bothered at least two people and Dirk sounds like he's had to implement a little hack to keep the path names down. Barry
______________________________________________ R-package-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
______________________________________________ R-package-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Peter Dalgaard, Professor, Center for Statistics, Copenhagen Business School Solbjerg Plads 3, 2000 Frederiksberg, Denmark Phone: (+45)38153501 Office: A 4.23 Email: pd.mes at cbs.dk Priv: PDalgd at gmail.com
I'm glad to see this discussion. Unfortunately in the short term I cannot remove the nested files (as Dirk implies with the BH package) as it would require restructuring the c++ library and make it far more difficult to maintain. Unless there are other suggestions I will need to hold off on submission until such a time that OSs that use obsolete tools are no longer supported out-of-the-box (as Peter says). I will likely just continue to maintain the package on my gitub until CRAN would accept the longer paths. Charles
On Tue, Oct 13, 2015 at 4:02 AM, peter dalgaard <pdalgd at gmail.com> wrote:
On 13 Oct 2015, at 09:42 , Uwe Ligges <ligges at statistik.tu-dortmund.de> wrote:
On 13.10.2015 09:01, Barry Rowlingson wrote:
On Tue, Oct 13, 2015 at 4:16 AM, Dirk Eddelbuettel <edd at debian.org>
wrote:
On 12 October 2015 at 13:13, Charles Determan wrote: | Greetings, | | I have a package which provides headers for a C++ library (similar
to the
| BH package). However, the C++ library has some heavily nested
components
| within its' structure so when I run R CMD check I get the following
NOTE:
| | Tarballs are only required to store paths of up to 100 bytes and
cannot
| store those of more than 256 bytes, with restrictions including to
100
| bytes for the final component. | | Is this a major problem? Yes. | As this is a 'NOTE' I am under the impression | that the long paths are not critical but I want to make sure. Wrong. 'BH' is called 'BH', believe it or not, to save a few letters, or else
I
would have a dozen or more files complain for the very same reason
(and CRAN
reject it for non-suitable path/filenames). Hence not 'BoostHeaders'
but
just 'BH'. And see the ChangeLog of BH and revel in how _each_ release
has to
rename one of the Runge-Kutta integration files to make the 'net' path
length
be suitable for R packing. There is no point in ignoring the output of R CMD check unless you
_really_
know better.
But is it worth asking if the NOTE's restriction could be lifted, given that from what I read on wikipedia that the "tar" file format has been happy with long path names since 2001? Given that R-Core/CRAN sometimes refer to R versions from a year ago as "obsolete", how can they complain about sticking with a file format restriction from the last century? I've just happily made a tar file with a path that is 1300 chars long containing a file with a name of 160 chars . Or are there still old versions of tar around, or other file systems with restrictive naming practices (in which case the error should be about the names/paths of the files themselves, not that old versions of tar can't cope with them)?
Yes, some Unixes seem to have old tar versions installed.
Yes, it is somewhat harder to deem entire OSs obsolete... But: How old versions and how common is the issue? Mainstream Unix is essentially Linux and OS X these days. OS X seems to be stuck in 2010 (BSD tar 2.8.3), at least up until Yosemite. Linux distros usually keep themselves up to date, except for the enterprise/long upgrade cycle ones that can be like 5 years outdated. Less common, but still in use, are FreeBSD, Solaris and AIX. It _is_ a valid question for how long we want out-of-the-box support for OSs that use obsolete tools. At some point we could decide that the more obscure Unices would have to install a recent version (with the R configuration arranging to set TAR=/usr/local/bin/gtar or somesuch). -pd
Best, Uwe Ligges
It now seems to have bothered at least two people and Dirk sounds like he's had to implement a little hack to keep the path names down. Barry
______________________________________________ R-package-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
______________________________________________ R-package-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
-- Peter Dalgaard, Professor, Center for Statistics, Copenhagen Business School Solbjerg Plads 3, 2000 Frederiksberg, Denmark Phone: (+45)38153501 Office: A 4.23 Email: pd.mes at cbs.dk Priv: PDalgd at gmail.com
______________________________________________ R-package-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Regardless of tar or not, note that there are other limits in R on how long *file* names can be, e.g.
pathname <- paste(rep("a", times=255), collapse="")
nchar(pathname)
[1] 255
cat("hello", file=pathname)
pathname <- paste(rep("a", times=256), collapse="")
nchar(pathname)
[1] 256
cat("hello", file=pathname)
Error in file(file, ifelse(append, "a", "w")) : cannot open the connection In addition: Warning message: In file(file, ifelse(append, "a", "w")) : cannot open file 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa': File name too long The above is the output from R 3.2.2 on x86_64-pc-linux-gnu (64-bit). I don't know if/where this is documented. Also, and possibly related to the above, the MS Windows operating system has a limit of 256 characters on the *absolute* *pathname* (directory and file names including slashes but not initial drive letter specification), cf. http://aroma-project.org/howtos/UseLongFilenamesOnWindows/. For instance, if your package is installed in, say, library "C:/Users/JohnDoe/R/win-library/3.2/", you've already used up 35 characters out of your 256 available ones just there. Then add the length of your package name. So, allowing for fairly long usernames on Windows, in order for your package to build/install on Windows as well, you might still want to aim for something like a maximum of 200-character *relative* pathnames. (You might get away with a bit longer src/ pathnames if you build a Windows binary package under, say, "T:/', because src/ is not part of the distributed binary, so the user can still install it, but I don't know how far you want to push that). /Henrik
On Tue, Oct 13, 2015 at 1:04 PM, Charles Determan <cdetermanjr at gmail.com> wrote:
I'm glad to see this discussion. Unfortunately in the short term I cannot remove the nested files (as Dirk implies with the BH package) as it would require restructuring the c++ library and make it far more difficult to maintain. Unless there are other suggestions I will need to hold off on submission until such a time that OSs that use obsolete tools are no longer supported out-of-the-box (as Peter says). I will likely just continue to maintain the package on my gitub until CRAN would accept the longer paths. Charles On Tue, Oct 13, 2015 at 4:02 AM, peter dalgaard <pdalgd at gmail.com> wrote:
On 13 Oct 2015, at 09:42 , Uwe Ligges <ligges at statistik.tu-dortmund.de> wrote:
On 13.10.2015 09:01, Barry Rowlingson wrote:
On Tue, Oct 13, 2015 at 4:16 AM, Dirk Eddelbuettel <edd at debian.org>
wrote:
On 12 October 2015 at 13:13, Charles Determan wrote: | Greetings, | | I have a package which provides headers for a C++ library (similar
to the
| BH package). However, the C++ library has some heavily nested
components
| within its' structure so when I run R CMD check I get the following
NOTE:
| | Tarballs are only required to store paths of up to 100 bytes and
cannot
| store those of more than 256 bytes, with restrictions including to
100
| bytes for the final component. | | Is this a major problem? Yes. | As this is a 'NOTE' I am under the impression | that the long paths are not critical but I want to make sure. Wrong. 'BH' is called 'BH', believe it or not, to save a few letters, or else
I
would have a dozen or more files complain for the very same reason
(and CRAN
reject it for non-suitable path/filenames). Hence not 'BoostHeaders'
but
just 'BH'. And see the ChangeLog of BH and revel in how _each_ release
has to
rename one of the Runge-Kutta integration files to make the 'net' path
length
be suitable for R packing. There is no point in ignoring the output of R CMD check unless you
_really_
know better.
But is it worth asking if the NOTE's restriction could be lifted, given that from what I read on wikipedia that the "tar" file format has been happy with long path names since 2001? Given that R-Core/CRAN sometimes refer to R versions from a year ago as "obsolete", how can they complain about sticking with a file format restriction from the last century? I've just happily made a tar file with a path that is 1300 chars long containing a file with a name of 160 chars . Or are there still old versions of tar around, or other file systems with restrictive naming practices (in which case the error should be about the names/paths of the files themselves, not that old versions of tar can't cope with them)?
Yes, some Unixes seem to have old tar versions installed.
Yes, it is somewhat harder to deem entire OSs obsolete... But: How old versions and how common is the issue? Mainstream Unix is essentially Linux and OS X these days. OS X seems to be stuck in 2010 (BSD tar 2.8.3), at least up until Yosemite. Linux distros usually keep themselves up to date, except for the enterprise/long upgrade cycle ones that can be like 5 years outdated. Less common, but still in use, are FreeBSD, Solaris and AIX. It _is_ a valid question for how long we want out-of-the-box support for OSs that use obsolete tools. At some point we could decide that the more obscure Unices would have to install a recent version (with the R configuration arranging to set TAR=/usr/local/bin/gtar or somesuch). -pd
Best, Uwe Ligges
It now seems to have bothered at least two people and Dirk sounds like he's had to implement a little hack to keep the path names down. Barry
______________________________________________ R-package-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
______________________________________________ R-package-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
-- Peter Dalgaard, Professor, Center for Statistics, Copenhagen Business School Solbjerg Plads 3, 2000 Frederiksberg, Denmark Phone: (+45)38153501 Office: A 4.23 Email: pd.mes at cbs.dk Priv: PDalgd at gmail.com
______________________________________________ R-package-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
[[alternative HTML version deleted]]
______________________________________________ R-package-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
On 13 October 2015 at 15:04, Charles Determan wrote:
| I'm glad to see this discussion. Unfortunately in the short term I cannot | remove the nested files (as Dirk implies with the BH package) as it would Rename, not remove. Of course I didn't remove a file. I changed a filename, and the include statement leading to it. | require restructuring the c++ library and make it far more difficult to | maintain. Unless there are other suggestions I will need to hold off on | submission until such a time that OSs that use obsolete tools are no longer | supported out-of-the-box (as Peter says). I will likely just continue to | maintain the package on my gitub until CRAN would accept the longer paths. Which still keeps it uninstallable on some systems. Dirk
http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org