Corrupting files while copying (was Re: saveRDS() and readRDS() Why? [solved, pretty much anyway])
Sequence of steps: Using R-3.5.1 and Windows 7 with the latest Rstudio, shared directory as working directory on Virtual Box host machine:
x <- airquality saveRDS(x, file = "x.rds") saveRDS(x, file = "y.rds")
On guest machine Mint Linux 17.3, KDE desktop, copy x.rds & y.rds to working directory PWD using file manager Dolphin. (Don't have the precise version of VirtualBox right now.)
x <- readRDS(file = "x.rds")
Error in readRDS(file = "x.rds") : error reading from connection
x <- readRDS(file = "y.rds")
tools::md5sum(c("x.rds", "y.rds"))
x.rds y.rds "5fef054848f39b4be02b7c54f1c71a20" "978a64d1dd342d16a381c9ca728d3665" Yet, if instead of using Dolphin, use bash commands from the shared directory $ cp *.rds ~/PWD/ no error reading from the connection or other differences between x.rds and y.rds. head(x)
head(datasets::airquality)
Ozone Solar.R Wind Temp Month Day 1 41 190 7.4 67 5 1 2 36 118 8.0 72 5 2 3 12 149 12.6 74 5 3 4 18 313 11.5 62 5 4 5 NA NA 14.3 56 5 5 6 28 NA 14.9 66 5 6
On Thu, 15-Nov-2018 at 09:53AM -0500, Ista Zahn wrote:
|> Hi Patrick, |> |> I think it would help to start from the beginning and give complete |> (but concise!) replication instructions, including telling us what |> host and gest operating systems you are using (including the |> versions), the version |> of virtualbox you used, and exactly what steps are needed to |> reproduce the surprising behavior. |> |> Best, |> Ista |> On Wed, Nov 14, 2018 at 2:36 AM Patrick Connolly
|> <p_connolly at slingshot.co.nz> wrote:
|> > |> > Thanks William, |> > |> > I've used Dolphin for years and never encountered that phenomenon. |> > Even so, that description doesn't fit what's going on here. 1.7 |> > kilobytes is hardly a 'large directory'. |> > |> > The problem seems to be with the way VirtualBox mounts directories |> > which isn't an R issue, nor is the fact that copying from Linux to |> > Windows isn't affected. But the fact that it happens only with rds |> > files that use the name of the R object as part of their own names |> > must be an R issue (that surfaces only when other conditions are |> > present). |> > |> > Theories short of divine intervention appreciated. |> > |> > |> >
|> > On Tue, 13-Nov-2018 at 02:22PM -0800, William Dunlap wrote:
|> > |> > |> Perhaps you got bitten by Dolphin's non-modal dialogs, as described in |> > |> https://userbase.kde.org/Dolphin/File_Management: |> > |> |> > |> Non Modal Dialogs |> > |> |> > |> When Moving, Copying or Deleting files/directories the dialog disappears |> > |> even when the operation has not yet completed. A progress bar then appears |> > |> in the bottom right of the screen, this then disappears also, if you want |> > |> see the progress you need to click a small (i) information icon in the |> > |> system tray. |> > |> |> > |> |> > |> Warning |> > |> New users who are not used to this way of working (and even experienced |> > |> users) can get caught out by this, if you are Moving, Copying or Deleting |> > |> large directories then you need to use the icon to monitor the progress of |> > |> your operation. If you don't then any subsequent actions you do, may well |> > |> use an incomplete file structure resulting in corrupted files. You have |> > |> been warned! |> > |> |> > |> Bill Dunlap |> > |> TIBCO Software |> > |> wdunlap tibco.com |> > |> |> > |> On Tue, Nov 13, 2018 at 2:10 PM, p_connolly <p_connolly at slingshot.co.nz>
|> > |> wrote:
|> > |> |> > |> > This is getting more strange. |> > |> > |> > |> > I normally copy from the shared folder to the appropriate directory using |> > |> > Dolphin, the KDE file manager. If instead I use the standard bash cp |> > |> > command, no corruption happens -- at least with the limited testing I have |> > |> > done. There also seems to be no problem copying from Linux to Windows. I |> > |> > installed R-3.5.1 for Windows just to eliminate that possible issue. |> > |> > |> > |> > However, R has *something* to do with it because it was used to make the |> > |> > .rds file. Just how the relationship between the name of the R object and |> > |> > the name of the .rds file comes into it, I can't imagine. |> > |> > |> > |> > Thanks for the suggestion William. |> > |> > |> > |> >
|> > |> > On 2018-11-14 06:26, William Dunlap wrote:
|> > |> > |> > |> >> It seems like copying the files corrupted them. How did you copy them |> > |> >> (with R |> > |> >> or cp or copy or ftp, etc.)? I don't see how this has anything to do |> > |> >> with R. |> > |> >> |> > |> >> Bill Dunlap |> > |> >> TIBCO Software |> > |> >> wdunlap tibco.com [1] |> > |> >> On Mon, Nov 12, 2018 at 7:10 PM, p_connolly
|> > |> >> <p_connolly at slingshot.co.nz> wrote:
|> > |> >>
|> > |> > [...]
|> > |> >
|> > |> >
|> >
|> > --
|> > ~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.
|> > ___ Patrick Connolly
|> > {~._.~} Great minds discuss ideas
|> > _( Y )_ Average minds discuss events
|> > (:_~*~_:) Small minds discuss people
|> > (_)-(_) ..... Eleanor Roosevelt
|> >
|> > ~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.
|> >
|> > ______________________________________________
|> > R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
|> > https://stat.ethz.ch/mailman/listinfo/r-help
|> > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
|> > and provide commented, minimal, self-contained, reproducible code.
~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.
___ Patrick Connolly
{~._.~} Great minds discuss ideas
_( Y )_ Average minds discuss events
(:_~*~_:) Small minds discuss people
(_)-(_) ..... Eleanor Roosevelt
~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.