NA command in a 'for' loop

Thank you all for your input.

This is an example of one data file (I have 74 data files):

2.90546E+11, threat,    1, 2, 1, 2, 1,        death,        stove,
  NA,           NA,  205,    0,  394
2.90546E+11, threat,    2, 2, 2, 1, 1,    emaciated,    shortened,
  NA,           NA,  205,    0,  502
2.90546E+11, threat,    3, 1, 1, 1, 2,     mutilate,     consider,
  NA,           NA,  205,    1,  468
2.90546E+11, threat,    6, 1, 2, 2, 1,         weep,         shop,
  NA,           NA,  203,    1,  345
2.90546E+11, threat,    9, 2, 1, 2, 2,    tormented,    easygoing,
  NA,           NA,  205,    1,  373
2.90546E+11, threat,   10, 1, 2, 2, 2,        snake,        table,
  NA,           NA,  205,    1,  343
2.90546E+11, threat,   11, 2, 2, 1, 1,       crisis,       faucet,
  NA,           NA,  203,    1,  437
2.90546E+11, threat,   12, 1, 1, 1, 1,       victim,      utensil,
  NA,           NA,  203,    1,  343
2.90546E+11, threat,   14, 1, 2, 2, 1,    depressed,    repentant,
  NA,           NA,  203,    1,  441
2.90546E+11, threat,   15, 2, 2, 1, 2,         scum,         shoe,
  NA,           NA,  205,    1,  475

?Column 13 has values of 0s and 1s which my cognitive task outputted.
Column 14 is the reaction time (ms) data. I want to get rid of the rows
that contain zeros so I thought I'd first replace zeros with NAs then use
complete.cases function to get rid of the NAs. I also wanted to apply other
functions so I included them all in a loop. All work fine except for the
one where I try to turn the zeros to NAs.

Jim when I tried your mockdata example, it worked fine. But when I
translated it to my data, I still get zeros in the output. Can you identify
any mistranslations I'm doing?

txt.files<-list.files(".",pattern="dotprobe") #all my data files are text
files in one folder
for(tf in txt.files) {
  d<-read.table(tf)
  d[,13][d[,13]==0]<-NA #column 13 contains zeros
  d<-d[ ,-c(10,11)] #get rid of columns 10 and 11
  write.table(d,sub("[.]",".tlbs.",tf),quote=FALSE, row.names=FALSE)
}

That's an example of one of the output I get:

V1 V2 V3 V4 V5 V6 V7 V8 V9 V12 V13 V14
2.90546E+11, threat, 1, 2, 1, 2, 1, death, stove, 205, 0, 394
2.90546E+11, threat, 2, 2, 2, 1, 1, emaciated, shortened, 205, 0, 502
2.90546E+11, threat, 3, 1, 1, 1, 2, mutilate, consider, 205, 1, 468
2.90546E+11, threat, 6, 1, 2, 2, 1, weep, shop, 203, 1, 345
2.90546E+11, threat, 9, 2, 1, 2, 2, tormented, easygoing, 205, 1, 373
2.90546E+11, threat, 10, 1, 2, 2, 2, snake, table, 205, 1, 343

Columns 10 and 11 were deleted. But zeros were not replaced by NAs.
After all the data cleaning, the functions I'm interested in including in
the loop are: get_tlbs and summarize_bias (and these also work fine in my
loop).

Thanks again ?
Sincerely
Helen
------------------------------
*From:* Jim Lemon <drjimlemon at gmail.com>
*Sent:* Tuesday, April 21, 2020 2:52 AM
*To:* Rui Barradas <ruipbarradas at sapo.pt>
*Cc:* Helen Sawaya <helensawaya at hotmail.com>; Michael Dewey <
lists at dewey.myzen.co.uk>; r-help at R-project.org <r-help at r-project.org>
*Subject:* Re: [R] NA command in a 'for' loop

Hi Helen,
Your problem may lie in using row.names=TRUE. I was puzzled when an
extra column kept popping up in the output files. For reading in and
replacing zeros with NAs, this seems to work:

for(mockdata in 1:3) {
 mdf<-data.frame(sample(2:20,10),sample(2:20,10),sample(0:1,10,TRUE))
 write.table(mdf,file=paste0("threat",mockdata,".txt"),quote=FALSE,
  row.names=FALSE,col.names=FALSE)
}
txt.files<-list.files(".",pattern="threat[1-3]")
for(tf in txt.files) {
 d<-read.table(tf)
 d[,3][d[,3]==0]<-NA
 write.table(d,sub("[.]",".tbls.",tf),quote=FALSE,row.names=FALSE)
}

Jim

On Tue, Apr 21, 2020 at 7:57 AM Rui Barradas <ruipbarradas at sapo.pt> wrote:

Hello,

I believe the only way we have to see what is happening is for you to
post the output of


dput(head(d, 20))  # or 30


or, with d2 a subset of d that includes zeros,


dput(head(d2, 20))


Hope this helps,

Rui Barradas

?s 17:48 de 20/04/20, Helen Sawaya escreveu:

I have one column that represents correct response versus error

(correct

is coded as 1 and error is coded as 0). Nowhere else in the dataset are
there values of 0. The vector is treated as an integer.

------------------------------------------------------------------------

*From:* Michael Dewey <lists at dewey.myzen.co.uk>
*Sent:* Monday, April 20, 2020 7:35 PM
*To:* Helen Sawaya <helensawaya at hotmail.com>; Rui Barradas
<ruipbarradas at sapo.pt>; r-help at R-project.org <r-help at R-project.org>
*Subject:* Re: [R] NA command in a 'for' loop
Just a thought Helen but is x being treated as a real and what you

think

are zero and are printed as zero are in fact some very small number? If
so you need to alter your test appropriately.

Michael

On 20/04/2020 17:25, Helen Sawaya wrote:

Thank you for your reply.

I tried d[] <- lapply(d, function(x) {is.na(x) <- x == 0; x})
but I am still getting zeros instead of NAs in my output..

I wonder if the problem is that some of my data files don't have any

zeros (participants made no errors)..

<r-help at R-project.org>

Subject: Re: [R] NA command in a 'for' loop

Hello,

Instead of

d[d == 0] <- NA

try

d[] <- lapply(d, function(x) {is.na(x) <- x == 0; x})


Also, in the first for loop

paste(i, sep = "")

does nothing, it's the same as i.
And the same for

(d2$V4 == 1) == TRUE

Since (d2$V4 == 1)  already is FALSE/TRUE there is no need for

(.) == TRUE


Hope this helps,

Rui Barradas



?s 20:52 de 19/04/20, Helen Sawaya escreveu:

Dear R experts,

I am using a 'for' loop to apply commands to multiple datasets (each

file is one participant). The only one not working is the command that
identifies zeros in my datasets and changes them to NAs. But when I look at
the output, zeros ("0") are still present.  Surprisingly, the functions
work fine when I apply them to a single

dataset (outside the loop). I've tried:

all.files <- list.files(".")
txt.files <- grep("threat.txt",all.files,value=T)

for(i in txt.files){
    d <- read.table(paste(i,sep=""),header=F)
    d[d==0] <- NA #replace zeros with NA
    write.table(d, paste0(i,".tlbs.txt"), quote=FALSE,

row.names=TRUE)}

    d<-d[ ,-c(10,11)]
    d2<-d[complete.cases(d), ]
    d2$V4<-as.numeric(d2$V4)
    congruent <- (d2$V4 == 1) == TRUE
    x <- get_tlbs(d2$V14, congruent, prior_weights = NULL, method =

"weighted", fill_gaps = FALSE)

    write.table(x, paste0(i,".tlbs.txt"), quote=FALSE,

row.names=TRUE)}

I've also tried:

for(i in txt.files){
    d <- read.table(paste(i,sep=""),header=F)
    if (0 %in% d)
    {replace_with_na(d,replace = list(x = 0))} # replace zeros with

NA

    d<-d[ ,-c(10,11)]
    d2<-d[complete.cases(d), ]
    d2$V4<-as.numeric(d2$V4)
    congruent <- (d2$V4 == 1) == TRUE
    x <- get_tlbs(d2$V14, congruent, prior_weights = NULL, method =

"weighted", fill_gaps = FALSE)

    write.table(x, paste0(i,".summaryoutput.txt"), quote=FALSE,

row.names=TRUE)}

Thank you for your help.
Sincerely
Helen

        [[alternative HTML version deleted]]