Skip to content

Antwort: Re: selecting columns from a data frame or data table by type, ie, numeric, integer

5 messages · Martin Maechler, G.Maubach at weinwolf.de, PIKAL Petr +1 more

#
Hi All,
Hi Carl,

I am not sure if this is useful to you, but I followed your conversation 
and thought of you when I read this:

for (i in 1:ncol(dataset)) {
  if(class(dataset) == "character|numeric|factor|or whatsoever") {
    dataset[, i] <- as.factor(dataset[, i])
  }
}
Source: Zumel, Nina / Mount, John: Practical Data Science with R, Manning 
Publications: Shelter Island, 2014, Chapter 2: Loading data into R, p. 25

This way you can select variables of a certain class only and do 
transformations. I found that this approach is not applicable if used with 
statistical functions like head(). Transformations worked fine for me.

I found reading the above given source worthwile.

Kind regards

Georg

PS: I am not related to the above given authors. I am just a reader 
reporting on - at least to me - a valuable ressource.



Von:    Carl Sutton via R-help <r-help at r-project.org>
An:     William Dunlap <wdunlap at tibco.com>, 
Kopie:  "r-help at r-project.org" <r-help at r-project.org>
Datum:  29.04.2016 22:08
Betreff:        Re: [R] selecting columns from a data frame or data table 
by type, ie, numeric, integer
Gesendet von:   "R-help" <r-help-bounces at r-project.org>



Thank you Bill Dunlap.  So simple I never tried that approach. Tried 
dozens of others though, read manuals till I was getting headaches, and of 
course the answer was simple when one is competent.   Learning, its a 
struggle, but slowly getting there.
Thanks again
 Carl Sutton CPA
 

    On Friday, April 29, 2016 10:50 AM, William Dunlap <wdunlap at tibco.com> 
wrote:
 
 

 > dt1[ vapply(dt1, FUN=is.numeric, FUN.VALUE=NA) ]    a   c1   1 1.12   2 
1.0...10 10 0.2


Bill Dunlap
TIBCO Software
wdunlap tibco.com
On Fri, Apr 29, 2016 at 9:19 AM, Carl Sutton via R-help
<r-help at r-project.org> wrote:
Good morning RGuru's
I have a data frame of 575 columns.  I want to extract only those columns 
that are numeric(double) or integer to do some machine learning with.  I 
have searched the web for a couple of days (off and on) and have not found 
anything that shows how to do this.   Lots of ways to extract rows, but 
not columns.  I have attempted to use "(x == y)" indices extraction method 
but that threw error that == was for atomic vectors and lists, and I was 
doing this on a data frame.

My test code is below

#  a technique to get column classes
library(data.table)
a <- 1:10
b <- c("a","b","c","d","e","f","g","h","i","j")
c <- seq(1.1, .2, length = 10)
dt1 <- data.table(a,b,c)
str(dt1)
col.classes <- sapply(dt1, class)
head(col.classes)
dt2 <- subset(dt1, typeof = "double" | "numeric")
str(dt2)
dt2   #  not subset
dt2 <- dt1[, list(typeof = "double")]
str(dt2)
class_data <- dt1[,sapply(dt1,is.integer) | sapply(dt1, is.numeric)]
class_data
sum(class_data)
typeof(class_data)
names(class_data)
str(class_data)
 Any help is appreciated
Carl Sutton CPA


______________________________________________
R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide 
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.




 

______________________________________________
R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide 
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
#
Ouch -- so many problems in such a short piece of R code !!!
Sorry, but after reading the above, I'd strongly recommend getting
better books about R...
       {{maybe do not take those containing "data science" ;-)}}

Compared to the nice and efficient solution of Bill Dunlap,
the above is really bad-bad-bad  in at least four ways :

0) They way you write it above, you cannot use it,
     <string> == "variant1|variant2|..."
   is pseudocode and does not really work

1) Note the missing "[, i]"  in the 2nd line: It should be
     if(class(dataset[, i]) ...

2) A for loop changing each column at a time is really slow for
   largish data sets

3) [last but not at all least!]
   Please ... many of you readers, do learn:
  
 Using checks such as
       if ( class(x) == "numeric" )
 are (almost) always wrong by design !!!

 Instead you really should (almost) always use

 	 if(inherits(x, "numeric"))

Why?  Because classes in R (S3 or S4) can *extend* other classes.
Example: Many of you know that after   fm <- glm(...)
class(fm) is   c("glm", "lm")   and so

    > if(class(fm) == "lm")
    + "yes"
    Warning message:
    In if (class(fm) == "lm") "yes" :
      the condition has length > 1 and only the first element will be used

Similarly, in your case

y <- 1:10
class(y) <- c("myNumber", "numeric")

when that 'y' is a column in your data frame,
the test for  if(class(dataset[,i]) == "numeric")  will *not*
work but actually produce the above warning.

However, one  could als have had

Num <- setClass("Num", contains="numeric")
N <- Num(1:10)

     > Num <- setClass("Num", contains="numeric")
     > N <- Num(1:10)
     > N
     An object of class "Num"
      [1]  1  2  3  4  5  6  7  8  9 10
     > if(class(N) == "numeric") "yes" else "no"
     [1] "no"
     > 

I hope that many of the readers --- including *MANY* authors of
R packages !! --- have understood the above and will fix their R
code -- and even more their books where applicable !!

Martin Maechler,
ETH Zurich & R Core Team
#
Hi Martin,

many thanks for your answer and your broad explanation. 

I am a newbie to "R" and got help on this list and thought I could give 
something back what looked OK to me.

regarding 0)
You're right, it's pseudo code. I assumed that anybody on the list would 
be able to adapt the code to their needs so that it worked. Next time I 
will post runnable code.

regarding 1)
Your right: "[, i]" is missing. My fault. Sorry.

regarding 3)
I got your point and will do better in the future.

One question: What books do you recommend to read to get to know "R" 
better?

Kind regards

Georg




Von:    Martin Maechler <maechler at stat.math.ethz.ch>
An:     <G.Maubach at weinwolf.de>, 
Kopie:  Carl Sutton <suttoncarl at ymail.com>, "r-help at r-project.org" 
<r-help at r-project.org>
Datum:  04.05.2016 09:05
Betreff:        [R] Antwort: Re: selecting columns from a data frame or 
data table      by type, ie, numeric, integer
Ouch -- so many problems in such a short piece of R code !!!
Manning
25

Sorry, but after reading the above, I'd strongly recommend getting
better books about R...
       {{maybe do not take those containing "data science" ;-)}}

Compared to the nice and efficient solution of Bill Dunlap,
the above is really bad-bad-bad  in at least four ways :

0) They way you write it above, you cannot use it,
     <string> == "variant1|variant2|..."
   is pseudocode and does not really work

1) Note the missing "[, i]"  in the 2nd line: It should be
     if(class(dataset[, i]) ...

2) A for loop changing each column at a time is really slow for
   largish data sets

3) [last but not at all least!]
   Please ... many of you readers, do learn:
 
 Using checks such as
       if ( class(x) == "numeric" )
 are (almost) always wrong by design !!!

 Instead you really should (almost) always use

                  if(inherits(x, "numeric"))

Why?  Because classes in R (S3 or S4) can *extend* other classes.
Example: Many of you know that after   fm <- glm(...)
class(fm) is   c("glm", "lm")   and so

    > if(class(fm) == "lm")
    + "yes"
    Warning message:
    In if (class(fm) == "lm") "yes" :
      the condition has length > 1 and only the first element will be used

Similarly, in your case

y <- 1:10
class(y) <- c("myNumber", "numeric")

when that 'y' is a column in your data frame,
the test for  if(class(dataset[,i]) == "numeric")  will *not*
work but actually produce the above warning.

However, one  could als have had

Num <- setClass("Num", contains="numeric")
N <- Num(1:10)

     > Num <- setClass("Num", contains="numeric")
     > N <- Num(1:10)
     > N
     An object of class "Num"
      [1]  1  2  3  4  5  6  7  8  9 10
     > if(class(N) == "numeric") "yes" else "no"
     [1] "no"
     > 

I hope that many of the readers --- including *MANY* authors of
R packages !! --- have understood the above and will fix their R
code -- and even more their books where applicable !!

Martin Maechler,
ETH Zurich & R Core Team
with
table
of
<wdunlap at tibco.com>
columns
found
method
#
Hi
Freely available from CRAN

These I can recommend as I went through them

?A Guide for the Unwilling S User? by Patrick Burns
?Using R for Data Analysis and Graphics - Introduction, Examples and Commentary?  by John Maindonald (PDF, data sets and scripts are available at JM's homepage).
?Practical Regression and Anova using R? by Julian Faraway (PDF, data sets and scripts are available at the book homepage).

There are also some German books but I do not have experience with them.

Books from bookstores
Modern Applied Statistics with S. Fourth Edition, by W. N. Venables and B. D. Ripley
Introductory Statistics with R, Authors: Dalgaard, Peter

Cheers
Petr
________________________________
Tento e-mail a jak?koliv k n?mu p?ipojen? dokumenty jsou d?v?rn? a jsou ur?eny pouze jeho adres?t?m.
Jestli?e jste obdr?el(a) tento e-mail omylem, informujte laskav? neprodlen? jeho odes?latele. Obsah tohoto emailu i s p??lohami a jeho kopie vyma?te ze sv?ho syst?mu.
Nejste-li zam??len?m adres?tem tohoto emailu, nejste opr?vn?ni tento email jakkoliv u??vat, roz?i?ovat, kop?rovat ?i zve?ej?ovat.
Odes?latel e-mailu neodpov?d? za eventu?ln? ?kodu zp?sobenou modifikacemi ?i zpo?d?n?m p?enosu e-mailu.

V p??pad?, ?e je tento e-mail sou??st? obchodn?ho jedn?n?:
- vyhrazuje si odes?latel pr?vo ukon?it kdykoliv jedn?n? o uzav?en? smlouvy, a to z jak?hokoliv d?vodu i bez uveden? d?vodu.
- a obsahuje-li nab?dku, je adres?t opr?vn?n nab?dku bezodkladn? p?ijmout; Odes?latel tohoto e-mailu (nab?dky) vylu?uje p?ijet? nab?dky ze strany p??jemce s dodatkem ?i odchylkou.
- trv? odes?latel na tom, ?e p??slu?n? smlouva je uzav?ena teprve v?slovn?m dosa?en?m shody na v?ech jej?ch n?le?itostech.
- odes?latel tohoto emailu informuje, ?e nen? opr?vn?n uzav?rat za spole?nost ??dn? smlouvy s v?jimkou p??pad?, kdy k tomu byl p?semn? zmocn?n nebo p?semn? pov??en a takov? pov??en? nebo pln? moc byly adres?tovi tohoto emailu p??padn? osob?, kterou adres?t zastupuje, p?edlo?eny nebo jejich existence je adres?tovi ?i osob? j?m zastoupen? zn?m?.

This e-mail and any documents attached to it may be confidential and are intended only for its intended recipients.
If you received this e-mail by mistake, please immediately inform its sender. Delete the contents of this e-mail with all attachments and its copies from your system.
If you are not the intended recipient of this e-mail, you are not authorized to use, disseminate, copy or disclose this e-mail in any manner.
The sender of this e-mail shall not be liable for any possible damage caused by modifications of the e-mail or by delay with transfer of the email.

In case that this e-mail forms part of business dealings:
- the sender reserves the right to end negotiations about entering into a contract in any time, for any reason, and without stating any reasoning.
- if the e-mail contains an offer, the recipient is entitled to immediately accept such offer; The sender of this e-mail (offer) excludes any acceptance of the offer on the part of the recipient containing any amendment or variation.
- the sender insists on that the respective contract is concluded only upon an express mutual agreement on all its aspects.
- the sender of this e-mail informs that he/she is not authorized to enter into any contracts on behalf of the company except for cases in which he/she is expressly authorized to do so in writing, and such authorization or power of attorney is submitted to the recipient or the person represented by the recipient, or the existence of such authorization is known to the recipient of the person represented by the recipient.
#
Hi Martin and list:
First let me thank you for thinking of me.?? It is probably apparent that my programming experience is limited, and the vector aspect of R has taken some getting used to.?? A very very long time ago I did some programming in Fortran and for loops and if statements were ordinary, useful, and used frequently.?? Now if I see a for loop and if statement together then it is flatly apparent that I need to rework that code to take advantage of R's strengths using vectoized functions on whatever object I am working on.??
It has also become apparent that one should read manuals, experiment with some code to firmly cement the knowledge, and build a solid foundation.?? My natural inclination is the opposite.? I am anxious to produce some code to solve a problem or satisfy curiosity or ....? R is not something one can learn and use productively in a few weeks or months.?? It is powerful, subtle, and takes some thinking.?? I started this trek into R with Jared Lander's book "R for Everyone",? progressed to Prof Norman Matloff's "The Art of R Programming" (which was way beyond my comprehension at the beginning) and Hadley Wickham's "ggplot2" book, and several courses with Data Camp and Couresra.? One day I will be at the point where I do know what I don't know about R and at that time I will almost be competent with the language.

I have taken the work from Bill Dunlap and Giorgio Garziano and have applied it to my little project and am just amazed that so little code can do so much.? I have also followed Bert Gunter's advice and taken that code and dissected it item by item to comprehend what each element is doing.?? 

The knowledge and help on this list is just amazing and I do appreciate the efforts of all involved.? I read the digest daily .
Carl Sutton CPA
On Wednesday, May 4, 2016 12:06 AM, Martin Maechler <maechler at stat.math.ethz.ch> wrote:
>>>>>? <G.Maubach at weinwolf.de>
Ouch -- so many problems in such a short piece of R code !!!
Sorry, but after reading the above, I'd strongly recommend getting
better books about R...
? ? ? {{maybe do not take those containing "data science" ;-)}}

Compared to the nice and efficient solution of Bill Dunlap,
the above is really bad-bad-bad? in at least four ways :

0) They way you write it above, you cannot use it,
? ? <string> == "variant1|variant2|..."
? is pseudocode and does not really work

1) Note the missing "[, i]"? in the 2nd line: It should be
? ? if(class(dataset[, i]) ...

2) A for loop changing each column at a time is really slow for
? largish data sets

3) [last but not at all least!]
? Please ... many of you readers, do learn:
? 
 Using checks such as
? ? ? if ( class(x) == "numeric" )
 are (almost) always wrong by design !!!

 Instead you really should (almost) always use

 ??? if(inherits(x, "numeric"))

Why?? Because classes in R (S3 or S4) can *extend* other classes.
Example: Many of you know that after? fm <- glm(...)
class(fm) is? c("glm", "lm")? and so

? ? > if(class(fm) == "lm")
? ? + "yes"
? ? Warning message:
? ? In if (class(fm) == "lm") "yes" :
? ? ? the condition has length > 1 and only the first element will be used

Similarly, in your case

y <- 1:10
class(y) <- c("myNumber", "numeric")

when that 'y' is a column in your data frame,
the test for? if(class(dataset[,i]) == "numeric")? will *not*
work but actually produce the above warning.

However, one? could als have had

Num <- setClass("Num", contains="numeric")
N <- Num(1:10)

? ? > Num <- setClass("Num", contains="numeric")
? ? > N <- Num(1:10)
? ? > N
? ? An object of class "Num"
? ? ? [1]? 1? 2? 3? 4? 5? 6? 7? 8? 9 10
? ? > if(class(N) == "numeric") "yes" else "no"
? ? [1] "no"
? ? > 

I hope that many of the readers --- including *MANY* authors of
R packages !! --- have understood the above and will fix their R
code -- and even more their books where applicable !!

Martin Maechler,
ETH Zurich & R Core Team