[Bioc-devel] Changes in the %in% function for DNAStringSet?

Hi Nico,

Last week I did some improvements/reorganization of the match(),
%in%, duplicated(), and unique() stuff in 
IRanges/GenomicRanges/Biostrings, and apparently forgot to define the 
"%in%" method
for ANY,Vector. Thanks for the catch!

This is fixed in IRanges 1.15.8. FWIW I also added an "%in%" method
for Vector,ANY so now this works too:

   > DNAStringSet(c("TTGCGA","ATGRCT","ACASTG")) %in% 
c("TTGCGA","ATGGCT","ACACTG")
   [1]  TRUE FALSE FALSE

<rant mode>

It is so sad that we have to redefine "%in%" methods that do exactly
the same thing as base::`%in%`:

   > base::`%in%`
   function (x, table)
   match(x, table, nomatch = 0L) > 0L
   <environment: namespace:base>

just because base::`%in%` cannot dispatch on the appropriate
"match" method. A well-known issue of the way generics, methods
and NAMESPACE interact with each other... but still an unfortunate
one.

</rant mode>

The good news is that we have on our TODO list to explicitly define
the match() and %in% generics in BiocGenerics so there will be an
opportunity to overwrite the "%in%" default method:

   setMethod("%in%", c("ANY", "ANY"), function (x, table) match(x, 
table, nomatch = 0L) > 0L)

(I'm still hesitant about this though. What could be the drawbacks
of overwriting the default method?)

Also last week at the same time I did the changes on match() and
family, I also reimplemented the "match" method for DNAStringSet
objects (which is called when either 'x' or 'table' or both are
DNAStringSet). The new implementation is in Biostrings 2.25.3.
It uses a hash-based algorithm instead of the quicksort-based algo
that was used so far. The resulting speedup varies a lot depending
on the sizes of 'x' and 'table', and will typically be important
(10x or more) for big (i.e. > 1M elements) DNAStringSet objects.

This benefits directly %in%, duplicated() and unique() on
DNAStringSet objects.

With Biostrings 2.25.3 (Bioc 2.11):

   > library(Biostrings)
   > probes <- DNAStringSet(hgu133aprobe)
   > system.time(isdup <- duplicated(probes))
      user  system elapsed
     0.048   0.000   0.050

With Bioc <= 2.10:

   > system.time(isdup <- duplicated(probes))
      user  system elapsed
     0.232   0.000   0.233

Finally I should mention that, even though the hash function I use
for DNAStringSet (and RNAStringSet, AAStringSet, BStringSet) is the
same as the function used in base R for hashing the strings of a
standard character vector, calling match(), %in%, duplicated() or
unique() on a standard character vector is still slightly faster
(2x) than on a DNAStringSet. This can probably be explained by the
fact that all the strings in all the character vectors defined in
a session are pre-hashed i.e. hashed the 1st time the string is
created and the result of the hash stored in the "global CHARSXP
hash table".

Cheers,
H.

[Bioc-devel] Changes in the %in% function for DNAStringSet?

Thread (3 messages)