Discovering patterns in textual strings

z <- c("abc",

pats <- sub("^(.+)[. _].*","\\1",z)

tapply(seq_along(z),pats,I)

"Does that help?"

No. I am not your private consultant. You need to reply to the list, which
I have cc'ed here, not just me.

I am still somewhat confused by your specifications, but others may not
be. Part of my confusion stems from your failure to provide a reproducible
example (see e.g. the posting guide linked below).  For example, I cannot
tell from your text whether the Abc and Bce strings contain one or more
spaces at the end. I shall assume they may but need not.

Anyway, here is a reproducible example and solution that assumes that the
substrings/patterns of interest to you occur at the beginning of the
strings and may or may not be followed by one of "." "_" or " "(space) and
then possibly further text which should be ignored. Assuming that you are
familiar with regular expressions, maybe this will help to get you started
even if I have misunderstood your specifications. If you aren't familiar
with regex's, maybe the stringr package may provide a gentler interface
than using R's raw regex functionality. Or maybe someone else can suggest a
better approach (which is another reason why you should reply to the list,
not just me).

z <- c("abc",
       "abc_def",
       "abc.def",
       "abc def",
       "abcd_ef",
       "abcd",
       "e","f")

pats <- unique(sub("^(.+)[. _]+.*", "\\1", z))
## gives:

pats

[1] "abc"  "abcd" "e"    "f"


This gives you the four separate patterns that you could then use to group
your records, perhaps by:

lapply(pats,function(x)grep(paste0("^", x,"([_. ]|$)"), z))

[[1]]
[1] 1 2 3 4

[[2]]
[1] 5 6

[[3]]
[1] 7

[[4]]
[1] 8

That is, indices 1-4 in z are the first group; 5 and 6 are the second; etc.



Cheers,
Bert

Bert Gunter

"The trouble with having an open mind is that people keep coming along and
sticking things into it."
-- Opus (aka Berkeley Breathed in his "Bloom County" comic strip )

On Fri, May 4, 2018 at 9:00 PM, Jeff Reichman <reichmanj at sbcglobal.net>
wrote:

Bert

Thank you for the  link.  Figured there might be something

Regarding your questions

This is from a large 53 Billion records.  The column in question are
AdNames (Real Time Bidding data)

#1. Generally yes, but not always

#2 Separators could be underscores  (_) or dots (.) as in 1.2.3_ABC ......

#3 Yes. So there could be Abc 123 could be a matching string

This would not be considered a match  ...
abc_something
this.is_a long stringwithabcinthemiddle

The sequence(s) are always are at the beginning (or so it appears).  Out
of the 54 billion records  I am able to pull (SparkR sql) 948,679 unique
strings.  It is from these unique strings that I (if possible)  want to
identify the "key" strings.

1.  Abc_1232.niok7j9hd
2.  Abc
3.  Abc.2#348hfk2.njilo
4.  Abc.2
5.  Abc.7
6.  BAdfr_kajdhf98#kjsdh
7.  BAdrf_gofer
948679 ....


So I may have a thousand individuals strings all of which have Abc as a
common string, or Badrf.  So I am looking to pull "Abc," "BAdrf", etc.  So
then I can go back and restructure the data to show that any record with
Abc_1232.niok7j9hd if part of the Abc "Group," or Family ???

Does that help

Jeff

-----Original Message-----
From: Bert Gunter <bgunter.4567 at gmail.com>
Sent: Friday, May 4, 2018 5:41 PM
To: reichmanj at sbcglobal.net
Cc: R-help <R-help at r-project.org>
Subject: Re: [R] Discovering patterns in textual strings

The answer is, of course, using regular expressions and/or libraries
therefor. However, I do not think you have defined your problem
sufficiently. Some questions I have:

1. Do possible patterns to be matched always appear at the beginning of
your strings?

2. Always together between specified separators ("_"  in your example);
or one of several specified separators; or otherwise?

3. Do spaces or other nonprinting characters occur in your strings?

e.g. would

abc_something
this.is_a long stringwithabcinthemiddle

be considered matching?
There are undoubtedly other possibilities that I've missed.



You may also find it useful to check this "task view" out for
possibilities:
https://cran.r-project.org/web/views/NaturalLanguageProcessing.html

Cheers,
Bert


Bert Gunter

"The trouble with having an open mind is that people keep coming along
and sticking things into it."
-- Opus (aka Berkeley Breathed in his "Bloom County" comic strip )


On Fri, May 4, 2018 at 3:25 PM, Jeff Reichman <reichmanj at sbcglobal.net>
wrote:

R Help Forum



Is there a R library (or a way) that I can extract unique character
strings, or repeating patterns in textual strings.  Say for example I
have the following records:



Abc_1234_kjhksh_276

Abc

Abc_1234_lakdofyo_324

Bce_876_skdhk_*&^%*&

Bce

Bce_454



And I would like to see the following results

Abc

Abc_1234

Bce





Jeff Reichman


        [[alternative HTML version deleted]]