missing value where TRUE/FALSE needed

Thanks for al your help with Jim,
I realise this is really a case of debugging (and so falls to the code
writer) using your help i have traced the NaN to be from the c1 calculation
in the FindGradient function.
there is a place where we use the calculation of c1...

which at the start was calculated as
c1 <- (2*pi) ** (n.cols/2)

when i use
c1 <- (2*pi) ** (811/2)

[1] Inf

So i think i have found a ceiling on the number of cols i can use to make a
calcuation.

Many thanks again for your help, especially on 23rd Dec!

Mike



On 23 December 2011 16:01, jim holtman <jholtman at gmail.com> wrote:

The next step is to now evaluate the objects used in the calculation;
e.g., look at the objects in the statement:

?change <- sqrt(sum((next.x - x) * (next.x - x)))

What do each of them show? ?Most likely you have something in the data
that causing this statement to evaluate to NaN (not a number) and this
is caused by the values of one of these as not being what you think it
is. ?From the value of 'i', it looks like the first iteration (or at
least when it has the value of 1). ?Notice that "X" is defined as a
matrix with NAs as the initial values and you are picking up the first
row (X[i,]) which on the initial pass is an NA. ?So start by examining
the values of all the objects used up to that point and what the
values of each expression is.

On Fri, Dec 23, 2011 at 10:38 AM, Michael Pearmain
<michael.pearmain at gmail.com> wrote:

Thanks for?guidance;

the return from the dump is below:

So i can see that the NaN is causing NA, as its evaluating to NA in the
ifelse.
So i guess my next question is why this happens for my large data set
but
not he iris dataset?

Thank you again for your help on this, i really appreciate you taking
the
time to help

1: DensitiesAndAttractors(data = cluster.data, smoothing = 0.34, tol =
1e-05,
2: denclue_par.R#136: sfSapply(rep(1:n.rows), function(i) {
3: sapply(x, fun, ..., simplify = simplify, USE.NAMES = USE.NAMES)
4: lapply(X, FUN, ...)
5: FUN(1:9985[[1]], ...)

Selection: 5
Called from: function ()
{
? ? if (.isMethodsDispatchOn()) {
? ? ? ? tState <- tracingState(FALSE)
? ? ? ? on.exit(tracingState(tState))
? ? }
? ? calls <- sys.calls()
? ? from <- 0L
? ? n <- length(calls)
? ? if (identical(sys.function(n), recover))
? ? ? ? n <- n - 1L
? ? for (i in rev(seq_len(n))) {
? ? ? ? calli <- calls[[i]]
? ? ? ? fname <- calli[[1L]]
? ? ? ? if (!is.na(match(deparse(fname)[1L], c("methods::.doTrace",
? ? ? ? ? ? ".doTrace")))) {
? ? ? ? ? ? from <- i - 1L
? ? ? ? ? ? break
? ? ? ? }
? ? }
? ? if (from == 0L)
? ? ? ? for (i in rev(seq_len(n))) {
? ? ? ? ? ? calli <- calls[[i]]
? ? ? ? ? ? fname <- calli[[1L]]
? ? ? ? ? ? if (!is.name(fname) || is.na(match(as.character(fname),
? ? ? ? ? ? ? ? c("recover", "stop", "Stop")))) {
? ? ? ? ? ? ? ? from <- i
? ? ? ? ? ? ? ? break
? ? ? ? ? ? }
? ? ? ? }
? ? if (from > 0L) {
? ? ? ? if (!interactive()) {
? ? ? ? ? ? try(dump.frames())
? ? ? ? ? ? cat(gettext("recover called non-interactively; frames
dumped,
use debugger() to view\n"))
? ? ? ? ? ? return(NULL)
? ? ? ? }
? ? ? ? else if (identical(getOption("show.error.messages"),
? ? ? ? ? ? FALSE))
? ? ? ? ? ? return(NULL)
? ? ? ? calls <- limitedLabels(calls[1L:from])
? ? ? ? repeat {
? ? ? ? ? ? which <- menu(calls, title = "\nEnter a frame number, or 0
to
exit ?")
? ? ? ? ? ? if (which)
? ? ? ? ? ? ? ? eval(substitute(browser(skipCalls = skip), list(skip = 7
-
? ? ? ? ? ? ? ? ? which)), envir = sys.frame(which))
? ? ? ? ? ? else break
? ? ? ? }
? ? }
? ? else cat(gettext("No suitable frames for recover()\n"))
}()
Browse[1]> ls()
[1] "change" ? ? ? ?"find.gradient" "i" ? ? ? ? ? ? "iters"
[5] "next.x" ? ? ? ?"notconverged" ?"x"
Browse[1]> change
[1] NaN
Browse[1]> i
[1] 1
Browse[1]>


On 23 December 2011 15:22, jim holtman <jholtman at gmail.com> wrote:

According to your dump, if you look in section 5 (this is the function
being call by 'sapply') you should do an 'ls()' and see the object
'notconverged' and if you type its name, I am guessing that is that
the value NA. ?Also look at value of 'change' and 'tol' to determine
how the 'ifelse' was evaluated.

Let me know how it goes.

On Fri, Dec 23, 2011 at 9:35 AM, Michael Pearmain
<michael.pearmain at gmail.com> wrote:

Hi Jim,

Sorry to email directly, i wasn't sure what i was looking for, can
you
give
me a little guidance then i can post back to the list? ?Many thanks

density.data <- DensitiesAndAttractors(data = cluster.data,

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?smoothing = 0.34,
? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?tol = 0.00001,
? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?parallel = FALSE)
density.data <- DensitiesAndAttractors(data = cluster.data,
+ ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?smoothing = 0.34,
+ ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?tol = 0.00001,
+ ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?parallel = FALSE)
Explicit sfStop() is missing: stop now.
snowfall 1.84 initialized: sequential execution, one CPU.

Error in while (notconverged) { : missing value where TRUE/FALSE
needed

Enter a frame number, or 0 to exit

1: DensitiesAndAttractors(data = cluster.data, smoothing = 0.34, tol
=
1e-05,
2: denclue_par.R#136: sfSapply(rep(1:n.rows), function(i) {
3: sapply(x, fun, ..., simplify = simplify, USE.NAMES = USE.NAMES)
4: lapply(X, FUN, ...)
5: FUN(1:9985[[1]], ...)

Selection: 1
Called from: sfSapply(rep(1:n.rows), function(i) {
? ? notconverged <- TRUE
? ? x <- (X[i, ])
? ? iters <- as.integer(1)
? ? while (notconverged) {
? ? ? ? find.gradient <- FindGradient(x)
? ? ? ? next.x <- x + stepsize * find.gradient$gradient
? ? ? ? change <- sqrt(sum((next.x - x) * (next.x - x)))
? ? ? ? notconverged <- ifelse(change > tol, TRUE, FALSE)
? ? ? ? x <- next.x
? ? ? ? iters <- iters + 1
? ? }
? ? return(c(densities[i, ] <- find.gradient$density, attractors[i,
? ? ? ? ] <- x))
})
Browse[1]> ls()
?[1] "attractors" ? "c1" ? ? ? ? ? "c2" ? ? ? ? ? "c3"
"cores"

?[6] "data" ? ? ? ? "densities" ? ?"FindGradient" "Kexp"
"n.cols"

[11] "n.rows" ? ? ? "parallel" ? ? "smoothing" ? ?"stepsize"
"tol"

[16] "X"
Browse[1]>

Enter a frame number, or 0 to exit

1: DensitiesAndAttractors(data = cluster.data, smoothing = 0.34, tol
=
1e-05,
2: denclue_par.R#136: sfSapply(rep(1:n.rows), function(i) {
3: sapply(x, fun, ..., simplify = simplify, USE.NAMES = USE.NAMES)
4: lapply(X, FUN, ...)
5: FUN(1:9985[[1]], ...)

Selection: 2
Called from: sapply(x, fun, ..., simplify = simplify, USE.NAMES =
USE.NAMES)
Browse[1]> ls()
[1] "fun" ? ? ? "simplify" ?"USE.NAMES" "x"
Browse[1]>




On 23 December 2011 13:26, jim holtman <jholtman at gmail.com> wrote:

Does this look similar to the error you are getting:

while(NA == TRUE) 1

Error in while (NA == TRUE) 1 : missing value where TRUE/FALSE
needed

SO 'notconverged' is probably equal to NA. ?BTW, what is the value
of
'tol'; I do not see it defined. ?So when computing 'notconverged'
you
have generated an NA. ?You can test it to see if this is true.

You can use the following command:

options(error=utils::recover)

and then learn how to use the 'browser' to examine variables when
the
error occurs.

On Fri, Dec 23, 2011 at 5:44 AM, Michael Pearmain
<michael.pearmain at gmail.com> wrote:

Merry Xmas to all,

I am writing a function and curiously this runs sometimes on one
data
set
and fails on another and i cannot figure out why.
Any help much appreciated.

If i run the code below with
data <- iris[ ,1:4]
The code runs fine, but if i run on a large dataset i get the
following
error (showing data structures as matrix is large)

str(cluster.data)

?num [1:9985, 1:811] 0 0 0 0 0 0 0 0 0 0 ...
?- attr(*, "dimnames")=List of 2
?..$ : NULL
?..$ : chr [1:811] "1073949105" "1073930585" "1073843224"
"1073792624"
...
#(This is intended to be chr)

str(iris)

'data.frame': 150 obs. of ?5 variables:
?$ Sepal.Length: num ?5.1 4.9 4.7 4.6 5 5.4 4.6 5 4.4 4.9 ...
?$ Sepal.Width : num ?3.5 3 3.2 3.1 3.6 3.9 3.4 3.4 2.9 3.1 ...
?$ Petal.Length: num ?1.4 1.4 1.3 1.5 1.4 1.7 1.4 1.5 1.4 1.5 ...
?$ Petal.Width : num ?0.2 0.2 0.2 0.2 0.2 0.4 0.3 0.2 0.2 0.1 ...
?$ Species ? ? : Factor w/ 3 levels "setosa","versicolor",..: 1 1
1 1
1
1 1
1 1 1 ...

str(as.matrix(iris[,1:4]))

?num [1:150, 1:4] 5.1 4.9 4.7 4.6 5 5.4 4.6 5 4.4 4.9 ...
?- attr(*, "dimnames")=List of 2
?..$ : NULL
?..$ : chr [1:4] "Sepal.Length" "Sepal.Width" "Petal.Length"
"Petal.Width"

n.cols <- ncol(data)
?n.rows <- nrow(data)
?X <- as.matrix(data)
?stepsize <- 0.05
?c1 <- (2 * pi) ** (n.cols / 2)
?c2 <- n.rows * (smoothing ** (n.cols + 2))
?c3 <- n.rows * (smoothing ** n.cols)

?Kexp <- function(sqs){
? ?return (exp((-1 * sqs) / (2 * smoothing ** 2)))
?}

?FindGradient <- function(x){
? ?XmY <- t(x - t(X))
? ?sqsum <- rowSums(XmY * XmY)
? ?K <- sapply(sqsum, Kexp)
? ?dens <- ((c1 * c3) ** -1) * sum(K)
? ?grad <- -1 * ((c1 * c2) ** -1) * colSums(K * XmY)
? ?return (list(gradient = grad,
? ? ? ? ? ? ? ? density = dens))
?}

?attractors <- matrix(0, n.rows, n.cols)
?densities <- matrix(0, n.rows)

density.attractors <-

? ?sapply(rep(1:n.rows), function(i) {
? ? ?notconverged <- TRUE
? ? ?# For each row loop through and find the attractor and
density
value.
? ? ?x <- (X[i, ])
? ? ?iters <- as.integer(1)
? ? ?# Run gradient ascent for each point to obtain x*
? ? ?while(notconverged == TRUE) {
? ? ? ?find.gradient <- FindGradient(x)
? ? ? ?next.x <- x + stepsize * find.gradient$gradient
? ? ? ?change <- sqrt(sum((next.x - x) * (next.x - x)))
? ? ? ?notconverged <- ifelse(change > tol, TRUE, FALSE)
? ? ? ?x <- next.x
? ? ? ?iters <- iters + 1
? ? ?}

? ? ?# store the attractor and density value
? ? ?return(c(densities[i, ] <- find.gradient$density,
? ? ? ? ? ? ? attractors[i, ] <- x))
? ?})

Error in while (notconverged == TRUE) { :
?missing value where TRUE/FALSE needed

Any help would be great

Mike

? ? ? ?[[alternative HTML version deleted]]

--
Jim Holtman
Data Munger Guru

What is the problem that you are trying to solve?
Tell me what you want to do, not how you want to do it.

--
Jim Holtman
Data Munger Guru

What is the problem that you are trying to solve?
Tell me what you want to do, not how you want to do it.

--
Jim Holtman
Data Munger Guru

What is the problem that you are trying to solve?
Tell me what you want to do, not how you want to do it.