Sorry for not being more clear - I'm interested in accessing these
indices from within the trainControl summaryFunction, not afterward
(from the train object).
As for the weights, I'm referring to the weights argument passed into
train.
On Fri, Feb 10, 2012 at 5:50 AM, Max Kuhn <mxkuhn at gmail.com> wrote:
I think you need to read the man pages and the four vignettes. A lot
of your questions have answers there.
If you don't specify the resampling indices, they ones generated for
you are saved in the train object:
data(iris)
TrainData <- iris[,1:4]
TrainClasses <- iris[,5]
knnFit1 <- train(TrainData, TrainClasses,
+ ? ? ? ? ? ? ? ? ?method = "knn",
+ ? ? ? ? ? ? ? ? ?preProcess = c("center", "scale"),
+ ? ? ? ? ? ? ? ? ?tuneLength = 10,
+ ? ? ? ? ? ? ? ? ?trControl = trainControl(method = "cv"))
Loading required package: class
Attaching package: ?class?
The following object(s) are masked from ?package:reshape?:
? ?condense
Warning message:
executing %dopar% sequentially: no parallel backend registered
str(knnFit1$control$index)
List of 10
?$ Fold01: int [1:135] 1 2 3 4 5 6 7 9 10 11 ...
?$ Fold02: int [1:135] 1 2 3 4 5 6 8 9 10 12 ...
?$ Fold03: int [1:135] 1 3 4 5 6 7 8 9 10 11 ...
?$ Fold04: int [1:135] 1 2 3 5 6 7 8 9 10 11 ...
?$ Fold05: int [1:135] 1 2 3 4 6 7 8 9 11 12 ...
?$ Fold06: int [1:135] 1 2 3 4 5 6 7 8 9 10 ...
?$ Fold07: int [1:135] 1 2 3 4 5 7 8 9 10 11 ...
?$ Fold08: int [1:135] 2 3 4 5 6 7 8 9 10 11 ...
?$ Fold09: int [1:135] 1 2 3 4 5 6 7 8 9 10 ...
?$ Fold10: int [1:135] 1 2 4 5 6 7 8 10 11 12 ...
There is also a savePredictions argument that gives you the hold-out results.
I'm not sure which weights you are referring to.
On Fri, Feb 10, 2012 at 4:38 AM, Yang Zhang <yanghatespam at gmail.com> wrote:
Actually, is there any way to get at additional information beyond the
classProbs? ?In particular, is there any way to find out the
associated weights, or otherwise the row indices into the original
model matrix corresponding to the tested instances?
On Thu, Feb 9, 2012 at 4:37 PM, Yang Zhang <yanghatespam at gmail.com> wrote:
Oops, found trainControl's classProbs right after I sent!
On Thu, Feb 9, 2012 at 4:30 PM, Yang Zhang <yanghatespam at gmail.com> wrote:
I'm dealing with classification problems, and I'm trying to specify a
custom scoring metric (recall at p, ROC, etc.) that depends on not just
the class output but the probability estimates, so that caret::train
can choose the optimal tuning parameters based on this metric.
However, when I supply a trainControl summaryFunction, the data given
to it contains only class predictions, so the only metrics possible
are things like accuracy, kappa, etc.
Is there any way to do this that I'm looking? ?If not, could I put
this in as a feature request? ?Thanks!
--
Yang Zhang
http://yz.mit.edu/