Can't see what i did wrong..

with

pred.pca<-predict(splits[[i]]$pca,trainingData at samples)[,1:nPCs]
dframe<-as.data.frame(cbind(pred.pca,class=isExplosive(trainingData,2)));
results[[i]]$classifier<-ksvm(class~.,data=dframe,scaled=T,kernel="polydot",type="C-svc",
		C=C,kpar=list(degree=degree,scale=scale,offset=offset),prob.model=T)

and a degree of 5 i get an error of 0 reported by the ksvm object. But when doing

pred.pca<-predict(splits[[i]]$pca,trainingData at samples)[,1:nPCs]
pred.svm<-kernlab::predict(results[[i]]$classifier,pred.pca,type="probabilities");
results[[i]]$trainResults$predicted<-pred.svm[,2]
		
the results vary widely from the class vector. Nearly all predictions are somewhat around 0.29. Its just strange. And i have no idea where things go wrong. They're in the same loop with i, so its probably not an indexing issue.

Maybe kernlabs predict doesn't scale the data or something? 

Its not scaling.. so.. 

I guess i'll stay severely frustrated, and yes i know this is probably not enough information for anyone to help.
Still, talking helps ;)

On 15.11.2012, at 15:15, Jessica Streicher wrote:

with

pred.pca<-predict(splits[[i]]$pca,trainingData at samples)[,1:nPCs]
dframe<-as.data.frame(cbind(pred.pca,class=isExplosive(trainingData,2)));
results[[i]]$classifier<-ksvm(class~.,data=dframe,scaled=T,kernel="polydot",type="C-svc",
		C=C,kpar=list(degree=degree,scale=scale,offset=offset),prob.model=T)

and a degree of 5 i get an error of 0 reported by the ksvm object. But when doing

pred.pca<-predict(splits[[i]]$pca,trainingData at samples)[,1:nPCs]
pred.svm<-kernlab::predict(results[[i]]$classifier,pred.pca,type="probabilities");
results[[i]]$trainResults$predicted<-pred.svm[,2]
		
the results vary widely from the class vector. Nearly all predictions are somewhat around 0.29. Its just strange. And i have no idea where things go wrong. They're in the same loop with i, so its probably not an indexing issue.

Maybe kernlabs predict doesn't scale the data or something? 

Now i let it run for one specific set and got the same bad result, then i deactivated the probabilities and got a good result, then i activated the probabilities again and got a good result .. huh???

On 15.11.2012, at 15:32, Jessica Streicher wrote:

Its not scaling.. so..

I guess i'll stay severely frustrated, and yes i know this is probably not enough information for anyone to help.
Still, talking helps ;)

On 15.11.2012, at 15:15, Jessica Streicher wrote:

with

pred.pca<-predict(splits[[i]]$pca,trainingData at samples)[,1:nPCs]
dframe<-as.data.frame(cbind(pred.pca,class=isExplosive(trainingData,2)));
results[[i]]$classifier<-ksvm(class~.,data=dframe,scaled=T,kernel="polydot",type="C-svc",
             C=C,kpar=list(degree=degree,scale=scale,offset=offset),prob.model=T)

and a degree of 5 i get an error of 0 reported by the ksvm object. But when doing

pred.pca<-predict(splits[[i]]$pca,trainingData at samples)[,1:nPCs]
pred.svm<-kernlab::predict(results[[i]]$classifier,pred.pca,type="probabilities");
results[[i]]$trainResults$predicted<-pred.svm[,2]

the results vary widely from the class vector. Nearly all predictions are somewhat around 0.29. Its just strange. And i have no idea where things go wrong. They're in the same loop with i, so its probably not an indexing issue.

Maybe kernlabs predict doesn't scale the data or something?

Hi Jessica,

Talking may help you, but it's kind of a waste of time for the
thousands of readers of R-help unless you provide a reproducible
example and full context. We'd like to help, but can't without
adequate information.

Sarah

On Thu, Nov 15, 2012 at 9:48 AM, Jessica Streicher
<j.streicher at micromata.de> wrote:

Now i let it run for one specific set and got the same bad result, then i deactivated the probabilities and got a good result, then i activated the probabilities again and got a good result .. huh???

On 15.11.2012, at 15:32, Jessica Streicher wrote:

Its not scaling.. so..

I guess i'll stay severely frustrated, and yes i know this is probably not enough information for anyone to help.
Still, talking helps ;)

On 15.11.2012, at 15:15, Jessica Streicher wrote:

with

pred.pca<-predict(splits[[i]]$pca,trainingData at samples)[,1:nPCs]
dframe<-as.data.frame(cbind(pred.pca,class=isExplosive(trainingData,2)));
results[[i]]$classifier<-ksvm(class~.,data=dframe,scaled=T,kernel="polydot",type="C-svc",
            C=C,kpar=list(degree=degree,scale=scale,offset=offset),prob.model=T)

and a degree of 5 i get an error of 0 reported by the ksvm object. But when doing

pred.pca<-predict(splits[[i]]$pca,trainingData at samples)[,1:nPCs]
pred.svm<-kernlab::predict(results[[i]]$classifier,pred.pca,type="probabilities");
results[[i]]$trainResults$predicted<-pred.svm[,2]

the results vary widely from the class vector. Nearly all predictions are somewhat around 0.29. Its just strange. And i have no idea where things go wrong. They're in the same loop with i, so its probably not an indexing issue.

Maybe kernlabs predict doesn't scale the data or something?

--
Sarah Goslee
http://www.functionaldiversity.org

I'll try .. lets see

dput(dframe)
structure(list(PC1 = c(-112.131127281872, -116.596689350263, 
-128.974817313418, 1196.74257699487, 1216.80975908625, -162.017759589637, 
-85.3191578683763, -130.528496524968, -6.27985683466795, -110.832877061623, 
-127.466759201992, -106.169095838249, -135.615623230809, -140.740670692038, 
-163.111656805033), PC2 = c(-116.519244989851, 127.45362265627, 
-111.091607020859, -90.5227270764559, -123.103694889442, -92.620202365279, 
-31.3353634789021, -24.4646703348531, 627.701800243656, -5.87156108239855, 
-88.8993352468245, -106.166416491185, -95.3261810128253, -112.315850116593, 
-141.197051597155), PC3 = c(64.9404052536739, 259.296520455491, 
81.2617131909924, -40.2969961195199, -49.959279821408, 114.342573713326, 
-127.625681151677, 293.080433739631, -130.564246301803, 243.838480231849, 
-188.819145619591, -184.73732844957, 68.0106824426824, -202.00600443294, 
-3.31973728306762), PC4 = c(56.5233383519436, -188.28891646169, 
36.982033091297, 2.96535930848599, 38.9643424847775, 66.1927347587241, 
-70.2278717299823, 24.2000870994265, 85.41705362572, 81.3070906919206, 
-49.4076806523943, 44.4291395459566, 44.4189881931216, -47.787706169273, 
60.3084338394971), PC5 = c(-24.4330445432631, 119.03248964697, 
25.3182533898001, 2.22712423872515, 20.1564988015345, 8.58984063196041, 
-73.6618858270628, 154.531152822823, 80.0172481588946, -77.3997650093847, 
12.5317198856102, 16.7807094602606, 21.9728292529402, 18.0679363256966, 
14.8070821345172), PC6 = c(-3.51236160660659, 4.01466215993921, 
-14.0560600152898, 79.9790057450278, 63.8459665084246, 70.1505245709296, 
-9.72399636564978, -37.3441332401865, -20.0211154965333, 34.8148111484049, 
19.6697124222742, -12.5006014423202, -35.9412566332911, 43.8525592643803, 
49.0837309296695), PC7 = c(25.0416790893669, 29.2636829211613, 
58.5827246416567, 24.5865832216293, -3.79787182743163, 32.269692748727, 
9.13653843864113, 22.2828779024266, -1.74050522229119, -81.2339879700331, 
-8.39734635571791, 4.90076673663989, 26.3079758222854, -16.6096446464957, 
27.2347172477201), PC8 = c(-7.21492487063905, 41.4415040543379, 
-6.8561527559211, -9.6515803005436, 40.5578239936188, 35.6268320766844, 
4.33982606720149, -27.7334062748555, 1.58916911406888, 42.2854747927154, 
-19.0049848668069, 53.1381176037003, -43.7208819443017, -27.4638912504461, 
26.0693753249626), PC9 = c(5.07013225478207, 3.80978415808406, 
-12.9265139792091, 22.3706065788694, 43.9527567691036, 44.7661368799448, 
-25.6398826082422, 4.37103700086919, -9.0919050870289, 11.9520700564903, 
-23.250383344361, -30.3220249879799, -46.3887501545308, -14.0817576379036, 
16.168581787112), PC10 = c(-17.5630832749372, 52.568055046381, 
-9.07626573878839, -14.0951768936723, -5.23003343696749, 35.0154213026854, 
0.272344896140687, 13.3950601982445, -1.5533667315822, -40.8292771335063, 
-6.65959388343316, 5.06195872128945, -22.7857465669081, 6.62863071798569, 
18.3995069374505), PC11 = c(18.6010453343854, -28.4637751927999, 
1.71806569355502, 46.1205174166346, 9.53199967063977, 41.2972754904467, 
9.33464956019139, 23.3603727930995, 10.9744437554614, -0.126735591637572, 
21.8896623331257, -10.457965922839, -18.8322478793736, 27.7321423838636, 
28.4679654920173), PC12 = c(-2.22649901627144, 2.14317119837832, 
8.11993688993784, -1.02101828008616, -3.02022884335462, 3.99260060747241, 
-3.98208300398649, -8.73435109273014, -15.4116191493509, -21.9916390100063, 
-24.5400095100928, 11.7508407539612, -4.82585090072434, -21.4330004806799, 
4.07445502977139), PC13 = c(12.0478528474097, 19.1463355000476, 
-13.7614159178043, -2.45636501215998, 10.7570602509335, -27.124462677835, 
2.50581414841615, -1.43645727084062, -9.92501401886999, 14.541265924675, 
4.61925247518254, 31.9444025444124, 15.0123077631076, 1.73726777225125, 
-21.7438739224478), PC14 = c(-24.3652304503883, 21.1785358765272, 
0.221058054284211, -28.7788948242716, -0.840491052309543, 40.1941623968201, 
-17.1958234033114, 7.5770624168308, 7.45003285540585, 15.5283651855595, 
-8.65376069406027, 9.56538784228923, 4.23345479961019, -1.50016336971708, 
19.7559666669304), PC15 = c(9.92727520847547, -16.0078543011331, 
3.37100252722285, 26.8689400893287, 16.2203912046009, -6.07396584265131, 
-7.32119707525572, 0.194098009190007, -10.3216068605436, -23.0845947221256, 
0.696161095884098, -5.29057246485069, -1.79824229741562, -3.26507441975894, 
1.08088312906883), class = c(0, 1, 0, 1, 1, 0, 0, 1, 0, 0, 0, 
0, 0, 0, 0)), .Names = c("PC1", "PC2", "PC3", "PC4", "PC5", "PC6", 
"PC7", "PC8", "PC9", "PC10", "PC11", "PC12", "PC13", "PC14", 
"PC15", "class"), row.names = c("2396", "2754", "3645", "7122", 
"7106", "7172", "6761", "1599", "2107", "1865", "6795", "6913", 
"3329", "6836", "7229"), class = "data.frame")

These are 15 of my samples, and it seems to work with just that.

classifier<-ksvm(class~.,data=dframe,scaled=T,kernel="polydot",type="C-svc",
			C=1,kpar=list(degree=5,scale=1,offset=0),prob.model=T)
pred.svm<-kernlab::predict(classifier,dframe[,1:15],type="probabilities");

cbind(pred.svm[,2],dframe$class)
           [,1] [,2]
[1,] 0.29135615    0
[2,] 0.03668269    1
[3,] 0.29135845    0
[4,] 0.03667823    1
[5,] 0.03667505    1
[6,] 0.35437992    0
[7,] 0.29135892    0
[8,] 0.03667943    1
[9,] 0.29121498    0
[10,] 0.29121019    0
[11,] 0.29132637    0
[12,] 0.29135683    0
[13,] 0.29134198    0
[14,] 0.29132484    0
[15,] 0.29132652    0

Now that i managed to get me (and you) a closer look - seems like class labels switched maybe? pred.svm[,2] is named "1", but seems to have the probability to be class 0 in it.
This does not always occur however, you may have to run it a few times.

On 15.11.2012, at 16:47, Sarah Goslee wrote:

Hi Jessica,

Talking may help you, but it's kind of a waste of time for the
thousands of readers of R-help unless you provide a reproducible
example and full context. We'd like to help, but can't without
adequate information.

Sarah

On Thu, Nov 15, 2012 at 9:48 AM, Jessica Streicher
<j.streicher at micromata.de> wrote:

Now i let it run for one specific set and got the same bad result, then i deactivated the probabilities and got a good result, then i activated the probabilities again and got a good result .. huh???

On 15.11.2012, at 15:32, Jessica Streicher wrote:

Its not scaling.. so..

I guess i'll stay severely frustrated, and yes i know this is probably not enough information for anyone to help.
Still, talking helps ;)

On 15.11.2012, at 15:15, Jessica Streicher wrote:

with

pred.pca<-predict(splits[[i]]$pca,trainingData at samples)[,1:nPCs]
dframe<-as.data.frame(cbind(pred.pca,class=isExplosive(trainingData,2)));
results[[i]]$classifier<-ksvm(class~.,data=dframe,scaled=T,kernel="polydot",type="C-svc",
           C=C,kpar=list(degree=degree,scale=scale,offset=offset),prob.model=T)

and a degree of 5 i get an error of 0 reported by the ksvm object. But when doing

pred.pca<-predict(splits[[i]]$pca,trainingData at samples)[,1:nPCs]
pred.svm<-kernlab::predict(results[[i]]$classifier,pred.pca,type="probabilities");
results[[i]]$trainResults$predicted<-pred.svm[,2]

the results vary widely from the class vector. Nearly all predictions are somewhat around 0.29. Its just strange. And i have no idea where things go wrong. They're in the same loop with i, so its probably not an indexing issue.

Maybe kernlabs predict doesn't scale the data or something?

--
Sarah Goslee
http://www.functionaldiversity.org

Guess it has something t do with the values in prob.model. The classifiers with bad predicitons have positive values and the ones with good predictions have negative values.

On 15.11.2012, at 17:18, Jessica Streicher wrote:

I'll try .. lets see

dput(dframe)
structure(list(PC1 = c(-112.131127281872, -116.596689350263, 
-128.974817313418, 1196.74257699487, 1216.80975908625, -162.017759589637, 
-85.3191578683763, -130.528496524968, -6.27985683466795, -110.832877061623, 
-127.466759201992, -106.169095838249, -135.615623230809, -140.740670692038, 
-163.111656805033), PC2 = c(-116.519244989851, 127.45362265627, 
-111.091607020859, -90.5227270764559, -123.103694889442, -92.620202365279, 
-31.3353634789021, -24.4646703348531, 627.701800243656, -5.87156108239855, 
-88.8993352468245, -106.166416491185, -95.3261810128253, -112.315850116593, 
-141.197051597155), PC3 = c(64.9404052536739, 259.296520455491, 
81.2617131909924, -40.2969961195199, -49.959279821408, 114.342573713326, 
-127.625681151677, 293.080433739631, -130.564246301803, 243.838480231849, 
-188.819145619591, -184.73732844957, 68.0106824426824, -202.00600443294, 
-3.31973728306762), PC4 = c(56.5233383519436, -188.28891646169, 
36.982033091297, 2.96535930848599, 38.9643424847775, 66.1927347587241, 
-70.2278717299823, 24.2000870994265, 85.41705362572, 81.3070906919206, 
-49.4076806523943, 44.4291395459566, 44.4189881931216, -47.787706169273, 
60.3084338394971), PC5 = c(-24.4330445432631, 119.03248964697, 
25.3182533898001, 2.22712423872515, 20.1564988015345, 8.58984063196041, 
-73.6618858270628, 154.531152822823, 80.0172481588946, -77.3997650093847, 
12.5317198856102, 16.7807094602606, 21.9728292529402, 18.0679363256966, 
14.8070821345172), PC6 = c(-3.51236160660659, 4.01466215993921, 
-14.0560600152898, 79.9790057450278, 63.8459665084246, 70.1505245709296, 
-9.72399636564978, -37.3441332401865, -20.0211154965333, 34.8148111484049, 
19.6697124222742, -12.5006014423202, -35.9412566332911, 43.8525592643803, 
49.0837309296695), PC7 = c(25.0416790893669, 29.2636829211613, 
58.5827246416567, 24.5865832216293, -3.79787182743163, 32.269692748727, 
9.13653843864113, 22.2828779024266, -1.74050522229119, -81.2339879700331, 
-8.39734635571791, 4.90076673663989, 26.3079758222854, -16.6096446464957, 
27.2347172477201), PC8 = c(-7.21492487063905, 41.4415040543379, 
-6.8561527559211, -9.6515803005436, 40.5578239936188, 35.6268320766844, 
4.33982606720149, -27.7334062748555, 1.58916911406888, 42.2854747927154, 
-19.0049848668069, 53.1381176037003, -43.7208819443017, -27.4638912504461, 
26.0693753249626), PC9 = c(5.07013225478207, 3.80978415808406, 
-12.9265139792091, 22.3706065788694, 43.9527567691036, 44.7661368799448, 
-25.6398826082422, 4.37103700086919, -9.0919050870289, 11.9520700564903, 
-23.250383344361, -30.3220249879799, -46.3887501545308, -14.0817576379036, 
16.168581787112), PC10 = c(-17.5630832749372, 52.568055046381, 
-9.07626573878839, -14.0951768936723, -5.23003343696749, 35.0154213026854, 
0.272344896140687, 13.3950601982445, -1.5533667315822, -40.8292771335063, 
-6.65959388343316, 5.06195872128945, -22.7857465669081, 6.62863071798569, 
18.3995069374505), PC11 = c(18.6010453343854, -28.4637751927999, 
1.71806569355502, 46.1205174166346, 9.53199967063977, 41.2972754904467, 
9.33464956019139, 23.3603727930995, 10.9744437554614, -0.126735591637572, 
21.8896623331257, -10.457965922839, -18.8322478793736, 27.7321423838636, 
28.4679654920173), PC12 = c(-2.22649901627144, 2.14317119837832, 
8.11993688993784, -1.02101828008616, -3.02022884335462, 3.99260060747241, 
-3.98208300398649, -8.73435109273014, -15.4116191493509, -21.9916390100063, 
-24.5400095100928, 11.7508407539612, -4.82585090072434, -21.4330004806799, 
4.07445502977139), PC13 = c(12.0478528474097, 19.1463355000476, 
-13.7614159178043, -2.45636501215998, 10.7570602509335, -27.124462677835, 
2.50581414841615, -1.43645727084062, -9.92501401886999, 14.541265924675, 
4.61925247518254, 31.9444025444124, 15.0123077631076, 1.73726777225125, 
-21.7438739224478), PC14 = c(-24.3652304503883, 21.1785358765272, 
0.221058054284211, -28.7788948242716, -0.840491052309543, 40.1941623968201, 
-17.1958234033114, 7.5770624168308, 7.45003285540585, 15.5283651855595, 
-8.65376069406027, 9.56538784228923, 4.23345479961019, -1.50016336971708, 
19.7559666669304), PC15 = c(9.92727520847547, -16.0078543011331, 
3.37100252722285, 26.8689400893287, 16.2203912046009, -6.07396584265131, 
-7.32119707525572, 0.194098009190007, -10.3216068605436, -23.0845947221256, 
0.696161095884098, -5.29057246485069, -1.79824229741562, -3.26507441975894, 
1.08088312906883), class = c(0, 1, 0, 1, 1, 0, 0, 1, 0, 0, 0, 
0, 0, 0, 0)), .Names = c("PC1", "PC2", "PC3", "PC4", "PC5", "PC6", 
"PC7", "PC8", "PC9", "PC10", "PC11", "PC12", "PC13", "PC14", 
"PC15", "class"), row.names = c("2396", "2754", "3645", "7122", 
"7106", "7172", "6761", "1599", "2107", "1865", "6795", "6913", 
"3329", "6836", "7229"), class = "data.frame")

These are 15 of my samples, and it seems to work with just that.

classifier<-ksvm(class~.,data=dframe,scaled=T,kernel="polydot",type="C-svc",
			C=1,kpar=list(degree=5,scale=1,offset=0),prob.model=T)
pred.svm<-kernlab::predict(classifier,dframe[,1:15],type="probabilities");

cbind(pred.svm[,2],dframe$class)
          [,1] [,2]
[1,] 0.29135615    0
[2,] 0.03668269    1
[3,] 0.29135845    0
[4,] 0.03667823    1
[5,] 0.03667505    1
[6,] 0.35437992    0
[7,] 0.29135892    0
[8,] 0.03667943    1
[9,] 0.29121498    0
[10,] 0.29121019    0
[11,] 0.29132637    0
[12,] 0.29135683    0
[13,] 0.29134198    0
[14,] 0.29132484    0
[15,] 0.29132652    0

Now that i managed to get me (and you) a closer look - seems like class labels switched maybe? pred.svm[,2] is named "1", but seems to have the probability to be class 0 in it.
This does not always occur however, you may have to run it a few times.

On 15.11.2012, at 16:47, Sarah Goslee wrote:

Hi Jessica,

Talking may help you, but it's kind of a waste of time for the
thousands of readers of R-help unless you provide a reproducible
example and full context. We'd like to help, but can't without
adequate information.

Sarah

On Thu, Nov 15, 2012 at 9:48 AM, Jessica Streicher
<j.streicher at micromata.de> wrote:

Now i let it run for one specific set and got the same bad result, then i deactivated the probabilities and got a good result, then i activated the probabilities again and got a good result .. huh???

On 15.11.2012, at 15:32, Jessica Streicher wrote:

Its not scaling.. so..

I guess i'll stay severely frustrated, and yes i know this is probably not enough information for anyone to help.
Still, talking helps ;)

On 15.11.2012, at 15:15, Jessica Streicher wrote:

with

pred.pca<-predict(splits[[i]]$pca,trainingData at samples)[,1:nPCs]
dframe<-as.data.frame(cbind(pred.pca,class=isExplosive(trainingData,2)));
results[[i]]$classifier<-ksvm(class~.,data=dframe,scaled=T,kernel="polydot",type="C-svc",
          C=C,kpar=list(degree=degree,scale=scale,offset=offset),prob.model=T)

and a degree of 5 i get an error of 0 reported by the ksvm object. But when doing

pred.pca<-predict(splits[[i]]$pca,trainingData at samples)[,1:nPCs]
pred.svm<-kernlab::predict(results[[i]]$classifier,pred.pca,type="probabilities");
results[[i]]$trainResults$predicted<-pred.svm[,2]

the results vary widely from the class vector. Nearly all predictions are somewhat around 0.29. Its just strange. And i have no idea where things go wrong. They're in the same loop with i, so its probably not an indexing issue.

Maybe kernlabs predict doesn't scale the data or something?

--
Sarah Goslee
http://www.functionaldiversity.org