hi all i'm new to the area of neural networks. i've been reading some references and seem to understand some of the learning algorithms. i am very familiar with regression and would just like to see how neural nets handle this problem so i've been using the nnet package. i simply want to use a 3 layer neural net, ie 1 input, 1 hidden layer (where the hidden layer is linear, since i want to basically perform regression analysis by means of neural nets) and 1 output layer. the x and y vector was simulated as follows: x<-1:100 x [1] 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 [19] 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 [37] 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 [55] 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 [73] 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 [91] 91 92 93 94 95 96 97 98 99 100 y<-2+5*x+rnorm(100)*5 y [1] 8.789605 11.151109 14.622276 30.381379 19.328647 29.317038 33.793720 39.557390 51.939294 45.045965 [11] 58.783991 63.191745 72.882202 79.184778 85.034551 94.446000 89.243004 88.223547 106.327683 104.424668 [21] 103.057648 112.855778 111.777823 108.359485 128.956152 127.369102 128.784481 139.760279 151.959887 152.014623 [31] 158.869586 167.030970 166.711160 177.415680 173.542293 182.484224 179.767128 192.284343 196.173830 202.353030 [41] 220.449623 213.410307 216.746041 219.812526 230.440402 230.759429 239.311279 244.151390 248.637023 254.648298 [51] 262.694237 253.619539 276.975714 280.395284 280.173787 286.813617 284.766870 296.705692 295.110064 304.709464 [61] 305.650793 310.128992 314.035624 314.649213 322.958865 333.640203 342.538307 340.546359 342.433629 344.720633 [71] 354.115051 363.631246 371.479886 367.066764 377.184512 386.634677 392.310577 386.151325 400.345393 408.831710 [81] 413.999148 405.009358 418.679828 418.388427 419.282955 432.329471 433.448313 444.166060 447.773185 455.103503 [91] 448.588598 464.410358 465.565875 478.677403 478.306390 479.565728 487.681689 491.422090 502.468491 500.385458 i then went about to use the nnet function:
a.net<-nnet(y~x,linout=T,size=1)
# weights: 4 initial value 8587786.130082 iter 10 value 2086583.979646 final value 2079743.529111 converged NOTE: the function said that four weights were estimated. This is correct as shown below. the model can be represented as: input ---(w1)---> hidden ---(w2)---> output x ----------> a1+w1*x ----------> a2+w2*(a1+w1*x) where: wi are the weights, i=1,2 x is an input pattern further results were: summary(a.net) a 1-1-1 network with 4 weights options were - linear output units b->h1 i1->h1 -276.48 -295.11 b->o h1->o 254.92 764.72 is the following statement correct? (i think that it is!) a1= "b->h1" w1= "i1->h1" a2= "b->o" w2= "h1->o" If the hidden layer and the output layers are both LINEAR then the following should be true: 1. 2= a2+a1*w2 2. 5= w1*w2 THIS IS NOT THE CASE, see the results. The only thing that i can think of thats happening is that the activation function from the hidden layer is not linear. Is this correct since i used the linout=T arguement? are we able to change the activation function from the hidden layer? two other questions: 1. with regard to the size arguement, how does one know how many nodes are in the hidden layer? (this might be a silly question.) e.g we might have 2 hidden layers both with 3 nodes. 2. are we able to plot the drop in the error function as a function of the epochs? hope someone can help thanking you!!!