Statistical Machine Learning

["Balance MSEP 150000 50000 02468 10 12 number of components # Comment: # We see that the smallest CV error occurs # when M = 11 components are used. This is # barely fewer than M = 12, which amounts # to simply performing least squares, because # when all of the components are used in PCR # no dimension reduction occurs. However, # from the plot we also see that the CV error # is roughly the same when only one component # is included in the model. This suggests # that a model that uses just a small # number of components might suffice. # Perform PCR on training data: set.seed(1) pcr.fit <- pcr( Balance ~., data = data, subset = train, scale = TRUE, validation = \\\"CV\\\" ) validationplot(pcr.fit, val.type = \\\"MSEP\\\") 201","Balance MSEP 150000 50000 0 2 4 6 8 10 12 number of components pcr.pred <- predict(pcr.fit,x[test,],ncomp = 10) mean((pcr.pred - y.test)^2) ## [1] 67014.07 # Fit PCR on full data set: pcr.fit <- pcr(y ~ x, scale = TRUE, ncomp = 7) summary(pcr.fit) ## Data: X dimension: 400 12 5 comps 6 comps 7 comps ## Y dimension: 400 1 64.23 72.34 80.36 ## Fit method: svdpc 61.39 62.34 67.36 ## Number of components considered: 7 ## TRAINING: % variance explained ## 1 comps 2 comps 3 comps 4 comps ## X 22.98 36.54 46.05 55.25 ## y 57.93 58.37 61.06 61.34 # Partial Least Squares: set.seed(1) pls.fit = plsr( Balance ~., data = data, subset = train, scale = TRUE,TRUEvalidation = \\\"CV\\\" ) summary(pls.fit) ## Data: X dimension: 200 12 202","## Y dimension: 200 1 ## Fit method: kernelpls ## Number of components considered: 12 ## TRAINING: % variance explained ## 1 comps 2 comps 3 comps 4 comps 5 comps 6 comps 7 comps 68.53 ## X 23.41 31.68 35.93 46.64 55.75 61.85 96.21 ## Balance 70.92 84.90 95.30 96.13 96.20 96.21 ## 8 comps 9 comps 10 comps 11 comps 12 comps ## X 77.28 80.13 85.75 93.52 100.00 ## Balance 96.21 96.25 96.26 96.26 96.26 validationplot(pls.fit, val.type = \\\"MSEP\\\") Balance MSEP 150000 0 50000 02468 10 12 number of components # Test set MSE: pls.pred <- predict(pls.fit, x[test,],ncomp = 2) mean((pls.pred - y.test)^2) ## [1] 34724.48 # Perform PLS using full data set: pls.fit <- plsr( Balance ~., data = data, scale = TRUE, ncomp = 2 ) summary(pls.fit) ## Data: X dimension: 400 12 ## Y dimension: 400 1 ## Fit method: kernelpls 203","## Number of components considered: 2 ## TRAINING: % variance explained ## 1 comps 2 comps ## X 22.54 29.96 ## Balance 69.67 86.42 204"]