Suppose you have implemented a classifier to predict whether…

Suppose you have implemented a classifier to predict whether a hospital patient has a certain condition. You would like to use the classifier’s results to develop a screening strategy for when additional medical testing (which can be costly) should be conducted on a patient. Using the estimated benefit of a true positive (value of diagnosing condition with additional medical testing) and the estimated cost of a false positive (cost of unnecessary medical testing), you construct a profit curve on a test set. Describe how to use the profit curve to establish whether additional medical testing should be conducted for a future patient.    

Consider a data set of 1000 observations to be used to train…

Consider a data set of 1000 observations to be used to train and validate a supervised learning algorithm. Compare how these data would be used in a 10-fold cross-validation procedure versus a static holdout procedure in which 70% of the data is used to train the model and 30% of the data is used to validate the model. For both of these approaches, compare: The number of different supervised learning models trained.  The size of the training set for each supervised learning model trained.  The total number of out-of-sample predictions on which performance measures are computed.